Site icon David Yin's Blog

Mastodon Self-hosting – Clean Media Storage with tootctl command

Check the current situation.

davidyin@instance-20230911-1538:~$ sudo su - mastodon
mastodon@instance-20230911-1538:~$ cd live
mastodon@instance-20230911-1538:~/live$ cd bin
mastodon@instance-20230911-1538:~/live/bin$ ls
bundle  heroku-web  rails  rake  retry  rspec  setup  tootctl  update  webpack  webpack-dev-server  yarn
mastodon@instance-20230911-1538:~/live/bin$ ./tootctl media usage
Attachments:    11.9 GB (23.8 MB local)
Custom emoji:   91.7 MB (8.5 KB local)
Preview cards:  481.8 MB
Avatars:        768.8 MB (18.8 KB local)
Headers:        1.5 GB (85.5 KB local)
Backups:        0 字节
Imports:        0 字节
Settings:       289.6 KB
mastodon@instance-20230911-1538:~/live/bin$

There are 1.5GB  of header images. none of which are from local accounts except 85.5KB which is my account header on the server.

For Avatars, it is 768.8 MB, only 18.8KB is from my instance.

 

 ./tootctl media remove --days 1 --prune-profiles --dry-run
6700/6700 |=============================================================================================================| Time: 00:00:09
Visited 6700 accounts and removed profile media totaling 1.8 GB (DRY RUN)

Do a dry-run to see what will happen to delete profile media from accounts I haven’t seen in one day or more.

Let’s run the actual purge operation.

mastodon@instance-20230911-1538:~/live/bin$ .//tootctl media remove --days 1 --prune-profiles
Error processing 111460850187830601: end of file reached
Error processing 111460829052463449: end of file reached
6702/6702 |=============================================================================================================| Time: 00:05:58
Visited 6702 accounts and removed profile media totaling 1.8 GB

Check the usage of storage now.

mastodon@instance-20230911-1538:~/live/bin$ ./tootctl media usage
Attachments:    11.9 GB (23.8 MB local)
Custom emoji:   91.7 MB (8.5 KB local)
Preview cards:  483.3 MB
Avatars:        170.2 MB (18.8 KB local)
Headers:        340.4 MB (85.5 KB local)
Backups:        0 字节
Imports:        0 字节
Settings:       289.6 KB
mastodon@instance-20230911-1538:~/live/bin$

OK. 1.8GB spaces is saved.


There are more commands for maintenance jobs.

# Prune remote accounts that never interacted with a local user
mastodon@instance-20230911-1538:~/live/bin$ ./live/bin/tootctl accounts prune
1195/1195 |=============================================================================================================| Time: 00:00:47
OK, pruned 1168 accounts


# Remove remote statuses that local users never interacted with older than 3 days
mastodon@instance-20230911-1538:~/live/bin$ ./tootctl statuses remove --days 3
Creating temporary database indices...
Extract the deletion target from statuses... This might take a while...
Removing temporary database indices to restore write performance...
Beginning statuses removal... This might take a while...
21/21 |=================================================================================================================| Time: 00:00:02
Done after 2.361309051513672s, removed 20643 out of 20643 statuses.
Removing temporary database indices to restore write performance...
Run ANALYZE to statuses...
Beginning removal of now-orphaned media attachments to free up disk space...
3318/3318 |=============================================================================================================| Time: 00:09:31
Done after 571.8121016025543s, removed 3318 out of 3318 media_attachments.
Creating temporary database indices...
Extract the deletion target from conversations... This might take a while...
Removing temporary database indices to restore write performance...
Beginning orphans removal... This might take a while...
18/18 |=================================================================================================================| Time: 00:00:00
Done after 0.8161077499389648s, removed 17359 out of 17359 conversations.
Removing temporary database indices to restore write performance...
Run ANALYZE to conversations...

mastodon@instance-20230911-1538:~/live/bin$ ./tootctl media usage
Attachments: 9.6 GB (23.8 MB local)
Custom emoji: 91.7 MB (8.5 KB local)
Preview cards: 483.3 MB
Avatars: 168.1 MB (18.8 KB local)
Headers: 331.0 MB (85.5 KB local)
Backups: 0 字节
Imports: 0 字节
Settings: 289.6 KB
# Remove media attachments older than 3 days
mastodon@instance-20230911-1538:~/live/bin$ ./tootctl media remove --days 3
2357/2357 |=============================================================================================================| Time: 00:01:36
Removed 2357 media attachments (approx. 1.7 GB)

# Remove all headers (including people I follow) Very aggressive
mastodon@instance-20230911-1538:~/live/bin$ ./tootctl media remove --remove-headers --include-follows --days 0
7235/7235 |=============================================================================================================| Time: 00:00:39
Visited 7235 accounts and removed profile media totaling 330.9 MB


# Remove link previews older than 3 days
mastodon@instance-20230911-1538:~/live/bin$ ./tootctl preview_cards remove --days 3
1926/1926 |=============================================================================================================| Time: 00:00:40
Removed 1926 preview cards (approx. 269.7 MB)


# Remove files not linked to any post
mastodon@instance-20230911-1538:~/live/bin$ ./tootctl media remove-orphans

#The orphan removing command took a little bit longer to remove these files. And I run the usage of storage command again to see the results.

mastodon@instance-20230911-1538:~/live/bin$ ./tootctl media usage
Attachments: 7.9 GB (23.8 MB local)
Custom emoji: 91.7 MB (8.5 KB local)
Preview cards: 213.6 MB
Avatars: 168.2 MB (18.8 KB local)
Headers: 85.5 KB (85.5 KB local)
Backups: 0 字节
Imports: 0 字节
Settings: 289.6 KB

 

After these procedures, the total space I saved is 12.99GB – 8.37GB =4.62GB.

 

My question is why Backblaze did not show the correct space occupied. Mastodon Administration Dashboard shows as 8.36 for media storage.

The Backblaze web interface, in Caps & Alers.

It is 28GB. I don’t know how to read it and why it is such a big number.

Exit mobile version