Automatically mirror folder from seedbox to local machine without deleting local files when they are deleted on the seedbox

Posted October 21st, 2018 5018 2

Hey,
Im currently trying to automatically setup a way to download files from my "TV" folder on the seedbox to my TV folder on my local machine.

I already tried out resilio sync but the problem is that resilio sync deletes files from my local machine if they are deleted on the seedbox. Apparently there is no way to turn this off.

I also got syncthing to work and via the "ignore deletes" option managed to get it not to delete files when they are deleted on my seedbox. Unfortunately syncthing is painfully slow for or doesnt sync at all.

Im currently trying to get lftp to work so that it automatically mirrors the specified folders but I just wanted to ask whether anyone has had the same problem and could provide an easy fix for me.

7 Replies

MrAudi

about 6 years

Accept Answer

I actually want to do the exact same thing as you and I've just started to look into Resilio for that very purpose. I have a thinking to solve this but I don't know if it'll work.

Can you add the individual sync folders by TV Show name? That way, say Gotham Season 01, is finished airing. All you have to do is delete the sync folder from Resilio in your seedbox so you can delete the files off your seedbox.

tony

Staff

almost 6 years

Accept Answer

You might want to try "Syncthing" instead of resilioSync, as it is probably able to do what you ask. In syncthing, you can use an ignoreDelete flag to the folder, which should ignore any deletions and only sync new and changed files

https://docs.syncthing.net/advanced/folder-ignoredelete.html?highlight=ignoredeletes

Syncthing is also available an an installable app in your client area of your seedbox

vaiz85

almost 6 years

Accept Answer

Resilio sync doesn't actually delete the files it just moves them by default to an archive folder in '\"yoursyncfolder".sync\archive' assuming you have the setting "Store Deleted Items in Folder Archive" enabled(its enabled by default). From there you just need create a simple move files script and add it to Windows scheduled tasks. For instance, mine is set to move anything in the archive folder to a permanent directory every morning at 3am.

It actually does delete them. I tested it without the move to archive option on and it still deletes them. The move to archive option does send them to an archive folder that will delete them automatically after 30 days or disabling the archiving deletes them instantly. Here's more info on it. Really wish this worked without deleting. https://help.resilio.com/hc/en-us/articles/205458125-Folder-Preferences - by somedude (over 5 years ago)

tony

Staff

almost 6 years

Accept Answer

I am not sure why syncthing is slow for you, it should not be. Nevertheless, if command line tools are also an option, then rsync or lftp are probably your best choice to do what you want. Both tools are very powerful and can work for any syncing scenario, however, they need to run on a predefined schedule and cannot detect a change in your folder in order to start syncing immediatelly

I just submitted an 'idea' to add incrond, which is a clean way to run a script/executable when there's a change in a folder. It's based on inotify, the same mechanism SyncThing uses to be told by the filesystem about changes, and very efficient - no need to poll. Hope you'll consider adding it :) - Paul - by CrushableMap (almost 6 years ago)

CrushableMap

almost 6 years

Accept Answer

One trick I've used in the past is to hardlink the files into another folder when the download is complete. I'm guessing you're doing something like 'remove when ratio x is reached' in a torrent client.

Hardlinking is very fast, and doesn't consume any additional disk space. You're essentially pointing to the same file on disk from two places in the folder heirarchy. Either one can be deleted without affecting the other; the file will only be removed when both links are removed. Note that hardlinking is not the same as symbolic linking.

In my case, one folder was called 'seeding' and the other 'complete'. I was using a deluge 'torrent complete' script to hardlink the just-completed torrent download from 'seeding' into 'complete'. 'complete' was being sync'd to a remote machine via SyncThing.

The remote machine was running Sonarr, which would notice a new file arriving in the remote 'complete' mirror folder being maintained by SyncThing, and import it. Which moved it out of the remote 'complete' folder, which SyncThing interpreted as it being deleted, and reflected the deletion back up to the seedbox where it would be removed from the 'complete' folder there. But it was still accessible in the 'seeding' folder, so deluge remained happy and would continue to seed it until it reached the target ratio, and then remove the link in 'seeding'. Only then would the file on disk actually be deleted.

If, on the other hand, SyncThing was unhappy for any reason, the file would sit in the 'complete' folder until SyncThing was happy again and transferred it (starting the chain of events above). In the meantime, deluge could delete the file without it affecting SyncThing in the slightest.

Long-winded explanation, I know, but hopefully it's clear how this sidesteps the problem. Basically it decouples the two peices of software from having to know when it's safe to remove the file.

Paul

CrushableMap

almost 6 years

Accept Answer

I dug up the 'torrent complete' script I wrote for deluge, to provide a concrete example. I posted it as a public 'gist' on github (with an improved version over the one I posted here first), so it's more widely visible than only here:

https://gist.github.com/paul-chambers/71ef48e40449ec73eef95430b9e4e6c7

markwillams180

over 5 years

Accept Answer

i have got movies on my pc i cant find away to put them on plex any help thanks