angoikon angoikon
Bookmark

How to create and use rclone remotes in your seedbox

Posted February 5th, 2020    833   0   rclone terminal remote ssh

Introduction

Rclone util is a command line program to (most commonly) move or copy files and directories from a source item (folder or file) to a destination folder.
It comes pre-installed with every seedbox.

Users might want this to utilize our fast network infrastructure in order to copy or move content from their local seedbox towards a remote drive or vice versa . The remote drive doesn't have to be a drive we currently support in seedbucket (Onedrive, Googledrive, Dropbox or any FTP/SFTP adapter), it can be anything you want - except 1Fichier or similar cloud drives that can't be installed in a headless machine.

Note: You need to know how to SSH to your seedbox before proceeding - we have an excellent guide for that here

Remote installation

In this guide, we will be connecting a Google drive remote to your local seedbox. It can be either already connected to your seedbucket or not, it doesn't really matter. You need however to have pre-created the project along with the necessary credentials through your Google API console. You can check out our Plex/Gdrive implementation guide here for information regarding the Google console.
We will use the default rclone configuration path which (will) reside in /home/user/.config/rclone/rclone.conf .
After you SSH to your seedbox terminal, follow the steps below

Create the remote

1) Issue the command;

rclone config

This will initialize the terminal wizard which will guide you through the rest of the process

file

2) Type n to create a new remote and press enter

3) Give it a distinguishable name (e.g. gdrive)

4) From the list below, type the appropriate drive number and press enter. In our case, it is 12 as shown in the picture, however that depends in rclone version installed, yours might differ

file

5) Open your Google cloud console and navigate to your project, click at Credentials and then click at the name of the OAuth 2.0 Client ID. This will open up your id and secret

file

Note: You must have already created the project, given your consent through the OAuth consent screen and created the credentials

6) Copy and paste both Client ID and Client secret at the given fields

file

7) Choose 1 to give the gdrive full access

8) At rootfolderid field, press Enter for using the default. This means that you will be at the root folder of your Google drive so you can see every single folder there.

9) At serviceaccountfile field, press Enter for using the default

10) Choose to not Edit advanced config by pressing n

11) At the Remote config, choose n as our seedbox is a headless machine. This will open a new tab at your browser so you can authenticate with Google. Accept/allow, and at the end it will give you a verification code. Copy that and paste it to your terminal.

Note: If you are not automatically redirected to your browser, manually copy the link you see at terminal and paste it in your browser

12) Choose whether this drive will be configured as a team drive or not, and at the end, you will have created a preview of the remote drive. If everything is typed successfully, choose y and this will create the remote drive

file

13) You wil now see the newly-created drive with the distinguishable name you gave it (gdrive), just press q and enter to exit the wizard. Congratulations you have created your remote drive!

Note: Depending on the drive, the above steps might differ, you can check the official rclone site for information regarding a specific cloud drive

Rclone commands

You need to familiarize yourself with lsd (List all directories/containers/buckets in the path) and ls (List the objects in the path with size and path) rclone commands, so you know where you should move or copy content to.

For example:

rclone lsd "gdrive:"

will list all directories under your "gdrive" remote. Notice the colon symbol ":" after gdrive. This is mandatory.

If you want to list all directories under a certain folder (e.g. "temp" folder that resides directly in your root folder), then you issue:

rclone lsd "gdrive:/temp"

and it will list all directories under (root) -> temp.

Same goes for ls command too.

Note: You can issue the "--max-depth 2" flag at the end of each command to control the depth (default is 1). Also, we use double quotes ( " ) so there is no need for escaping characters.

IMPORTANT: Linux is case-sensitive, which means that temp folder and TEMP folder are two different folders. You need to type the file/folder name exactly as you see it

Rclone move

You will most probably needing to move content between your local seedbox and your remote drive (or vice versa).
However, before initiating a move, you need to set some flags in order for the move to be efficient.
At your terminal paste this:

ARGS=(-P --checkers 3 --log-file /home/user/.config/rclone/upload_rclone.log -v --tpslimit 3 --transfers 3 --drive-chunk-size 32M --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36" --delete-empty-src-dirs)

We are setting a progress bar along with the necessary log-to-be-created and a User Agent, that will be used in the rclone move command afterwards. Bear in mind that we have also set to delete empty source directories, after content is successfully moved to the destination drive. If however you want to keep the empty source directory, then remove the last flag.

IMPORTANT: Each time you log to your terminal via SSH, you need to set that variable ARGS like above. Afterwards, you can initiate the rclone move (single or multiple times until you log out).

EXAMPLE:
Lets say you want to move the contents of a folder that reside in "~/files/downloads/my pictures" at your local seedbox, towards your Googledrive "temp" folder. You need to issue the following:

rclone move "/home/user/files/downloads/my pictures" "gdrive:/temp" "${ARGS[@]}"

This, however, won't move the folder "my pictures" itself, so if you want to move that folder too, you need to append it to your destination like this:

rclone move "/home/user/files/downloads/my pictures" "gdrive:/temp/my pictures" "${ARGS[@]}"

Note: You have to use double quotes everywhere as shown! This way, you don't have to escape characters.

To check if the move was successful, you can do that with two ways: Either

rclone ls "gdrive:/temp" (or rclone ls "gdrive:/temp/my pictures" if you moved the folder too), or

cat /home/user/.config/rclone/upload_rclone.log to check the logs

If you want to move content from your Google drive towards your local seedbox, the process is similar, just swap source with destination. So:

rclone move "gdrive:/temp" "/home/user/files/downloads/my pictures" "${ARGS[@]}"

Rclone copy

Copy works similarly to move, but this time the arguments mustn't have the "--delete-empty-src-dirs" flag.
At your terminal paste this:

ARGS=(-P --checkers 3 --log-file /home/user/.config/rclone/upload_rclone.log -v --tpslimit 3 --transfers 3 --drive-chunk-size 32M --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36")

We are setting a progress bar along with the necessary log-to-be-created and a User Agent, that will be used in the rclone copy command afterwards.

IMPORTANT: Each time you log to your terminal via SSH, you need to set that variable ARGS like above. Afterwards, you can initiate the rclone copy (single or multiple times until you log out).

EXAMPLE:
Lets say you want to copy the contents of a folder that reside in "~/files/downloads/my pictures" at your local seedbox, towards your Googledrive "temp" folder. You need to issue the following:

rclone copy "/home/user/files/downloads/my pictures" "gdrive:/temp" "${ARGS[@]}"

This, however, won't copy the folder "my pictures" itself, so if you want to copy that folder too, you need to append it to your destination like this:

rclone copy "/home/user/files/downloads/my pictures" "gdrive:/temp/my pictures" "${ARGS[@]}"

Note: You have to use double quotes everywhere as shown! This way, you don't have to escape characters.

To check if the copy was successful, you can do that with two ways: Either

rclone ls "gdrive:/temp" (or rclone ls "gdrive:/temp/my pictures" if you copied the folder too), or

cat /home/user/.config/rclone/upload_rclone.log to check the logs

If you want to copy content from your Google drive towards your local seedbox, the process is similar, just swap source with destination. So:

rclone copy "gdrive:/temp" "/home/user/files/downloads/my pictures" "${ARGS[@]}"

2 Comments

markelkin
markelkin

Hi there
It seems i followed the instructions to mount my tdrive - yet in plex it only sees G drive folders and not the T drive. Pleaase advise what i am doing wrong.

angoikon
angoikon
Staff

Unfortunately you can't create an rclone mount point in our seedboxes. We don't allow mount points with the exception of Plex and the Google drive Plex was integrated to.

This guide is just for utilizing our fast network with several -either supported or unsupported- cloud drives