A Beginner's Guide To Rclone

foreign [Music] a program that you can use to backup and sync files to the cloud over a network or to storage devices while I'm pleased

with the response of the video looking back I know I could have done better so here's my updated beginner's guide to our clone the first thing we need to do is go ahead and download our clone you can download our clone by going to

rclone.org and going to downloads I'm on windows so I'll get the 64-bit version and download the zip you can also download our clone directly from GitHub and I have links in the

description and you just go to latest releases and download our clone for your supported system once downloaded we'll go ahead and extract the package

and the next thing we need to do on Windows is ADD our clone to our system path this is so we can use our clone in the command line what I'll do is go to my C drive I like to place our

clone in the folder at the root called arcline and I'll just paste the executable here

and then in Windows settings you want to go to about Advanced system settings and under environment variables

go to your path and edit I'm going to click on new and locate that folder that you place our clone my

case is going to be at the root in the folder are clone now if I open the terminal and type

arcline we see that our clone is installed because it's giving us all these options now that we have our clone set up let's go ahead and set up a cloud service so that we can sync some data in this

example I'll be using Amazon S3 but as you can see our clone supports a wide range of cloud services anyway to set up a cloud servers we'll type our clone config

this gives us a couple of options such as creating a new remote or cloud service so we'll hit in the option in to create a new remote now we'll have to give it a name you

want to keep the name short and simple because this is the name you're going to be referencing for your source and destination paths so I'll just call it S3

and then it gives us a whole list of services I want Amazon S3 or Amazon S3 compliant so I'll just hit five the option five

in the internet now there are some additional options for Amazon S3 I'll just stick with number one for web service next we need to set up our secret and

access keys if you have these stored in your environment variables you could just hit two for True in my case I'm going to hit one so I can manually enter the keys

and I'll Supply the access key and the secret key now we need to set the region I'll choose number one for us East one

and we can set the end point I'll just go ahead and hit enter for the defaults now we can set up permissions or access

number one is full access public read access and so on I'll hit number one then hit enter on these defaults

you want to edit Advanced config copy enter for no okay with these settings yes enter for yes and queen out of this with q

now if I type R cone with remotes it shows us our services that we're connected to and if I do article LSD on

S3 colon it shows all the buckets in my Amazon S3 account let's look at uploading and downloading files we can do this with the copy command in our clone if I do R

clone LSD again on S3 we get a list of our bucket and if I use the ls command on

tree net backup we get nothing because there's nothing in that folder I'm going to go ahead and copy or upload my videos folder at the

root of my C drive to the drainet backup bucket in a folder called videos so to do this I'll just do our clone copy and we need to set the source in the

destination The Source in this case is going to be C videos and the destination is going to be to my

S3 colon the bucket name in this case spray and backup videos now there isn't a folder called videos in that backup but it will be

created by our clone enter it's complete and if I bring up my S3 browser here and go to that bucket

we can see in videos has been created I have the two videos that I uploaded if I do LS or drain that back up again

you see it gives us two results the two videos that we just upload next I want to upload this sample.txt file then to drain that back up we do our clone

copy from the desktop sample .txt to S3 colon the bucket name

and I'll place it at the root and so I can see what's happening I'll use the flag e for progress and it gives us some information about the transfer and the amount of time it

took if I do LS on drain that backup again now we see not only do we have the videos that we uploaded we now have the

sample.txt file uploaded if I go back into S3 browser we see sample.txt let's look at downloading files downloading is the same as uploading except we switched

the source and destination lsf command on netbackup we have our videos and Sample I'm going to go ahead and delete the videos folder on the root of my drive

I'm going to go ahead and download that folder we'll do our clone copy this time from S3 net backup videos

to my sea drawing videos NSP to get to progress and we can see here the progress and we're downloading these files the videos folder has been created

and it's complete we now have our files back if I wanted to download a single file let's go ahead and copy the sample

.txt into the C videos folder and we now have sample.txt downloaded let's look at syncing how syncing works

is that every time you run the sync command any changes made to the source will be mirrored to the destination so if you edit or remove a file at the source those changes will be made at the

destination if I run the command lsf on drainet backup we get the videos folder in sample.txt

I'm going to use the sync command or clone sync and set the destination and set the source

I'm going to sync my software folder which has a zip and a text file do S3 colon net backup software

I'm going to use the option minus B this will show each step of the process along with minus capital P so we can see the

progress of this sync and we can see what's happening here it went really fast but basically

we transferred two files five megabytes and if I do lsf on that backup again we now have the software folder LS7

software shows the data now I'm gonna go ahead and modify this downloads.txt that has Simple Start

off back text now add Photoshop save that out and re-run the sync and we see a check two files one file

was transferred and if we do our clone cat on that file we can see the contents of downloads.txt

which says start on that photoshop so the changes were made to downloads.txt that's stored on Amazon S3 if I go ahead

and delete this file and run the sink again we'll see one file was deleted lsf on software and we no longer have that text file be

very careful with sync because you can lose data this way or you screw up the source and destination paths when using the sync and copy commands I suggest you use a dry run this allows you to test

your sync and copies to make sure that the proper files are being transferred you can do this by applying the dry run flag

to a command I've gone ahead and deleted the software folder in the net backup on Amazon S3

and I'm going to resync the software folder but this time we're going to try run it first clear the screen again and we'll use the sync command with dry run

and basically it went through the motions of syncing without actually syncing any data so no data was transferred instead it tells us this ZIP file will be transferred and this text

file will be transferred another thing to ensure that the proper files are being transferred this would be minus all right option or interactive

what this will do is ask you if you're sure you want to upload or transfer the file for each item so in this case it starts with start all

back.zip uh we sure we want to upload this file we can say yes or skip it and so forth I'll say yes now do we want to copy the downloads.txt

I'll say yes and transferred let's look at filtering you can create some incredibly complex and complicated filters to ensure that the proper files and folders get

transferred when you copy and sync file or when you do things like delete move or List files I will do a video on filtering in more detail but in this video we'll look at

some simple filters that you can use to better control your syncing at the root of my C drive I have a pictures folder with a folder called wallpapers in this

background image I'm going to sync the pictures folder to Amazon S3 in the folder called pictures in my wallpapers folder I have Brave folder with

wallpapers and they're all varying different types momentum wallpapers now I want to sync this pictures folder but I don't want any files

that are webp so we have some webp images here and I don't want those synced up so we can do let's use the exclude flag

and tell our clone to exclude all files to our webp we could do this with star Asterix indeed file extension web p

I'll use the dry run on this so we can see what would happen and we should see no web P files

being transferred let's go ahead and run this command transfer is complete and sure enough if I go into the pictures folder here

and wallpapers Brave we should see no webp images let's run this command again except this time I want to filter out the bray folder for the filter item folder we can

do forward slash and this is going to be in wallpapers for age followed by a forward slash and two asterisks

and everything is being copied except the brave folder in its contents it's complete lsf on pictures

and we have of all papers and the image but no grave and the wall of thickness just momentum now bear in mind that our

clone is case sensitive so if you use the sync command such as this you have to make sure the name is the exact same as in your file system if this is all lowercase

then Brave here needs to be all lowercase or it won't work you can also use the ignore case flag which will tell our clone to ignore

K sensitive you can also use ignore case think when syncing that way you won't have to worry about case sensitivity let's look at include so back at the pictures folder and I've gone ahead and

deleted it again on Amazon I want to only include files that are of type webp we can use include so include

asterisks by e this will include all files only files that are a web P key difference and include is that it will only include

the files that you specify and ignore everything else so even though we're copying this pictures folder only folders that have webp images will be copied and it's done

it looks like only three files were copied lsf on wallpapers grade and we see only these three files in an

S3 browser we only have the wallpapers folder with brave and those files we don't have anything else

remove this and we're going to run this again and now I only want the brave folder so we're just going to do this again

and only include wallpapers but and only the brave folder should be included in the sink

not this image and not this folder momentum it's complete an S3 browser if I refresh

we have wallpapers and just a braid folder with its contents these include and exclude Flags can be repeated

so if you wanted to include a folder here and then another folder if we stack them like that same with exclude however you should not

combine the two so if you use include in a command don't also use x blue in cases where you want to include and exclude you use the filter flag

pictures is gone and I'm going to sync the pictures while filtering out everything except momentum and this file

at the root of pictures to start I'm going to filter out Brave in the filter command and you want to filter something out you use a minus

sign followed by the path in this case I don't want wallpapers Brave so I'll do papers breathe

and I'm also going to filter out this image by type it's a jpeg so I'll do filter

minus and I'm going to Target the root of the pictures folder on any file that's a JPEG but that's a file type jpeg and then I'll use the filter

to include everything else in wallpapers we won't include something we can use the plus sign followed by the path and we'll do a dry run on this let's see

what happens and we can see only the momentum folder is going to be copy transferred there's no picture at the root of wallpapers and

there's no brain folder so this sync is going to work so let's get rid of the dry run and run this command foreign pictures

wallpapers and we just have momentum with no other file or folders now this filtering looks a bit archaic and in the upcoming video

I'll show you a much better way for filtering using a filter file similar to what's in this file where we

specify what we don't want included in our backup and sync and what we do and that would be done with the filter

from and our filter.txt file so be looking out for that video let's look at creating folders you can use our clone to create folders at a

given point you do R clone LK dir I'll create a folder on my desktop here and call it ready

and we see the braid folder here if we do the same command on say Amazon S3 my remote we get an error because Amazon S3 cannot

have empty directories some Services support this some don't you can't create empty folders or USB devices and network drives I'll go ahead

and create a folder on my Nas here labeled in mkdir on in and we see the folders created

now I'd like to show you how to move files and folders what Amazon does three here I have the brave folder and I want to now move the momentum folder over

into pictures wallpaper you can use the move command rclowing Moon if you want to move momentum

notice I'm on my desktop so I'm using the relative path here the S3 and let's run then and if we open up this folder

we can see here the files are being deleted from the destination as they are being transferred it's complete and if we refresh in pictures

wallpapers we have momentum let's look at deleting files and folders we'll start by deleting this folder momentum we can do our clones rmbir and the path

of the folder or file and momentum is gone simple if that rmdir only deletes empty folders so

if I did this again and copied the sample file ran this command again you get an error because this folder is

not empty if we wanted to delete the folder in all of its contents we can use the purge command our phone Verge momentum and

it's gone next I want to delete a file and if I do lsf on my S3 pictures wallpapers break you've seen some files

here and I want to delete this expensive more desert image this webp image Quran to run the delete command

and tell our clone delete to delete this image we do lsf

we see the image is gone let's tell our cone to delete the brave folder itself on Amazon S3 so we'll do

our clone delete lsf one wallpapers and we just have momentum now usually

when you use the delete command it will only delete the content and leave the folder structure itself alone however we can't have empty folders in Amazon S3 so that's why it's completely gone I've

added a brain folder here on the desktop with an image in it and if I run this command again but this time Brave on my desktop and we click it we see that it's empty

but the folder is still present on my desktop while our clone is commonly known for being able to access cloud storage you can actually use our clone to sync and backup data over a network

or do a USB device here we have a USB drive labeled e and I've got some documents on here I'm going to go ahead and delete that and I'm going to sync over my documents that I have on my C

drive to the drive sync documents to E and I'll create a folder called documents and we can see the progress

and document folder has been created on that drive likewise if I wanted to send files over to the network

on my nav here like we did earlier I simply use the letter the letter drive itself or I can use the IP address of the device

plus the share name in this case is called backup in documents and again

on over my Nas I have documents with my Nas storage reports being copied you can also use the name of the server

I'll do now is copy the videos folder and instead of using the IP address I'll use the name of the server

plus the share name and where I want to store the data again we have videos using our clone you can actually Mount

all the cloud connected services to your PC as network drive if I do arcone mounts on my Amazon S3

and give her the letter name I'll give it J colon and it entered we'll get an error because in order for this to work we'll

need to install winf SP it's a very simple program that simply allows our clone to mount cloud storage as network drive so for this all we have to do is

type winget install when fsp this will go ahead and install that little program and we'll walk through the installation it's complete

clear the screen here now when we run this command again it started the service if I go to my PC we see we have my drive here and this is

my Amazon S3 with all of its buckets where I can go into net backup and look at all the files now notice this drive is listed under devices and drives if

you didn't want that you can actually use network mode to have it listed under Network locations I'll use Ctrl C to get out of this and run the command again

this time of pinned Network Mode now we will see it listed under Network locations some programs might not work well with files served in this way when you run

into issues like that you could simply use the VFS cache mode and set the level in this case I'll use fool and you can also use rights

you can learn more about the differences between rights and full and the documentation just know that for best compatibility you want to include the VFS cache mode if this is a feature you

want to use more often it'd be best to set up a task to automatically run this when you log on your PC if I open up task scheduler and create a new task I'll go ahead and create a task to

automatically run this script so that I can connect to my S3 the moment I log into my computer not just call it Mount Oracle under actions I'll create a new action

and we'll start a program and I'll point to the autoclone executable so so see autoclone our clone

EDC and then the command so we're going to mount our S3 to J and network mode I'm also going to set the flag no constant to hide the console window when

it starts hit okay yes if I close this log out and sign back in

we should see we're connected to S3 now I'd like to show you how to create a backup and sync script that we can use to automatically back up at a

given time or manually I've gone ahead and deleted everything in the tradenet backup bucket on Amazon S3 and I want to now sync or back up my documents

videos and software folders what I'll do is create a new file on my desktop and I'll call it backup .bat or CMD for script I'll do a batch

.bat or CMD for script I'll do a batch file and all we have to do is just put the script so first I'm going to reference the

executable c r clone our clone.exe

our clone.exe and the command that we want to run if you want to do a traditional backup you could just use the copy command which

will only upload changes and new files or sync if you want the destination to mirror your source in this case I just want to do a backup

so I just do copy and since we know everything is going to be coming from the C drive we'll say c onto S3 handy Gray

in that backup bucket now we're going to apply filters because we're not going to obviously upload the entire C drive so when used to include filter to

include our three documents so include documents will also include software and videos

I'll apply ignore paste Linux P for progress and to see how this is going to work we'll use dry run and I don't want this to exit out so

I'll do CMD okay and Save and I forgot this needs to be net backup not net back

so we'll save that out and run this and we see our dry run it's complete and it's going to back up

400 megabytes so now let's go ahead get rid of the Subscribe run and now we know our script is going to work so I'll get rid of this because we're not going to need this it's going

to be automatic backup and get rid of this minus p save this again and now the next thing we need to do is create a task so that windows can run this script this batch

file actually one other option I want to add here is no console this will prevent the console from opening

in our face when the file runs now we'll go back to task scheduler and create a new task for our backup I'll go

ahead and delete this mount create a new task I'll have it set to 12. now we're going to run it manually but just to show you how to set this up

so run the backup at 12 am every day and we can repeat the task I'm just gonna say okay and under actions we'll start a program scripts and we

will run the backup.bat

and hit OK now I'll manually trigger this by right clicking and say run now I hit the console but it is backing up

if I go to the S3 browser here and refresh we can already see files have been copied over and if I go into my videos we don't have

those videos files yet but they will be coming the console is here running and it's complete and closed

back at Amazon S3 if I refresh go to videos I have all the data that I said I want to back up so the backup was successful it was a

very simple and basic example of how to use autoclone for backing up and syncing the true power in our clone is with its filtering let's look at serving files over a network or the internet using our

clone you can set up your cloud storage as a simple file server with support for

HTTP webdav FTP SFTP even rustic and docker all we have to do is do our film with the serve command and the protocol

that we want to use in this case I'll do HTTP in the cloud service Coleman then if I go to my browser here

and under unlockable holes I believe the default is 8080 Port but you can customize this and it opens up her S3

and I have access to my S3 buckets you can customize the port if you want or the entire IP address with the address flat

how to change the port to any 88 and I'll set a user with the user flag and set a password the past flag

you might get this little window security warning allow our clone access and now when we go to Port 8088

we have to put in our password our username and password and we've got access you can then set this up to work outside

your network but I strongly recommend against that as this is really for basic file but it is a really cool feature one last major feature I want to show you

with our clone is the web interface foreign rcd with the flash RC or web GUI

will actually start up the web interface that you can access and this is a highly functional little web interface that you can use for managing our clone as well as

monitoring transfers under config you can see all of your connected cloud services I have S3 you can also go ahead here and add a service

from here what if I wanted to add B2 you can select our type Supply some keys and now I have B2

under explore I can browse my cloud services maybe I want to look at B2 open that up and now I can look at all my files S3

and here you can actually upload files download the file backend gives you a whole bunch of information and settings about our bloom

under mounts you can actually quickly mount a cloud storage what if I wanted to go ahead and mount B2 give it a letter maybe eight you can set all these options and

parameters and then it mounts on my desktop here where I can go and browse my buckets you

can unmount directly from this interface so this is a really convenient way to manage our clone without having to always use the command line lastly I want to show you how to update

our clone blue arc loan you can see self update command self update we can update our clone and it says we're up to date if there was an update

it would go ahead and download the latest version and by the way all of these commands you can see and get help with using our clone the command itself followed by eight and this will give you

information about a particular command and how to use all of our clones configuration data is stored in the autoclone config file to find this file you can use our clone

and the fee with the option y app data roaming or clone we can see the file it is effectively a DOT ini file that

contains information about all your cloud storage connections and their options our clone is an exceptionally powerful piece of software and in my opinion easily one of the best backup and sync

Solutions you're gonna find understand that I've only scratched the surface of what this lightweight cross-platform little program can do I didn't demonstrate it but you can even copy and sync files from one cloud provider to

another in an upcoming video I'll be covering more advanced filtering and syncing options

[Music] thank you foreign

[Music]