rclone and rsync

- rclone browser
  https://github.com/kapitainsky/RcloneBrowser
  https://rclone.org/gui/
  `rclone rcd --rc-web-gui`
- sync to remote storage
  https://research.it.iastate.edu/guides/pronto/file_transfers/rclone/
  https://rclone.org/internetarchive/

	[Box - Pronto User Documentation.pdf](../../../_resources/Box%20-%20Pronto%20User%20Documentation.pdf)
	```bash
	rclone config
	# Select new remote and when ask for a name, give one
	# After that it will ask what cloud storage u want be able to sync
	# Then it will ask you for api cresentials or something else
	# but choose to rather login with your brower for simplicity to authorize access
		
	```
	
	`rclone sync /notes remote_cloud_name:/syncF/`
	`remote_cloud_name` is the name of remote 

- simply copy
  `rclone copy source dest`
 
- encryption
   - For configuring select `crypt` when asked for cloud provider then select another config like `remote_box` as `remote_box` is the name of config for box account
  https://www.maketecheasier.com/use-rclone-crypt-encrypt-files/
  To encrypt before uploading to host
  https://small.fileditchstuff.me/s6/FXtvAmOlrKQaaPVabACZ.mp4
- completely deleting directory with its contents
  `rclone purge remote_box:/syncF`
- removing files
  `rclone delete remote:box/syncF/filesList.txt`
- list contents of remote directory
  `rclone lsf remote_name:path/to/directory`
- excluding files
  `rclone copy . ./destination/ --exclude="node_modules/**"  --exclude=".*/"`
  `.*/` represents directories which starts with dot
 - creating a shatable link
  `rclone link remote:/path/to/file`
   
 - mount
    ` rclone mount keepass_drop:/ ~/Dropbox_keepaas  --daemon`
- unmount
  `fusermount -u ~/Dropbox_keepaas`
	
  
 **Internet archive**
 - When configuring provide access key and secret key. Go to this url https://archive.org/account/s3.php to find 
 - For uploading in internet archive first we need to give a unique identifier which acts like bucket
   Choose any name like `itemqwerzxc` and use this command
   `rclone copy Wow\!\ That\'s\ Loud.zip  rearchive:itemqwerzxc`
- For downloading
  `rclone copy   rearchive:itemqwerzxc downloaded_from_archive/`
- encryption wont be easy and might not work. better to use zipped file maybe. Also use different itemId (unique identifier than prev unencrypted one like `rearchive:itemqwerzxcenc`)
- to serve directory through http
 `rclone serve http /path/to/dir --addr :8081`  
  
  
**Supported cloud providers**
```text
 • 1Fichier

       • Akamai Netstorage

       • Alibaba Cloud (Aliyun) Object Storage System (OSS)

       • Amazon Drive

       • Amazon S3

       • Backblaze B2

       • Box

       • Ceph

       • China Mobile Ecloud Elastic Object Storage (EOS)

       • Arvan Cloud Object Storage (AOS)

       • Citrix ShareFile

       • Cloudflare R2

       • DigitalOcean Spaces

       • Digi Storage

       • Dreamhost

       • Dropbox

       • Enterprise File Fabric

       • Fastmail Files

       • FTP

       • Google Cloud Storage

       • Google Drive

       • Google Photos

       • HDFS

       • Hetzner Storage Box

       • HiDrive

       • HTTP

       • ImageKit

       • Internet Archive

       • Jottacloud

       • IBM COS S3

       • IDrive e2

       • IONOS Cloud

       • Koofr

       • Leviia Object Storage

       • Liara Object Storage

       • Linkbox

       • Linode Object Storage

       • Mail.ru Cloud

       • Memset Memstore

       • Mega

       • Memory

       • Microsoft Azure Blob Storage

       • Microsoft Azure Files Storage

       • Microsoft OneDrive

       • Minio

       • Nextcloud

       • OVH

       • Blomp Cloud Storage

       • OpenDrive

       • OpenStack Swift

       • Oracle Cloud Storage Swift

       • Oracle Object Storage

       • ownCloud

       • pCloud

       • Petabox

       • PikPak

       • premiumize.me

       • put.io

       • Proton Drive

       • QingStor

       • Qiniu Cloud Object Storage (Kodo)

       • Quatrix by Maytech

       • Rackspace Cloud Files

       • rsync.net

       • Scaleway

       • Seafile

       • Seagate Lyve Cloud

       • SeaweedFS

       • SFTP

       • Sia

       • SMB / CIFS

       • StackPath

       • Storj

       • Synology

       • SugarSync

       • Tencent Cloud Object Storage (COS)

       • Uptobox

       • Wasabi

       • WebDAV

       • Yandex Disk

       • Zoho WorkDrive

       • The local filesystem
```

 

### Categorized
```text
1 / 1Fichier
   \ (fichier)
 2 / Akamai NetStorage
   \ (netstorage)
 3 / Alias for an existing remote
   \ (alias)
 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, ChinaMobile, Cloudflare, DigitalOcean, Dreamhost, GCS, HuaweiOBS, IBMCOS, IDrive, IONOS, LyveCloud, Leviia, Liara, Linode, Minio, Netease, Petabox, RackCorp, Rclone, Scaleway, SeaweedFS, StackPath, Storj, Synology, TencentCOS, Wasabi, Qiniu and others
   \ (s3)
 5 / Backblaze B2
   \ (b2)
 6 / Better checksums for other remotes
   \ (hasher)
 7 / Box
   \ (box)
 8 / Cache a remote
   \ (cache)
 9 / Citrix Sharefile
   \ (sharefile)
10 / Combine several remotes into one
   \ (combine)
11 / Compress a remote
   \ (compress)
12 / Dropbox
   \ (dropbox)
13 / Encrypt/Decrypt a remote
   \ (crypt)
14 / Enterprise File Fabric
   \ (filefabric)
15 / FTP
   \ (ftp)
16 / Google Cloud Storage (this is not Google Drive)
   \ (google cloud storage)
17 / Google Drive
   \ (drive)
18 / Google Photos
   \ (google photos)
19 / HTTP
   \ (http)
20 / Hadoop distributed file system
   \ (hdfs)
21 / HiDrive
   \ (hidrive)
22 / ImageKit.io
   \ (imagekit)
23 / In memory object storage system.
   \ (memory)
24 / Internet Archive
   \ (internetarchive)
25 / Jottacloud
   \ (jottacloud)
26 / Koofr, Digi Storage and other Koofr-compatible storage providers
   \ (koofr)
27 / Linkbox
   \ (linkbox)
28 / Local Disk
   \ (local)
29 / Mail.ru Cloud
   \ (mailru)
30 / Mega
   \ (mega)
31 / Microsoft Azure Blob Storage
   \ (azureblob)
32 / Microsoft Azure Files
   \ (azurefiles)
33 / Microsoft OneDrive
   \ (onedrive)
34 / OpenDrive
   \ (opendrive)
35 / OpenStack Swift (Rackspace Cloud Files, Blomp Cloud Storage, Memset Memstore, OVH)
   \ (swift)
36 / Oracle Cloud Infrastructure Object Storage
   \ (oracleobjectstorage)
37 / Pcloud
   \ (pcloud)
38 / PikPak
   \ (pikpak)
39 / Proton Drive
   \ (protondrive)
40 / Put.io
   \ (putio)
41 / QingCloud Object Storage
   \ (qingstor)
42 / Quatrix by Maytech
   \ (quatrix)
43 / SMB / CIFS
   \ (smb)
44 / SSH/SFTP
   \ (sftp)
45 / Sia Decentralized Cloud
   \ (sia)
46 / Storj Decentralized Cloud Storage
   \ (storj)
47 / Sugarsync
   \ (sugarsync)
48 / Transparently chunk/split large files
   \ (chunker)
49 / Union merges the contents of several upstream fs
   \ (union)
50 / Uptobox
   \ (uptobox)
51 / WebDAV
   \ (webdav)
52 / Yandex Disk
   \ (yandex)
53 / Zoho
   \ (zoho)
54 / premiumize.me
   \ (premiumizeme)
55 / seafile
   \ (seafile)

```


new

```txt
 1 / 1Fichier
   \ (fichier)
 2 / Akamai NetStorage
   \ (netstorage)
 3 / Alias for an existing remote
   \ (alias)
 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, ArvanCloud, BizflyCloud, Ceph, ChinaMobile, Cloudflare, Cubbit, DigitalOcean, Dreamhost, Exaba, Fastly, FileLu, FlashBlade, GCS, HCP, Hetzner, HuaweiOBS, IBMCOS, IDrive, ImpossibleCloud, Intercolo, IONOS, Leviia, Liara, Linode, LyveCloud, Magalu, Mega, Minio, Netease, Outscale, OVHcloud, Petabox, Qiniu, Rabata, RackCorp, Rclone, Scaleway, SeaweedFS, Selectel, Servercore, SpectraLogic, Storj, Synology, TencentCOS, US3, Wasabi, Zadara, Zata, Other
   \ (s3)
 5 / Backblaze B2
   \ (b2)
 6 / Better checksums for other remotes
   \ (hasher)
 7 / Box
   \ (box)
 8 / Cache a remote
   \ (cache)
 9 / Citrix Sharefile
   \ (sharefile)
10 / Cloudinary
   \ (cloudinary)
11 / Combine several remotes into one
   \ (combine)
12 / Compress a remote
   \ (compress)
13 / DOI datasets
   \ (doi)
14 / Drime
   \ (drime)
15 / Dropbox
   \ (dropbox)
16 / Encrypt/Decrypt a remote
   \ (crypt)
17 / Enterprise File Fabric
   \ (filefabric)
18 / FTP
   \ (ftp)
19 / FileLu Cloud Storage
   \ (filelu)
20 / Filen
   \ (filen)
21 / Files.com
   \ (filescom)
22 / Gofile
   \ (gofile)
23 / Google Cloud Storage (this is not Google Drive)
   \ (google cloud storage)
24 / Google Drive
   \ (drive)
25 / Google Photos
   \ (google photos)
26 / HTTP
   \ (http)
27 / Hadoop distributed file system
   \ (hdfs)
28 / HiDrive
   \ (hidrive)
29 / Huawei Drive
   \ (huaweidrive)
30 / ImageKit.io
   \ (imagekit)
31 / In memory object storage system.
   \ (memory)
32 / Internet Archive
   \ (internetarchive)
33 / Internxt Drive
   \ (internxt)
34 / Jottacloud
   \ (jottacloud)
35 / Koofr, Digi Storage and other Koofr-compatible storage providers
   \ (koofr)
36 / Linkbox
   \ (linkbox)
37 / Local Disk
   \ (local)
38 / Mail.ru Cloud
   \ (mailru)
39 / Mega
   \ (mega)
40 / Microsoft Azure Blob Storage
   \ (azureblob)
41 / Microsoft Azure Files
   \ (azurefiles)
42 / Microsoft OneDrive
   \ (onedrive)
43 / OpenDrive
   \ (opendrive)
44 / OpenStack Swift (Rackspace Cloud Files, Blomp Cloud Storage, Memset Memstore, OVH)
   \ (swift)
45 / Oracle Cloud Infrastructure Object Storage
   \ (oracleobjectstorage)
46 / Pcloud
   \ (pcloud)
47 / PikPak
   \ (pikpak)
48 / Pixeldrain Filesystem
   \ (pixeldrain)
49 / Proton Drive
   \ (protondrive)
50 / Put.io
   \ (putio)
51 / QingCloud Object Storage
   \ (qingstor)
52 / Quatrix by Maytech
   \ (quatrix)
53 / Read archives
   \ (archive)
54 / SMB / CIFS
   \ (smb)
55 / SSH/SFTP
   \ (sftp)
56 / Shade FS
   \ (shade)
57 / Sia Decentralized Cloud
   \ (sia)
58 / Storj Decentralized Cloud Storage
   \ (storj)
59 / Sugarsync
   \ (sugarsync)
60 / Transparently chunk/split large files
   \ (chunker)
61 / Uloz.to
   \ (ulozto)
62 / Union merges the contents of several upstream fs
   \ (union)
63 / WebDAV
   \ (webdav)
64 / Yandex Disk
   \ (yandex)
65 / Zoho
   \ (zoho)
66 / iCloud Drive and Photos
   \ (iclouddrive)
67 / premiumize.me
   \ (premiumizeme)
68 / seafile
   \ (seafile)

```

 

# rsync
- unidirectional sync
  \# a for archiving(preserving metadata), v for verbose, z for compressing 
  `rsync -avz --delete "$SOURCE/" "$DESTINATION/"`
- bidirectional sync but without syncing deletes, only modifiation to file
  ```bash
  # Function to sync directories
	sync_directories() {
		rsync -avu --ignore-existing "$SOURCE/" "$DESTINATION/"
	}

	sync_directories2() {
		   rsync -avu --ignore-existing  "$DESTINATION/" "$SOURCE/"
	}
	
	sync_directories
	sync_directories2
  ```