Skip to content
Products
Platform
Developer Tools
Portal

Migrate data using rclone

Prerequisites

Section titled “Prerequisites”

You will need to have the following things ready before you start:

Migrate your data

Section titled “Migrate your data”

Setup source and target on rclone

Section titled “Setup source and target on rclone”

You can easily setup the source and target using the configuration file ”~/.config/rclone/rclone.conf”

Just add source / target including the access and secret key to this config file and safe.

For example:

cat ~/.config/rclone/rclone.conf [source] type = s3 provider = Other access_key_id = ***************** secret_access_key = ***************** endpoint = https://platform.cloud.schwarz:8080 acl = private [destination] type = s3 provider = Other access_key_id = ***************** secret_access_key = ***************** endpoint = https://object.storage.eu01.onstackit.cloud acl = private

Afterwards you can verify that both target and destination have been added correctly using the command “rclone config”

We also recommend to encrypt your configuration file to protect the access and secretkeys within your config by selecting “Set configuration password” on the “rclone config” command.

For example:

rclone config Current remotes: Name Type ==== ==== destination s3 source s3 e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> s Your configuration is not encrypted. If you add a password, you will protect your login information to cloud services. a) Add Password q) Quit to main menu a/q> a Enter NEW configuration password: password: Confirm NEW configuration password: password: Password set Your configuration is encrypted.

Migrate the data

Section titled “Migrate the data”

You can now migrate the data from source to target using “clone copy source:/$BUCKET_NAME destination:/$BUCKET_NAME”.

On our example we also added a few usefull parameters for statistics, logging and multi part uploading.

rclone copy source:/directory1 destination:/directory1 --s3-upload-cutoff 0 --progress --log-file /tmp/rclone.log Enter configuration password: Transferred: 300 / 300 Bytes, 100%, 218 Bytes/s, ETA 0s Errors: 0 Checks: 0 / 0, - Transferred: 50 / 50, 100% Elapsed time: 1.3s

Once finished, review the statistics and the log file for any error.

Further Information

Section titled “Further Information”