Migrate Data from a Block Storage

Migrating OpenStack Volumes Between Providers Using Rsync

Migrate a volume from one OpenStack provider to another using rsync to ensure data consistency and minimize transfer time.

Prerequisites

1. Active accounts with both the source and target OpenStack providers.
2. OpenStack CLI tools installed on your local machine or a management server.
3. Rsync installed on both the source and target instances.
4. SSH key-based authentication set up between the source and target instances for secure, password-less transfers.
5. Adequate network connectivity between the source and target OpenStack instances.

Steps

Preparation

On the source and target instance, ensure that rsync is installed:

sudo apt-get install rsync

SSH Key Setup

If you haven’t already, set up SSH key-based authentication between the source and target instances.

Click here to expand…

Generate a New SSH Key Pair on the Target Instance.
SSH into the target instance and generate a new SSH key pair that will be used only for the rsync transfer:

ssh user@target-instance-ip
ssh-keygen -t rsa -b 4096 -C "rsync-transfer" -f ~/.ssh/rsync-key

Download the Public Key to the Local Machine.
From your local machine, use the scp command to download the public key from the target instance:

scp user@target-instance-ip:~/.ssh/rsync-key.pub.

Transfer the Public Key to the Source Instance.
From your local machine, use the scp command to transfer the downloaded public key to the source instance:

scp ~/path/to/downloaded/rsync-key.pub user@source-instance-ip:~

Append the Public Key to the authorized_keys file.
SSH into the source instance, then append the public key to the ~/.ssh/authorized_keys file:

ssh user@source-instance-ip
cat ~/rsync-key.pub >> ~/.ssh/authorized_keys

In these commands:

• Replace user with your username on the source and target instances
• Replace source-instance-ip and target-instance-ip with the IP addresses of the source and target instances, respectively.
• Replace \~/path/to/downloaded/rsync-key.pub with the path to the public key file you downloaded to the target instance.

Now, the target instance should have passwordless SSH access to the source instance, allowing for seamless rsync transfer between the two.

Stop Services and Detach Volume (Source)

Stop any services or applications on the source instance that are writing to or accessing the volume to ensure data consistency during the migration:

sudo service <service-name> stop

Mount Volume (Target)

On the target instance, create and mount a new volume with the same or greater size as the source volume.

If the target instance is an OpenStack instance you can follow these instructions. Click here to expand…

Create a new volume on the target OpenStack instance:

openstack volume create --availability-zone <zone> --size <size-in-GB> <volume-name>

List the available volumes to find the ID of the new volume:

openstack volume list

Attach the new volume to the target instance:

openstack server add volume <instance-id> <volume-id>

Ensure that the volume is also mounted inside the VM.

Rsync

Run the rsync command to synchronize the data from the source to the target volume:

rsync -avz -e user@source-instance-ip:/path/to/source/volume/ /path/to/target/volume/

Verify Data

Once rsync completes, verify that the data on the target volume matches the source.

Remove temporary SSH-Key

After successful transfer ensure that the temporary SSH-Keys created in Step 1.2.2 are removed (if no longer needed).

Start Services (Source and Target)

Restart any services that were stopped on the source instance, and start necessary services on the target instance to begin using the migrated volume.

sudo service <service-name> start