Share storage over a network

Tags: internet, nbd, block device, storage, howto,

Added: 2025-02-21T00:00

Share storage over a network

This shares some storage from one computer completely privately with another. You and a friend can exchange storage so that you've both got a free offsite backup.
If you use LUKS (cryptsetup), the data written by the client goes through an encryption layer before being transferred, so the server never sees the unencrypted data.
You -> encryption -> nbd-client -> network -> nbd-server -> remote block device

Warning - some of these commands can delete data - be really careful with the device names

SERVER

# This toy example uses a 1GB file for the block device. You can substitute it with /dev/sda2 or whatever if you want to use a real disk partition

apt-get install nbd-server
dd if=/dev/zero of=/home/nbd-test bs=1G count=1 # Destructive command - be careful
sync
chown nbd:nbd /home/nbd-test
chmod 600 /home/nbd-test

cat /etc/nbd-server/conf.d/share.conf
[NAME] exportname = /home/nbd-test blocksize = 4096 authfile = /etc/nbd-server/allow
cat /etc/nbd-server/allow
<the client IP address>
/etc/init.d/nbd-server start

CLIENT

apt-get install nbd-client cryptsetup
modprobe nbd
nbd-client <server_ip> -connections 8 -timeout 60 -block-size 4096 -persist -name NAME

At this point, you have now effectively "plugged in" a disk to your machine over the internet (probably accessible as /dev/nbd0)
You can now set up the encryption and format it ready for use.

cryptsetup -v luksFormat --type luks2 --sector-size 4096 /dev/nbd0 # Destructive command - be careful
cryptsetup open /dev/nbd0 offsite
mkfs.ext4 -m 0 /dev/mapper/offsite # Destructive command - be careful

You can now mount it, and use it as a normal disk.

mkdir /offsite
mount -o errors=remount-ro,noatime,commit=30 /dev/mapper/offsite /offsite

Now whatever you copy to and from /offsite will be encrypted and sent over the network to be stored on the remote block device.

Backup the LUKS header

The first couple of megabytes of the disk contain the LUKS header. If this is overwritten/corrupted, then all the data on the disk is permanently lost. It might be worth backing up the header locally. cryptsetup luksHeaderBackup /dev/nbd0 --header-backup-file /path/to/local/luks_header_backup.bin

Asynchronicity

When you copy a file to it, it might return the command prompt before the file is fully transferred. Run the sync command afterwards to ensure data is flushed fully.

Security

This example doesn't support any authentication - use firewalling to prevent randoms being able to connect and overwrite your data. (They won't be able to decrypt, but they could overwrite).

Performance

Remember the storage is running over a network, so it will be a lot slower than locally attached disks. Network slowness/packet loss/reordering may cause strange issues. MTU can also be an issue. Test you're not fragmenting packets with ping -6 -M do -s 1452 <server_ip>
Wireshark/tcpdump will be able to give you more info.

You could use a VPN to protect the traffic between the two endpoints (although the data is encrypted before leaving the client)
Add a comment

Your IP:
Please enter 3941217 here: