Openrsync knows --include and --exclude:
https://man.openbsd.org/openrsync
A central server (for example your home NAS) will pull all your backups from stfp sources and store them on a local disk.
Then the backup directory gets synced with remote sftp backup servers (for example a NAS at your parents home).
Web server ---+ +--- friends home NAS
| |
DB server ---+--- pull --- Home NAS --- push ---+--- server in the cloud
| |
Mail server ---+ +--- parents home NAS
. | | .
. | | .
. | | .
server N ---+ +--- backup location N
/var/backups/sftp/data.Your home NAS. Everything gets synced here first for easy access.
Create Mountpoint: mkdir -p /mnt/backup
Create Partition:
fdisk -iy sd1
disklabel -E sd1 <<EOF
a a
*
*
RAID
w
q
EOF
Create CryptoFS:
bioctl -c C -l sd1a softraid0
Create Filesystem on CryptoFS:
fdisk -iy sd2
disklabel -E sd2 <<EOF
a a
*
*
4.2BSD
w
q
EOF
newfs sd2a
bioctl -d sd2
Fstab:
echo "/dev/sd2a /mnt/backup ffs rw,nodev,nosuid,softdep,noauto 1 0" >>/etc/fstab
Mount Command:
echo "mc() { bioctl -c C -l sd1a softraid0 && { sleep 1; fsck /dev/sd2a && mount /mnt/backup; }; }" >>/root/.profile
Unmount Command:
echo "uc() { sync; umount /mnt/backup && bioctl -d sd2; }" >>/root/.profile
Allow others to store their backups at your location.
Create group: groupadd sftp
SSH Config:
cat <<EOF>>/etc/ssh/sshd_config
Match Group sftp
ChrootDirectory /mnt/backup
ForceCommand internal-sftp
AllowAgentForwarding no
AllowStreamLocalForwarding no
PasswordAuthentication no
PermitRootLogin no
AllowTcpForwarding no
EOF
Create user and backup directory for the allowed to store backups.
Additionally you should configure one of these directories for your own user that runs the sync cronjob.
useradd -m -g sftp <user>
mkdir -p /mnt/backup/<user>
chown <user>:sftp /mnt/backup/<user>
chmod 700 /mnt/backup/<user>
Store the users ssh public key in /home/<user>/.ssh/authorized_keys.
Optionally add a disk quota to limit space usage:
Add the userquota option to the fstab entry of yout /mnt/backup mount.
Edit quotas: edquota <user>
Here you need to place all servers, you want to pull/push backups from/to.
Replace <user> by the user you chose to run the sync cronjob:
cat <<EOF>> /home/<user>/.ssh/config
Host mail
Hostname mail.example.com
IdentityFile /mnt/backup/<user>/id_ed25519
User sftp
Host db
Hostname db.example.com
IdentityFile /mnt/backup/<user>/id_ed25519
User sftp
Host www
Hostname www.example.com
IdentityFile /mnt/backup/<user>/id_ed25519
User sftp
Host parents
Hostname parents.dynip
IdentityFile /mnt/backup/<user>/id_ed25519
User <user>
Host friend
Hostname friend.dynip
IdentityFile /mnt/backup/<user>/id_ed25519
User <user>
EOF
Create the ssh key for your sync user:
ssh-keygen -N "" -t ed25519 -f /mnt/backup/<user>/id_ed25519
Give the ssh public key /mnt/backup/<user>/id_ed25519.pub to your parents/friends so they can allow you to store your backups at thier location.
Install: pkg_add sshfs-fuse
Create the base mount directory: mkdir /mnt/sshfs
Note: the mount scripts have to be called using the absolute path:
doas /usr/local/bin/mount_sshfs <host> <rdir>
doas /usr/local/bin/umount_sshfs <host>
doas config for the user which is permitted to mount an sshfs:
permit nopass setenv { USER=<username> } <username> as root cmd /usr/local/bin/mount_sshfs
permit nopass setenv { USER=<username> } <username> as root cmd /usr/local/bin/umount_sshfs
Secured script to mount an sshfs:
cat <<'EOF'> /usr/local/bin/mount_sshfs
#!/bin/sh
prefix=/mnt/sshfs
[ $# -ne 2 ] && { echo "usage: mount_sshfs <host> <remote_dir>" >&2; exit 1; }
home=$(/usr/bin/getent passwd "${USER}" | cut -d: -f6)
host=$(echo "$1" | /usr/bin/sed 's/[^a-zA-Z0-9._-]*//g')
user_dir="${prefix}/${USER}"
[ ! -d "${user_dir}" ] && /bin/mkdir "${user_dir}"
mount_dir="${user_dir}/${host}"
[ ! -d "${mount_dir}" ] && /bin/mkdir "${mount_dir}"
/usr/local/bin/sshfs "${host}:$2" "${mount_dir}" -F ${home}/.ssh/config \
-o workaround=rename \
-o allow_other \
-o uid=$(/usr/bin/id -u "${USER}") \
-o gid=$(/usr/bin/id -g "${USER}") \
-o disable_hardlink \
-o kernel_cache \
-o reconnect \
-o UserKnownHostsFile=${home}/.ssh/known_hosts \
-o ServerAliveInterval=15
EOF
chmod 755 /usr/local/bin/mount_sshfs
Secured script to unmount an sshfs:
cat <<'EOF'> /usr/local/bin/umount_sshfs
#!/bin/sh
prefix=/mnt/sshfs
[ $# -ne 1 ] && { echo "usage: umount_sshfs <host>" >&2; exit 1; }
host=$(echo "$1" | /usr/bin/sed 's/[^a-zA-Z0-9._-]*//g')
/sbin/umount "${prefix}/${USER}/${host}"
EOF
chmod 755 /usr/local/bin/umount_sshfs
Install: pkg_add rsync
Note: secret and persistent are special directories:
secret will not be cleaned up nor synced to remote hosts.persistent will not be cleaned up but synced to remote hosts like the other normal directories.Do NOT name a server secret or persistent to not confuse with these directories.
These functions should be called in this order: pull, clean, push
pull pulls backup files from remote servers to the local backup directory and deletes the file on the server after the transfer has finished.clean cleans the local backup directory and deletes any file older than 35 days.push synchronizes the local backup directory to the remote backup location excluding the secrets directory. It will also delete remote files that have been cleanup up before.Create the script and edit for your needs:
cat <<'EOF'> /usr/local/bin/backup_sync
#!/bin/sh
backup_dir="/mnt/backup/${USER}"
sshfs_dir="/mnt/sshfs/${USER}"
mount_cmd() { doas /usr/local/bin/mount_sshfs $@; }
umount_cmd() { doas /usr/local/bin/umount_sshfs $@; }
pull() {
mount_cmd $1 "data" || return 1
rsync -a --remove-source-files ${sshfs_dir}/$1 ${backup_dir}
umount_cmd $1
}
push() {
mount_cmd $1 "$2" || return 1
rsync -a --exclude=secret --delete-after ${backup_dir}/* ${sshfs_dir}/$1
umount_cmd $1
}
clean() {
find ${backup_dir} -type f -mtime +35 \! -path "*secret*" \! -path "*persistent*" -delete
}
pull "db"
pull "mail"
pull "www"
clean
push "parents" "${USER}"
push "friend" "${USER}"
EOF
chmod 755 /usr/local/bin/backup_sync
Weekly cronjob on sunday:
echo "0 3 * * 0 /usr/local/bin/backup_sync >/dev/null 2>&1" >> /var/cron/tabs/<user>
Your web, mail, db, whatever server you want to pull backups from.
Create user and group:
groupadd sftp
useradd -g sftp -m -s /sbin/nologin sftp
Create sftp directory:
mkdir -p /var/backups/sftp/data
chown sftp: /var/backups/sftp/data
SSH Config:
cat <<EOF>> /etc/ssh/sshd_config
Match User sftp
ChrootDirectory /var/backups/sftp
ForceCommand internal-sftp
AllowAgentForwarding no
AllowStreamLocalForwarding no
PasswordAuthentication no
PermitRootLogin no
AllowTcpForwarding no
EOF
Example Backup Script:
cat <<'EOF'> /root/backup.sh
#!/bin/sh
recipient="<GPG Key ID>"
set -A paths \
/root/backup.sh \
/root/.gnupg \
/root/.profile \
/etc/pf.conf \
/etc/doas.conf \
/etc/ntpd.conf \
/etc/resolv.conf \
/etc/acme \
/etc/acme-client.conf \
/etc/ssl \
/etc/rc.conf.local \
/etc/ssh \
/var/cron/tabs
file="/var/backups/sftp/data/$(hostname -s)_backup_$(date +%Y%m%d).tgz"
umask 0077
tar pcfz ${file} ${paths[@]}
/usr/local/bin/gpg2 --homedir /root/.gnupg --encrypt --cipher-algo TWOFISH --digest-algo SHA512 --recipient ${recipient} ${file} 2>/dev/null
chown sftp ${file}.gpg
rm -f ${file}
EOF
Place the SSH public key /mnt/backup/<user>/id_ed25519.pub in /home/sftp/.ssh/authorized_keys on your server.