This time I would like to explain how I backup my server with the help of some build in tools and a script written by myself. As I’m running my blogs and my mail server, which is Scalix, on a Linux server I need to make sure, that no data is lost during an outage and that I will be back online in a short period of time. Therefore I have two strategies to backup my stuff.
- Using Proxmox, it is very easy to do an online backup of my virtual machine, as described here:
HowTo: Proxmox Live Backup
The drawback with this method is, that it takes some time to do the backup, it consumes a lot of system resources (I/O) and the backup is very large in size. - Using a
file based backup from within the virtual machine, backing up only thereal important stuff.
The drawback here, you cannot restore the system from this backup. But in combination with method one, it works perfectly.
I will not describe the first method, as this is already described in the mentioned post. I do this kind of full backup once a week and use method two to get the increment data from every day between two full backups. As method two will not work out of the box, you have to make sure the data is consistent, I will cover this method in this post.
Requirements for the Backup
First of all, you need to have an external backup server to make sure the data is protected against loss. I will use an external
In addition to an external storage, you should also have LVM (Logical Volume Manager) installed on your system. This is the only option to make online backups of your system and keep data consistent using LVM snapshots. Instead of LVM, some hardware raid systems support a similar feature.
LVM (Logical Volume Manager)
I will not describe how to configure LVM, this is outside of the scope of this post, but I will explain shortly, what LVM is. LVM is included in the Linux kernel and provides a layer between the file system and the block device. LVM has features like encryption, easy expanding the volume and many more, but we would like to use only the snapshot function in the first place.
To use the snapshot function of LVM you need to make sure, that you still have some free unallocated space on your drive. This space will be used during the snapshot phase to write new changes to the disk. Make sure to leave enough free space on the disk, as you can expand you volume any time. There is no reason to oversize your volume in the first place. Keep in mind, if the space is not enough to hold all the data changes during the snapshot phase, you can be in big trouble.
Consistent LVM Snapshot
Creating
Scalix
To get a consistent backup of the Scalix message store, you need to suspend Scalix before creating the backup. This could be accomplished using
omsuspend -s 120 &
sleep 2
sync; sync; sync
omsuspend -rThis first line will suspend Scalix and no user interaction with the Scalix system is possible. The last line will resume the Scalix system again and Scalix will run normal.
MySQL
To avoid the situation,
FLUSH TABLES WITH READ LOCK;This will write the database to disk and lock the tables. Do this with a permanent session, as the lock is removed as soon as the session is terminated.
To unlock the tables again, after the snapshot is created, use this command:
UNLOCK TABLES;You should now have a consistent snapshot and you can start the backup process… wait, I didn’t told you, how to create the snapshot.
Create the Snapshot
The snapshot is created very easy and needs only one command, two if you actually would like to get access to the snapshot data. It takes some more effort to get all the information needed, before creating the snapshot.
First of all, you need to know, the name of the volume for which you would like to create the snapshot
df /var/opt/scalix/sx/
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/scalix-root 96816868 6806528 85092276 8% /This will tell me, that the Scalix message store is on the volume group “
lvscan
ACTIVE '/dev/scalix/swap_1' [4.09 GiB] inherit
ACTIVE '/dev/scalix/root' [93.80 GiB] inheritAs you can see, there are two volumes, but as the command from above told us to use the volume “root” we have now the complete path to the volume, which is
/dev/scalix/rootYou also need a name for the snapshot volume. I chose “backup-snapshot”. Using
lvcreate -l100%FREE -s -n backup-snapshot /dev/scalix/rootThis will use 100% of the free space on the disk and create the snapshot. If you issue
lvscan
ACTIVE '/dev/scalix/swap_1' [4.09 GiB] inherit
ACTIVE Original '/dev/scalix/root' [93.80 GiB] inherit
ACTIVE Snapshot '/dev/scalix/backup-snapshot' [1.87 GiB] inheritThe last step would be to mount the snapshot to get access to the files. We can use a simple mount command here:
mount /dev/scalix/backup-snapshot /mnt/backupMake sure, to create the mount folder before using this command. After backing up everything you need, you can simply unmount and drop the snapshot:
unmount /mnt/backup
lvremove -f /dev/scalix/backup-snapshotBackup
As we now know, how to create the snapshot and remove it, I would like to share my thoughts about the backup itself. I run this kind of backup every day, except the day, when I run the full VM backup. I use rsync and create a backup folder for every day. This works fast and reliable and I can benefit from just backing up the changes.
My backup includes all configuration files within “/etc” and all logs in “/var/log”. To include the Scalix messages store I also backup “/var/opt” and to get the MySQL files I include “/var/lib/
From my point of view, this includes all files needed to recover from a crash, using the full VM backup plus the file backup.
The Backup Script
To do all the above steps automatically I wrote a script for my usage. Feel free to use it and adapt the script to your own needs.
#!/bin/bash
#set -x
#is mysql present
#0=no
#1=yes
mysql=1
#is scalix present
#0=no
#1=yes
scalix=1
#mysql user name
mysqlusername="mysql"
#mysql user file
mysql_user_file=/etc/mysql/debian.cnf
#Path to the Volume to make the snapshot of
lvmpath="/dev/scalix/root"
#Logical Volume Name for the snapshot
lvmname="backup-snapshot"
#Path to the mountfolder
mountfolder="/mnt/snapshot"
#create path to snapshot volume
lvmsnapshotpath=${lvmpath%/*}"/"$lvmname
#list of folders to backup, separate with space
backupfolder="/etc /var/log /var/opt /var/lib/mysql /var/www"
#backup destination.
backupdestination="rsync://backup-server/folder/"
#get day of week
dow=$(date +%a)
#clean up if something goes wrong
function clean_up {
#unlock mysql
unlock_mysql
#resume scalix
activate_scalix
#remove the snapshot
remove_snapshot
#exit the script
exit
}
trap clean_up SIGHUP SIGINT SIGTERM
#lock mysql
function lock_mysql {
if [ $mysql -eq 0 ]; then
return
fi
#create a screen session to lock mysql during snapshot
screen -S "mysql" -d -m
#pause for 1sec to get the session created
sleep 1
#connect to mysql
screen -r "mysql" -X stuff "/usr/bin/mysql --defaults-extra-file=$mysql_user_file -u $mysqlusername
"
#lock mysql
screen -r "mysql" -X stuff "FLUSH TABLES WITH READ LOCK;
"
#wait for 1sec to fully lock mysql
sleep 1
}
#unlock mysql
function unlock_mysql {
if [ $mysql -eq 0 ]; then
return
fi
#check if the screen session is still tunning
if ! screen -list | grep -q "mysql"; then
return
fi
#unlock mysql again
screen -r "mysql" -X stuff "UNLOCK TABLES;
"
#quit from mysql
screen -r "mysql" -X stuff "quit
"
#end the screen session
screen -r "mysql" -X stuff "exit
"
}
#create the snapshot
function create_snapshot {
#create the snapshot
lvcreate -l100%FREE -s -n $lvmname $lvmpath
}
#mount the snapshot
function mount_snapshot {
#create the mountfolder
mkdir -p $mountfolder
#moun the snapshot
mount $lvmsnapshotpath $mountfolder
}
#do the backup
function backup_everything {
#do some backup
rsync -apE --delete --stats $backupfolder $backupdestination$dow
}
#remove the snapshot
function remove_snapshot {
#unmount the snapshot folder
umount $mountfolder
#remove the snapshot
lvremove -f $lvmsnapshotpath
}
#suspend scalix
function suspend_scalix {
if [ $scalix -eq 0 ]; then
return
fi
/opt/scalix/bin/omsuspend -s 120 &
sleep 2
sync; sync; sync
}
#activate scalix again
function activate_scalix {
if [ $scalix -eq 0 ]; then
return
fi
/opt/scalix/bin/omsuspend -r
}
lock_mysql
suspend_scalix
create_snapshot
activate_scalix
unlock_mysql
mount_snapshot
backup_everything
remove_snapshot
exitYou need to modify some of the variables at the top of the script, to make it work for you environment.
You can run the script as a cron job to run it automatically. I run my script every night at 04:30am.
If you have any questions, regarding this post or if you would like to provide feedback, please use the comment function below.