Building Automated Ubuntu Server Backups with Restic and AWS S3
What happens if your production server crashes tomorrow?
Do you have a backup? More importantly, have you actually tested restoring it?
Server backups are one of those things that everyone knows are important, but they are often overlooked until something goes wrong.
A server can fail. A database can become corrupted. An administrator can accidentally delete an important directory. A deployment can break an application.
And sometimes, the biggest problem isn't the failure itself. It's discovering that your backups are incomplete or cannot be restored.
I recently implemented an automated backup solution for Ubuntu servers using Restic, Amazon S3, MySQL/MariaDB and systemd.
In this article, I'll walk through the implementation, from initial installation to automated scheduling and restoration.
The goal is to build something practical, secure and reusable across multiple servers.
1. Understanding the architecture
The backup workflow is straightforward:
- Export MySQL/MariaDB databases into compressed SQL files.
- Back up the database dumps and selected server directories.
- Encrypt the backup repository using Restic.
- Upload the repository to Amazon S3.
- Apply a retention policy.
- Schedule everything to run automatically using systemd.
The architecture:
Ubuntu Server | v MySQL / MariaDB | v mysqldump / mariadb-dump | v Compressed SQL files | v Restic Backup | +--------+--------+ | | v v Server Files Database Dumps | | +--------+--------+ | v Restic Encryption | v Amazon S3 | v Retention Management Technologies used
| Component | Responsibility |
|---|---|
| Ubuntu Server | Production environment |
| Restic | Backup, encryption and restoration |
| Amazon S3 | Remote backup storage |
| MySQL/MariaDB | Database source |
| systemd | Automated scheduling |
| Bash | Backup orchestration |
One important detail: Restic uses snapshots and deduplication. Unchanged data does not need to be uploaded again unnecessarily.
2. Prepare Amazon S3
First, create an S3 bucket or select an existing one.
For this example, I'll use:
Bucket: my-company-backups Region: eu-central-1 Prefix: restic The repository will be stored under:
s3:s3.eu-central-1.amazonaws.com/my-company-backups/restic Using a dedicated prefix helps isolate backup storage from other objects.
Configure AWS IAM permissions
Create a dedicated IAM identity for the backup process.
Avoid using administrator credentials or an IAM identity with unnecessary permissions.
Here is an example policy:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ResticBucketAccess", "Effect": "Allow", "Action": [ "s3:GetBucketLocation", "s3:ListBucket" ], "Resource": "arn:aws:s3:::my-company-backups", "Condition": { "StringLike": { "s3:prefix": [ "restic", "restic/*" ] } } }, { "Sid": "ResticObjectAccess", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::my-company-backups/restic/*" } ] } This restricts access to the relevant repository prefix.
If you use SSE-KMS, additional KMS permissions and key-policy configuration may be necessary.
Install AWS CLI
sudo apt update sudo apt install -y awscli Configure the credentials for the account that will execute the backup.
For this implementation, the systemd service runs as root:
sudo aws configure Verify the AWS identity:
sudo aws sts get-caller-identity Test access to the bucket:
sudo aws s3 ls s3://my-company-backups/ \ --region eu-central-1 Where possible, prefer role-based credentials instead of long-lived access keys.
Security note: Never put AWS secret keys directly inside your backup script.
3. Install Restic
Install Restic from Ubuntu's package repositories:
sudo apt update sudo apt install -y restic Verify the installation:
restic version If the available Ubuntu package is too old for your requirements, use a current official Restic release.
Keep Restic updated as part of normal server maintenance.
4. Configure Restic
We need three things:
- A protected configuration directory.
- A repository password.
- The repository location and AWS region.
Create the configuration directory
sudo mkdir -p /root/.config/restic sudo chmod 700 /root/.config/restic Create the repository password
sudo nano /root/.config/restic/password Enter a long, unique password.
Protect the file:
sudo chmod 600 /root/.config/restic/password sudo chown root:root /root/.config/restic/password Restic encrypts repository data using keys derived from the repository password.
Keep a secure offline copy of this password.
If you lose the password and have no usable recovery copy, you will not be able to decrypt your backups.
Create the environment configuration
sudo nano /root/.config/restic/env Add:
export AWS_DEFAULT_REGION="eu-central-1" export RESTIC_REPOSITORY="s3:s3.eu-central-1.amazonaws.com/my-company-backups/restic" export RESTIC_PASSWORD_FILE="/root/.config/restic/password" Protect the configuration:
sudo chown root:root /root/.config/restic/env sudo chmod 600 /root/.config/restic/env The S3 bucket, region and repository prefix must match your AWS configuration and IAM permissions.
Initialize the repository
sudo bash -c \ 'source /root/.config/restic/env; restic init' This operation should be performed only once for a new repository.
If a repository already exists, do not initialize another one or casually change its location.
5. Configure MySQL/MariaDB backups
Backing up application files is not enough.
Databases contain important information that must be exported using database-aware backup tools.
For this implementation, I'll use logical database dumps.
Check the database utilities
mysql --version command -v mysqldump || command -v mariadb-dump Configure database credentials securely
If your MySQL root account uses password authentication, create:
sudo nano /root/.my.cnf Add:
[client] user=root password="YOUR_MYSQL_PASSWORD" Protect the file:
sudo chown root:root /root/.my.cnf sudo chmod 600 /root/.my.cnf Do not put database passwords inside the backup script or command-line arguments.
If MySQL root uses Unix socket authentication, configure an appropriate dedicated backup account and adjust the backup script accordingly.
Test database access
sudo mysql \ --defaults-extra-file=/root/.my.cnf \ -e "SHOW DATABASES;" Test the database dump:
sudo mysqldump \ --defaults-extra-file=/root/.my.cnf \ --all-databases \ --single-transaction \ > /dev/null For MariaDB, use mariadb-dump if mysqldump is unavailable.
The --single-transaction option provides a consistent point-in-time dump for transactional tables such as InnoDB.
It does not guarantee consistency for non-transactional tables such as MyISAM.
6. Choose which directories to back up
This is an important part of the implementation.
You do not necessarily need to back up the entire filesystem.
Instead, identify the directories that contain important application data and configuration.
For example:
/etc /root /home /var/www /opt /usr/local /var/backups/mysql Create the paths configuration file:
sudo nano /etc/restic-backup.paths Add the directories you want to protect.
Should you back up the entire /var/www directory?
Not necessarily.
If your server hosts multiple websites, you can configure individual website directories instead.
For example:
/etc /root /var/www/website-one /var/www/website-two /var/www/website-three /var/backups/mysql This gives you more control over what is included in each backup and makes individual website restoration easier.
Remember to include other important directories such as uploaded files, application storage, custom configurations and persistent data stored outside the website directories.
Avoid backing up transient filesystems such as:
/proc /sys /dev /run Also, do not blindly include /var/lib/mysql in a normal file-level backup of a running database. Use database-aware backup procedures instead.
Protect the paths file:
sudo chmod 600 /etc/restic-backup.paths 7. Create the automated backup script
Now we combine everything into one script.
The script will:
- Load the Restic configuration.
- Prevent overlapping backup executions.
- Create a compressed MySQL/MariaDB dump.
- Validate the generated dump.
- Back up the configured directories.
- Apply snapshot retention.
- Remove the temporary database dump after successful completion.
- Write execution details to a log file.
Create the script:
sudo nano /usr/local/bin/restic-backup.sh Add:
#!/bin/bash set -Eeuo pipefail # Load Restic configuration source /root/.config/restic/env BACKUP_DIR="/var/backups/mysql" LOG_DIR="/var/log/restic" LOG_FILE="$LOG_DIR/backup.log" PATHS_FILE="/etc/restic-backup.paths" LOCK_FILE="/run/lock/restic-backup.lock" TIMESTAMP="$(date '+%Y-%m-%d_%H-%M-%S')" MYSQL_BACKUP="$BACKUP_DIR/mysql-all-$TIMESTAMP.sql.gz" mkdir -p "$BACKUP_DIR" "$LOG_DIR" /run/lock chmod 700 "$BACKUP_DIR" "$LOG_DIR" # Log all output exec >> "$LOG_FILE" 2>&1 echo echo "==================================================" echo "Backup started: $(date)" echo "Host: $(hostname)" echo "Repository: $RESTIC_REPOSITORY" echo "==================================================" # Prevent overlapping executions exec 9>"$LOCK_FILE" if ! flock -n 9; then echo "Another Restic backup is already running; exiting." exit 0 fi # Validate configuration : "${RESTIC_REPOSITORY:?RESTIC_REPOSITORY is not set}" : "${RESTIC_PASSWORD_FILE:?RESTIC_PASSWORD_FILE is not set}" if [[ ! -r "$RESTIC_PASSWORD_FILE" ]]; then echo "ERROR: Restic password file is missing." exit 1 fi if [[ ! -s "$PATHS_FILE" ]]; then echo "ERROR: Backup paths file is missing or empty." exit 1 fi # Find available database dump utility if command -v mariadb-dump >/dev/null 2>&1; then MYSQL_DUMP="$(command -v mariadb-dump)" elif command -v mysqldump >/dev/null 2>&1; then MYSQL_DUMP="$(command -v mysqldump)" else echo "ERROR: No database dump utility found." exit 1 fi # Create database dump echo "[1/3] Creating MySQL/MariaDB dump..." "$MYSQL_DUMP" \ --defaults-extra-file=/root/.my.cnf \ --all-databases \ --single-transaction \ --routines \ --triggers \ --events \ --quick \ | gzip -c > "$MYSQL_BACKUP" # Validate database dump if [[ ! -s "$MYSQL_BACKUP" ]] || ! gzip -t "$MYSQL_BACKUP"; then echo "ERROR: SQL dump validation failed." exit 1 fi chmod 600 "$MYSQL_BACKUP" echo "Database dump created: $MYSQL_BACKUP" ls -lh "$MYSQL_BACKUP" # Execute Restic backup echo "[2/3] Running Restic backup..." restic backup \ --files-from "$PATHS_FILE" \ --exclude-caches \ --exclude '/var/cache/**' \ --exclude '/tmp/**' \ --exclude '/var/tmp/**' # Apply retention policy echo "[3/3] Applying snapshot retention..." restic forget \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 12 \ --prune # Remove local SQL dump only after success rm -f -- "$MYSQL_BACKUP" echo "Backup completed successfully: $(date)" echo "==================================================" Set the correct ownership and permissions:
sudo chown root:root /usr/local/bin/restic-backup.sh sudo chmod 700 /usr/local/bin/restic-backup.sh The script uses set -Eeuo pipefail so that failures, including failures in the database dump pipeline, are not silently ignored.
One useful detail is that the database dump is retained locally if the backup or retention process fails. This helps with troubleshooting.
8. Configure systemd automation
We now have a working backup script.
Instead of manually executing it every day, we'll use systemd.
This requires two files:
- A systemd service.
- A systemd timer.
Create the systemd service
sudo nano /etc/systemd/system/restic-backup.service Add:
[Unit] Description=Restic S3 Backup Wants=network-online.target After=network-online.target [Service] Type=oneshot User=root ExecStart=/usr/local/bin/restic-backup.sh The service executes the backup script and exits when the process finishes.
Create the systemd timer
sudo nano /etc/systemd/system/restic-backup.timer Add:
[Unit] Description=Daily Restic S3 Backup [Timer] OnCalendar=*-*-* 02:00:00 Persistent=true Unit=restic-backup.service [Install] WantedBy=timers.target This schedules the backup every day at 02:00 server-local time.
The Persistent=true setting allows systemd to trigger a missed scheduled run when the server becomes available again.
Enable the timer
Reload systemd:
sudo systemctl daemon-reload Enable and start the timer:
sudo systemctl enable --now restic-backup.timer Check its status:
sudo systemctl is-enabled restic-backup.timer sudo systemctl list-timers restic-backup.timer Your backup process is now scheduled to run automatically every day.
9. Run your first backup manually
Do not wait until the following day to discover whether something is wrong.
Run the service manually:
sudo systemctl start restic-backup.service Check the service:
sudo systemctl status restic-backup.service Review the backup log:
sudo tail -100 /var/log/restic/backup.log A successful systemd oneshot service normally shows:
Active: inactive (dead) code=exited, status=0/SUCCESS This is expected. The service is designed to execute a task and terminate.
Now verify that Restic created a snapshot:
sudo bash -c \ 'source /root/.config/restic/env; restic snapshots' Check repository statistics:
sudo bash -c \ 'source /root/.config/restic/env; restic stats' You should now see your first backup snapshot.
10. Configure backup retention
Keeping every backup forever is not always practical.
Storage costs increase, and older snapshots may no longer be necessary.
The script uses this retention policy:
restic forget \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 12 \ --prune This retains:
| Retention | Snapshots |
|---|---|
| Daily | 7 |
| Weekly | 4 |
| Monthly | 12 |
Restic removes snapshots that are no longer needed and prunes unreferenced repository data.
The retention policy should reflect your recovery requirements and storage budget.
Important: If you enable S3 Object Lock or restrictive versioning policies, evaluate how they interact with Restic pruning. Object Lock can prevent Restic from deleting data when required.
11. Verify repository integrity
A successful backup does not automatically mean that the repository is healthy.
Restic provides commands to verify repository integrity.
Run:
sudo bash -c \ 'source /root/.config/restic/env; restic check' For deeper verification, you can also read and verify the repository's stored data:
sudo bash -c \ 'source /root/.config/restic/env; restic check --read-data' The second operation can take considerably more time and consume additional bandwidth.
I recommend running it periodically during a quiet maintenance window rather than automatically performing a full data read after every daily backup.
12. Restore files and databases
This is probably the most important section of the entire implementation.
Backups are useful only if you can restore them.
List available snapshots
sudo bash -c \ 'source /root/.config/restic/env; restic snapshots' Browse a snapshot
sudo bash -c \ 'source /root/.config/restic/env; restic ls latest' Restore a complete snapshot
Create a separate restoration directory:
sudo mkdir -p /restore-test Restore the latest snapshot:
sudo bash -c \ 'source /root/.config/restic/env; restic restore latest --target /restore-test' Your restored files will be available under /restore-test.
This is safer than immediately overwriting production files.
You can inspect individual websites, configuration files and database dumps before restoring anything to the live environment.
Restore an individual website
If your backup includes individual website directories, you can restore only the required path.
For example:
sudo bash -c \ 'source /root/.config/restic/env; restic restore latest --target /restore-test --include /var/www/website-one' This is particularly useful when managing multiple websites on the same server.
Restore a MySQL database dump
The database dump is stored as a compressed SQL file.
First, validate it:
gzip -t /restore-test/var/backups/mysql/mysql-all-YYYY-MM-DD_HH-MM-SS.sql.gz You can then import it into an appropriate MySQL instance:
gunzip -c /restore-test/var/backups/mysql/mysql-all-YYYY-MM-DD_HH-MM-SS.sql.gz | sudo mysql Production warning: Do not blindly execute an all-databases import against a live production server.
For recovery testing, use an isolated MySQL instance first. Review the dump and confirm the destination before importing it.
13. Monitoring and troubleshooting
Once the system is running, you should be able to quickly identify whether the backup process is working.
Here are the commands I use most frequently.
| Task | Command |
|---|---|
| Check timer | sudo systemctl is-enabled restic-backup.timer |
| Next execution | sudo systemctl list-timers restic-backup.timer |
| Service status | sudo systemctl status restic-backup.service |
| Backup log | sudo tail -100 /var/log/restic/backup.log |
| Systemd journal | sudo journalctl -u restic-backup.service -n 100 --no-pager |
| Run manually | sudo systemctl start restic-backup.service |
| List snapshots | sudo bash -c 'source /root/.config/restic/env; restic snapshots' |
Common problems
AWS AccessDenied or HTTP 403
Check:
- IAM permissions.
- S3 bucket policies.
- The repository prefix.
- KMS permissions, if applicable.
- Any explicit Deny statements.
The repository path and IAM policy must match.
Restic cannot find the repository
You might see:
Fatal: Please specify repository location Check that the environment file is loaded correctly:
sudo cat /root/.config/restic/env Make sure RESTIC_REPOSITORY is exported.
MySQL authentication failure
For example:
mysqldump: Got error: 1045 Access denied for user Check the credentials:
sudo cat /root/.my.cnf Verify the username, password and permissions.
Also check that the file has the correct ownership and permissions:
sudo chown root:root /root/.my.cnf sudo chmod 600 /root/.my.cnf systemd reports 203/EXEC
Check that:
- The script starts with
#!/bin/bash. - The script is executable.
- The file uses Unix LF line endings.
If the script was edited on Windows, convert its line endings:
sudo sed -i 's/\r$//' /usr/local/bin/restic-backup.sh The service shows inactive (dead)
For a successful oneshot service, this is normal.
Check the exit status and backup log rather than expecting the service to remain active.
AWS works for a normal user but not root
Remember that the systemd service runs as root.
AWS credentials configured for another Linux account are not automatically available to root.
Verify the identity:
sudo aws sts get-caller-identity 14. Additional security recommendations
There are several things worth considering when deploying this solution on production servers.
- Protect the Restic password. Store a secure recovery copy outside the server, preferably in an approved password manager or secrets vault.
- Use dedicated AWS permissions. Do not give your backup identity access to unrelated S3 buckets or resources.
- Enable S3 Block Public Access. Backup repositories should never be publicly accessible.
- Consider S3 versioning. Versioning can provide additional protection against accidental deletion, but evaluate storage costs and interaction with repository maintenance.
- Monitor backup failures. A log file is useful, but it does not notify anyone when a backup fails.
- Protect your database backups. The compressed SQL dump temporarily exists on the server until the Restic backup succeeds. Ensure sufficient disk space and appropriate permissions.
- Test restoration regularly. A successful backup job is not proof of a successful disaster recovery process.
15. Final deployment checklist
Before considering the implementation complete, verify the following:
- [ ] S3 bucket and region confirmed.
- [ ] IAM permissions restricted to the correct repository prefix.
- [ ] AWS identity verified as root.
- [ ] Restic installed and version checked.
- [ ] Restic password created and protected.
- [ ] Restic environment configuration created.
- [ ] Repository initialized.
- [ ] MySQL/MariaDB dump utility available.
- [ ] Database credentials configured and tested.
- [ ] Backup paths reviewed.
- [ ] Backup script installed and executable.
- [ ] systemd service created.
- [ ] systemd timer enabled.
- [ ] First manual backup completed successfully.
- [ ] Snapshot visible in Restic.
- [ ] Repository integrity checked.
- [ ] Test restoration completed.
Conclusion
With Restic, Amazon S3, MySQL/MariaDB and systemd, it is possible to build a reliable and relatively simple automated backup solution for Ubuntu servers.
What I particularly like about this approach is that it does not require an expensive backup platform or complicated infrastructure.
You have encrypted remote backups, database exports, configurable backup paths, automatic retention and straightforward restoration procedures.
The same implementation can be reused across multiple servers, with each server having its own configuration, S3 repository and backup schedule.
One final thought:
Don't measure your backup strategy by how many successful backups you have. Measure it by how confidently you can restore your systems when something goes wrong.
If you found this useful, follow me for more practical guides covering Ubuntu Server, Linux administration, Laravel, infrastructure and software development.
Have you tested restoring your production backups recently? I'd be interested to hear what backup solutions you use.
