Jenkins Backup Setup Documentation

Document Version: 1.0
Last Updated: January 20, 2025
Author: Jay Shenkar

Table of Contents

1. Executive Summary
2. System Overview
3. Backup Strategy
4. On-Server Backup Configuration
5. Off-Server Backup Configuration
6. Backup Execution Procedures
7. Restore Procedures
8. Configuration Reference
9. Appendices

Overview

This document outlines the comprehensive backup strategy for our Jenkins environment to ensure system integrity and business continuity. The dual-tier backup approach combines automated on-server backups with manual off-server backups to AWS S3, providing robust protection against hardware failures, data corruption, and system migration scenarios.

Key Features:

• Automated daily backups with 2-day retention
• External S3 backup for disaster recovery
• Average backup size: 1.7GB (uncompressed), ~600MB (compressed)
• Minimal downtime recovery procedures

System Overview

Environment Details

Backup Architecture

Jenkins Server (jenkins.byrny.com)

├── On-Server Backups (Daily, 2-day retention)

│   └── /var/lib/jenkins/backup/

└── Off-Server Backups (Manual, Long-term)

    └── AWS S3: smart360-jenkins-backup

Backup Strategy

Two-Tier Backup Approach

Tier 1: On-Server Automated Backups

• Purpose: Immediate recovery from recent configuration changes or minor corruption
• Frequency: Daily (2:30 AM - 3:30 AM IST)
• Retention: 2 days
• Storage: Local Jenkins server
• Risk Mitigation: Configuration drift, accidental job deletion, plugin issues

Tier 2: Off-Server Manual Backups

• Purpose: Disaster recovery and complete system restoration
• Frequency: On-demand via Jenkins pipeline
• Retention: Long-term (managed by S3 lifecycle policies)
• Storage: AWS S3 bucket
• Risk Mitigation: Hardware failure, server corruption, data center issues

On-Server Backup Configuration

Plugin Setup

ThinBackup Plugin

The ThinBackup plugin handles automated Jenkins configuration, job, and plugin backups.

Installation:

1. Navigate to Manage Jenkins → Manage Plugins
2. Search for "ThinBackup" and install
3. Restart Jenkins after installation

Backup Settings

What Gets Backed Up

• Jenkins configuration files
• Job configurations and build history
• Plugin configurations
• User settings and security configurations
• Workspace data (if configured)

Off-Server Backup Configuration

AWS S3 Setup

S3 Bucket Configuration

Bucket Name: smart360-jenkins-backup

Region: [Configure based on your AWS region]

Storage Class: Standard

Versioning: Enabled (recommended)

S3 Bucket Permissions

Ensure the bucket has appropriate permissions for the Jenkins service account:

{

  "Version": "2012-10-17",

  "Statement": [

    {

      "Effect": "Allow",

      "Principal": {

        "AWS": "arn:aws:iam::ACCOUNT-ID:user/monitoring_user"

      },

      "Action": [

        "s3:PutObject",

        "s3:GetObject",

        "s3:DeleteObject",

        "s3:ListBucket"

      ],

      "Resource": [

        "arn:aws:s3:::smart360-jenkins-backup",

        "arn:aws:s3:::smart360-jenkins-backup/*"

      ]

    }

  ]

}

Backup Script Configuration

The backup script is located at /home/jay_shenkar/jenkins-backup-script/jenkins-backup-script.sh and handles:

• Compression of backup files (~600MB compressed)
• Transfer to S3 bucket
• Error handling and logging

Backup Execution Procedures

Automated On-Server Backup

No manual intervention required. Backups run automatically daily between 2:30 AM and 3:30 AM IST.

Verification:

1. Check backup directory: ls -la /var/lib/jenkins/backup/
2. Verify recent backup timestamp
3. Check ThinBackup logs in Jenkins UI

Manual Off-Server Backup

Triggering S3 Backup via Jenkins Pipeline

1. Navigate to Jenkins dashboard
2. Locate the backup pipeline job
3. Click Build Now
4. Monitor pipeline execution
5. Verify backup upload to S3

Manual Script Execution

# Navigate to script directory

cd /home/jay_shenkar/jenkins-backup-script/

# Execute backup script

./jenkins-backup-script.sh

# Verify S3 upload

aws s3 ls s3://smart360-jenkins-backup/

Restore Procedures

Restoring from On-Server Backup

Prerequisites

• Jenkins server accessible
• ThinBackup plugin installed
• Valid backup files in /var/lib/jenkins/backup/

Steps

1. Access Jenkins Dashboard
2. • Navigate to Manage Jenkins
2. Open ThinBackup
3. • Under Tools and Actions, click ThinBackup
3. Select Backup
4. • Click Restore
   • Browse to /var/lib/jenkins/backup/
   • Select desired backup file
4. Execute Restore
5. • Click Restore to begin process
   • Wait for completion confirmation

Restart Jenkins
jenkins.bynry.com/restart

Restoring from S3 Backup

Prerequisites

• New Jenkins instance with ThinBackup plugin
• AWS CLI configured with monitoring_user credentials
• Access to S3 bucket

Step-by-Step Process

Install ThinBackup Plugin

# Access Jenkins UI and install ThinBackup plugin

# Restart Jenkins after installation

1. Configure AWS CLI
aws configure

# Enter monitoring_user credentials

2. Download Backup from S3

# List available backups

aws s3 ls s3://smart360-jenkins-backup/

# Download specific backup

aws s3 cp s3://smart360-jenkins-backup/<backup-file>.zip /home/jay_shenkar/jenkins-backup-script/

3. Extract Backup

cd /home/jay_shenkar/jenkins-backup-script/

unzip <backup-file>.zip

4. Prepare Backup Directory

# Create backup directory

sudo mkdir -p /var/lib/jenkins/backup/

# Move extracted files

sudo mv /home/jay_shenkar/jenkins-backup-script/backup/* /var/lib/jenkins/backup/

# Set proper permissions

sudo chown -R jenkins:jenkins /var/lib/jenkins/backup/

5. Configure ThinBackup

Navigate to Manage Jenkins → System Configuration

Locate ThinBackup Configuration

Set Backup Directory to /var/lib/jenkins/backup/

Save configuration

Execute Restore

Go to Manage Jenkins → ThinBackup

Click Restore

Select backup file

Click Restore

6. Restart Jenkins

jenkins.bynry.com/restart

Configuration Reference

Directory Structure

/var/lib/jenkins/

├── backup/                          # ThinBackup storage directory

│   ├── FULL-2025-01-20_03-00/      # Full backup timestamp

│   └── FULL-2025-01-19_03-00/      # Previous backup

├── config.xml                       # Jenkins main configuration

├── jobs/                           # Job configurations

└── plugins/                        # Installed plugins

/home/jay_shenkar/jenkins-backup-script/

├── jenkins-backup-script.sh        # S3 backup script

└── logs/                           # Script execution logs

Key Configuration Files

ThinBackup Configuration

Location: Jenkins UI → Manage Jenkins → System Configuration → ThinBackup

Appendices

Appendix A: Backup Size Estimates

Appendix B: Emergency Contacts