Introduction: In the world of DevOps, efficiency and automation are paramount. Shell scripts are powerful tools that enable DevOps engineers to automate a wide range of tasks, from server management to continuous integration and deployment. This article presents 15 essential shell scripts, complete with code examples, that can significantly enhance your productivity and streamline your operations.

1. Automating Backup of a Directory

This script creates a compressed backup of a directory and stores it with a timestamp.

#!/bin/bash

# Variables
SOURCE_DIR="/path/to/source"
BACKUP_DIR="/path/to/backup"
TIMESTAMP=$(date +"%Y%m%d%H%M%S")
BACKUP_FILE="backup-$TIMESTAMP.tar.gz"
# Create a backup
tar -czvf $BACKUP_DIR/$BACKUP_FILE $SOURCE_DIR
# Output message
echo "Backup completed: $BACKUP_DIR/$BACKUP_FILE"

2. Basic CI/CD Script

This script pulls the latest code from a Git repository, builds the project, and restarts the service.

#!/bin/bash

# Variables
REPO_DIR="/path/to/repo"
SERVICE_NAME="your-service-name"
# Pull latest code
cd $REPO_DIR
git pull origin main
# Build the project (Example: Maven for a Java project)
mvn clean install
# Restart the service
systemctl restart $SERVICE_NAME
# Output message
echo "Deployment completed and service restarted."

3. Monitoring Disk Usage

This script checks disk usage and sends an email if usage exceeds a certain threshold.

#!/bin/bash

# Variables
THRESHOLD=80
EMAIL="admin@example.com"
CURRENT_USAGE=$(df / | grep / | awk '{ print $5 }' | sed 's/%//g')
# Check if usage exceeds threshold
if [ $CURRENT_USAGE -gt $THRESHOLD ]; then
  echo "Disk usage is at $CURRENT_USAGE% on $(hostname)" | mail -s "Disk Space Alert" $EMAIL
fi

4. Automating MySQL Database Backup

This script creates a backup of a MySQL database and stores it with a timestamp.

#!/bin/bash
# Variables
DB_USER="yourusername"
DB_PASS="yourpassword"
DB_NAME="yourdbname"
BACKUP_DIR="/path/to/backup"
TIMESTAMP=$(date +"%Y%m%d%H%M%S")
BACKUP_FILE="db-backup-$DB_NAME-$TIMESTAMP.sql"
# Create a database backup
mysqldump -u $DB_USER -p$DB_PASS $DB_NAME > $BACKUP_DIR/$BACKUP_FILE
# Compress the backup
gzip $BACKUP_DIR/$BACKUP_FILE
# Output message
echo "Database backup completed: $BACKUP_DIR/$BACKUP_FILE.gz"

5. Automating Docker Container Cleanup

This script removes all stopped Docker containers, dangling images, and unused networks.

#!/bin/bash

# Remove all stopped containers
docker container prune -f
# Remove all dangling images
docker image prune -f
# Remove all unused networks
docker network prune -f
# Output message
echo "Docker cleanup completed."

6. System Update and Upgrade

This script updates and upgrades the system packages on a Debian/Ubuntu system.

#!/bin/bash

# Update package list
sudo apt-get update
# Upgrade packages
sudo apt-get upgrade -y
# Clean up
sudo apt-get autoremove -y
sudo apt-get clean
# Output message
echo "System update and upgrade completed."

7. Checking Service Status and Restarting if Down

This script checks if a service is running, and if not, restarts it.

#!/bin/bash
# Variables
SERVICE_NAME="your-service-name"# Check if the service is runningif ! systemctl is-active --quiet $SERVICE_NAME; then
  # Restart the service if it's not running
  systemctl restart $SERVICE_NAME
  echo "$SERVICE_NAME was down and has been restarted."else
  echo "$SERVICE_NAME is running."fi

8. Automating Log Rotation

This script rotates logs for a specified directory, keeping only a certain number of backups.

#!/bin/bash

# Variables
LOG_DIR="/path/to/logs"
MAX_BACKUPS=7
# Rotate logs
find $LOG_DIR -type f -name "*.log" | while read LOG_FILE; do
  # Rotate logs and delete old backups
  mv $LOG_FILE $LOG_FILE.$(date +"%Y%m%d%H%M%S")
  ls -t $LOG_FILE.* | sed -e "1,${MAX_BACKUPS}d" | xargs -d '\n' rm -f
done
# Output message
echo "Log rotation completed."

9. Automated User Account Creation

This script creates multiple user accounts from a list and sets their passwords.

#!/bin/bash

# Variables
USER_LIST="/path/to/userlist.txt"
DEFAULT_PASSWORD="changeme"
# Loop through the list of users
while IFS= read -r USERNAME; do
  # Create user account
  useradd -m $USERNAME
  echo $USERNAME:$DEFAULT_PASSWORD | chpasswd
  echo "User $USERNAME created with default password."
done < $USER_LIST

10. Automated SSL Certificate Renewal with Certbot

This script renews SSL certificates using Certbot and reloads the web server.

#!/bin/bash

# Variables
WEB_SERVER="nginx"
# Renew SSL certificates
certbot renew --quiet
# Reload web server to apply new certificates
systemctl reload $WEB_SERVER
# Output message
echo "SSL certificates renewed and $WEB_SERVER reloaded."

11. Monitoring CPU and Memory Usage

This script monitors CPU and memory usage and logs it to a file.

#!/bin/bash

# Variables
LOG_FILE="/var/log/resource-usage.log"
THRESHOLD_CPU=80
THRESHOLD_MEM=80
# Get CPU and memory usage
CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print $2 + $4}')
MEM_USAGE=$(free | grep Mem | awk '{print $3/$2 * 100.0}')
# Log the usage
echo "$(date): CPU: $CPU_USAGE%, Memory: $MEM_USAGE%" >> $LOG_FILE
# Alert if usage is above threshold
if (( $(echo "$CPU_USAGE > $THRESHOLD_CPU" | bc -l) )) || (( $(echo "$MEM_USAGE > $THRESHOLD_MEM" | bc -l) )); then
  echo "$(date): High resource usage detected. CPU: $CPU_USAGE%, Memory: $MEM_USAGE%" | mail -s "Resource Usage Alert" admin@example.com
fi

12. Automated File Syncing Between Servers

This script uses rsync to synchronize files between two servers.

#!/bin/bash

# Variables
SOURCE_DIR="/path/to/source"
DEST_SERVER="user@remote-server"
DEST_DIR="/path/to/destination"
LOG_FILE="/var/log/rsync.log"
# Sync files
rsync -avz --delete $SOURCE_DIR $DEST_SERVER:$DEST_DIR >> $LOG_FILE 2>&1
# Output message
echo "File sync completed. Check $LOG_FILE for details."

13. Database Health Check

This script checks if a MySQL database is up and running.

#!/bin/bash

# Variables
DB_HOST="localhost"
DB_USER="yourusername"
DB_PASS="yourpassword"
DB_NAME="yourdbname"
# Check database connection
if mysqladmin ping -h $DB_HOST -u $DB_USER -p$DB_PASS --silent; then
  echo "Database $DB_NAME is up and running."
else
  echo "Database $DB_NAME is down."
  # Restart the database service (optional)
  # systemctl restart mysql
fi

14. Automating Cleanup of Old Files

This script deletes files older than a certain number of days in a specified directory.

#!/bin/bash

# Variables
TARGET_DIR="/path/to/directory"
DAYS_OLD=30
# Find and delete files older than the specified number of days
find $TARGET_DIR -type f -mtime +$DAYS_OLD -exec rm -f {} \;
# Output message
echo "Cleanup completed. Files older than $DAYS_OLD days have been deleted."

15. Automated Security Updates

This script installs security updates on a Debian/Ubuntu system.

#!/bin/bash
# Update package list
sudo apt-get update
# Install security updates
sudo apt-get upgrade -y --with-new-pkgs
# Clean up
sudo apt-get autoremove -y
sudo apt-get clean
# Output message
echo "Security updates installed."

These scripts provide a starting point and can be customized further based on your specific needs. Always test scripts in a safe environment before running them in production to avoid any unintended consequences.