系统运维脚本：定时采集、定时备份、远程重启、磁盘空间监控

系统运维脚本定时采集、定时备份、远程重启、磁盘空间监控工控设备部署在野外你不可能天天往现场跑——运维自动化不是选择题是必答题。一、工控运维为什么必须自动化工业控制设备的典型运行环境工厂车间、户外机柜、偏远站点7×24小时不间断运行。现场没人懂技术出了问题只能远程排查或者派人去。如果每次数据采集、备份、磁盘清理都靠人工那维护成本会随设备数量线性增长。自动化运维脚本的核心目标让设备自己照顾自己人只管看报警和处理异常。二、crontab定时任务配置Linux下的定时任务调度器每个用户一份crontab格式固定。2.1 crontab格式# ┌──────── 分钟 (0-59) # │ ┌────── 小时 (0-23) # │ │ ┌──── 日 (1-31) # │ │ │ ┌── 月 (1-12) # │ │ │ │ ┌ 周 (0-7, 0和7都是周日) # │ │ │ │ │ * * * * * command2.2 特殊符号符号含义示例*任意值* * * * *每分钟,列表0,15,30,45 * * * *每15分钟-范围0 9-18 * * 1-5工作日9-18点/步长*/5 * * * *每5分钟2.3 常用操作# 编辑当前用户的crontabcrontab-e# 查看当前用户的crontabcrontab-l# 系统级定时任务直接编辑文件vim/etc/crontab# /etc/crontab 比crontab -e多一个用户字段# m h dom mon dow user command三、实战脚本1定时数据采集场景每隔5分钟从串口读取传感器数据存入CSV文件。#!/bin/bash# /opt/scripts/serial_collect.sh# 串口数据采集脚本SERIAL_DEV/dev/ttyS1BAUDRATE9600DATA_DIR/data/serialDATE$(date%Y%m%d)CSV_FILE${DATA_DIR}/sensor_${DATE}.csvmkdir-p$DATA_DIR# 使用stty设置串口参数stty-F$SERIAL_DEV$BAUDRATEcs8-cstopb-parenb-ixon-ixoff# 读取串口数据3秒超时读取一行RAW_DATA$(timeout3cat$SERIAL_DEV|head-n1)if[-z$RAW_DATA];thenecho$(date%Y-%m-%d %H:%M:%S)[WARN] No data from serial$DATA_DIR/collect.logexit1fi# 写入CSV首行表头自动创建if[!-f$CSV_FILE];thenechotimestamp,raw_data$CSV_FILEfiecho$(date%Y-%m-%d %H:%M:%S),${RAW_DATA}$CSV_FILEecho$(date%Y-%m-%d %H:%M:%S)[INFO] Data saved:${RAW_DATA}$DATA_DIR/collect.logcrontab配置# 每5分钟采集一次*/5 * * * * /opt/scripts/serial_collect.sh四、实战脚本2定时备份场景每天凌晨2点打包关键数据保留最近7天备份。#!/bin/bash# /opt/scripts/auto_backup.sh# 定时备份脚本BACKUP_SRC/data/config /data/serial /opt/appBACKUP_DST/backupKEEP_DAYS7DATE$(date%Y%m%d_%H%M%S)BACKUP_FILE${BACKUP_DST}/backup_${DATE}.tar.gzLOG_FILE${BACKUP_DST}/backup.logmkdir-p$BACKUP_DSTecho$(date%Y-%m-%d %H:%M:%S)[INFO] Start backup...$LOG_FILE# 打包压缩tar-czf$BACKUP_FILE$BACKUP_SRC2$LOG_FILEif[$?-eq0];thenFILESIZE$(du-h$BACKUP_FILE|awk{print $1})echo$(date%Y-%m-%d %H:%M:%S)[OK] Backup done:${BACKUP_FILE}(${FILESIZE})$LOG_FILEelseecho$(date%Y-%m-%d %H:%M:%S)[ERROR] Backup failed!$LOG_FILEexit1fi# 清理过期备份find$BACKUP_DST-namebackup_*.tar.gz-mtime${KEEP_DAYS}-deleteecho$(date%Y-%m-%d %H:%M:%S)[INFO] Old backups (${KEEP_DAYS}d) cleaned$LOG_FILE# 列出当前备份echo--- Current backups ---$LOG_FILEls-lh$BACKUP_DST/backup_*.tar.gz$LOG_FILE21crontab配置# 每天凌晨2:00执行备份02* * * /opt/scripts/auto_backup.sh五、实战脚本3远程重启场景远程通过SSH重启目标设备重启后等待设备恢复并验证连通性。#!/bin/bash# /opt/scripts/remote_reboot.sh# 远程重启脚本TARGET_IP192.168.1.100SSH_USERrootSSH_PORT22SSH_KEY/root/.ssh/id_rsaMAX_WAIT120# 最大等待秒数CHECK_INTERVAL5echo$(date)[INFO] Rebooting${TARGET_IP}...# 远程执行rebootssh-i$SSH_KEY-p$SSH_PORT-oConnectTimeout10\-oStrictHostKeyCheckingno\${SSH_USER}${TARGET_IP}sync; reboot2/dev/nullif[$?-ne0][$?-ne255];thenecho$(date)[ERROR] SSH command failedexit1fiecho$(date)[INFO] Reboot command sent, waiting for device to come back...# 等待设备下线sleep10# 轮询检测设备恢复WAIT0while[$WAIT-lt$MAX_WAIT];doifping-c1-W2$TARGET_IP/dev/null21;then# 再等几秒让SSH服务起来sleep5ifssh-i$SSH_KEY-p$SSH_PORT-oConnectTimeout5\-oStrictHostKeyCheckingno\${SSH_USER}${TARGET_IP}echo ok2/dev/null|grep-qok;thenecho$(date)[OK] Device${TARGET_IP}is back online!# 获取设备uptime确认确实重启过ssh-i$SSH_KEY-p$SSH_PORT\${SSH_USER}${TARGET_IP}uptime2/dev/nullexit0fifiWAIT$((WAITCHECK_INTERVAL))sleep$CHECK_INTERVALecho$(date)[INFO] Waiting... (${WAIT}s elapsed)doneecho$(date)[ERROR] Device${TARGET_IP}did not come back within${MAX_WAIT}s!exit1六、实战脚本4磁盘空间监控场景监控磁盘使用率超过80%报警超过90%自动清理旧日志。#!/bin/bash# /opt/scripts/disk_monitor.sh# 磁盘空间监控脚本WARN_THRESHOLD80# 报警阈值(%)CLEAN_THRESHOLD90# 自动清理阈值(%)CLEAN_DIR/var/log# 待清理目录LOG_FILE/var/log/disk_monitor.log# 获取根分区使用率USAGE$(df-h/|awkNR2 {print $5}|tr-d%)echo$(date%Y-%m-%d %H:%M:%S)[INFO] Disk usage:${USAGE}%$LOG_FILEif[$USAGE-ge$CLEAN_THRESHOLD];thenecho$(date%Y-%m-%d %H:%M:%S)[WARN] Disk usage${USAGE}% ${CLEAN_THRESHOLD}%, auto cleaning...$LOG_FILE# 清理7天前的日志文件find$CLEAN_DIR-name*.log-mtime7-delete2/dev/nullfind$CLEAN_DIR-name*.log.*-mtime3-delete2/dev/null# 清理压缩的旧日志find$CLEAN_DIR-name*.gz-mtime3-delete2/dev/null# 截断大日志文件保留最后1000行find$CLEAN_DIR-name*.log-size50M-execsh-c\tail -n 1000 $1 $1.tmp mv $1.tmp $1_{}\;# 清理后重新检测USAGE_AFTER$(df-h/|awkNR2 {print $5}|tr-d%)echo$(date%Y-%m-%d %H:%M:%S)[INFO] After cleanup:${USAGE_AFTER}%$LOG_FILEif[$USAGE_AFTER-ge$CLEAN_THRESHOLD];then# 清理后仍然超标发送报警echo$(date%Y-%m-%d %H:%M:%S)[CRITICAL] Disk still full after cleanup:${USAGE_AFTER}%$LOG_FILE# 调用报警见下一节/opt/scripts/send_alert.shCRITICALDisk usage${USAGE_AFTER}% on$(hostname)fielif[$USAGE-ge$WARN_THRESHOLD];thenecho$(date%Y-%m-%d %H:%M:%S)[WARN] Disk usage${USAGE}% ${WARN_THRESHOLD}%!$LOG_FILE/opt/scripts/send_alert.shWARNINGDisk usage${USAGE}% on$(hostname)ficrontab配置# 每小时检查一次磁盘0* * * * /opt/scripts/disk_monitor.sh七、报警集成7.1 邮件报警#!/bin/bash# /opt/scripts/send_alert.sh# 报警发送脚本邮件 HTTP回调LEVEL$1MESSAGE$2MAIL_TOadminexample.comWEBHOOK_URLhttps://your-api.example.com/alertTIMESTAMP$(date%Y-%m-%d %H:%M:%S)HOSTNAME$(hostname)# 方式1mail命令发邮件echo[${LEVEL}]${TIMESTAMP}${HOSTNAME}:${MESSAGE}|\mail-s[${LEVEL}] Alert from${HOSTNAME}$MAIL_TO2/dev/null# 方式2curl调用Webhook钉钉/企业微信/飞书等curl-s-XPOST$WEBHOOK_URL\-HContent-Type: application/json\-d{\level\:\${LEVEL}\,\host\:\${HOSTNAME}\,\message\:\${MESSAGE}\,\timestamp\:\${TIMESTAMP}\}/dev/null21echo$(date%Y-%m-%d %H:%M:%S)[INFO] Alert sent: [${LEVEL}]${MESSAGE}/var/log/alert.log7.2 钉钉机器人报警示例# 钉钉自定义机器人WebhookDINGTALK_URLhttps://oapi.dingtalk.com/robot/send?access_tokenYOUR_TOKENcurl-s$DINGTALK_URL\-HContent-Type: application/json\-d{ msgtype: text, text: { content: [${LEVEL}] ${HOSTNAME}: ${MESSAGE}}}八、日志轮转除了脚本手动清理Linux自带的logrotate是更规范的日志管理工具# /etc/logrotate.d/myapp/var/log/myapp/*.log{daily rotate7compress delaycompress missingok notifempty copytruncate size 50M}配置说明每天轮转保留7份压缩旧日志文件超过50M也轮转copytruncate在不重启服务的情况下截断当前日志。九、工控运维脚本部署清单序号脚本功能定时周期优先级1serial_collect.sh串口数据采集5分钟高2auto_backup.sh数据备份每天2:00高3disk_monitor.sh磁盘空间监控每小时高4remote_reboot.sh远程重启手动触发中5send_alert.sh报警通知被调用高6logrotate日志轮转每天中7ntp_sync.shNTP时间同步每小时中部署步骤# 1. 创建脚本目录mkdir-p/opt/scripts# 2. 上传所有脚本cp*.sh /opt/scripts/chmodx /opt/scripts/*.sh# 3. 配置crontabcrontab-e# 粘贴所有定时任务# 4. 验证脚本权限和依赖/opt/scripts/auto_backup.sh --dry-run# 5. 检查crontab服务systemctl status crond systemctlenablecrond运维脚本不追求花哨追求可靠、可追溯、可报警。每条定时任务执行结果都要写日志出问题能查到这才是工控运维的基本功。