Automation Workflows

Transform archiving from a manual chore into a self-sustaining system. Progress from manual commands to fully automated pipelines.

Level 1

Manual

Level 2

Semi-Auto

Level 3

Fully Auto

Level 1: Manual Workflows

Beginner

Using command-line tools for specific, one-off tasks. This is the foundation for all automation.

Single video download

# Best quality video+audio
yt-dlp -f 'bv*+ba' https://www.youtube.com/watch?v=dQw4w9WgXcQ

Level 2: Semi-Automated Workflows

Intermediate

Use system schedulers to run manual commands automatically at regular intervals.

The classic Unix scheduler. Edit your crontab with crontab -e.

Cron examples

# At 3:00 AM every Sunday, update gallery archives
0 3 * * 0 /home/user/scripts/update_galleries.sh

# Every hour, import new links into ArchiveBox
0 * * * * /usr/local/bin/archivebox add < /home/user/new_links.txt

Level 3: Fully Automated Systems

Advanced

Create reactive systems that monitor for changes and trigger archiving pipelines automatically.

Create a "watch folder" and automatically process new files as they arrive.

watch_folder.sh

#!/bin/bash
# Monitor a directory and process new files
WATCH_DIR="/home/user/Downloads/incoming"

inotifywait -m -e create -e moved_to --format "%w%f" "$WATCH_DIR" |
while read FILEPATH
do
    echo "New file detected: $FILEPATH"
    # 1. Scrub metadata
    mat2 "$FILEPATH"
    # 2. Add the cleaned file to ArchiveBox
    archivebox add "$FILEPATH"
    # 3. Move to processed directory
    mv "$FILEPATH" "/home/user/Downloads/processed/"
done

Run this script as a background systemd service for fully automated processing.