Timelapse automation using Shinobi, ffmpeg and Discord on Unraid

Introduction

Shinobi allows for timelapse captures of footage recorded by a surveillance camera, as a series of JPG still images. That's a good method if you want to look at pictures, but not as handy if you want to review a whole day or several hours of footage quickly.
I have installed Shinobi in a docker container on my Unraid server, and was looking for a solution to automate conversion of still images to a timelapse video. My requirements were:

1. Hands-off, automated conversion.
2. Conversion task should be scheduled daily, at a given hour.
3. Conversion should cover the whole previous day's worth of still images.
4. Each step of the process should be sent as a notification to my personal Discord server.
5. Still images would be kept for 7 days, in case conversion process fails, as well as to allow reviewing a still image separately, without fumbling in the generated video frame by frame. After 7 days, older still images would be deleted from the server.

With those requirements in mind, and not knowing a whole lot about Linux, Bash, Shinoby or ffmpeg, I started working towards the goals above. Please note that the article applies to one as well as many monitors, and there is some manual configuration involved, as well as some changes you need to perform in the scripts detailed below.

Prerequisites

You need to have the following things in working order:

• An Unraid server you have root access to;
• An Unraid share where your still images and timelapse captures reside;
• Shinobi installed and working in a container on the Unraid server, with at least one monitor configured and working;
• Timelapse option enabled for Shinobi;
• [optional] a Discord account with access to your own Discord server (granted by default under your Discord account).

Software installation

There are several pieces of software you need to install beforehand. Make sure they are all installed. These will be detailed below:

1. Community Applications plugin for Unraid
2. User Scripts plugin for Unraid, installable through the Community Applications plugin (search for "User Scripts" and install)
3. One of the ffmpeg static builds appropriate for your underlying hardware configuration

Software configuration

First, you need to make sure you know the paths to Shinobi timelapse images and ffmpeg binaries. If you want notifications to be sent to your Discord server, you need to configure the server to have at least one webhook and create a bash script which uses that webhook to send out notifications. Documentation is here. I am going to assume you want those, and will present the process including them. If that's not the case, you can simply delete the extra lines and ignore the configuration steps involving Discord. I will prefix those steps with the [Discord] tag.

I am going to use my own paths in the scripts below, they will be italicized so that you'd know what needs to be changed. Generally, all text in italics would need to be adjusted to match your own configuration (namely, paths to stuff).

[Discord] The first thing to do is create a bash script which allows you to send a string message to your Discord server. Open an Unraid  terminal and use a text editor (I use nano) to create a bash script for discord notifications:

sudo nano /home/discord-notify.sh

Then, you need to paste this text in it:

message="$@"
## format to parse to curl
## echo Sending message: $message
msg_content=\"$message\"
## discord webhook
url='https://discord.com/api/webhooks/your_webhook_dir/your_webhook'
curl -H "Content-Type: application/json" -X POST -d "{\"content\": $msg_content}" $url

Note that your_webhook_dir and your_webhook must match the actual webhook you have generated after following the Discord webhook documentation above.
Save the file and remember where you placed it. I recommend placing all necessary components and script files in the same folder. It's not mandatory, but makes one's life easier.

The next step is to create the main script which would automate everything.
Go to your Unraid Server, go to Plugins tab and click the icon next to "User Scripts".

Click "Add New Script", give it a name, you will see your script appearing in the list of User Scripts.
Hover with your mouse pointer over the cog next to the script name, and pick "Edit Script" (you can also change name and description, use something meaningful).
Paste the script below:

#!/bin/bash

sh /home/discord-notify.sh "$(date +%T): Timelapse generation started. Renaming files..."
iii=1
cd /mnt/user/cctv/monitor1_timelapse/$(date -d "-1 days" '+%Y-%m-%d')
pwd

for x in * ; do

case ${#iii} in
1 )
prefix='00000' ;;
2 )
prefix='0000' ;;
3 )
prefix='000' ;;
4 )
prefix='00' ;;
5 )
prefix='0' ;;
esac

mv -- "$x" "$prefix$iii.jpg"

((iii=iii+1))

done

discordmessage="$(date +%T): $iii files renamed; video generation started..."

sh /home/discord-notify.sh "$discordmessage"

/home/ffmpeg-git-20210611-amd64-static/ffmpeg -framerate 30 -i /mnt/user/cctv/monitor1/monitor1_timelapse/$(date -d "-1 days" '+%Y-%m-%d')/%06d.jpg -c:v libx265 -preset medium -crf 28 -r 30 -c:a aac -b:a 128k /mnt/user/cctv/TL/$(date -d "-1 days" '+%Y-%m-%d').mp4
discordmessage="$(date +%T): Video generated. Deleting folder /mnt/user/cctv/monitor1/monitor1_timelapse/$(date -d "-7 days" '+%Y-%m-%d')..."

sh /home/discord-notify.sh "$discordmessage"

rm -r /mnt/user/cctv/monitor1/monitor1_timelapse/$(date -d "-7 days" '+%Y-%m-%d')

discordmessage="$(date +%T): Done."

sh /home/discord-notify.sh "$discordmessage"

Script documentation

sh /home/discord-notify.sh "$(date +%T): Timelapse generation started. Renaming files..."

The line above calls discord-notify.sh, which sends a message to your Discord server using a webhook. 
Example: 02:00:01: Timelapse generation started. Renaming files...

"iii" is a variable used for renaming timelapse images to an integer counter, preparing each file for conversion into a timelapse video.

cd /mnt/user/cctv/monitor1_timelapse/$(date -d "-1 days" '+%Y-%m-%d')

The line above goes to the folder where images are stored, for the previous day. The folder is in form of a date, and the line dynamically determines and formats yesterday's date to match the date folder. "monitor1" is the name of your configured monitor, you would likely have to change this to accommodate your folder name.
If you want to convert the images from X days ago, instead of yesterday, you need to change "-1" to "-X", where X represent how many days ago you want to go to.

The case statement dynamically determines the leading zeroes for each file (five leading zeroes for image 1 through 9, four leading zeroes for image 10 through 99, etc). This allows for up to 999,999 files to be renamed in sequence with the proper leading zeroes, preparing the set of images for ffmpeg conversion.

mv -- "$x" "$prefix$iii.jpg"

This line renames the files in sequence, according to generated prefix and counter.

discordmessage="$(date +%T): $iii files renamed; video generation started..."

This sends a Discord notification with a timestamp after file renaming is complete for all image files.

/home/ffmpeg-git-20210611-amd64-static/ffmpeg -framerate 30 -i /mnt/user/cctv/monitor1/monitor1_timelapse/$(date -d "-1 days" '+%Y-%m-%d')/%06d.jpg -c:v libx265 -preset medium -crf 28 -r 30 -c:a aac -b:a 128k /mnt/user/cctv/TL/$(date -d "-1 days" '+%Y-%m-%d').mp4

This is where you can make a host of changes, depending what you want to achieve. ffmpeg has a lot of options for tuning output, the settings above reflect my preference. They might be okay to you, however in case you want to adjust things, I suggest reading ffmpeg documentation and applying those changes as you see fit. I'll explain mine.

"-framerate 30" tells ffmpeg which is the input's framerate. Since we are dealing with a series of images, instead of an input video file, there is no implicit framerate. My Shinobi timelapse configuration captures one file every 0.1 minutes, that means 6 images per minute and a total of 14,400 images per day. I have chosen 30 FPS input framerate and 30 FPS output framerate, meaning that one second of a timelapse video is equal to 180 seconds (3 minutes) in real time.

"-i /mnt/user/cctv/monitor1/monitor1_timelapse/$(date -d "-1 days" '+%Y-%m-%d')/%06d.jpg" is the input path and file names for ffmpeg conversion. You need to change "monitor1/monitor1_timelapse" to your own path.

"-c:v libx265 -preset medium -crf 28 -r 30 -c:a aac -b:a 128k" are X.265 encoder options. I use X.265 because I want to generate smaller files and don't need full compatibilty when playing on various pieces of older hardware. "-crf 28" is the quality parameter, feel free to play with this value. "-r 30" is the output framerate. "-c:a aac -b:a 128k" are the audio parameters, which obviously do nothing because we have plain images as input. I left them there because they don't negatively affect conversion and I use the same parameters elsewhere. It was simply easier to not touch them.

discordmessage="$(date +%T): Video generated. Deleting folder /mnt/user/cctv/monitor1/monitor1_timelapse/$(date -d "-7 days" '+%Y-%m-%d')..."

sh /home/discord-notify.sh "$discordmessage"

This sends another notification to the Discord server, after video is converted and before archive images are deleted. Note the "-7 days" string, you will have to change the value to the actual value if you decide to delete files from a different number of days ago.

rm -r /mnt/user/cctv/monitor1/monitor1_timelapse/$(date -d "-7 days" '+%Y-%m-%d')

This deletes the folder containing images (still captures) from 7 days ago. Change the "-7" to the value you want (e.g. -10 for 10 days ago, etc.)

discordmessage="$(date +%T): Done."
sh /home/discord-notify.sh "$discordmessage"

Final message sent to Discord.

Discord output example

shinobi [BOT]
— Today at 2:00 AM
02:00:01: Timelapse generation started. Renaming files...
02:00:25: 14401 files renamed; video generation started...
shinobi [BOT]
— Today at 2:23 AM
02:23:39: Video generated. Deleting folder /mnt/user/cctv/mjwVb9pFuK/8sHGyfFGu6_timelapse/2021-07-11...
02:23:42: Done.

As you can see, the whole process took almost 24 minutes on my server which has a Ryzen 5 2400G CPU. I have configured the script to run every day at 2:00 AM.

Scheduling the script

Scheduling the script is easy and the User Scripts plugin takes care of that as well. All you need to do is set the script schedule to "Custom" and enter 5 values in form of a cron job configuration. My values are "0 2 * * *" which mean "minute zero, hour 2, every day, every month, every day of week". If you want to run the script daily, at 4:15 AM, you need to enter values like this: "15 4 * * *".

Notes

If you have multiple monitors, you have two possibilities to use this process:

1. Create a separate script for each monitor (you only need to change paths). It's recommended to have separate schedules, to avoid the scripts running all at once and overloading your server resources
2. Copy the whole script, except "#!/bin/bash" line, again, at the end, and modify paths to point to your next monitor.

Right now I only have one surveillance camera connected, but soon enough I will have more, and I'll use option #1, because it allows me to deactivate video conversion separately for each monitor.

A few words about space usage with my own setup. Shinobi takes captures in 1920x1080 resolution, a full day's worth of image stills (14,400 files) uses between 8 GB and 9 GB of disk space, and the resulted timelapse video uses up between 650 and 700 MB of space for a day. 
I plan to keep all timelapses for 10 years' retention time, which, using a back-of-the-napkin quick calculation, would take around 2.5 TB of data.
If you want to keep all images for the last 7 days, they should not take more than 70 GB for one monitor.

That's all. Good luck and have fun!