How to Cluster multiple Shinobi (Child Nodes)
Use multiple computers as if they were one big machine.
What are Child Nodes? What is Clustering?
Child Nodes are the machines that are connected to the main Shinobi engine to provide added CPU and GPU power to the workload.
Clustering is when you have multiple machines working together.
The childNodes object in conf.json
There are a few option to set. Options are only required if you choose to use the childNodes object.
| Option | Required | Type | Description | Default |
|---|---|---|---|---|
| enabled | no | bool | Whether or not to enable the childNode system. For Master and Child. | true |
| mode | yes | string | Type of module in the childNode system. For Master and Child. | master |
| port | no | integer | The port running on the Master Node in which Child Nodes connect on. For Master. | 8288 |
| key | yes | string | This string is sent back and forth to authenticate each request. For Master and Child. | none |
| host | yes | string | The connection address of the Master Node including the port. For Child. | none |
| masterDoWorkToo | no | bool | This flag decides if Master Node should be used for Monitor Processes. For Master. | false |
| maxCpuPercent | no | integer | This decides at what CPU percentage a Child Node cannot be used. For Master. | 75 |
How to setup a Master Node
1. Open your conf.json and add the following options to activate the Child Node connection port. The default connection port is 8288.
Remember to set the key value to something random, anything other than what is provided by default.
"childNodes": {
"enabled" : true,
"mode" : "master",
"port" : 8288,
"key" : "aMatchingKey1234",
"masterDoWorkToo": false,
"maxCpuPercent": 75
}
2. Then restart Shinobi to activated the changes.
pm2 restart camera
3. Now you can attach Child Nodes on port 8288.
How to setup a Child Node
Warning : You must never set a Master Node to a child unless it is a clean Shinobi install with no videos remaining in the file system.
1. As an example let's assume that our Master Node has the IP address 192.168.1.48 and is listening for children on port 8288. You will need to setup your configuration as follows. Remember to match the key value or it will not connect.
"childNodes": {
"enabled" : true,
"mode" : "child",
"host" : "192.168.1.48:8288",
"key" : "aMatchingKey1234"
}
2. Now you must restart Shinobi to turn it into a Child Node.
pm2 restart camera
3. Now you when you start a monitor (or restart) it will use the first available child node with less than 80% CPU use. The automatic method for choosing a Child Node may base off other details instead of just the CPU in the future.
Where is the repository for the childNode code?
It is all bult into camera.js so you just need to download another copy of Shinobi on another machine and set the childNodes options to Child Node mode. You just need the following installed on the machine before attempting to run Shinobi in Child Node Mode.
- Node.js 12.x (12.22.7 Stable)
- FFmpeg 3.2.4 to 4.1
Database engine is not required. The Child Node will proxy all database requests through the Master Node.
What about the database?
You do not need to install any database engine on the Child Nodes. No MariaDB, No MySQL, No SQLite, etc.
All children will use the Master Node to execute database queries. Only the Master Node needs to have connection details.
You can leave the db as default for all child nodes because they will proxy all database requests through the Websocket to the master.
What about the Detectors?
The Built-In Motion Detector by Kevin Godell will operate the same. The motion detection will occur on the Node the monitor is running on. If a monitor is running on a Child Node then motion detection will occur on that same Child Node.
The Plugin based detectors will be a little different. Each Node, Child and Master, will have to be connected to an instance of a plugin.
You can run multiple instances of the same plugin on one machine or each machine run a version of the plugin and have them connect accordingly.