How to run Detector Plugins in Cluster Mode

What is this? Plugins in a Cluster?

In short : Sharing the load for detection requests between plugins, instead of sending to all at one time.

Generally when you connect a plugin, like tensorflow or yolo, you could only send images in certain ways to those plugins. Those methods are as follows.

• Directly to all connected plugins at the same time.
• Check for Motion First and on success send to all connected plugins at the same time.

Plugins being able to run in Cluster Mode adds the following ways to send images to your detection engines.

• Send to the plugin with the least processing images at time of request.
  • This is the default method when enabling plugins in Cluster Mode.
  • Recommended method.
• Send to the plugin with the least GPU usage.
  • This calculation averages utilization of available GPUs on the plugin's side.
  • This requires nvidia-smi.
  • This does not take into account specified GPUs.
• Send to the plugin with the least CPU usage.
  • Only do this if all your plugins are running on CPU usage.
  • Not recommended. Please use the default method.

In gist these methods work as follows :

• Send to connected plugin with lowest utilization.
• Check for Motion First and on success send to connected plugin with lowest utilization.

What is the benefit of doing this?

Using a detection plugin means eat resources on the machine it runs on.

For Example : You may have a server with a lot of load already and decide to move the plugin to another machine but then realize that detection has now met the maximum and you want to do more. Enter Plugins in Cluster Mode.

Now you can setup multiple machines, each running a plugin. Shinobi will disperse image requests between the connected machines automatically.

Connect as many as you want

If one dies, no big deal. Shinobi will just start sending to the other ones as if nothing happened.

Once you revive your plugin server and reconnect: Shinobi will automatically engage it with active processes and their detection requests.

How to enable Plugins in Cluster Mode

To do this through the web panel :

1. Go to your Super User panel and open the Configuration tab

2. Add the following value in the JSON editing tool.

"detectorPluginsCluster": true,

3. Restart Shinobi.

To do this through command line :

1. Run the following inside the Shinobi folder.

node tools/modifyConfiguration.js addToConfig='{"detectorPluginsCluster":true}'

2. Restart Shinobi.

Adding Multiple Plugins with the same Name

The idea behind this is to use multiple plugins of the same type so you get a better experience out of your choice. The issue is that a name for a plugin can only be registered once. To get around this just open your plugin's conf.json and edit the plug value.

Example Scenario :

I have 3 machines ready to use tensorflow. Every machine is setup and ready to go, it just needs the plugins enabled and the main application just needs the keys added to allow connection.

• On my first machine leave the plug value as Tensorflow.
• Second machine I open conf.json and set plug to Tensorflow2.
• Third machine I set plug to Tensorflow3.

Second machine's conf.json would look like this :

{
  "plug":"Tensorflow2",
  "host":"10.1.103.113",
  "tfjsBuild":"gpu",
  "port":8080,
  "hostPort":8082,
  "key":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "mode":"client",
  "type":"detector"
}

Third machine's conf.json would look like this :

{
  "plug":"Tensorflow3",
  "host":"10.1.103.113",
  "tfjsBuild":"gpu",
  "port":8080,
  "hostPort":8082,
  "key":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "mode":"client",
  "type":"detector"
}

• plug : The unique name for the plugin when it is registered in Shinobi. It should be different for each running plugin to avoid overriding.
• host : The IP or hostname of the Shinobi server. 10.1.103.113 is an example IP Address.
• port : The port that you connect to the Shinobi server on. 8080 is the default port.
• key : This needs to match what is added in the main conf.json to allow connection.
• tfjsBuild : This option is unique to the tensorflow based plugins. It can be cpu or gpu.
• mode : The way the plugin connects to the Shinobi server. This can be client or host.
• hostPort : If in host mode then this will be where the Shinobi server connects to for plugin registration.

How to enable checking utilization based on CPU

To do this through the web panel :

1. Go to your Super User panel and open the Configuration tab

2. Add the following value in the JSON editing tool.

"clusterUseBasicFrameCount": false,

3. Restart Shinobi.

To do this through command line :

1. Run the following inside the Shinobi folder.

node tools/modifyConfiguration.js addToConfig='{"clusterUseBasicFrameCount":false}'

2. Restart Shinobi.

How to enable checking utilization based on GPU

The same steps as above for enabling CPU checking are required before enabling checking based on GPU.

To do this through the web panel :

1. Go to your Super User panel and open the Configuration tab

2. Add the following value in the JSON editing tool.

"clusterBasedOnGpu": true,

3. Restart Shinobi.

To do this through command line :

1. Run the following inside the Shinobi folder.

node tools/modifyConfiguration.js addToConfig='{"clusterBasedOnGpu":true}'

2. Restart Shinobi.