How to create your own Detector Plugin

What is a Detector Plugin?

Detector Plugins are image analysis engines. Usually Object Detection or Facial Recognition.

See a list of included Detector Plugins.

Plugin Pieces

The Important Piece

To explain this we will look at the Tensorflow plugin.

In this Repository you will find the shinobi-tensorflow.js file. This is the detection function set for the TensorFlow plugin.

Below is the key portion of this file. You'll see the declaration of s.detectObject. This is what makes the plugin do the detection and send back the response to the main process.

You'll notice there is a slight amount of post-processing after the detection result is offered for the given frame. This is to conform the drawn boxes to the format Shinobi recognizes. This way it can draw the boxes on the live stream and all display them over the recordings.

const ObjectDetectors = require('./ObjectDetectors.js')(config);

s.detectObject = function(buffer,d,tx,frameLocation,callback){
    new ObjectDetectors(buffer).process().then((resp)=>{
        var results = resp.data
        if(results[0]){
            var mats = []
            results.forEach(function(v){
                mats.push({
                    x: v.bbox[0],
                    y: v.bbox[1],
                    width: v.bbox[2],
                    height: v.bbox[3],
                    tag: v.class,
                    confidence: v.score,
                })
            })
            var isObjectDetectionSeparate = d.mon.detector_pam === '1' && d.mon.detector_use_detect_object === '1'
            var width = parseFloat(isObjectDetectionSeparate  && d.mon.detector_scale_y_object ? d.mon.detector_scale_y_object : d.mon.detector_scale_y)
            var height = parseFloat(isObjectDetectionSeparate  && d.mon.detector_scale_x_object ? d.mon.detector_scale_x_object : d.mon.detector_scale_x)
            tx({
                f:'trigger',
                id:d.id,
                ke:d.ke,
                details:{
                    plug:config.plug,
                    name:'Tensorflow',
                    reason:'object',
                    matrices:mats,
                    imgHeight:width,
                    imgWidth:height,
                    time: resp.time
                },
                frame: buffer
            })
        }
        callback()
    })
}

Making your own Detector Plugin

First you need to have the following to do this.

• Know how to setup a plugin (Try setting up the Tensorflow plugin)
• Knowledge of how to use Node.js (JavaScript)
• Have a Detection Engine that operates in Node.js or another language, like Python.

Let's Begin.

1. Navigate to your Shinobi folder. Default is /home/Shinobi.

cd /home/Shinobi

1. We're going to copy the tensorflow plugin and rename it to being a new plugin. For this example we will be making a Gun Detector Plugin.

cp plugins/tensorflow plugins/gundetector

2. Open the newly copied folder.

cd plugins/gundetector

3. Now we need to edit some files to make them designated as "gundetector".

4. Now that you have your plugin setup. You can try it out by running the following :

node shinobi-gundetector.js

You can daemonize it alongside Shinobi like this

pm2 start shinobi-gundetector.js

Be sure you are the OS user that installed Shinobi, so if you installed while as root then become root before starting the plugin.

5. Great, it's now running! The Object Detection section should say GunDetect Connected appearing in the section header. You can now follow any of the Object Detection setup guides to use your new plugin!