Plugin Basics

In this section:

Plugin Architecture

After the findface-video-worker component detects a face, the face is posted to the findface-facerouter component via an HTTP API request. To process this request, each findface-facerouter plugin must export the activate(app, ctx, plugin_name, plugin_source) function.

The activate function has the following parameters:

  • app: a tornado.web.Application entity of the findface-facerouter component.

  • ctx: data context to be passed to a plugin upon activation.

  • plugin_name: the name of the plugin to be activated.

  • plugin_source: source object to load the plugin from.

Upon activation, a plugin is passed the following data context:

  1. request.ctx.sfapi: a set up ntech.sfapi_client.Client instance that can be invoked directly to process the result of video face detection (for example, to create a new gallery, add a face to a gallery, etc.).

  2. plugins: OrderedDict with all the plugins as (key: plugin name, value: the result returned by the activate function).

  3. idgen: id generator that can be invoked as ctx.idgen().

The activate(app, ctx, plugin_name, plugin_source) function must return an object with the following methods:

  1. preprocess,

  2. process,

  3. shutdown (optional).

The preprocess method

In this method, a findface-facerouter plugin decides if it is interested in the face received from the findface-video-worker component. If so, it returns a tuple or a list that contains one or several strings 'facen', 'gender', 'age', 'emotions'. This means that it is necessary to extract a feature vector, recognize gender, age, emotions respectively. If the returned tuple/list is non-empty, the findface-facerouter redirects the face to the findface-sf-api in a /detect POST request with relevant query string parameters (facen=on, gender=on, age=on, emotions=on).

The basic preprocess method to inherit from has the following syntax (see the Plugin class):

preprocess(self, request: FrHTTPRequest, labels: Mapping[str, str]) → Tuple[str]
Parameters

  • FrHTTPRequest (tornado.httpserver.HTTPRequest) – a HTTP API request that includes an extra argument params

  • labels (dictionary) – a custom set of a frame labels, which are initially specified in a job parameters for findface-video-worker and then assigned to the frame

The params argument of FrHTTPRequest includes the following fields:

Parameters

  • photo (bytes) – JPEG video frame featuring a detected face

  • face0 (bytes) – normalized face image

  • bbox (list of integers [[x1,y1,x2,y2]], where x1: x coordinate of the top-left corner, y1: y coordinate of the top-left corner, x2: x coordinate of the bottom-right corner, y2: y coordinate of the bottom-right corner) – coordinates of the face region in the video frame

  • cam_id (string) – camera id

  • timestamp (datetime.datetime) – video frame timestamp

  • detectorParams (dictionary) – debug information from the video face detector

  • bs_type (string) – best face search mode. Available options: overall (the findface-video-worker posts only one snapshot per track, but of the highest quality.), realtime (the findface-video-worker posts the best snapshot within each of consecutive time intervals).

  • labels (dictionary) – (duplicates params.labels) a custom set of a frame labels, which are specified in a job parameters for findface-video-worker and then assigned to the frame

The decision about face processing is made based on the data in the request.params, including the custom set of labels, as well as for any other reasons.

The process method

This method is called if the preprocess method returns a non-empty tuple or list (i.e. with ‘facen’, ‘gender’, ‘age’, an/or ‘emotions’ strings). After the findface-sf-api returns a response with the result of face detection (see the /detect POST request) with all the requested face features, the findface-facerouter component calls the process method of the plugin in order to the perform face processing itself.

To process a face, a plugin uses request.ctx.sfapi.

The basic process method to inherit from has the following syntax (see the Plugin class):

process(self, request: FrHTTPRequest, photo: bytes, bbox: List[int], event_id: int, detection: DetectFace)

The shutdown method

This method is only called before the findface-facerouter shutdown.

The basic shutdown method to inherit from has the following syntax (see the Plugin class):

shutdown(self)