Video face detection: `findface-video-manager` and `findface-video-worker`¶

Note

The findface-video-worker is delivered in a CPU-accelerated (findface-video-worker) and a GPU-accelerated (findface-video-worker-gpu) packages.

In this section:

Functions of findface-video-manager
Functions of findface-video-worker
Configure Video Face Detection

Functions of `findface-video-manager`¶

The findface-video-manager service is the part of the video face detection module that is used for managing the video face detection functionality.

The findface-video-manager service interfaces with findface-video-worker as follows:

It supplies findface-video-worker with settings and the list of to-be-processed video streams. To do so, it issues a so called job, a video processing task which contains configuration settings and stream data.
In a distributed system, it distributes video streams (jobs) across vacant findface-video-worker instances.

Note

Configuration settings passed via jobs have priority over the findface-video-manager configuration file.

The findface-video-mananger service functioning requires ETCD, third-party software that implements a distributed key-value store for findface-video-manager. In the FindFace core, ETCD is used as a coordination service, providing the video face detector with fault tolerance.

Functionality:

allows for configuring video face detection parameters,
allows for managing the list of to-be-processed video streams,
implements HTTP API for video face detection management.

Functions of `findface-video-worker`¶

The findface-video-worker (or findface-video-worker-gpu) service is the part of the video face detection module, which recognizes faces in video. It can work with both live streams and files, and supports most video formats and codecs that can be decoded by FFmpeg.

The findface-video-worker service interfaces with the findface-video-manager and findface-facerouter services as follows:

By request, findface-video-worker gets a job with settings and the list of to-be-processed video streams from findface-video-manager.
The findface-video-worker posts extracted normalized face images, along with the full frames and meta data (such as bbox, camera ID and detection time) to the findface-facerouter service for further processing.

Functionality:

detects faces in video,
extracts normalized face images,
searches for the best face snapshot,
snapshot deduplication (only one snapshot per face detection event).

When processing video, findface-video-worker consequently uses the following algorithms:

Motion detection. Used to reduce resource consumption. Only when the motion detector recognizes motion of certain intensity that the face tracker can be triggered.
Face tracking. The face tracker tracks, detects and captures faces in video. It can simultaneously be working with several faces. It also searches for the best face snapshot, using an embedded neural network. After the best face snapshot is found, it is posted to findface-facerouter.

The best face snapshot can be found in one of the following modes:

Real-time
Offline

Real-Time Mode

In the real-time mode, findface-video-worker posts a face immediately after it appears in the camera field of view.

If rt-perm=True, the face tracker searches for the best face snapshot within each time period equal to rt-delay and posts it to findface-facerouter.
If rt-perm=False, the face tracker searches for the best face snapshot dynamically:
1. First, the face tracker estimates whether the quality of a face snapshot exceeds a pre-defined threshold value. If so, the snapshot is posted to findface-facerouter.
2. The threshold value increases after each post. Each time the face tracker gets a higher quality snapshot of the same face, it is posted.
3. When the face disappears from the camera field of view, the threshold value resets to default.

By default, the real-time mode is disabled (realtime=false in the /etc/findface-video-manager.conf file).

Offline Mode

The offline mode is less storage intensive than the real-time one as in this mode findface-video-worker posts only one snapshot per track, but of the highest quality. In this mode, the face tracker buffers a video stream with a face in it until the face disappears from the camera field of view. Then the face tracker picks up the best face snapshot from the buffered video and posts it to findface-facerouter.

By default, the offline mode is enabled (overall=true in the /etc/findface-video-manager.conf file).

Configure Video Face Detection ¶

The video face detector configuration is done through the following configuration files:

The findface-video-manager configuration file /etc/findface-video-manager.conf:

etcd:
  dial_timeout: 3s
  endpoints: 127.0.0.1:2379
exp_backoff:
  enabled: false
  factor: 2
  flush_interval: 2m0s
  max_delay: 1m0s
  min_delay: 1s
job_scheduler_script: ''
kafka:
  enabled: false
  endpoints: 127.0.0.1:9092
listen: 127.0.0.1:18810
master:
  lease_ttl: 10
  self_url: 127.0.0.1:18811
  self_url_http: 127.0.0.1:18811
ntls:
  enabled: false
  update_interval: 1m0s
  url: http://127.0.0.1:3185/
prometheus:
  jobs_processed_duration_buckets:
  - 1
  - 30
  - 60
  - 500
  - 1800
  - 3600
  - 21600
  - .inf
router_url: http://127.0.0.1:18820/v0/frame
rpc:
 heart_beat_timeout: 4s
 listen: 127.0.0.1:18811
stream_settings:
  additional_body: []
  additional_headers: []
  api_ssl_verify: true
  api_timeout: 15000
  det_period: 8
  disable_drops: false
  draw_track: false
  fd_frame_height: -1
  ffmpeg_format: ''
  ffmpeg_params: []
  image_arg: photo
  jpeg_quality: 95
  max_candidates: 0
  max_face_size: 0
  md_scale: 0.3
  md_threshold: 0.002
  min_d_score: -1000
  min_face_size: 0
  min_score: -2
  npersons: 4
  only_norm: false
  overall: true
  parse_sei: false
  post_uniq: true
  realtime: false
  realtime_dly: 500
  realtime_post_perm: false
  roi: ''
  rot: ''
  send_track: 0
  tracker_threads: 4
  uc_max_avg_shift: 10
  uc_max_dup: 3
  uc_max_time_diff: 30
stream_settings_gpu:
  ffmpeg_format: ''
  ffmpeg_params: []
  filter_max_face_size: 8192
  filter_min_face_size: 1
  filter_min_quality: -2
  imotion_threshold: 0
  jpeg_quality: 95
  normalized_only: false
  overall_only: false
  play_speed: -1
  realtime_post_every_interval: false
  realtime_post_interval: 1
  roi: ''
  rot: ''
  router_body: []
  router_headers: []
  router_timeout_ms: 15000
  router_verify_ssl: true
  start_stream_timestamp: 0
  use_stream_timestamp: false

When configuring findface-video-manager, refer to the following parameters:

Option	Description
`router_url`	IP address and port of the `findface-facerouter` host to receive detected faces from `findface-video-worker`. Default value: `http://127.0.0.1:18820/v0/frame`.
`etcd` -> `endpoints`	IP address and port of the `etcd` service. Default value: `127.0.0.1:2379`.
`ntls` -> `enabled`	If true, `findface-video-manager` will send a job to `findface-video-worker` only if the total number of processed cameras does not exceed the allowed number of cameras from the license. Default value: false.
`ntls` -> `url`	IP address and port of the `findface-ntls` host. Default value: `http://127.0.0.1:3185/`.

You can also configure the following parameters:

Note

In the stream_settings(-gpu) section of the file, you will find settings common to all video streams. To make settings of a certain stream, pass them in a job, a video processing task that findface-video-manager issues to findface-video-worker (see Job Object).

Option	Description
`ffmpeg_format`	Pass FFMPEG format (mxg, flv, etc.) if it cannot be detected automatically.
`ffmpeg_params`	List of a video stream ffmpeg options with their values as a key=value array: [“rtsp_transpotr=tcp”, .., “ss=00:20:00”]. Check out the FFmpeg web site for the full list of options. Default value: options not specified.
`md_threshold`, `imotion_threshold`	Minimum motion intensity to be detected by the motion detector. The threshold value is to be fitted empirically. Empirical units: zero and positive rational numbers. Milestones: 0 = detector disabled, 0.002 = default value, 0.05 = minimum intensity is too high to detect motion.
`md_scale`	Video frame scaling coefficient for the motion detector, relative to the original size from 0 to 1. Scale down in the case of high resolution cameras, or close up faces, or if the CPU load is too high, to reduce the system resources consumption. Make sure that the scaled face size exceeds the `min-face-size` value. Default value: 1 (original size).
`fd_frame_height`	Video frame height for the face tracker, in pixels. Scale down in the case of high resolution cameras, or close up faces, or if the CPU load is too high, to reduce the system resources consumption. Make sure that the scaled face size exceeds the `min-face-size` value. Default value: -1 (negative values corresponds to the original size). Optimal value to reduce load: 640-720.
`uc_max_time_diff`	Only if `post_uniq: true` (face deduplication enabled). Maximum time period in seconds during which a number of similar faces are considered as belonging to one person. Default value: 30.
`uc_max_dup`	Only if `post_uniq: true` (face deduplication enabled). Maximum number of faces during the `uc_max_time_diff` period that is posted for a person. Default value: 3.
`uc_max_avg_shift`	Only if `post_uniq: true` (face deduplication enabled). Distance in pixels within which a number of similar faces are considered as belonging to one person. Default value: 10.
`realtime`	Enables the real-time mode for the best face search. Default value: false.
`npersons`	Maximum number of faces simultaneously tracked by the face tracker. This parameter severely affects performance. Default value: 4.
`disable_drops`	Enables posting all appropriate faces without drops. By default, if `findface-video-worker` does not have enough resources to process all frames with faces, it drops some of them. If this option is active, `findface-video-worker` puts odd frames on the waiting list to process them later. Default value: false.
`tracker_threads`	Number of tracking threads for the face tracker. This value should be less or equal to the `npersons` value. We recommend you to set them equal. If the number of tracking threads is less than the maximum number of tracked faces, resource consumption is reduced but so is the tracking speed. Default value: 1.
`image_arg`	Name of the argument containing a bbox with a face, in an API request. Default value: `photo`.
`additional_headers`	Additional header fields in a POST request when posting a face: [“key = value”]. Default value: headers not specified.
`additional_body`	Additional body fields in the request body when posting a face: [“key = value”]. Default value: body fields not specified.
`api_timeout`	Timeout for a `findface-facerouter` response to a `findface-video-worker` API request, in milliseconds. If the timeout has expired, the system will log an error. Default value: 15000.
`api_ssl_verify`, `router_verify_ssl`	Enables a https certificate verification when `findface-video-worker` and `findface-facerouter` interact over https. Default value: true. If false, a self-signed certificate can be accepted.
`post_uniq`	Enables face deduplication, i.e. posting only a certain number of faces belonging to one person, during a certain period of time. In this case, if `findface-video-worker` posts a face to `findface-facerouter` and then tracks another one within the time period `uc_max_time_diff`, and the distance between the two faces doesn’t exceed `uc_max_avg_shift`, `findface-video-worker` estimates their similarity. If the faces are similar and the total number of similar faces during the `uc_max_time_diff` period does not exceed the number `uc_max_dup`, `findface-video-worker` posts the other face. Otherwise, the other face is not posted. Default value: true.
`min_score`, `filter_min_quality`	Minimum threshold value for a face image quality. A face is posted if it has better quality. The threshold value is to be fitted empirically. Empirical units: negative rational numbers to zero. Milestones: 0 = high quality faces, -1 = good quality, -2 = satisfactory quality, -5 = face recognition maybe inefficient. Default value: -7.
`min_d_score`	Maximum deviation of a face from its frontal position. A face is posted if its deviation is less than this value. The deviation is to be fitted empirically. Empirical units: negative rational numbers to zero. Milestones: -3.5 = large face angles, face recognition may be inefficient, -2.5 = satisfactory deviation, -0.05 = close to the frontal position, 0 = frontal face. Default value: -1000.
`realtime_dly`, `realtime_post_interval`	Only for the real-time mode. If `realtime_post_perm=True`, defines the time period in milliseconds within which the face tracker picks up the best snapshot and posts it to `findface-facerouter`. If `realtime_post_perm=False`, defines the minimum time period between 2 posts of the same face with increased quality. Default value: 500.
`realtime_post_perm`, `realtime_post_every_interval`	Only for the realtime mode. Post best snapshots obtained within each `realtime_dly` time period. If false, search for the best snapshot dynamically and send snapshots in order of increasing quality. Default value: false.
`rot`	Enables detecting and tracking faces only inside a clipping rectangle WxH+X+Y. You can use this option to reduce `findface-video-worker` load. Default value: rectangle not specified.
`roi`	Enable posting faces detected only inside a region of interest WxH+X+Y. Default value: region not specified.
`draw_track`	Enables drawing a face motion track in a bbox. Default value: false.
`send_track`	Enables posting a face motion track as array of the bbox center coordinates. As the `send_track` value, specify the number of dots in the motion track. Default value: 0 (array not posted).
`min_face_size`, `filter_min_face_size`	Minimum size of a face in pixels. Undersized faces are not posted. Default value: 0 (filter disabled).
`max_face_size`, `filter_max_face_size`	Maximum size of a face in pixels. Oversized faces are not posted. Default value: 0 (filter disabled).
`overall`	Enables the offline mode for the best face search. Default value: true.
`only_norm`	Enable posting only normalized face images without full frames. Default value: false.
`jpeg_quality`	Quality of an original frame JPEG compression, in percents. Default value: 95%.
`use_stream_timestamp`	If true, retrieve and post timestamps from a video stream. If false, post the actual date and time.

If you opt for the CPU-accelerated package findface-video-worker, use the /etc/findface-video-worker.ini configuration file:

ntls-addr=127.0.0.1:3133
mgr-static=127.0.0.1:18811
capacity=10
#mgr-exec=shell command with arguments

If you opt for the GPU-accelerated package findface-video-worker-gpu, use the /etc/findface-video-worker-gpu.ini configuration file.

## cuda device number
device_number = 0

## read streams from file, do not use VideoManager
input =

## exit on first finished job, only when --input specified
exit_on_first_finished = false

## models directory
models_dir = /usr/share/findface-gpudetector/models

## batch size
batch_size = 1

## http server port for metrics, 0=do not start server
metrics_port = 0

## resize scale, 1=do not resize
resize_scale = 1.0

## maximum number of streams
capacity = 30

## command to obtain videomanager's grpc ip:port
mgr_cmd =

## videomanager grpc ip:port
mgr_static = 127.0.0.1:18811

## ntls server ip:port
ntls_addr = 127.0.0.1:3133

## debug: save faces to dir
save_dir =

## minimum face size
min_face_size = 60

## preinit detector for specified resolutions: "640x480;1920x1080"
resolutions =

# worker labels: "k=v;group=enter"
labels =

## use timestamps from SEI packet
use_time_from_sei = false

#------------------------------
[streamer]
#------------------------------
## streamer server port, 0=disabled
port = 9999

## streamer url - how to access this worker on streamer_port
url = ws://127.0.0.1:9999/stream/

#------------------------------
[liveness]
#------------------------------
## path to liveness fnk
fnk =

## liveness threshold
threshold = 0.945

## liveness internal algo param
interval = 1.0

## liveness internal algo param
stdev_cnt = 1

#------------------------------
[video_decoder]
#------------------------------
## decode video on cpu
cpu = false

#------------------------------
[send]
#------------------------------
## posting faces threads
threads = 8

## posting faces maximum queue size
queue_limit = 256

#------------------------------
[tracker]
#------------------------------
## max face miss duration, sec
miss_interval = 1.0

## overlap threshold
overlap_threshold = 0.25```

When configuring findface-video-worker/findface-video-worker-gpu, refer to the following parameters:

Parameter	Description
`ntls-addr`	IP address and port of the `findface-ntls` host.
`mgr-static`	IP address of the `findface-video-manager` host to provide `findface-video-worker` with settings and the list of to-be-processed streams.
`capacity`	Maximum number of video streams to be processed by `findface-video-worker`.
`mgr-exec`	(Optional, instead of the `mgr-static` parameter) A script to describe dynamic IP address of the `findface-video-manager` host.
`fnk`	(Only for GPU, optional). Path to the face liveness detector.

Video face detection: findface-video-manager and findface-video-worker¶

Functions of findface-video-manager¶

Functions of findface-video-worker¶

Configure Video Face Detection¶

Video face detection: `findface-video-manager` and `findface-video-worker`¶

Functions of `findface-video-manager`¶

Functions of `findface-video-worker`¶

Configure Video Face Detection ¶