Video face detection: findface-video-manager and findface-video-worker

Note

The findface-video-worker is delivered in a CPU-accelerated (findface-video-worker) and a GPU-accelerated (findface-video-worker-gpu) packages.

In this section:

Functions of findface-video-manager

The findface-video-manager service is the part of the video face detection module that is used for managing the video face detection functionality.

The findface-video-manager service interfaces with findface-video-worker as follows:

  • It supplies findface-video-worker with settings and the list of to-be-processed video streams. To do so, it issues a so called job, a video processing task which contains configuration settings and stream data.
  • In a distributed system, it distributes video streams (jobs) across vacant findface-video-worker instances.

Note

Configuration settings passed via jobs have priority over the findface-video-manager configuration file.

The findface-video-mananger service functioning requires ETCD, third-party software that implements a distributed key-value store for findface-video-manager. In the FindFace core, ETCD is used as a coordination service, providing the video face detector with fault tolerance.

Functionality:

  • allows for configuring video face detection parameters,
  • allows for managing the list of to-be-processed video streams,
  • implements video face detection management.

Functions of findface-video-worker

The findface-video-worker (or findface-video-worker-gpu) service is the part of the video face detection module, which recognizes faces in video. It can work with both live streams and files, and supports most video formats and codecs that can be decoded by FFmpeg.

The findface-video-worker service interfaces with the findface-video-manager and findface-facerouter services as follows:

  • By request, findface-video-worker gets a job with settings and the list of to-be-processed video streams from findface-video-manager.
  • The findface-video-worker posts extracted normalized face images, along with the full frames and meta data (such as bbox, camera ID and detection time) to the findface-facerouter service for further processing.

Note

In FindFace Security, the findface-facerouter functions are performs by findface-security.

Functionality:

  • detects faces in video,
  • extracts normalized face images,
  • searches for the best face snapshot,
  • snapshot deduplication (only one snapshot per face detection event).

When processing video, findface-video-worker consequently uses the following algorithms:

  • Motion detection. Used to reduce resource consumption. Only when the motion detector recognizes motion of certain intensity that the face tracker can be triggered.
  • Face tracking. The face tracker tracks, detects and captures faces in video. It can simultaneously be working with several faces. It also searches for the best face snapshot, using an embedded neural network. After the best face snapshot is found, it is posted to findface-facerouter.

The best face snapshot can be found in one of the following modes:

  • Real-time
  • Offline

Real-Time Mode

In the real-time mode, findface-video-worker posts a face immediately after it appears in the camera field of view.

  • If rt-perm=True, the face tracker searches for the best face snapshot within each time period equal to rt-delay and posts it to findface-facerouter.
  • If rt-perm=False, the face tracker searches for the best face snapshot dynamically:
    1. First, the face tracker estimates whether the quality of a face snapshot exceeds a pre-defined threshold value. If so, the snapshot is posted to findface-facerouter.
    2. The threshold value increases after each post. Each time the face tracker gets a higher quality snapshot of the same face, it is posted.
    3. When the face disappears from the camera field of view, the threshold value resets to default.

By default, the real-time mode is disabled (realtime=false in the /etc/findface-video-manager.conf file).

Offline Mode

The offline mode is less storage intensive than the real-time one as in this mode findface-video-worker posts only one snapshot per track, but of the highest quality. In this mode, the face tracker buffers a video stream with a face in it until the face disappears from the camera field of view. Then the face tracker picks up the best face snapshot from the buffered video and posts it to findface-facerouter.

By default, the offline mode is enabled (overall=true in the /etc/findface-video-manager.conf file).

Configure Video Face Detection

The video face detector configuration is done through the following configuration files:

  1. The findface-video-manager configuration file /etc/findface-video-manager.conf:

    etcd:
      dial_timeout: 3s
      endpoints: 127.0.0.1:2379
    exp_backoff:
      enabled: false
      factor: 2
      flush_interval: 2m0s
      max_delay: 1m0s
      min_delay: 1s
    job_scheduler_script: ''
    kafka:
      enabled: false
      endpoints: 127.0.0.1:9092
    listen: 127.0.0.1:18810
    master:
      lease_ttl: 10
      self_url: 127.0.0.1:18811
      self_url_http: 127.0.0.1:18811
    ntls:
      enabled: false
      update_interval: 1m0s
      url: http://127.0.0.1:3185/
    prometheus:
      jobs_processed_duration_buckets:
      - 1
      - 30
      - 60
      - 500
      - 1800
      - 3600
      - 21600
      - .inf
    router_url: http://127.0.0.1:18820/v0/frame
    rpc:
     heart_beat_timeout: 4s
     listen: 127.0.0.1:18811
    stream_settings:
      additional_body: []
      additional_headers: []
      api_ssl_verify: true
      api_timeout: 15000
      det_period: 8
      disable_drops: false
      draw_track: false
      fd_frame_height: -1
      ffmpeg_format: ''
      ffmpeg_params: []
      image_arg: photo
      jpeg_quality: 95
      max_candidates: 0
      max_face_size: 0
      md_scale: 0.3
      md_threshold: 0.002
      min_d_score: -1000
      min_face_size: 0
      min_score: -2
      npersons: 4
      only_norm: false
      overall: true
      parse_sei: false
      post_uniq: true
      realtime: false
      realtime_dly: 500
      realtime_post_perm: false
      roi: ''
      rot: ''
      send_track: 0
      tracker_threads: 4
      uc_max_avg_shift: 10
      uc_max_dup: 3
      uc_max_time_diff: 30
    stream_settings_gpu:
      ffmpeg_format: ''
      ffmpeg_params: []
      filter_max_face_size: 8192
      filter_min_face_size: 1
      filter_min_quality: -2
      imotion_threshold: 0
      jpeg_quality: 95
      normalized_only: false
      overall_only: false
      play_speed: -1
      realtime_post_every_interval: false
      realtime_post_interval: 1
      roi: ''
      rot: ''
      router_body: []
      router_headers: []
      router_timeout_ms: 15000
      router_verify_ssl: true
      start_stream_timestamp: 0
      use_stream_timestamp: false
    

    When configuring findface-video-manager, refer to the following parameters:

    Option Description
    router_url IP address and port of the findface-facerouter host to receive detected faces from findface-video-worker. In FindFace Security, findface-facerouter functions are performed by findface-security. Default value: http://127.0.0.1:18820/v0/frame.
    etcd -> endpoints IP address and port of the etcd service. Default value: 127.0.0.1:2379.
    ntls -> enabled If true, findface-video-manager will send a job to findface-video-worker only if the total number of processed cameras does not exceed the allowed number of cameras from the license. Default value: false.
    ntls -> url IP address and port of the findface-ntls host. Default value: http://127.0.0.1:3185/.
  2. If you opt for the CPU-accelerated package findface-video-worker, use the /etc/video-worker.ini configuration file:

    ntls-addr=127.0.0.1:3133
    mgr-static=127.0.0.1:18811
    capacity=10
    #mgr-exec=shell command with arguments
    

    If you opt for the GPU-accelerated package findface-video-worker-gpu, use the etc/findface-video-worker-gpu.ini configuration file.

    ## cuda device number
    device_number = 0
    
    ## read streams from file, do not use VideoManager
    input =
    
    ## exit on first finished job, only when --input specified
    exit_on_first_finished = false
    
    ## models directory
    models_dir = /usr/share/findface-gpudetector/models
    
    ## batch size
    batch_size = 1
    
    ## http server port for metrics, 0=do not start server
    metrics_port = 0
    
    ## resize scale, 1=do not resize
    resize_scale = 1.0
    
    ## maximum number of streams
    capacity = 100
    
    ## videomanager grpc ip:port
    #mgr_static = 172.17.47.14:18811
    mgr_static = 127.0.0.1:18811
    
    ## ntls server ip:port
    #ntls_addr = 172.17.47.14:3133
    ntls_addr = 127.0.0.1:3133
    
    ## debug: save faces to dir
    save_dir =
    
    ## minimum face size
    min_face_size = 60
    
    ## preinit detector for specified resolutions: "640x480;1920x1080"
    resolutions =
    
    ## worker labels: "k=v;group=enter"
    labels =
    
    #------------------------------
    [video_decoder]
    #------------------------------
    ## decode video on cpu
    cpu = false
    
    #------------------------------
    [send]
    #------------------------------
    ## posting faces threads
    threads = 8
    
    ## posting faces maximum queue size
    queue_limit = 256
    
    #------------------------------
    [tracker]
    #------------------------------
    ## max face miss duration, sec
    miss_interval = 1.0
    
    ## overlap threshold
    overlap_threshold = 0.25
    
    ## use timestamps from SEI packet (Netris only)
    use_time_from_sei = false
    

    Whether you configure findface-video-worker or findface-video-worker-gpu, refer to the following parameters:

    Parameter Description
    ntls-addr IP address and port of the findface-ntls host.
    mgr-static IP address of the findface-video-manager host to provide findface-video-worker with settings and the list of to-be-processed streams.
    capacity Maximum number of video streams to be processed by findface-video-worker.
    mgr-exec (Optional, instead of the mgr-static parameter) A script to describe dynamic IP address of the findface-video-manager host.