.. _face-features:

Enable Face and Face Attribute Recognition
===========================================

FindFace Server allows you to recognize human faces and face attributes. Subject to your needs, you can enable recognition of such face attributes as age, gender, emotions, beard, glasses, medical masks, head position, or liveness.

Face and face attribute recognition can be manually enabled and configured. Face and face attribute recognition works on both GPU- and CPU-acceleration.

.. note::

   There are examples of how the sections should look like below. It may vary depending on the models that you have selected.


#. To enable face recognition, do the following:

   Specify neural network models for face object detection in the ``/opt/ffserver/configs/extraction-api.yaml`` configuration file.

   .. important::
      Be sure to choose the right acceleration type for each model, matching the acceleration type of ``extraction-api``: CPU or GPU. Be aware that ``extraction-api`` on CPU can work only with CPU-models, while ``extraction-api`` on GPU supports both CPU- and GPU-models.

   #. Open the ``extraction-api.yaml`` configuration file.

      .. code:: bash

         sudo vi /opt/ffserver/configs/extraction-api.yaml

   #. Specify the face detector model in the ``detectors -> models`` section by pasting the following code:

      .. rubric:: GPU

      .. code:: bash

         detectors:
           ...
           models:
             ...
             face_jasmine:
               aliases:
               - face
               - nnd
               - cheetah
               model: detector/facedet.jasmine_fast.004.gpu.fnk
               options:
                 min_object_size: 32
                 resolutions:
                 - 2048x2048
           ...

      .. rubric:: CPU

      .. code:: bash

         detectors:
           ...
           models:
             ...
             face_jasmine:
               aliases:
               - face
               - nnd
               - cheetah
               model: detector/facedet.jasmine_fast.004.cpu.fnk
               options:
                 min_object_size: 32
                 resolutions:
                 - 2048x2048
           ...


   #. Make sure that the ``objects -> face`` section contains the ``quality_attribute: face_quality`` and the  ``base_normalizer: facenorm/crop2x.v2_maxsize400.gpu.fnk`` or the ``base_normalizer: facenorm/crop2x.v2_maxsize400.cpu.fnk``, depending on your acceleration type:

      .. rubric:: GPU

      .. code:: bash

         objects:
           ...
           face:
             base_normalizer: facenorm/crop2x.v2_maxsize400.gpu.fnk
             quality_attribute: face_quality
           ...

      .. rubric:: CPU

      .. code:: bash

         objects:
           ...
           face:
             base_normalizer: facenorm/crop2x.v2_maxsize400.cpu.fnk
             quality_attribute: face_quality
           ...

   #. Specify the face normalizer models in the ``normalizers`` section by pasting the following code:

      .. rubric:: GPU

      .. code:: bash

         normalizers:
           ...
           models:
             crop1x:
               model: facenorm/crop1x.v2_maxsize400.gpu.fnk
             crop2x:
               model: facenorm/crop2x.v2_maxsize400.gpu.fnk
             cropbbox:
               model: facenorm/cropbbox.v2.gpu.fnk
             multicrop_full_center:
               model: ''
             multicrop_full_crop2x:
               model: facenorm/facenorm.multicrop_full_crop2x_size400.gpu.fnk
             norm200:
               model: facenorm/bee.v3.gpu.fnk
           ...

      .. rubric:: CPU

      .. code:: bash

         normalizers:
           ...
           models:
             crop1x:
               model: facenorm/crop1x.v2_maxsize400.cpu.fnk
             crop2x:
               model: facenorm/crop2x.v2_maxsize400.cpu.fnk
             cropbbox:
               model: facenorm/cropbbox.v2.cpu.fnk
             multicrop_full_center:
               model: ''
             multicrop_full_crop2x:
               model: facenorm/facenorm.multicrop_full_crop2x_size400.cpu.fnk
             norm200:
               model: facenorm/bee.v3.cpu.fnk
           ...


   #. .. note::

         This step is required to enable face attribute recognition.

      To enable face attribute recognition, do the following:

      In the ``/opt/ffserver/configs/extraction-api.yaml`` configuration file, specify the extraction models in the ``extractors`` section, subject to the extractors you want to enable. Be sure to indicate the right acceleration type for each model, matching the acceleration type of ``extraction-api``: CPU or GPU.

      .. rubric:: GPU

      .. code:: bash

         extractors:
           ...
           models:
             face_age:
               default:
                 model: faceattr/faceattr.age.v3.gpu.fnk
             face_beard:
               default:
                 model: faceattr/beard.v0.gpu.fnk
             face_emben:
               default:
                 model: face/nectarine_xl_320.gpu.fnk
             face_emotions:
               default:
                 model: faceattr/emotions.v1.gpu.fnk
             face_gender:
               default:
                 model: faceattr/faceattr.gender.v3.gpu.fnk
             face_glasses3:
               default:
                 model: faceattr/glasses3.v0.gpu.fnk
             face_headpose:
               default:
                 model: faceattr/headpose.v3.gpu.fnk
             face_liveness:
               default:
                 model: faceattr/faceattr.liveness_web.v1.gpu.fnk
             face_medmask3:
               default:
                 model: faceattr/medmask3.v2.gpu.fnk
             face_quality:
               default:
                  model: faceattr/faceattr.quality.v5.gpu.fnk
           ...


      .. rubric:: CPU

      .. code:: bash

         extractors:
           ...
           models:
             face_age:
               default:
                 model: faceattr/faceattr.age.v3.cpu.fnk
             face_beard:
               default:
                 model: faceattr/beard.v0.cpu.fnk
             face_emben:
               default:
                 model: face/nectarine_xl_320.cpu.fnk
             face_emotions:
               default:
                 model: faceattr/emotions.v1.cpu.fnk
             face_gender:
               default:
                 model: faceattr/faceattr.gender.v3.cpu.fnk
             face_glasses3:
               default:
                 model: faceattr/glasses3.v0.cpu.fnk
             face_headpose:
               default:
                 model: faceattr/headpose.v3.cpu.fnk
             face_liveness:
               default:
                 model: faceattr/faceattr.liveness_web.v1.cpu.fnk
             face_medmask3:
               default:
                 model: faceattr/medmask3.v2.cpu.fnk
             face_quality:
               default:
                 model: faceattr/faceattr.quality.v5.cpu.fnk
           ...



      The most used extraction models are the following:

      +------------------+--------------+--------------------------------------------------------------------------------+
      | Extractor        | Acceleration | Configure as follows                                                           |
      +==================+==============+================================================================================+
      | age              | CPU          | ``face_age: faceattr/faceattr.age.v3.cpu.fnk``                                 |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_age: faceattr/faceattr.age.v3.gpu.fnk``                                 |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | beard            | CPU          | ``face_beard: faceattr/beard.v0.cpu.fnk``                                      |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_beard: faceattr/beard.v0.gpu.fnk``                                      |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | CPU          | ``face_beard4: faceattr/faceattr.beard4.v1.cpu.fnk``                           |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_beard4: faceattr/faceattr.beard4.v1.cpu.fnk``                           |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | individual face  | CPU          | ``face_emben: face/nectarine_xl_320.cpu.fnk``                                  |
      | feature vector   +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_emben: face/nectarine_xl_320.gpu.fnk``                                  |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | gender           | CPU          | ``face_gender: faceattr/faceattr.gender.v3.cpu.fnk``                           |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_gender: faceattr/faceattr.gender.v3.gpu.fnk``                           |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | emotions         | CPU          | ``face_emotions: faceattr/emotions.v1.cpu.fnk``                                |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_emotions: faceattr/emotions.v1.gpu.fnk``                                |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | glasses          | CPU          | ``face_glasses3: faceattr/glasses3.v0.cpu.fnk``                                |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_glasses3: faceattr/glasses3.v0.gpu.fnk``                                |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | CPU          | ``face_glasses4: faceattr/faceattr.glasses4.v0.cpu.fnk``                       |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | head position    | CPU          | ``face_headpose: faceattr/headpose.v3.cpu.fnk``                                |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_headpose: faceattr/headpose.v3.gpu.fnk``                                |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | face liveness    | CPU          | ``face_liveness: faceattr/faceattr.liveness_web.v1.cpu.fnk``                   |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_liveness: faceattr/faceattr.liveness_web.v1.gpu.fnk``                   |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | face mask        | CPU          | ``face_medmask3: faceattr/medmask3.v2.cpu.fnk``                                |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_medmask3: faceattr/medmask3.v2.gpu.fnk``                                |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | CPU          | ``face_medmask4: faceattr/faceattr.medmask4.v0.cpu.fnk``                       |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_medmask4: faceattr/faceattr.medmask4.v0.gpu.fnk``                       |
      +------------------+--------------+--------------------------------------------------------------------------------+
      | face quality     | CPU          | ``face_quality: faceattr/faceattr.quality.v5.cpu.fnk``                         |
      |                  +--------------+--------------------------------------------------------------------------------+
      |                  | GPU          | ``face_quality: faceattr/faceattr.quality.v5.gpu.fnk``                         |
      +------------------+--------------+--------------------------------------------------------------------------------+


#. To enable face recognition, modify the ``/opt/ffserver/configs/video-worker.yaml`` configuration file.

   #. In the ``models`` section, specify the face neural network models by analogy with the example below:

      .. rubric:: GPU

      .. code:: bash

         sudo vi /opt/ffserver/configs/video-worker.yaml

      .. code:: bash

         models:
           ...
           detectors:
             ...
             face:
               fnk_path: /usr/share/findface-data/models/detector/facedet.jasmine_fast.004.gpu.fnk
               min_size: 60
             ...
           normalizers:
             ...
             face_norm:
               fnk_path: /usr/share/findface-data/models/facenorm/crop2x.v2_maxsize400.gpu.fnk
             face_norm_quality:
               fnk_path: /usr/share/findface-data/models/facenorm/crop1x.v2_maxsize400.gpu.fnk
             ...
           extractors:
             ...
             face_quality:
               fnk_path: /usr/share/findface-data/models/faceattr/faceattr.quality.v5.gpu.fnk
               normalizer: face_norm_quality
             ...

      .. rubric:: CPU

      .. code:: bash

         models:
           ...
           detectors:
             ...
             face:
               fnk_path: /usr/share/findface-data/models/detector/facedet.jasmine_fast.004.cpu.fnk
               min_size: 60
             ...
           normalizers:
             ...
             face_norm:
               fnk_path: /usr/share/findface-data/models/facenorm/crop2x.v2_maxsize400.cpu.fnk
             face_norm_quality:
               fnk_path: /usr/share/findface-data/models/facenorm/crop1x.v2_maxsize400.cpu.fnk
             ...
           extractors:
             ...
             face_quality:
               fnk_path: /usr/share/findface-data/models/faceattr/faceattr.quality.v5.cpu.fnk
               normalizer: face_norm_quality
             ...


   #. Make sure that the ``objects -> face`` section is included:

      .. code:: bash

           objects:
             ...
             face:
               normalizer: face_norm
               quality: face_quality
               track_features: ''


#. To enable face recognition, open the ``/opt/ffserver/configs/video-manager.yaml`` configuration file and make sure it contains the ``face`` section in ``detectors`` that looks similar to the example below.


  .. code:: bash

      sudo vi /opt/ffserver/configs/video-manager.yaml

  .. code:: bash

      detectors:
        face:
          filter_min_quality: 0.45
          filter_min_size: 1
          filter_max_size: 8192
          roi: ""
          fullframe_crop_rot: false
          fullframe_use_png: false
          jpeg_quality: 95
          overall_only: false
          realtime_post_first_immediately: false
          realtime_post_interval: 1
          realtime_post_every_interval: false
          track_interpolate_bboxes: true
          track_miss_interval: 1
          track_overlap_threshold: 0.25
          track_max_duration_frames: 0
          track_send_history: false
          post_best_track_frame: true
          post_best_track_normalize: true
          post_first_track_frame: false
          post_last_track_frame: false
          tracker_type: simple_iou
          track_deep_sort_matching_threshold: 0.65
          track_deep_sort_filter_unconfirmed_tracks: true
          track_object_is_principal: false
          track_history_active_track_miss_interval: 0
          filter_track_min_duration_frames: 1
          extractors_track_triggers: {}