.. _cars:

Enable Vehicle and Vehicle Attribute Recognition
=================================================

FindFace Server allows you to recognize a single vehicle and its attributes.

The vehicle attributes are as follows:

* license plate number (for selected countries),

  .. note::
     Currently, the following countries are supported:

     **Europe:** Russia, Lithuania, Latvia, Estonia, Finland, Czech Republic, Serbia, Belarus, Ukraine, Moldova, Georgia, Azerbaijan, Armenia.

     **Asia:** the UAE, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Uzbekistan, Saudi Arabia, Vietnam, India, Pakistan, Thailand.

     **North America:** Mexico.

     **South America:** Argentina, Brazil.

* color,
* make,
* model,
* body style,
* vehicle category,
* vehicle orientation (front, rear, or side),
* special vehicle type (taxi, route transport, carsharing, ambulance, police, rescue service, gas service, military, road service, other special),
* vehicle weight and body size.

  .. note::
     Special vehicle recognition as well as vehicle weight and body size recognition work for selected countries only and may not work for your country. For more information please contact your manager or our support team (support@ntechlab.com).

This section describes how to enable vehicle and vehicle attribute recognition.

To enable recognition of vehicles and their attributes, do the following:

#. Specify neural network models for vehicle recognition and vehicle attribute recognition in the ``/opt/ffserver/configs/extraction-api.yaml`` configuration file.

   .. important::
      Be sure to choose the right acceleration type for each model, matching the acceleration type of ``extraction-api``: CPU or GPU. Be aware that ``extraction-api`` on CPU can work only with CPU-models, while ``extraction-api`` on GPU supports both CPU- and GPU-models.

   #. Open the ``extraction-api.yaml`` configuration file.

      .. code:: bash

         sudo vi /opt/ffserver/configs/extraction-api.yaml

   #. Specify the vehicle detector model in the ``detectors -> models`` section by pasting the following code:

      .. rubric:: GPU

      .. code:: bash

         detectors:
           ...
           models:
             ...
             gustav:
               aliases:
               - car
               - efreitor
               model: detector/cardet.kali.008.gpu.fnk
               options:
                 min_object_size: 32
                 resolutions:
                 - 2048x2048
           ...

      .. rubric:: CPU

      .. code:: bash

         detectors:
           ...
           models:
             ...
             gustav:
               aliases:
               - car
               - efreitor
               model: detector/cardet.kali.008.cpu.fnk
               options:
                 min_object_size: 32
                 resolutions:
                 - 2048x2048
           ...


   #. Make sure that the ``objects -> car`` section contains the ``quality_attribute: car_quality`` and the ``base_normalizer: facenorm/cropbbox.v2.gpu.fnk`` or the ``base_normalizer: facenorm/cropbbox.v2.cpu.fnk``, depending on your acceleration type:

      .. rubric:: GPU

      .. code:: bash

         objects:
           ...
           car:
             base_normalizer: facenorm/cropbbox.v2.gpu.fnk
             quality_attribute: car_quality
           ...

      .. rubric:: CPU

      .. code:: bash

         objects:
           ...
           car:
             base_normalizer: facenorm/cropbbox.v2.cpu.fnk
             quality_attribute: car_quality
           ...

   #. Specify the normalizers required for the extractors. If you need license plate recognition, specify the ``carlicplate`` normalizer. For other extractors, specify the ``cropbbox`` normalizer.

      +-------------+-------------------------------------------+-------------------------------------------------------------------+
      | Normalizer  | Normalizer model                          | Used for extractors                                               |
      +=============+===========================================+===================================================================+
      | carlicplate | ``carnorm/carnorm.briacon.v2.gpu.fnk``    | ``car_license_plate``                                             |
      |             | ``carnorm/carnorm.briacon.v2.cpu.fnk``    |                                                                   |
      +-------------+-------------------------------------------+-------------------------------------------------------------------+
      | cropbbox    | ``facenorm/cropbbox.v2.gpu.fnk``          | ``car_license_plate_quality``, ``car_description``,               |
      |             | ``facenorm/cropbbox.v2.cpu.fnk``          | ``car_quality``, ``car_special_types11``, ``car_categories``,     |
      |             |                                           | ``car_orientation``, ``car_weight_types7``                        |
      +-------------+-------------------------------------------+-------------------------------------------------------------------+

      .. rubric:: GPU

      .. code:: bash

         normalizers:
           ...
           models:
             ...
             carlicplate:
               model: carnorm/carnorm.briacon.v2.gpu.fnk
             ...
             cropbbox:
               model: facenorm/cropbbox.v2.gpu.fnk
             ...

      .. rubric:: CPU

      .. code:: bash

         normalizers:
           ...
           models:
             ...
             carlicplate:
               model: carnorm/carnorm.briacon.v2.cpu.fnk
             ...
             cropbbox:
               model: facenorm/cropbbox.v2.cpu.fnk
             ...


   #. Specify the extraction models in the ``extractors -> models`` section, subject to the extractors you want to enable:

      .. rubric:: GPU

      .. code:: bash

         extractors:
            ...
            models:
              car_categories:
                default:
                  model: carattr/carattr.categories.v1.gpu.fnk
              car_description:
                default:
                  model: carattr/carattr.description.v1.gpu.fnk
              car_emben:
                default:
                  model: carrec/bottas.gpu.fnk
              car_license_plate:
                default:
                  model: carattr/carattr.license_plate.v9.gpu.fnk
              car_license_plate_quality:
                default:
                  model: carattr/carattr.license_plate_quality.v3.gpu.fnk
              car_orientation:
                default:
                  model: carattr/carattr.orientation.v0.gpu.fnk
              car_quality:
                default:
                  model: carattr/carattr.quality.v1.gpu.fnk
              car_special_types11:
                default:
                  model: carattr/carattr.special_types11.v1.gpu.fnk
              car_weight_types7:
                default:
                  model: carattr/carattr.weight_types7.v0.gpu.fnk
            ...

      .. rubric:: CPU

      .. code:: bash

         extractors:
            ...
            models:
              car_categories:
                default:
                  model: carattr/carattr.categories.v1.cpu.fnk
              car_description:
                default:
                  model: carattr/carattr.description.v1.cpu.fnk
              car_emben:
                default:
                  model: carrec/bottas.cpu.fnk
              car_license_plate:
                default:
                  model: carattr/carattr.license_plate.v9.cpu.fnk
              car_license_plate_quality:
                default:
                  model: carattr/carattr.license_plate_quality.v3.cpu.fnk
              car_orientation:
                default:
                  model: carattr/carattr.orientation.v0.cpu.fnk
              car_quality:
                default:
                  model: carattr/carattr.quality.v1.cpu.fnk
              car_special_types11:
                default:
                  model: carattr/carattr.special_types11.v1.cpu.fnk
              car_weight_types7:
                default:
                  model: carattr/carattr.weight_types7.v0.cpu.fnk
            ...


      The most used extraction models are the following:

      +------------------+----------------------------------------------------------------------------------+
      | Extractor        | Configure as follows                                                             |
      +==================+==================================================================================+
      | vehicle feature  | ``car_emben: carrec/bottas.cpu.fnk``                                             |
      | vector           +----------------------------------------------------------------------------------+
      |                  | ``car_emben: carrec/bottas.gpu.fnk``                                             |
      +------------------+----------------------------------------------------------------------------------+
      | license plate    | ``car_license_plate: carattr/carattr.license_plate.v9.cpu.fnk``                  |
      | number           | ``car_license_plate_quality: carattr/carattr.license_plate_quality.v3.cpu.fnk``  |
      |                  +----------------------------------------------------------------------------------+
      |                  | ``car_license_plate: carattr/carattr.license_plate.v9.gpu.fnk``                  |
      |                  | ``car_license_plate_quality: carattr/carattr.license_plate_quality.v3.gpu.fnk``  |
      +------------------+----------------------------------------------------------------------------------+
      | set of           | ``car_description: carattr/carattr.description.v1.cpu.fnk``                      |
      | attributes:      |                                                                                  |
      | make / color     +----------------------------------------------------------------------------------+
      | / model / body   | ``car_description: carattr/carattr.description.v1.gpu.fnk``                      |
      | style            |                                                                                  |
      +------------------+----------------------------------------------------------------------------------+
      | vehicle image    | ``car_quality: carattr/carattr.quality.v1.cpu.fnk``                              |
      | quality          +----------------------------------------------------------------------------------+
      |                  | ``car_quality: carattr/carattr.quality.v1.gpu.fnk``                              |
      +------------------+----------------------------------------------------------------------------------+
      | special vehicle  | ``car_special_types11: carattr/carattr.special_types11.v1.cpu.fnk``              |
      |                  +----------------------------------------------------------------------------------+
      |                  | ``car_special_types11: carattr/carattr.special_types11.v1.gpu.fnk``              |
      +------------------+----------------------------------------------------------------------------------+
      | vehicle category | ``car_categories: carattr/carattr.categories.v1.cpu.fnk``                        |
      |                  +----------------------------------------------------------------------------------+
      |                  | ``car_categories: carattr/carattr.categories.v1.gpu.fnk``                        |
      +------------------+----------------------------------------------------------------------------------+
      | vehicle weight   | ``car_weight_types7: carattr/carattr.weight_types7.v0.cpu.fnk``                  |
      | and body size    +----------------------------------------------------------------------------------+
      |                  | ``car_weight_types7: carattr/carattr.weight_types7.v0.gpu.fnk``                  |
      +------------------+----------------------------------------------------------------------------------+
      | vehicle          | ``car_orientation: carattr/carattr.orientation.v0.cpu.fnk``                      |
      | orientation      +----------------------------------------------------------------------------------+
      |                  | ``car_orientation: carattr/carattr.orientation.v0.gpu.fnk``                      |
      +------------------+----------------------------------------------------------------------------------+


#. Modify the ``/opt/ffserver/configs/video-worker.yaml`` configuration file.

   #. In the ``models`` section, specify the vehicle detector, normalizer, and extractor models as shown in the example below:

      .. rubric:: GPU

      .. code:: bash

         sudo vi /opt/ffserver/configs/video-worker.yaml

      .. code:: bash

         models:
           ...
           detectors:
             ...
             car:
               fnk_path: /usr/share/findface-data/models/detector/cardet.kali.008.gpu.fnk
               min_size: 60
             ...
           normalizers:
             ...
             car_norm:
               fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk
             car_norm_quality:
               fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk
             ...
           extractors:
             ...
             car_quality:
               fnk_path: /usr/share/findface-data/models/carattr/carattr.quality.v1.gpu.fnk
               normalizer: car_norm_quality

      .. rubric:: CPU

      .. code:: bash

         sudo vi /opt/ffserver/configs/video-worker.yaml

      .. code:: bash

         models:
           ...
           detectors:
             ...
             car:
               fnk_path: /usr/share/findface-data/models/detector/cardet.kali.008.cpu.fnk
               min_size: 60
             ...
           normalizers:
             ...
             car_norm:
               fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk
             car_norm_quality:
               fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk
             ...
           extractors:
             ...
             car_quality:
               fnk_path: /usr/share/findface-data/models/carattr/carattr.quality.v1.cpu.fnk
               normalizer: car_norm_quality


   #. Make sure that the ``objects -> car`` section is included:

      .. code:: bash

           objects:
             ...
             car:
               normalizer: car_norm
               quality: car_quality
               track_features: ''

#. Open the ``/opt/ffserver/configs/video-manager.yaml`` configuration file and make sure it contains the ``car`` section in ``detectors`` that looks similar to the example below. Note that the ``filter_min_quality`` parameter is set to ``0.65`` by default. You can increase it to get more accurate results from cameras and other video sources. The best practice is to set this value to ``0.73``.

   .. code:: bash

      sudo vi /opt/ffserver/configs/video-manager.yaml

   .. code:: bash

      detectors:
        ...
        car:
          filter_min_quality: 0.65
          filter_min_size: 100
          filter_max_size: 8192
          roi: ''
          fullframe_crop_rot: false
          fullframe_use_png: false
          jpeg_quality: 95
          overall_only: true
          realtime_post_first_immediately: false
          realtime_post_interval: 1
          realtime_post_every_interval: false
          track_interpolate_bboxes: true
          track_miss_interval: 1
          track_overlap_threshold: 0.25
          track_max_duration_frames: 0
          track_send_history: false
          post_best_track_frame: true
          post_best_track_normalize: true
          post_first_track_frame: false
          post_last_track_frame: false
          tracker_type: simple_iou
          track_deep_sort_matching_threshold: 0.65
          track_deep_sort_filter_unconfirmed_tracks: true
          track_object_is_principal: false
          track_history_active_track_miss_interval: 0