Enable Vehicle and Vehicle Attribute Recognition

FindFace Multi allows you to recognize a single vehicle and its attributes.

The vehicle attributes are as follows:

  • license plate number (for selected countries),

    Note

    Currently, the following countries are supported:

    Europe: Russia, Lithuania, Latvia, Estonia, Finland, Czech Republic, Serbia, Belarus, Ukraine, Moldova, Georgia, Azerbaijan, Armenia.

    Asia: the UAE, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Uzbekistan, Saudi Arabia, Vietnam, India, Pakistan, Thailand.

    North America: Mexico.

    South America: Argentina, Brazil.

  • color,

  • make,

  • model,

  • body style,

  • vehicle category,

  • vehicle orientation (front, rear, or side),

  • special vehicle type (taxi, route transport, carsharing, ambulance, police, rescue service, gas service, military, road service, other special),

  • vehicle weight and body size.

    Note

    Special vehicle recognition as well as vehicle weight and body size recognition work for selected countries only and may not work for your country. For more information please contact your manager or our support team (support@ntechlab.com).

Vehicle recognition along with the attributes choice can be configured at the installation level. This section describes how to enable vehicle and vehicle attribute recognition in case this step has been skipped during installation.

To enable recognition of vehicles and their attributes, do the following:

  1. Specify neural network models for vehicle recognition and vehicle attribute recognition in the /opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml configuration file.

    Important

    Be sure to choose the right acceleration type for each model, matching the acceleration type of findface-extraction-api: CPU or GPU. Be aware that findface-extraction-api on CPU can work only with CPU-models, while findface-extraction-api on GPU supports both CPU- and GPU-models.

    1. Open the findface-extraction-api.yaml configuration file.

      sudo vi /opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml
      
    2. Specify the vehicle detector model in the detectors -> models section by pasting the following code:

      GPU

      detectors:
      
        ...
        models:
          ...
          gustav:
            aliases:
            - car
            - efreitor
            model: detector/car.gustav_accurate.006.gpu.fnk
            options:
              min_object_size: 32
              resolutions:
              - 256x256
              - 384x384
              - 512x512
              - 768x768
              - 1024x1024
              - 1536x1536
              - 2048x2048
        ...
      

      CPU

      detectors:
      
        ...
        models:
          ...
          gustav:
            aliases:
            - car
            - efreitor
            model: detector/car.gustav_accurate.006.cpu.fnk
            options:
              min_object_size: 32
              resolutions:
              - 256x256
              - 384x384
              - 512x512
              - 768x768
              - 1024x1024
              - 1536x1536
              - 2048x2048
        ...
      
    3. Make sure that the objects -> car section contains the quality_attribute: car_quality and the base_normalizer: facenorm/cropbbox.v2.gpu.fnk or the base_normalizer: facenorm/cropbbox.v2.cpu.fnk, depending on your acceleration type:

      GPU

      objects:
        ...
        car:
          base_normalizer: facenorm/cropbbox.v2.gpu.fnk
          quality_attribute: car_quality
        ...
      

      CPU

      objects:
        ...
        car:
          base_normalizer: facenorm/cropbbox.v2.cpu.fnk
          quality_attribute: car_quality
        ...
      
    4. Specify the normalizers required for the extractors. If you need license plate recognition, specify the carlicplate normalizer. For other extractors, specify the cropbbox normalizer.

      Normalizer

      Normalizer model

      Used for extractors

      carlicplate

      carnorm/briacon.v0.gpu.fnk carnorm/briacon.v0.cpu.fnk

      car_license_plate

      cropbbox

      facenorm/cropbbox.v2.gpu.fnk facenorm/cropbbox.v2.cpu.fnk

      car_license_plate_quality, car_description, car_quality, car_special_types11, car_categories, car_orientation, car_weight_types7

      GPU

      normalizers:
        ...
      
        models:
          ...
          carlicplate:
            model: carnorm/briacon.v0.gpu.fnk
          ...
          cropbbox:
            model: facenorm/cropbbox.v2.gpu.fnk
          ...
      

      CPU

      normalizers:
        ...
      
        models:
          ...
          carlicplate:
            model: carnorm/briacon.v0.cpu.fnk
          ...
          cropbbox:
            model: facenorm/cropbbox.v2.cpu.fnk
          ...
      
    5. Specify the extraction models in the extractors -> models section, subject to the extractors you want to enable:

      GPU

      extractors:
         ...
         models:
          car_categories: carattr/carattr.categories.v0.gpu.fnk
          car_color: ''
          car_container_number: ''
          car_description: carattr/description.v0.gpu.fnk
          car_emben: carrec/bottas.gpu.fnk
          car_license_plate: carattr/carattr.license_plate.v7.gpu.fnk
          car_license_plate_quality: carattr/carattr.license_plate_quality.v1.gpu.fnk
          car_license_plate_visibility: ''
          car_make: ''
          car_orientation: carattr/carattr.orientation.v0.gpu.fnk
          car_quality: carattr/carattr.quality.v0.gpu.fnk
          car_special_types: ''
          car_special_types11: carattr/carattr.special_types11.v1.gpu.fnk
          car_trash: ''
          car_weight_types7: carattr/carattr.weight_types7.v0.gpu.fnk
      

      CPU

      extractors:
         ...
         models:
          car_categories: carattr/carattr.categories.v0.cpu.fnk
          car_color: ''
          car_container_number: ''
          car_description: carattr/description.v0.cpu.fnk
          car_emben: carrec/bottas.cpu.fnk
          car_license_plate: carattr/carattr.license_plate.v7.cpu.fnk
          car_license_plate_quality: carattr/carattr.license_plate_quality.v1.cpu.fnk
          car_license_plate_visibility: ''
          car_make: ''
          car_orientation: carattr/carattr.orientation.v0.cpu.fnk
          car_quality: carattr/carattr.quality.v0.cpu.fnk
          car_special_types: ''
          car_special_types11: carattr/carattr.special_types11.v1.cpu.fnk
          car_trash: ''
          car_weight_types7: carattr/carattr.weight_types7.v0.cpu.fnk
      

      The following extractors are available:

      Extractor

      Configure as follows

      vehicle feature vector

      car_emben: carrec/bottas.cpu.fnk

      car_emben: carrec/bottas.gpu.fnk

      license plate number

      car_license_plate: carattr/carattr.license_plate.v7.cpu.fnk car_license_plate_quality: carattr/carattr.license_plate_quality.v1.cpu.fnk

      car_license_plate: carattr/carattr.license_plate.v7.gpu.fnk car_license_plate_quality: carattr/carattr.license_plate_quality.v1.gpu.fnk

      set of attributes: make / color / model / body style

      car_description: carattr/description.v0.cpu.fnk

      car_description: carattr/description.v0.gpu.fnk

      vehicle image quality

      car_quality: carattr/carattr.quality.v0.cpu.fnk

      car_quality: carattr/carattr.quality.v0.gpu.fnk

      special vehicle

      car_special_types: carattr/carattr.special_types11.v1.cpu.fnk

      car_special_types: carattr/carattr.special_types11.v1.gpu.fnk

      vehicle category

      car_categories: carattr/carattr.categories.v0.cpu.fnk

      car_categories: carattr/carattr.categories.v0.gpu.fnk

      vehicle weight and body size

      car_weight_types7: carattr/carattr.weight_types7.v0.cpu.fnk

      car_weight_types7: carattr/carattr.weight_types7.v0.gpu.fnk

      vehicle orientation

      car_orientation: carattr/carattr.orientation.v0.cpu.fnk

      car_orientation: carattr/carattr.orientation.v0.gpu.fnk

      Tip

      To leave a model disabled, pass the empty value '' to the relevant parameter. Do not remove the parameter itself. Otherwise, the system will be searching for the default model.

      extractors:
         ...
         models:
          car_categories: ''
          car_color: ''
          car_container_number: ''
          car_description: ''
          car_emben: ''
          car_license_plate: ''
          car_license_plate_quality: ''
          car_license_plate_visibility: ''
          car_make: ''
          car_orientation: ''
          car_quality: ''
          car_special_types: ''
          car_special_types11: ''
          car_trash: ''
          car_weight_types7: ''
      
    6. Restart the findface-multi-findface-extraction-api-1 container.

      sudo docker container restart findface-multi-findface-extraction-api-1
      
  2. Modify the /opt/findface-multi/configs/findface-video-worker/findface-video-worker.yaml configuration file.

    1. In the models section, specify the vehicle detector, normalizer, and extractor models as shown in the example below:

      GPU

      sudo vi /opt/findface-multi/configs/findface-video-worker/findface-video-worker.yaml
      
      models:
        ...
        detectors:
          ...
          car:
            fnk_path: /usr/share/findface-data/models/detector/car.gustav_normal.004.gpu.fnk
            min_size: 60
          ...
        normalizers:
          ...
          car_norm:
            fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk
          car_norm_quality:
            fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk
          ...
        extractors:
          ...
          car_quality:
            fnk_path: /usr/share/findface-data/models/carattr/carattr.quality.v0.gpu.fnk
            normalizer: car_norm_quality
      

      CPU

      sudo vi /opt/findface-multi/configs/findface-video-worker/findface-video-worker.yaml
      
      
      models:
        ...
        detectors:
          ...
          car:
            fnk_path: /usr/share/findface-data/models/detector/car.gustav_normal.004.cpu.fnk
            min_size: 60
          ...
        normalizers:
          ...
          car_norm:
            fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk
          car_norm_quality:
            fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk
          ...
        extractors:
          ...
          car_quality:
            fnk_path: /usr/share/findface-data/models/carattr/carattr.quality.v0.cpu.fnk
            normalizer: car_norm_quality
      
    2. Make sure that the objects -> car section is included:

      objects:
        ...
        car:
          normalizer: car_norm
          quality: car_quality
          track_features: ''
      
    3. Restart the findface-multi-findface-video-worker-1 container.

      sudo docker container restart findface-multi-findface-video-worker-1
      
  3. Open the /opt/findface-multi/configs/findface-video-manager/findface-video-manager.yaml configuration file and make sure it contains the car section in detectors that looks similar to the example below. Note that the filter_min_quality parameter is set to 0.65 by default. You can increase it to get more accurate results from cameras and other video sources. The best practice is to set this value to 0.73.

    sudo vi /opt/findface-multi/configs/findface-video-manager/findface-video-manager.yaml
    
    detectors:
      ...
      car:
        filter_min_quality: 0.65
        filter_min_size: 100
        filter_max_size: 8192
        roi: ''
        fullframe_crop_rot: false
        fullframe_use_png: false
        jpeg_quality: 95
        overall_only: true
        realtime_post_first_immediately: false
        realtime_post_interval: 1
        realtime_post_every_interval: false
        track_interpolate_bboxes: true
        track_miss_interval: 1
        track_overlap_threshold: 0.25
        track_max_duration_frames: 0
        track_send_history: false
        post_best_track_frame: true
        post_best_track_normalize: true
        post_first_track_frame: false
        post_last_track_frame: false
        tracker_type: simple_iou
        track_deep_sort_matching_threshold: 0.65
        track_deep_sort_filter_unconfirmed_tracks: true
        track_object_is_principal: false
        track_history_active_track_miss_interval: 0
    

    Note

    Configuration of the filter_min_quality parameter affects the results that you get from video sources only and does not affect vehicle counting results and quality of the vehicle images that are used in the records. To get more accurate counting results or vehicle records with better quality images, configure the MINIMUM_CAR_QUALITY parameter individually in the findface-multi-legacy.py configuration file. This setting will be described below.

  4. Enable recognition of vehicles and their attributes in the /opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py configuration file. Do the following:

    1. In the FFSECURITY section, set 'ENABLE_CARS': True.

      sudo vi /opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py
      
      FFSECURITY = {
          ...
          # optional objects to detect
          'ENABLE_CARS': True,
          ...
      
    2. In the same section, you can configure the MINIMUM_CAR_QUALITY parameter. By default, it’s set to 0.65, which is the recommended minimum value for vehicle image quality, sufficient to make a record or use an object in a counter.

      FFSECURITY = {
          ...
          # minimum car quality sufficient to add it to a card and use object in counter
          # p.s. set this parameter equals to `0.73` for more accurate results
          'MINIMUM_CAR_QUALITY': 0.65,  # model: [carattr.quality.v0]
          ...
      

      If you need more accurate counting results or vehicle records with better quality images, you can increase the MINIMUM_CAR_QUALITY threshold. The best practice is to set this value to 0.73.

    3. In the FFSECURITY section, specify the attributes you want to display for vehicle recognition events.

      # available features are: category, description, license_plate, orientation, special_vehicle_type, weight_type
      'CAR_EVENTS_FEATURES': ['license_plate', 'category', 'special_vehicle_type', 'description', 'weight_type', 'orientation'],
      
    4. Restart all FindFace Multi containers.

      cd /opt/findface-multi/
      
      sudo docker-compose restart
      
  5. In the web interface, navigate to Video Source. Select a camera in the Cameras tab (or an uploaded file in the Uploads tab, or an external detector in the corresponding tab). Navigate to the General tab. Select Vehicles in the Detectors section.