Enable Car and Car Attribute Recognition

FindFace Multi allows you to recognize individual cars and car attributes.

The car attributes are the following:

  • license plate number (for selected countries),

  • color,

  • make,

  • model,

  • car body style.

Important

Recognition of individual cars is an experimental feature. Therefore, we highly recommend you enable the additional attribute analysis to improve the recognition quality. In this case, the system compares not only the feature vectors of two cars but also their attributes such as color, body style, make, and model. A conclusion about the cars’ match is only made if both the feature vectors and attributes of the cars coincide.

See the detailed description of how to enable the additional attribute analysis in the step-by-step instructions below.

To enable recognition of cars and their attributes, do the following:

  1. Specify neural network models for car object and car attribute recognition in the /etc/findface-extraction-api.ini configuration file. Do the following:

    Important

    Be sure to choose the right acceleration type for each model, matching the acceleration type of findface-extraction-api: CPU or GPU. Be aware that findface-extraction-api on CPU can work only with CPU-models, while findface-extraction-api on GPU supports both CPU- and GPU-models.

    1. Open the /etc/findface-extraction-api.ini configuration file.

      sudo vi /etc/findface-extraction-api.ini
      
    2. Specify the car detector model in the detectors -> models section by pasting the following code:

      GPU

      detectors:
      
        ...
        models:
          ...
          efreitor:
            aliases:
            - car
            model: cadet/efreitor.gpu.fnk
            options:
              min_object_size: 32
              resolutions: [256x256, 384x384, 512x512, 768x768, 1024x1024, 1536x1536, 2048x2048]
      
        ...
      

      CPU

      detectors:
      
        ...
        models:
          ...
          efreitor:
            aliases:
            - car
            model: cadet/efreitor.cpu.fnk
            options:
              min_object_size: 32
              resolutions: [256x256, 384x384, 512x512, 768x768, 1024x1024, 1536x1536, 2048x2048]
      
        ...
      
    3. Specify the extraction models in the extractors -> models section, subject to the extractors you want to enable:

      GPU

      extractors:
         ...
         models:
          car_color: ''
          car_description: carattr/description.v0.gpu.fnk
          car_emben: carrec/alonso.gpu.fnk
          car_license_plate: carattr/carattr.license_plate.v2.gpu.fnk
          car_license_plate_quality: carattr/carattr.license_plate_quality.v0.gpu.fnk
          car_make: ''
          car_quality: carattr/carattr.quality.v0.gpu.fnk
      

      CPU

      extractors:
         ...
         models:
          car_color: ''
          car_description: carattr/description.v0.cpu.fnk
          car_emben: carrec/alonso.cpu.fnk
          car_license_plate: carattr/carattr.license_plate.v2.cpu.fnk
          car_license_plate_quality: carattr/carattr.license_plate_quality.v0.cpu.fnk
          car_make: ''
          car_quality: carattr/carattr.quality.v0.cpu.fnk
      

      The following extractors are available:

      Extractor

      Configure as follows

      individual car object

      car_emben: carrec/alonso.cpu.fnk

      car_emben: carrec/alonso.gpu.fnk

      license plate number

      car_license_plate: carattr/carattr.license_plate.v2.cpu.fnk car_license_plate_quality: carattr/carattr.license_plate_quality.v0.cpu.fnk

      car_license_plate: carattr/carattr.license_plate.v2.gpu.fnk car_license_plate_quality: carattr/carattr.license_plate_quality.v0.gpu.fnk

      set of attributes: make / color / model / body style

      car_description: carattr/description.v0.cpu.fnk

      car_description: carattr/description.v0.gpu.fnk

      car image quality

      car_quality: carattr/carattr.quality.v0.cpu.fnk

      car_quality: carattr/carattr.quality.v0.gpu.fnk

      Tip

      To leave a model disabled, pass the empty value '' to the relevant parameter. Do not remove the parameter itself. Otherwise, the system will be searching for the default model.

      extractors:
         ...
         models:
          car_color: ''
          car_description: ''
          car_emben: ''
          car_license_plate: ''
          car_license_plate_quality: ''
          car_make: ''
          car_quality: ''
          car_trash: ''
      
    4. Specify the normalizers required for the extractors specified in the previous step. For example, if you need license plate recognition, specify the carlicplate normalizer.

      Normalizer

      Normalizer model

      Used for extractors

      carlicplate

      carnorm/anaferon.v1.gpu.fnk carnorm/anaferon.v1.cpu.fnk

      car_license_plate

      cropbbox

      facenorm/cropbbox.v2.gpu.fnk facenorm/cropbbox.v2.cpu.fnk

      car_license_plate_quality, car_description, car_quality

      GPU

      normalizers:
        ...
      
        models:
          carlicplate:
            model: carnorm/anaferon.v1.gpu.fnk
          ...
          cropbbox:
            model: facenorm/cropbbox.v2.gpu.fnk
      

      CPU

      normalizers:
        ...
      
        models:
          carlicplate:
            model: carnorm/anaferon.v1.cpu.fnk
          ...
          cropbbox:
            model: facenorm/cropbbox.v2.cpu.fnk
      
    5. Make sure that the objects -> car section contains the quality_attribute: car_quality:

      GPU

      objects:
        ...
        car:
          base_normalizer: facenorm/cropbbox.v2.gpu.fnk
          quality_attribute: car_quality
        ...
      

      CPU

      objects:
        ...
        car:
          base_normalizer: facenorm/cropbbox.v2.cpu.fnk
          quality_attribute: car_quality
      
    6. After completing the configuration file modification, make sure it looks similar to the following example.

    7. Restart findface-extraction-api.

      sudo systemctl restart findface-extraction-api
      
  2. Modify the /etc/findface-video-worker-gpu.ini (/etc/findface-video-worker-cpu.ini) configuration file. In the car section, specify the neural network models by analogy with the example below. Restart findface-video-worker-gpu (findface-video-worker-cpu).

    GPU

    sudo vi /etc/findface-video-worker-gpu.ini
    
    
    #------------------------------
    [car]
    #------------------------------
    ## detector param
    ## type:number env:CFG_CAR_MIN_SIZE longopt:--car-min-size
    min_size = 60
    
    ## path to car detector
    ## type:string env:CFG_CAR_DETECTOR longopt:--car-detector
    detector = /usr/share/findface-data/models/cadet/efreitor.gpu.fnk
    
    ## path to normalizer (usually crop2x)
    ## type:string env:CFG_CAR_NORM longopt:--car-norm
    norm = /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk
    
    ## path to car quality extractor
    ## type:string env:CFG_CAR_QUALITY longopt:--car-quality
    quality = /usr/share/findface-data/models/carattr/carattr.quality.v0.gpu.fnk
    
    ## path to car quality normalizer
    ## type:string env:CFG_CAR_NORM_QUALITY longopt:--car-norm-quality
    norm_quality = /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk
    
    ## path to car track features extractor
    ## type:string env:CFG_CAR_TRACK_FEATURES longopt:--car-track-features
    track_features =
    
    ## path to car track features normalizer
    ## type:string env:CFG_CAR_TRACK_FEATURES_NORM longopt:--car-track-features-norm
    track_features_norm =
    
    sudo systemctl restart findface-video-worker-gpu.service
    

    CPU

    sudo vi /etc/findface-video-worker-cpu.ini
    
    
    #------------------------------
    [car]
    #------------------------------
    ## detector param
    ## type:number env:CFG_CAR_MIN_SIZE longopt:--car-min-size
    min_size = 60
    
    ## path to car detector
    ## type:string env:CFG_CAR_DETECTOR longopt:--car-detector
    detector = /usr/share/findface-data/models/cadet/efreitor.cpu.fnk
    
    ## path to normalizer (usually crop2x)
    ## type:string env:CFG_CAR_NORM longopt:--car-norm
    norm = /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk
    
    ## path to car quality extractor
    ## type:string env:CFG_CAR_QUALITY longopt:--car-quality
    quality = /usr/share/findface-data/models/carattr/carattr.quality.v0.cpu.fnk
    
    ## path to car quality normalizer
    ## type:string env:CFG_CAR_NORM_QUALITY longopt:--car-norm-quality
    norm_quality = /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk
    
    sudo systemctl restart findface-video-worker-cpu.service
    
  3. Open the /etc/findface-video-manager.conf configuration file and make sure it contains the car section in detectors that looks similar to the example below.

    Tip

    As a reference value for the filter_min_quality parameter, you can take the MINIMUM_CAR_QUALITY parameter value from the /etc/findface-security/config.py configuration file.

    sudo vi /etc/findface-video-manager.conf
    
    detectors:
      ...
      car:
        filter_min_quality: 0.65
        filter_min_size: 1
        filter_max_size: 8192
        roi: ""
        fullframe_crop_rot: false
        fullframe_use_png: false
        jpeg_quality: 95
        overall_only: false
        realtime_post_first_immediately: false
        realtime_post_interval: 1
        realtime_post_every_interval: false
        track_interpolate_bboxes: true
        track_miss_interval: 1
        track_overlap_threshold: 0.25
        track_max_duration_frames: 0
        track_send_history: false
        post_best_track_frame: true
        post_best_track_normalize: true
        post_first_track_frame: false
        post_last_track_frame: false
        tracker_type: simple_iou
        track_deep_sort_matching_threshold: 0.65
        track_deep_sort_filter_unconfirmed_tracks: true
    
  4. Enable the recognition of cars and car attributes in the /etc/findface-security/config.py configuration file. Do the following:

    1. In the FFSECURITY section, set 'ENABLE_CARS': True.

      sudo vi /etc/findface-security/config.py
      
      FFSECURITY = {
          ...
      
          # optional objects to detect
          'ENABLE_CARS': True,
          ...
      
    2. In the same section, specify the car attributes you want to display for the car recognition events.

      # available features are: description, license_plate
      'CAR_EVENTS_FEATURES': ['description', 'license_plate'],
      
    3. To improve the quality of individual car recognition, we highly recommend you enable the additional attribute analysis. In this case, the system compares not only the feature vectors of two cars but also their attributes. A conclusion about the cars’ match is only made if both the feature vectors and attributes of the cars coincide.

      You can use the following attributes for additional analysis:

      • color: car color,

      • body: body style,

      • make: make,

      • model: model.

      To enable the additional attribute analysis, set True in the FFSECURITY -> EXTRA_CAR_MATCHING section for the attributes that you want to compare.

      FFSECURITY = {
          # use additional features for extra confidence when matching cars by emben
          'EXTRA_CAR_MATCHING': {
              'color': {'enabled': False, 'min_confidence': 0},
              'body': {'enabled': False, 'min_confidence': 0},
              'make': {'enabled': False, 'min_confidence': 0},
              'model': {'enabled': False, 'min_confidence': 0},
          },
      

      Important

      For the attribute analysis to function, the description model must be enabled in the /etc/findface-extraction-api.ini and /etc/findface-security/config.py configuration files (see above).

      Note

      Enabling the additional attribute analysis reduces the number of false positives. However, the system might miss out on some real matches as well.

      Warning

      Do not change the default values of min_confidence without consulting with our technical experts (support@ntechlab.com).

    4. Restart the findface-security service.

      sudo systemctl restart findface-security.service