Enable Car and Car Attribute Recognition

FindFace Multi allows you to recognize individual cars and car attributes.

The car attributes are the following:

license plate number (for selected countries),
color,
make,
model,
car body style,
whether a car is a special vehicle (police, ambulance, rescue service, or taxi).

Important

Recognition of individual cars is an experimental feature. Therefore, we highly recommend enabling the additional attribute analysis to improve the recognition quality. In this case, the system compares not only the feature vectors of two cars but also their attributes such as color, body style, make, model, and belonging to a special vehicle type. A conclusion about the cars’ match is only made if both the feature vectors and attributes of the cars coincide.

See the detailed description of how to enable the additional attribute analysis in the step-by-step instructions below.

To enable recognition of cars and their attributes, do the following:

Specify neural network models for individual car recognition and car attribute recognition in the /etc/findface-extraction-api.ini configuration file. Do the following:

Important

Be sure to choose the right acceleration type for each model, matching the acceleration type of findface-extraction-api: CPU or GPU. Be aware that findface-extraction-api on CPU can work only with CPU-models, while findface-extraction-api on GPU supports both CPU- and GPU-models.

Open the /etc/findface-extraction-api.ini configuration file.
```
sudo vi /etc/findface-extraction-api.ini
```

Specify the car detector model in the detectors -> models section by pasting the following code:

GPU

detectors:

  ...
  models:
    ...
    efreitor:
      aliases:
      - car
      model: cadet/efreitor.gpu.fnk
      options:
        min_object_size: 32
        resolutions: [256x256, 384x384, 512x512, 768x768, 1024x1024, 1536x1536, 2048x2048]

  ...

CPU

detectors:

  ...
  models:
    ...
    efreitor:
      aliases:
      - car
      model: cadet/efreitor.cpu.fnk
      options:
        min_object_size: 32
        resolutions: [256x256, 384x384, 512x512, 768x768, 1024x1024, 1536x1536, 2048x2048]

  ...

Specify the extraction models in the extractors -> models section, subject to the extractors you want to enable:

GPU

extractors:
   ...
   models:
    car_color: ''
    car_description: carattr/description.v0.gpu.fnk
    car_emben: carrec/alonso.gpu.fnk
    car_license_plate: carattr/carattr.license_plate.v4.gpu.fnk
    car_license_plate_quality: carattr/carattr.license_plate_quality.v0.gpu.fnk
    car_make: ''
    car_quality: carattr/carattr.quality.v0.gpu.fnk
    car_special_types: carattr/carattr.special_types.v0.gpu.fnk

CPU

extractors:
   ...
   models:
    car_color: ''
    car_description: carattr/description.v0.cpu.fnk
    car_emben: carrec/alonso.cpu.fnk
    car_license_plate: carattr/carattr.license_plate.v4.cpu.fnk
    car_license_plate_quality: carattr/carattr.license_plate_quality.v0.cpu.fnk
    car_make: ''
    car_quality: carattr/carattr.quality.v0.cpu.fnk
    car_special_types: carattr/carattr.special_types.v0.cpu.fnk

The following extractors are available:

Extractor	Configure as follows
individual car object	`car_emben: carrec/alonso.cpu.fnk`
individual car object	`car_emben: carrec/alonso.gpu.fnk`
license plate number	`car_license_plate: carattr/carattr.license_plate.v4.cpu.fnk` `car_license_plate_quality: carattr/carattr.license_plate_quality.v0.cpu.fnk`
license plate number	`car_license_plate: carattr/carattr.license_plate.v4.gpu.fnk` `car_license_plate_quality: carattr/carattr.license_plate_quality.v0.gpu.fnk`
set of attributes: make / color / model / body style	`car_description: carattr/description.v0.cpu.fnk`
set of attributes: make / color / model / body style	`car_description: carattr/description.v0.gpu.fnk`
car image quality	`car_quality: carattr/carattr.quality.v0.cpu.fnk`
car image quality	`car_quality: carattr/carattr.quality.v0.gpu.fnk`
special vehicle	`car_special_types: carattr/carattr.special_types.v0.cpu.fnk`
special vehicle	`car_special_types: carattr/carattr.special_types.v0.gpu.fnk`

Tip

To leave a model disabled, pass the empty value '' to the relevant parameter. Do not remove the parameter itself. Otherwise, the system will be searching for the default model.

extractors:
   ...
   models:
    car_color: ""
    car_description: ""
    car_emben: ""
    car_license_plate: ""
    car_license_plate_quality: ""
    car_make: ""
    car_quality: ""
    car_special_types: ""

Specify the normalizers required for the extractors specified in the previous step. For example, if you need license plate recognition, specify the carlicplate normalizer.

Normalizer	Normalizer model	Used for extractors
carlicplate	`carnorm/anaferon.v3.gpu.fnk` `carnorm/anaferon.v3.cpu.fnk`	`car_license_plate`
cropbbox	`facenorm/cropbbox.v2.gpu.fnk` `facenorm/cropbbox.v2.cpu.fnk`	`car_license_plate_quality`, `car_description`, `car_quality`, `car_special_types`

GPU

normalizers:
  ...

  models:
    carlicplate:
      model: carnorm/anaferon.v3.gpu.fnk
    ...
    cropbbox:
      model: facenorm/cropbbox.v2.gpu.fnk

CPU

normalizers:
  ...

  models:
    carlicplate:
      model: carnorm/anaferon.v3.cpu.fnk
    ...
    cropbbox:
      model: facenorm/cropbbox.v2.cpu.fnk

Make sure that the objects -> car section contains the quality_attribute: car_quality:

GPU

objects:
  ...
  car:
    base_normalizer: facenorm/cropbbox.v2.gpu.fnk
    quality_attribute: car_quality
  ...

CPU

objects:
  ...
  car:
    base_normalizer: facenorm/cropbbox.v2.cpu.fnk
    quality_attribute: car_quality

Restart findface-extraction-api.

sudo systemctl restart findface-extraction-api

Modify the /etc/findface-video-worker-gpu.ini (/etc/findface-video-worker-cpu.ini) configuration file. In the car section, specify the neural network models by analogy with the example below. Restart findface-video-worker-gpu (findface-video-worker-cpu).

GPU

sudo vi /etc/findface-video-worker-gpu.ini


#------------------------------
[car]
#------------------------------
## detector param
## type:number env:CFG_CAR_MIN_SIZE longopt:--car-min-size
min_size = 60

## path to car detector
## type:string env:CFG_CAR_DETECTOR longopt:--car-detector
detector = /usr/share/findface-data/models/cadet/efreitor.gpu.fnk

## path to normalizer (usually crop2x)
## type:string env:CFG_CAR_NORM longopt:--car-norm
norm = /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk

## path to car quality extractor
## type:string env:CFG_CAR_QUALITY longopt:--car-quality
quality = /usr/share/findface-data/models/carattr/carattr.quality.v0.gpu.fnk

## path to car quality normalizer
## type:string env:CFG_CAR_NORM_QUALITY longopt:--car-norm-quality
norm_quality = /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk

## path to car track features extractor
## type:string env:CFG_CAR_TRACK_FEATURES longopt:--car-track-features
track_features =

## path to car track features normalizer
## type:string env:CFG_CAR_TRACK_FEATURES_NORM longopt:--car-track-features-norm
track_features_norm =

sudo systemctl restart findface-video-worker-gpu.service

CPU

sudo vi /etc/findface-video-worker-cpu.ini


#------------------------------
[car]
#------------------------------
## detector param
## type:number env:CFG_CAR_MIN_SIZE longopt:--car-min-size
min_size = 60

## path to car detector
## type:string env:CFG_CAR_DETECTOR longopt:--car-detector
detector = /usr/share/findface-data/models/cadet/efreitor.cpu.fnk

## path to normalizer (usually crop2x)
## type:string env:CFG_CAR_NORM longopt:--car-norm
norm = /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk

## path to car quality extractor
## type:string env:CFG_CAR_QUALITY longopt:--car-quality
quality = /usr/share/findface-data/models/carattr/carattr.quality.v0.cpu.fnk

## path to car quality normalizer
## type:string env:CFG_CAR_NORM_QUALITY longopt:--car-norm-quality
norm_quality = /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk

sudo systemctl restart findface-video-worker-cpu.service

Open the /etc/findface-video-manager.conf configuration file and make sure it contains the car section in detectors that looks similar to the example below.

Tip

As a reference value for the filter_min_quality parameter, you can take the MINIMUM_CAR_QUALITY parameter value from the /etc/findface-security/config.py configuration file.

sudo vi /etc/findface-video-manager.conf

detectors:
  ...
  car:
    filter_min_quality: 0.65
    filter_min_size: 1
    filter_max_size: 8192
    roi: ""
    fullframe_crop_rot: false
    fullframe_use_png: false
    jpeg_quality: 95
    overall_only: false
    realtime_post_first_immediately: false
    realtime_post_interval: 1
    realtime_post_every_interval: false
    track_interpolate_bboxes: true
    track_miss_interval: 1
    track_overlap_threshold: 0.25
    track_max_duration_frames: 0
    track_send_history: false
    post_best_track_frame: true
    post_best_track_normalize: true
    post_first_track_frame: false
    post_last_track_frame: false
    tracker_type: simple_iou
    track_deep_sort_matching_threshold: 0.65
    track_deep_sort_filter_unconfirmed_tracks: true

Enable the recognition of cars and car attributes in the /etc/findface-security/config.py configuration file. Do the following:
1. In the FFSECURITY section, set 'ENABLE_CARS': True.
```
sudo vi /etc/findface-security/config.py

FFSECURITY = {
    ...

    # optional objects to detect
    'ENABLE_CARS': True,
    ...
```
2. In the same section, specify the car attributes you want to display for the car recognition events.
```
# available features are: description, license_plate, special_vehicle_type
'CAR_EVENTS_FEATURES': ['description', 'license_plate', 'special_vehicle_type'],
```
3. To improve the quality of individual car recognition, we highly recommend you enable the additional attribute analysis. In this case, the system compares not only the feature vectors of two cars but also their attributes. A conclusion about the cars’ match is only made if both the feature vectors and attributes of the cars coincide.
  
  You can use the following attributes for additional analysis:
  - color: car color,
  - body: body style,
  - make: make,
  - model: model,
  - special_vehicle_type: belonging to a special vehicle type.
  To enable the additional attribute analysis, set True in the FFSECURITY -> EXTRA_CAR_MATCHING section for the attributes that you want to compare.
```
FFSECURITY = {
    # use additional features for extra confidence when matching cars by emben
    'EXTRA_CAR_MATCHING': {
        'color': {'enabled': False, 'min_confidence': 0},
        'body': {'enabled': False, 'min_confidence': 0},
        'make': {'enabled': False, 'min_confidence': 0},
        'model': {'enabled': False, 'min_confidence': 0},
        'special_vehicle_type': {'enabled': False, 'min_confidence': 0}
    },
```
  Important
  
  For the attribute analysis to function, the description model must be enabled in the /etc/findface-extraction-api.ini and /etc/findface-security/config.py configuration files (see above).
  
  Note
  
  Enabling the additional attribute analysis reduces the number of false positives. However, the system might miss out on some real matches as well.
  
  Warning
  
  Do not change the default values of min_confidence without consulting with our technical experts (support@ntechlab.com).
4. Restart the findface-security service.
```
sudo systemctl restart findface-security.service
```