.. _cars: Enable Vehicle and Vehicle Attribute Recognition ================================================= FindFace Multi allows you to recognize a single vehicle and its attributes. The vehicle attributes are as follows: * license plate number (for selected countries), .. note:: Currently, the following countries are supported: **Europe:** Russia, Lithuania, Latvia, Estonia, Finland, Czech Republic, Serbia, Belarus, Ukraine, Moldova, Georgia, Azerbaijan, Armenia, Türkiye. **Asia:** the UAE, Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Uzbekistan, Saudi Arabia, Vietnam, India, Pakistan, Thailand, China, Japan, Mongolia, Singapore, South Korea, Bahrain, Qatar, Palestine, Iran, Iraq, Jordan, Kuwait, Oman. **Africa:** Egypt, Morocco, Tunisia, South Africa. **North America:** Mexico. **South America:** Argentina, Brazil, Colombia, Peru. * color, * make, * model, * body style, * vehicle category, * vehicle orientation (front, rear, or side), * :ref:`special vehicle type ` (taxi, route transport, carsharing, ambulance, police, rescue service, gas service, military, road service, cash-in-transit vehicles, other special), * vehicle weight and body size. .. note:: Special vehicle recognition as well as vehicle weight and body size recognition work for selected countries only and may not work for your country. For more information please contact your manager or our support team (support@ntechlab.com). Vehicle recognition along with the attributes choice can be configured at the :ref:`installation ` level. The standard deployment of the ``car`` object and its attributes configures the system to recognize by default license plate numbers, not vehicles. It means that a vehicle event will only be created if a license plate number is recognized. This section describes how to enable vehicle and vehicle attribute recognition in case this step has been skipped during installation. Also, you will learn how to configure the ``findface-video-worker`` service to get detection events when either a vehicle or a license plate number is recognized. To enable recognition of vehicles and their attributes, do the following: #. Specify neural network models for vehicle recognition and vehicle attribute recognition in the ``/opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml`` configuration file. .. important:: Be sure to choose the right acceleration type for each model, matching the acceleration type of ``findface-extraction-api``: CPU or GPU. Be aware that ``findface-extraction-api`` on CPU can work only with CPU-models, while ``findface-extraction-api`` on GPU supports both CPU- and GPU-models. #. Open the ``findface-extraction-api.yaml`` configuration file. .. code:: sudo vi /opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml #. Specify the vehicle detector model in the ``detectors → models`` section by pasting the following code: .. rubric:: GPU .. code:: detectors: ... models: ... gustav: aliases: - car - efreitor model: detector/cardet.kali_accurate.008.gpu.fnk options: min_object_size: 32 resolutions: - 2048x2048 ... .. rubric:: CPU .. code:: detectors: ... models: ... gustav: aliases: - car - efreitor model: detector/cardet.kali_accurate.008.cpu.fnk options: min_object_size: 32 resolutions: - 2048x2048 ... #. In the ``objects → car`` section, specify the ``quality_attribute: car_quality`` and the ``base_normalizer: facenorm/cropbbox.v2.gpu.fnk`` or the ``base_normalizer: facenorm/cropbbox.v2.cpu.fnk``, depending on your acceleration type: .. rubric:: GPU .. code:: objects: ... car: base_normalizer: facenorm/cropbbox.v2.gpu.fnk quality_attribute: car_quality ... .. rubric:: CPU .. code:: objects: ... car: base_normalizer: facenorm/cropbbox.v2.cpu.fnk quality_attribute: car_quality ... #. Specify the normalizers required for the extractors. If you need license plate recognition, specify the ``carlicplate`` normalizer. For other extractors, specify the ``cropbbox`` normalizer. +-------------+-------------------------------------------+-------------------------------------------------------------------+ | Normalizer | Normalizer model | Used for extractors | +=============+===========================================+===================================================================+ | carlicplate | ``carnorm/carnorm.briacon.v3.gpu.fnk`` | ``car_license_plate`` | | | ``carnorm/carnorm.briacon.v3.cpu.fnk`` | | +-------------+-------------------------------------------+-------------------------------------------------------------------+ | cropbbox | ``facenorm/cropbbox.v2.gpu.fnk`` | ``car_license_plate_quality``, ``car_description``, | | | ``facenorm/cropbbox.v2.cpu.fnk`` | ``car_quality``, ``car_special_types12``, ``car_categories``, | | | | ``car_orientation``, ``car_weight_types7`` | +-------------+-------------------------------------------+-------------------------------------------------------------------+ .. rubric:: GPU .. code:: normalizers: ... models: ... carlicplate: model: carnorm/carnorm.briacon.v3.gpu.fnk ... cropbbox: model: facenorm/cropbbox.v2.gpu.fnk ... .. rubric:: CPU .. code:: normalizers: ... models: ... carlicplate: model: carnorm/carnorm.briacon.v3.cpu.fnk ... cropbbox: model: facenorm/cropbbox.v2.cpu.fnk ... #. Specify the extraction models in the ``extractors → models`` section, subject to the extractors you want to enable: .. rubric:: GPU .. code:: extractors: ... models: car_categories: default: model: carattr/carattr.categories.v1.gpu.fnk car_color: default: model: '' car_container_number: default: model: '' car_description: default: model: carattr/carattr.description.v1.gpu.fnk car_dumptruck_types: default: model: '' car_emben: default: model: carrec/bottas.gpu.fnk car_license_plate: default: model: carattr/carattr.license_plate.v11.gpu.fnk car_license_plate_quality: default: model: carattr/carattr.license_plate_quality.v4.gpu.fnk car_license_plate_visibility: default: model: '' car_make: default: model: '' car_orientation: default: model: carattr/carattr.orientation.v0.gpu.fnk car_quality: default: model: carattr/carattr.quality.v1.gpu.fnk car_special_types: default: model: '' car_special_types11: default: model: '' car_special_types12: default: model: carattr/carattr.special_types12.v0.gpu.fnk car_trash: default: model: '' car_weight_types7: default: model: carattr/carattr.weight_types7.v0.gpu.fnk .. rubric:: CPU .. code:: extractors: ... models: car_categories: default: model: carattr/carattr.categories.v1.cpu.fnk car_color: default: model: '' car_container_number: default: model: '' car_description: default: model: carattr/carattr.description.v1.cpu.fnk car_dumptruck_types: default: model: '' car_emben: default: model: carrec/bottas.cpu.fnk car_license_plate: default: model: carattr/carattr.license_plate.v11.cpu.fnk car_license_plate_quality: default: model: carattr/carattr.license_plate_quality.v4.cpu.fnk car_license_plate_visibility: default: model: '' car_make: default: model: '' car_orientation: default: model: carattr/carattr.orientation.v0.cpu.fnk car_quality: default: model: carattr/carattr.quality.v1.cpu.fnk car_special_types: default: model: '' car_special_types11: default: model: '' car_special_types12: default: model: carattr/carattr.special_types12.v0.cpu.fnk car_trash: default: model: '' car_weight_types7: default: model: carattr/carattr.weight_types7.v0.cpu.fnk The following extractors are available: +------------------+----------------------------------------------------------------------------------+ | Extractor | Configure as follows | +==================+==================================================================================+ | vehicle feature | ``car_emben: carrec/bottas.cpu.fnk`` | | vector +----------------------------------------------------------------------------------+ | | ``car_emben: carrec/bottas.gpu.fnk`` | +------------------+----------------------------------------------------------------------------------+ | license plate | ``car_license_plate: carattr/carattr.license_plate.v11.cpu.fnk`` | | number | ``car_license_plate_quality: carattr/carattr.license_plate_quality.v4.cpu.fnk`` | | +----------------------------------------------------------------------------------+ | | ``car_license_plate: carattr/carattr.license_plate.v11.gpu.fnk`` | | | ``car_license_plate_quality: carattr/carattr.license_plate_quality.v4.gpu.fnk`` | +------------------+----------------------------------------------------------------------------------+ | set of | ``car_description: carattr/carattr.description.v1.cpu.fnk`` | | attributes: | | | make / color +----------------------------------------------------------------------------------+ | / model / body | ``car_description: carattr/carattr.description.v1.gpu.fnk`` | | style | | +------------------+----------------------------------------------------------------------------------+ | vehicle image | ``car_quality: carattr/carattr.quality.v1.cpu.fnk`` | | quality +----------------------------------------------------------------------------------+ | | ``car_quality: carattr/carattr.quality.v1.gpu.fnk`` | +------------------+----------------------------------------------------------------------------------+ | special vehicle | ``car_special_types: carattr/carattr.special_types12.v0.cpu.fnk`` | | +----------------------------------------------------------------------------------+ | | ``car_special_types: carattr/carattr.special_types12.v0.gpu.fnk`` | +------------------+----------------------------------------------------------------------------------+ | vehicle category | ``car_categories: carattr/carattr.categories.v1.cpu.fnk`` | | +----------------------------------------------------------------------------------+ | | ``car_categories: carattr/carattr.categories.v1.gpu.fnk`` | +------------------+----------------------------------------------------------------------------------+ | vehicle weight | ``car_weight_types7: carattr/carattr.weight_types7.v0.cpu.fnk`` | | and body size +----------------------------------------------------------------------------------+ | | ``car_weight_types7: carattr/carattr.weight_types7.v0.gpu.fnk`` | +------------------+----------------------------------------------------------------------------------+ | vehicle | ``car_orientation: carattr/carattr.orientation.v0.cpu.fnk`` | | orientation +----------------------------------------------------------------------------------+ | | ``car_orientation: carattr/carattr.orientation.v0.gpu.fnk`` | +------------------+----------------------------------------------------------------------------------+ .. note:: The default special vehicle recognition model is ``carattr.special_types12.v0``, specified in the ``car_special_types12`` extractor. It predicts :ref:`12 classes `. However, you may decide to bring a special vehicle recognition model that predicts fewer classes. If you use the ``carattr.special_types11.v1`` model, specify it in the ``car_special_types11`` extractor in the ``/opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml`` file. If you use the ``carattr.special_types.v0`` model, specify it in the ``car_special_types`` extractor in the ``/opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml`` file. In the ``/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py`` file, configure the default value of the ``CAR_SPECIAL_VEHICLE_TYPE_EXTRACTOR`` parameter, which is set to ``car_special_types12``, to specify the enabled special vehicle recognition extractor. E.g., if you enabled the ``carattr.special_types11.v1`` model, specify ``'CAR_SPECIAL_VEHICLE_TYPE_EXTRACTOR': 'car_special_types11'``. Standard FindFace Multi installation pack includes ``carattr.special_types12.v0`` special vehicle recognition model. If you use another special vehicle recognition model, copy it to the ``/opt/findface-multi/models/carattr/`` directory before editing the configuration files. .. important:: The enabled attributes will be recognized by FindFace Multi. The confidence value of the recognized attribute depends on the neural network models used. For more information please contact our support team at support@ntechlab.com. #. Modify the ``/opt/findface-multi/configs/findface-video-worker/findface-video-worker.yaml`` configuration file. Depending on your needs, you may configure the ``findface-video-worker`` service so that detection events are created when either a vehicle or a license plate number is recognized. The example below shows how to configure the ``findface-video-worker`` service to get vehicle detection events only when a license plate number is recognized. If you want to get vehicle detection events based on the recognition of the vehicle itself, specify the ``carattr.quality.v1`` model in the ``extractors`` → ``car_quality`` section in the example below. The rest of the configuration will remain unchanged. #. In the ``models`` section, specify the vehicle detector, normalizer, and extractor models as shown in the example below: .. rubric:: GPU .. code:: sudo vi /opt/findface-multi/configs/findface-video-worker/findface-video-worker.yaml models: ... detectors: ... car: fnk_path: /usr/share/findface-data/models/detector/car.gustav_normal.007.gpu.fnk min_size: 60 ... normalizers: ... car_norm: fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk car_norm_quality: fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.gpu.fnk ... extractors: ... car_quality: fnk_path: /usr/share/findface-data/models/carattr/carattr.license_plate_quality.v4.gpu.fnk normalizer: car_norm_quality .. rubric:: CPU .. code:: sudo vi /opt/findface-multi/configs/findface-video-worker/findface-video-worker.yaml models: ... detectors: ... car: fnk_path: /usr/share/findface-data/models/detector/car.gustav_normal.007.cpu.fnk min_size: 60 ... normalizers: ... car_norm: fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk car_norm_quality: fnk_path: /usr/share/findface-data/models/facenorm/cropbbox.v2.cpu.fnk ... extractors: ... car_quality: fnk_path: /usr/share/findface-data/models/carattr/carattr.license_plate_quality.v4.cpu.fnk normalizer: car_norm_quality #. Add the ``car`` section within ``objects``: .. code:: objects: ... car: normalizer: car_norm quality: car_quality track_features: '' .. important:: If the ``findface-video-worker`` service is configured for license plate recognition, a vehicle detection event may not be created in case the license plate is not recognized. #. Open the ``/opt/findface-multi/configs/findface-video-manager/findface-video-manager.yaml`` configuration file and make sure it contains the ``car`` section in ``detectors`` that looks similar to the example below. Note that the ``filter_min_quality`` parameter is set to ``0.6`` by default. You can increase it to get more accurate results from cameras and other video sources. The best practice is to set this value to ``0.73``. .. code:: sudo vi /opt/findface-multi/configs/findface-video-manager/findface-video-manager.yaml detectors: ... car: filter_min_quality: 0.6 filter_min_size: 100 filter_max_size: 8192 roi: '' fullframe_crop_rot: false fullframe_use_png: false jpeg_quality: 95 overall_only: true realtime_post_first_immediately: false realtime_post_interval: 1 realtime_post_every_interval: false track_interpolate_bboxes: true track_miss_interval: 1 track_overlap_threshold: 0.25 track_max_duration_frames: 0 track_send_history: false post_best_track_frame: true post_best_track_normalize: true post_first_track_frame: false post_last_track_frame: false tracker_type: simple_iou track_deep_sort_matching_threshold: 0.65 track_deep_sort_filter_unconfirmed_tracks: true track_object_is_principal: false track_history_active_track_miss_interval: 0 filter_track_min_duration_frames: 1 tracker_settings: oc_sort: filter_unconfirmed_tracks: true high_quality_detects_threshold: 0.6 momentum_delta_time: 3 smooth_factor: 0.5 time_since_update: 0 extractors_track_triggers: {} .. note:: Configuration of the ``filter_min_quality`` parameter affects the results that you get from video sources only and does not affect vehicle counting results and quality of the vehicle images that are used in the records. To get more accurate counting results or vehicle records with better quality images, configure the ``MINIMUM_CAR_QUALITY`` parameter individually in the ``findface-multi-legacy.py`` configuration file. This setting will be described below. #. Enable recognition of vehicles and their attributes in the ``/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py`` configuration file. Do the following: #. In the ``FFSECURITY`` section, set ``'ENABLE_CARS': True``. .. code:: sudo vi /opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py FFSECURITY = { ... # optional objects to detect 'ENABLE_CARS': True, ... #. In the same section, you can configure the ``MINIMUM_CAR_QUALITY`` parameter. By default, it's set to ``0.6``, which is the recommended minimum value for vehicle image quality, sufficient to make a record or use an object in a counter. .. code:: FFSECURITY = { ... # minimum car quality sufficient to add it to a card and use object in counter # p.s. set this parameter equals to `0.73` for more accurate results 'MINIMUM_CAR_QUALITY': 0.6, # model: [carattr.quality.v1] ... If you need more accurate counting results or vehicle records with better quality images, you can increase the ``MINIMUM_CAR_QUALITY`` threshold. The best practice is to set this value to ``0.73``. #. In the ``FFSECURITY`` section, specify the attributes you want to display for vehicle recognition events. .. code:: # available features are: category, description, license_plate, orientation, special_vehicle_type, weight_type 'CAR_EVENTS_FEATURES': ['license_plate', 'category', 'special_vehicle_type', 'description', 'weight_type', 'orientation'], #. Restart all FindFace Multi containers. .. code:: cd /opt/findface-multi/ sudo docker-compose restart #. In the web interface, navigate to :guilabel:`Video Sources`. Select a camera in the :guilabel:`Cameras` tab (or an uploaded file in the :guilabel:`Uploads` tab, or an external detector in the corresponding tab). Navigate to the :guilabel:`General` tab. Select :guilabel:`Vehicles` in the :guilabel:`Detectors` section.