Person Recognition

FindFace Multi allows for automatic person recognition. The system on the fly recognizes faces belonging to the same person and clusters them, creating a person gallery. You can work with the person gallery on the Persons tab.

Important

By default, the person clusterization is disabled. Enable and configure it via the /etc/findface-security/config.py configuration file.

In this section:

Clusterization Methods

FindFace Multi uses the following methods to cluster faces belonging to the same person:

  • Dynamic clusterization. The clusterization takes place on the fly after an episode is closed. The result of dynamic clusterization is shown in real-time on the Persons tab.

    Note

    The technical details are as follows. Not every episode is qualified: the number of events in it must be equal or greater than PERSON_EVENT_MIN_EPISODE_EVENTS (set up via the /etc/findface-security/config.py configuration file). If an episode meets this requirement, the system selects the best quality event and performs the following operations:

    • Creates a new entity person event in the main system database PostgreSQL. The entity contains the event metadata, a link to the parent episode, face feature vector, and thumbnail.

    • Searches for a similar face centroid in the person_events gallery of the Tarantool feature vector database. A face centroid is a virtual feature vector averaged across all person’s faces that have been detected so far. If a similar centroid is found, the system updates it using the new event. Otherwise, it creates a new centroid.

  • Scheduled clusterization. We recommend scheduling it on late night hours as it takes up a lot of CPU resources and time.

    Note

    The schedule is defined in the RRULE format as PERSONS_CLUSTERIZATION_SCHEDULE in the /etc/findface-security/config.py configuration file. The rest of the technical implementation resembles the dynamic method. However, the face centroid quality is better in the scheduled method as centroids are averaged across a larger array of accumulated feature vectors.

    Important

    The scheduled clusterization completely overwrites the person gallery, including ids, unless the PIN_MATCHED_PERSONS parameter is enabled (see description below).

Enable and Configure Person Clusterization

By default, the person clusterization is disabled. To enable and configure it, do the following:

  1. Open the /etc/findface-security/config.py configuration file.

    sudo vi /etc/findface-security/config.py

  2. Enable the dynamic clusterization by setting ENABLE_PERSONS_REALTIME_CLUSTERIZATION: True.

  3. If necessary, enable the scheduled clusterization by setting ENABLE_PERSONS_NIGHT_CLUSTERIZATION: True.

    Important

    Enabling the scheduled clusterization only makes sense if the dynamic clusterization is enabled. Otherwise, the system won’t generate new persons, as only the dynamic clusterization can supply it with unique person events.

    ...
# -- Persons configuration --
'ENABLE_PERSONS_NIGHT_CLUSTERIZATION': True,
'ENABLE_PERSONS_REALTIME_CLUSTERIZATION': True,
...

  4. If necessary, configure the person clusterization with the following parameters:

    • PERSONS_CLUSTERIZATION_SCHEDULE: recurrence rule (RRULE) for the scheduled person clusterization. If the RRULE is not specified, the clusterization automatically starts at 00:00 GMT.

      Tip

      See the RRULE calculator here.

    • PERSONS_CONFIDENCE_THRESHOLD: confidence threshold to match a face to a person.

      Warning

      Consult with our experts by support@ntechlab.com before changing this parameter.

    • PERSON_EVENT_MIN_QUALITY: minimum quality of faces used in the person clusterization.

    • PERSON_EVENT_MIN_EPISODE_EVENTS: minimum number of events in episodes used in the person clusterization.

    • PIN_MATCHED_PERSONS: by default, the scheduled clusterization overwrites the entire person gallery. Enable this parameter to keep the matched persons and the associated person events intact during this process.

      Note

      The dynamic clusterization will still be creating new person events for the matched persons.

    # rrule (recurrence rule) for scheduling persons clusterization
# night clusterizer only works when 'ENABLE_PERSONS_REALTIME_CLUSTERIZATION': True
# WARNING: all scheduling works with UTC time and NOT aware of any timezone
'PERSONS_CLUSTERIZATION_SCHEDULE': 'RRULE:FREQ=DAILY;INTERVAL=1;WKST=MO;BYHOUR=0;BYMINUTE=0',

# face to person matching confidence threshold
'PERSONS_CONFIDENCE_THRESHOLD': 0.723,  # model: [kiwi_320]

# minimum required face quality for person creation
'PERSON_EVENT_MIN_QUALITY': 0.45,  # model: [quality.v1]
# minimum required number events in episode for person creation
'PERSON_EVENT_MIN_EPISODE_EVENTS': 1,
# coefficient of dependence of the clustering threshold on the person event's quality
'PERSONS_SOFT_CLUSTERIZATION_COEFFICIENT': 0.1,
# pin persons with matched events
'PIN_MATCHED_PERSONS': False,

  5. Restart the findface-security service. You will see the Persons tab appear in the FindFace Multi web interface.

    sudo systemctl restart findface-security.service