Person Recognition and Social Interaction Analysis

FindFace Security allows for automatic person recognition. The system on-the-fly recognizes faces belonging to the same person and clusters them, building a person gallery. You can work with the person gallery on the Persons tab.

It is possible to see a circle of people with whom a person has previously been in contact. For each person from the first circle, the system determines another circle of connected people, and so on. Overall, social interaction analysis is three-circle deep. It is available right from the person gallery and on the tab Relations.

Note

In the person clusterization is enabled, the system databases will hold the person event entity linked to all episodes that feature a person’s face.

Important

By default, person clusterization is disabled. Enable and configure it via the findface-security configuration file.

In this section:

Clusterization Methods

FindFace Security uses the following methods to cluster faces belonging to the same person:

  • Dynamic clusterization. The clusterization takes place on-the-fly after an episode is closed. The result of dynamic clusterization is shown in real-time on the Persons tab.

    Note

    The technical details are as follows. Not every episode is qualified: the number of events in it must be equal or greater than PERSON_EVENT_MIN_EPISODE_EVENTS (set up via the findface-security configuration file). If an episode meets this requirement, the system selects the best quality event and performs the following operations:

    • Creates a new entity PersonEvent in the main system database PostgreSQL. The entity contains the event metadata, a link to the parent episode, face biometric sample, and thumbnail.
    • Searches for a similar face centroid in the person_events gallery of the Tarantool biometric database. A face centroid is a virtual biometric sample averaged across all person’s faces that have been detected so far. If a similar centroid is found, the system updates it using the new event. Otherwise, it creates a new centroid.

  • Scheduled clusterization. We recommend scheduling it on late night hours as it takes up a lot of CPU resources and time.

    Note

    The schedule is defined in the RRULE format as PERSONS_CLUSTERIZATION_SCHEDULE in the findface-security configuration file. The rest of the technical implementation resembles the dynamic method. However, the face centroid quality is better in the scheduled method as centroids are averaged across a larger array of accumulated biometric samples.

    Important

    The scheduled clusterization completely overwrites the person gallery, including ids.

Enable and Configure Person Clusterization

By default, person clusterization is disabled. To enable it, open the findface-security configuration file and modify the SERVICES section as such:

sudo vi /etc/findface-security/config.py

...
SERVICES = {
    "ffsecurity": {
        ...
        "persons": True,
    }
...

You will see the Persons tab appear in the FindFace Security web interface.

In the same configuration file, you can modify the following parameters:

  • PERSON_EVENT_MIN_QUALITY: minimum quality of faces used in person clusterization.

  • PERSON_EVENT_MIN_EPISODE_EVENTS: minimum number of events in episodes used in person clusterization.

  • PERSONS_CONFIDENCE_THRESHOLD: confidence threshold to match a face to a person.

    Warning

    Consult with our experts by support@ntechlab.com before changing this parameter.

  • PERSONS_CLUSTERIZATION_SCHEDULE: recurrence rule (RRULE) for scheduling person clusterization.

    Tip

    See the RRULE calculator here.

# -- Persons configuration --
# rrule (recurrence rule) for scheduling persons clusterization
# WARNING: all scheduling works with UTC time and NOT aware of any timezone
'PERSONS_CLUSTERIZATION_SCHEDULE': 'RRULE:FREQ=DAILY;INTERVAL=1;WKST=MO;BYHOUR=0;BYMINUTE=0',
# face to person matching confidence threshold
'PERSONS_CONFIDENCE_THRESHOLD': 0.739,
# minimum required face quality for person creation
'PERSON_EVENT_MIN_QUALITY': 0.45,
# minimum required number events in episode for person creation
'PERSON_EVENT_MIN_EPISODE_EVENTS': 1,

Social Interaction Analysis

The social interaction analysis is available on the Relations tab.

Tip

You can also display the circle of connected people right from the Persons tab by clicking on the handshake icon.

persons_handshake_button_en

relations_en

On the Relations tab, click on a person to display their first circle of relations. Keep on to unveil the entire tree of social interactions.

circles_en

You can apply available filters to every circle.

Tip

For example, you can find older adults or people without a face mask who are directly or indirectly related to a potentially contagious person.

When searching through a circle of relations, apply the following settings:

  • Use the last event: use the last event of an episode to analyze contacts between individuals. In this case, having found truly associated people is most probable as they simultaneously leave a camera’s field of view. If the option is disabled, the system will use the best event of an episode for relations search.

  • Relations threshold: maximum time in seconds between the appearance of individuals to consider them related.

    circles_settings_en