.. _persons: ******************************************************** Person Recognition and Social Interaction Analysis ******************************************************** FindFace Security allows for automatic person recognition. The system on-the-fly recognizes faces belonging to the same person and clusters them, building a person gallery. You can work with the person gallery on the :guilabel:`Persons` tab. It is possible to see a circle of people with whom a person has previously been in contact. For each person from the first circle, the system determines another circle of connected people, and so on. Overall, social interaction analysis is three-circle deep. It is available right from the person gallery and on the tab :guilabel:`Relations`. .. note:: In the person clusterization is enabled, the system databases will hold the ``person event`` entity linked to all :ref:`episodes ` that feature a person's face. .. important:: By default, person clusterization is disabled. :ref:`Enable and configure it ` via the ``findface-security`` configuration file. .. rubric:: In this section: .. contents:: :local: Clusterization Methods ============================ FindFace Security uses the following methods to cluster faces belonging to the same person: * Dynamic clusterization. The clusterization takes place on-the-fly after an episode is closed. The result of dynamic clusterization is shown in real-time on the :guilabel:`Persons` tab. .. note:: The technical details are as follows. Not every episode is qualified: the number of events in it must be equal or greater than ``PERSON_EVENT_MIN_EPISODE_EVENTS`` (set up via the ``findface-security`` configuration file). If an episode meets this requirement, the system selects the best quality event and performs the following operations: * Creates a new entity ``PersonEvent`` in the main system database :program:`PostgreSQL`. The entity contains the event metadata, a link to the parent episode, face biometric sample, and thumbnail. * Searches for a similar face centroid in the ``person_events`` gallery of the :program:`Tarantool` biometric database. A face centroid is a virtual biometric sample averaged across all person's faces that have been detected so far. If a similar centroid is found, the system updates it using the new event. Otherwise, it creates a new centroid. * Scheduled clusterization. We recommend scheduling it on late night hours as it takes up a lot of CPU resources and time. .. note:: The schedule is defined in the RRULE format as ``PERSONS_CLUSTERIZATION_SCHEDULE`` in the ``findface-security`` configuration file. The rest of the technical implementation resembles the dynamic method. However, the face centroid quality is better in the scheduled method as centroids are averaged across a larger array of accumulated biometric samples. .. important:: The scheduled clusterization completely overwrites the person gallery, including ids. .. _enable-clusterization: Enable and Configure Person Clusterization ================================================== By default, person clusterization is disabled. To enable it, open the ``findface-security`` configuration file and modify the ``SERVICES`` section as such: .. code:: sudo vi /etc/findface-security/config.py ... SERVICES = { "ffsecurity": { ... "persons": True, } ... You will see the :guilabel:`Persons` tab appear in the FindFace Security web interface. In the same configuration file, you can modify the following parameters: * ``PERSON_EVENT_MIN_QUALITY``: minimum quality of faces used in person clusterization. * ``PERSON_EVENT_MIN_EPISODE_EVENTS``: minimum number of events in episodes used in person clusterization. * ``PERSONS_CONFIDENCE_THRESHOLD``: confidence threshold to match a face to a person. .. warning:: Consult with our experts by support@ntechlab.com before changing this parameter. * ``PERSONS_CLUSTERIZATION_SCHEDULE``: recurrence rule (RRULE) for scheduling person clusterization. .. tip:: See the RRULE calculator `here `_. .. code:: # -- Persons configuration -- # rrule (recurrence rule) for scheduling persons clusterization # WARNING: all scheduling works with UTC time and NOT aware of any timezone 'PERSONS_CLUSTERIZATION_SCHEDULE': 'RRULE:FREQ=DAILY;INTERVAL=1;WKST=MO;BYHOUR=0;BYMINUTE=0', # face to person matching confidence threshold 'PERSONS_CONFIDENCE_THRESHOLD': 0.739, # minimum required face quality for person creation 'PERSON_EVENT_MIN_QUALITY': 0.45, # minimum required number events in episode for person creation 'PERSON_EVENT_MIN_EPISODE_EVENTS': 1, Work with Person Gallery ================================= To see the person gallery, navigate to the :guilabel:`Persons` tab. |person_gallery_en| .. |person_gallery_en| image:: /_static/person_gallery_en.png :scale: 45% .. |person_gallery_ru| image:: /_static/person_gallery_ru.png :scale: 45% To work with the person gallery, use the following filters: * Dossier * Matches * Cameras * Camera groups * Watch lists * Time period * Person id * Face features (if enabled) * Liveness (if enabled) .. seealso:: * :ref:`Configuration file of findface-security ` * :ref:`webhooks` * :ref:`analytics` Social Interaction Analysis ================================= The social interaction analysis is available on the :guilabel:`Relations` tab. .. tip:: You can also display the circle of connected people right from the :guilabel:`Persons` tab by clicking on the handshake icon. |persons_handshake_button_en| .. |persons_handshake_button_en| image:: /_static/persons_handshake_button_en.png :scale: 60% .. |persons_handshake_button_ru| image:: /_static/persons_handshake_button_ru.png :scale: 60% |relations_en| .. |relations_en| image:: /_static/relations_en.png :scale: 50% .. |relations_ru| image:: /_static/relations_ru.png :scale: 50% On the :guilabel:`Relations` tab, click on a person to display their first circle of relations. Keep on to unveil the entire tree of social interactions. |circles_en| .. |circles_en| image:: /_static/circles_en.png :scale: 50% .. |circles_ru| image:: /_static/circles_ru.png :scale: 50% You can apply available filters to every circle. .. tip:: For example, you can find older adults or people without a face mask who are directly or indirectly related to a potentially contagious person. When searching through a circle of relations, apply the following settings: * :guilabel:`Use the last event`: use the last event of an episode to analyze contacts between individuals. In this case, having found truly associated people is most probable as they simultaneously leave a camera’s field of view. If the option is disabled, the system will use the best event of an episode for relations search. * :guilabel:`Relations threshold`: maximum time in seconds between the appearance of individuals to consider them related. |circles_settings_en| .. |circles_settings_en| image:: /_static/circles_settings_en.png :scale: 60% .. |circles_settings_ru| image:: /_static/circles_settings_ru.png :scale: 60%