.. _architecture:
*************************************
Architecture
*************************************
Though you mostly interact with FindFace Multi through its web interface, be sure to take a minute to learn the FindFace Multi architecture. This knowledge is essential for the FindFace Multi deployment, integration, maintenance, and troubleshooting.
.. rubric:: In this chapter:
.. contents::
:local:
Recognition Objects and Recognition Process
==================================================
FindFace Multi can recognize the following objects and their attributes:
* human faces
* human bodies (silhouettes)
* vehicles
FindFace Multi detects an object in the photo or video and prepares its image through normalization. The normalized image is then used for extracting the object's feature vector (an n-dimensional vector of numerical features that represent the object). Object feature vectors are stored in the database and further used for verification and identification purposes.
.. _docker-platform:
Docker Platform
======================
FindFace Multi is deployed in Docker, a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Each FindFace Multi service runs in a Docker container.
Architectural Elements
===============================
FindFace Multi consists of the following fundamental architectural elements:
* FindFace Core, a cutting-edge AI-based recognition technology that can be used as a separate product `FindFace Enterprise Server `_.
* FindFace Multi, which is a turnkey application module for FindFace Enterprise Server.
The FindFace Core internal architecture allows for embedding Video Recorder, an additional functionality that records, stores, and plays back video data from cameras.
Architecture scheme
------------------------
|architecture_en|
.. |architecture_en| image:: /_static/architecture_en.png
.. |architecture_ru| image:: /_static/architecture_ru.png
FindFace Core
-------------------------
The FindFace Core includes the following components:
+---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+
| Component | Ports in use | Description | Vendor |
+===========================+==============+======================================================================================+============================================+
| findface-extraction-api | 18666 | Service that uses neural networks to detect an object in an image and extract its | NtechLab own deployment |
| | | feature vector. It also recognizes object attributes | |
| | | (for example, gender, age, emotions, beard, glasses, face mask - for face objects). | |
| | | CPU- or GPU-acceleration. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-sf-api | 18411 | Service that implements the internal HTTP API for object detection and recognition. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-tarantool-server | 32001, | Service that provides interaction between the ``findface-sf-api`` service and the | |
| | shard ports | feature vector database (the Tarantool-powered database that stores object | |
| | (default | feature vectors. | |
| | 330xx, 81xx) | | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-upload | 3333 | NginX-based web server used as a storage for original images, thumbnails, and | |
| | | normalized object images. If Video Recorder is installed, ``findface-upload`` also | |
| | | stores video data from cameras. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-facerouter | 18820 | Service used to define processing directives for detected objects. In FindFace | |
| | | Multi, its functions are performed by ``findface-multi-legacy`` | |
| | | (see :ref:`application-module`). | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-video-manager | 18810, | Service, part of the video object detection module, that is used for managing the | |
| | 18811 | video object detection functionality, configuring the video object detector | |
| | | settings and specifying the list of to-be-processed video streams. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-video-worker | 18999 | Service, part of the video object detection module, that recognizes an object in | |
| | | the video and posts its normalized image, full frame and metadata (such as | |
| | | detection time) to the ``findface-facerouter`` service for further processing | |
| | | according to given directives. If :ref:`Video Recorder ` is | |
| | | enabled, ``findface-video-worker`` sends video over to ``findface-video-storage``. | |
| | | Provides :ref:`face liveness detection ` if enabled. | |
| | | CPU- or GPU-acceleration. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-ntls | 443 | License server that interfaces with the NtechLab Global License Server, a USB | |
| | (TCP), | dongle, or hardware fingerprint to verify the :ref:`license ` | |
| | 3133, | of your FindFace Multi instance. | |
| | 3185 | | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-counter | 18300 | Service used for event deduplication. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+ |
| findface-liveness-api | 18301 | Besides the embedded functionality provided by ``findface-video-worker``, face | |
| | | liveness detection can also be harnessed as a standalone service | |
| | | ``findface-liveness-api``. The service takes a specific number of frames from a | |
| | | video chunk and returns the best quality face, and decimal liveness result averaged | |
| | | across the taken frames. The service is also involved | |
| | | in the course of user :ref:`authentication ` by face. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+
| Tarantool | Shard ports | Third-party software that implements the feature vector database that stores | `Tarantool `_ |
| | (default | extracted object feature vectors and identification events. The system data, | |
| | 330xx, 81xx) | records, user accounts, and camera settings are stored in PostgreSQL | |
| | | (part of the FindFace Multi application module). | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+
| etcd | 2379 | Third-party software that implements a distributed key-value store for | `etcd `_ |
| | | ``findface-video-manager``. Used as a coordination service in the distributed | |
| | | system, providing the video object detector with fault tolerance. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+
| NginX | 80; | Third-party software that implements the system web interfaces. | `nginx `_ |
| | SSL: 8002, | | |
| | 8003, 443, | | |
| | 80 | | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+
| memcached | 11211 | Third-party software that implements a distributed memory caching system. | `memcached `_ |
| | | Used by ``findface-sf-api`` as a temporary storage for extracted object | |
| | | feature vectors before they are written to the feature vector database powered by | |
| | | Tarantool. | |
+---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+
.. _application-module:
FindFace Multi Application Module
---------------------------------------------
The FindFace Multi application module includes the following components:
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+--------------------------------------------+
| Component | Ports in use | Description | Vendor |
+===================================+==================+=======================================================================================+============================================+
| findface-multi-legacy | Configurable | Service that serves as a gateway to the FindFace Core. | NtechLab own deployment |
| | | Provides interaction between the FindFace Core and the web interface, the system | |
| | | functioning as a whole, HTTP and web socket, object monitoring, event | |
| | | notifications, episodes, webhooks, object clusterization, and counters. | |
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+ |
| findface-multi-pause | n/a | Internal services that assist ``findface-multi-legacy``. The ``findface-multi-audit`` | |
+-----------------------------------+------------------+ service is created for the future. It will become full-fledged in the upcoming | |
| findface-multi-audit | 8012, 8013, 8014 | versions. As for now, it partly duplicates the ``findface-multi-legacy`` | |
+-----------------------------------+------------------+ functionality. Use ``findface-multi-legacy`` to work with it. The | |
| findface-multi-identity-provider | 8022, 8023, 8024 | ``findface-multi-identity-provider`` is the service for authentication, user | |
+-----------------------------------+------------------+ management, and management of role-based model of accesses. | |
| findface-onvif-discovery | n/a | | |
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+ |
| findface-multi-ui | Configurable | Main web interface used to interact with FindFace Multi. | |
| | | Based on the `Django framework `_. | |
| | | Allows you to work with object recognition events, search for | |
| | | objects, manage cameras, users, records, watch lists, collect real-time | |
| | | statistics, and many more. | |
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+--------------------------------------------+
| NATS | 4222 | Third-party software that implements a message broker in ``findface-multi-legacy``. | `NATS `_ |
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+--------------------------------------------+
| etcd | 2379 | Third-party software that implements locks in the ``findface-multi-legacy`` service | `etcd `_ |
| | | e.g. locks in license checker, reports, video processing, object clusterization, etc. | |
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+--------------------------------------------+
| Pgbouncer | 5439 | Third-party software, a lightweight connection pooler for PostgreSQL. Optional, | `PgBouncer `_ |
| | | used to increase the database performance under high load. | |
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+--------------------------------------------+
| PostgreSQL | 5432 | Third-party software that implements the main system database. This database stores | `PostgreSQL `_ |
| | | records of persons and vehicles and data for internal use, including user accounts | |
| | | and camera settings. | |
| | | The object feature vectors and object identification events are stored in Tarantool | |
| | | (part of the FindFace Core). | |
+-----------------------------------+------------------+---------------------------------------------------------------------------------------+--------------------------------------------+
.. _architecture-vms:
Video Recorder
-------------------------------------------------------
Video Recorder is an optional part of the FindFace Core. It includes the following services:
+-----------------------------+--------------+---------------------------------------------------------------------------------------+--------------------------------------------+
| Component | Ports in use | Description | Vendor |
+=============================+==============+=======================================================================================+============================================+
| findface-video-storage | 18611 | Service that implements video chunk management. It takes video chunks from the | NtechLab own deployment |
| | | ``findface-video-worker`` service, puts them into the storage | |
| | | (``findface-upload``), and writes their meta-information and whereabouts to the Video | |
| | | Recorder database (MongoDB). By request from ``findface-multi-legacy``, it issues | |
| | | info about existing video chunks in the form of Websocket-links to the corresponding | |
| | | streams. These links are further used by ``findface-video-streamer-cpu`` to deliver | |
| | | the video to a user for viewing and downloading. | |
+-----------------------------+--------------+---------------------------------------------------------------------------------------+ |
| findface-video-streamer | 9000 | After receiving a ``findface-multi-ui`` request, this service uses Websocket to | |
| | | extract requested video chunks from ``findface-video-storage`` and | |
| | | ``findface-video-worker`` (only the last chunk if it's not yet recorded to the | |
| | | storage). Then it merges the video chunks into a one-piece video and delivers it to a | |
| | | user for viewing and downloading. | |
+-----------------------------+--------------+---------------------------------------------------------------------------------------+--------------------------------------------+
| MongoDB | 27017 | Third-party software that implements the Video Recorder database. The database stores | `MongoDB `_ |
| | | meta-information of the video chunks, including their location. The video chunks | |
| | | themselves are stored in the ``findface-upload`` service. | |
+-----------------------------+--------------+---------------------------------------------------------------------------------------+--------------------------------------------+
.. seealso::
* :ref:`storages`
Single- and Multi-Host Deployment
==================================
You can deploy FindFace Multi on a single host or in a multi-host environment. If you opt for the latter, we offer you one of the following deployment schemes:
* Deploy FindFace Multi on a principal host and distribute additional ``findface-video-worker`` instances across remote hosts.
See :ref:`video-worker-installer`.
* Distribute the FindFace Multi services across multiple hosts. If necessary, set up load balancing. Contact our experts by support@ntechlab.com for guidance.
See :ref:`multi-host-installation`.
CPU- and GPU-acceleration
==========================================
The ``findface-extraction-api`` and ``findface-video-worker`` services can be either CPU- or GPU-based. You will have an opportunity to choose the acceleration type you need during the installation.
.. important::
Refer to :ref:`requirements` when choosing hardware configuration.
.. important::
If the resolution of a camera(s) in use is more than 1280x720px, it is strongly recommended to use the GPU-accelerated package ``findface-video-worker``.
.. note::
The :ref:`liveness detector ` is much slower on CPU than on GPU.