.. _architecture: ************************************* Architecture ************************************* Please, take a minute to learn the FindFace Server architecture. This knowledge is essential for the FindFace Server deployment, integration, maintenance, and troubleshooting. .. rubric:: In this chapter: .. contents:: :local: Recognition Objects and Recognition Process ================================================== FindFace Server can recognize a lot of objects and their attributes: * human faces * human bodies (silhouettes) * vehicles * snow pile * streetlight * graffiti * pet faces * shopping cart * visible gas * fire * bear * container * garbage * trash bin * and other FindFace Server detects an object in the photo or video and prepares its image through normalization. The normalized image is then used for extracting the object's feature vector (an n-dimensional vector of numerical features that represent the object). Object feature vectors are stored in the database and further used for verification and identification purposes. .. _docker-platform: Docker Platform ====================== FindFace Server is delivered as a set of packages (container images), that are expected to be deployed using an OS-level virtualization technology commonly referred as "containers". Each FindFace Server service runs in a separate container. This manual uses Docker in all of its examples, but any OCI-compatible runtime may be used. Architectural Elements =============================== FindFace Server is a set of services for video analytics. You can implement the business logic and UI through application modules. .. note:: Application modules are not available in the basic configuration. To learn more about building a turnkey application with the help of our team, contact our experts by info@ntechlab.com. FindFace Server implements a :ref:`Video Recorder `, an additional functionality that records, stores, and plays back video data from cameras. Architecture scheme ------------------------ |ffserver-components| .. |ffserver-components| image:: /_static/ffserver-components.svg :scale: 70% FindFace Server Components -------------------------------------- FindFace Server includes the following components: +---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+ | Component | Ports in use | Description | Vendor | +===========================+==============+======================================================================================+============================================+ | extraction-api | 18666 | Service that uses neural networks to detect an object in an image and extract its | NtechLab own deployment | | | | feature vector. It also recognizes object attributes | | | | | (for example, gender, age, emotions, beard, glasses, face mask - for face objects). | | | | | CPU- or GPU-acceleration. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | sf-api | 18411 | Service that implements a unified user-friendly HTTP API | | | | | for the ``extraction-api`` and ``tntapi``. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | tntapi | 8001 (API), | Feature vector and metadata storage built on top of | | | | 32001 | `Tarantool in-memory database `_. | | | | (replication)| | | | | | | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | upload | 3333 | ``nginx`` -based web server used as a storage for | | | | | normalized object images. If :ref:`Video Recorder ` is enabled, | | | | | ``upload`` can be used to store video data from cameras. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | facerouter | 18820 | Service used to define processing directives for detected objects. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | video-manager | 18810 (API), | Service, part of the video object detection module, that is used for managing the | | | | 18811 | video object detection functionality, configuring the video object detector | | | | (worker) | settings and specifying the list of to-be-processed video streams. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | video-worker | 18999 | Service, part of the video object detection module, that recognizes an object in | | | | | the video and posts its normalized image, full frame and metadata (such as | | | | | detection time) to the router service (``facerouter``) for further processing | | | | | according to given directives. If :ref:`Video Recorder ` is | | | | | enabled, ``video-worker`` sends video over to ``video-storage``. | | | | | Provides :ref:`face liveness detection ` if enabled. | | | | | CPU- or GPU-acceleration. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | ntls | 3185 | License server that interfaces with the NtechLab Global License Server, a USB | | | | (API & UI), | dongle, or hardware fingerprint to verify the :ref:`license ` | | | | 3133 | of your FindFace Server instance. | | | | (licensing) | | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | counter | 18300 | Feature vector deduplication, counting and aggregation service. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | liveness-api | 18301 | Besides the embedded functionality provided by ``video-worker``, face | | | | | liveness detection can also be harnessed as a standalone service | | | | | ``liveness-api``. The service takes a specific number of frames from a | | | | | video chunk and returns the best quality face, and decimal liveness result. | | | | | The service can be used | | | | | in the course of user authentication by face. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+ | | deduplicator | 18310 | Feature vector deduplication service. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+ | etcd | 2379 | Third-party software that implements a distributed key-value store for | `etcd `_ | | | | ``video-manager``. Used as a coordination service in the distributed | | | | | system, providing the video object detector with fault tolerance. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+ | redis | 6379 | Third-party in-memory database that can be used by ``sf-api`` as a temporary storage | `redis `_ | | | | for extracted object feature vectors before they are written to the feature vector | | | | | storage. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+ | memcached | 11211 | Third-party software that implements a distributed memory caching system. | `memcached `_ | | | | Used by ``sf-api`` as a temporary storage for extracted object | | | | | feature vectors before they are written to the feature vector database powered by | | | | | Tarantool. | | +---------------------------+--------------+--------------------------------------------------------------------------------------+--------------------------------------------+ .. _architecture-vms: Video Recorder ------------------------------------------------------- A Video Recorder includes the following services: +-----------------------------+--------------+---------------------------------------------------------------------------------------+--------------------------------------------+ | Component | Ports in use | Description | Vendor | +=============================+==============+=======================================================================================+============================================+ | video-storage | 18611 | Service that implements video chunk management. It takes video chunks from the | NtechLab own deployment | | | | ``video-worker`` service, puts them into the storage | | | | | (``upload``), and writes their meta-information to the MongoDB database. | | | | | By request, it issues | | | | | info about existing video chunks and Websocket-links to the corresponding | | | | | streams. These links are further used by ``video-streamer-cpu`` to deliver | | | | | the video to a user for viewing and downloading. | | +-----------------------------+--------------+---------------------------------------------------------------------------------------+ | | video-streamer | 9000 | After receiving an API request, this service | | | | | extracts the requested video chunks from ``video-storage`` and | | | | | ``video-worker`` (only the last chunk if it's not yet recorded to the | | | | | storage). Then it merges the video chunks into a one-piece video and delivers it to a | | | | | user for viewing and downloading using WebSocket. | | +-----------------------------+--------------+---------------------------------------------------------------------------------------+--------------------------------------------+ | MongoDB | 27017 | Third-party software that implements the Video Recorder database. The database stores | `MongoDB `_ | | | | meta-information of the video chunks, including their location. The video chunks | | | | | themselves are stored in the ``upload`` service. | | +-----------------------------+--------------+---------------------------------------------------------------------------------------+--------------------------------------------+ .. seealso:: :ref:`components` Single- and Multi-Host Deployment ================================== You can deploy FindFace Server on a single host or in a multi-host environment. If you opt for the latter, we offer you one of the following deployment schemes: * Deploy FindFace Server on a principal host and distribute additional ``video-worker`` instances across remote hosts. * Distribute the FindFace Server services across multiple hosts. If necessary, set up load balancing. Contact our experts by support@ntechlab.com for guidance. CPU- and GPU-acceleration ========================================== The ``extraction-api`` and ``video-worker`` services can be either CPU- or GPU-based. .. important:: Refer to :ref:`requirements` when choosing hardware configuration. If the cameras you are using have the resolution of ``1920x1080@25``, and you need to process less than 5 such cameras, then it is enough to use a ``video-worker-cpu``. But if you need to process more cameras, it is strongly recommended to use a ``video-worker-gpu``. When selecting CPU/GPU-acceleration of ``extraction-api``, you should consider the number of objects in the camera frame and the number of features extracted from each object. When there is a high density of objects (about 50 objects in the frame of each camera), one instance of the ``extraction-api-cpu`` can extract features of objects from about 17 cameras, if you need more features or cameras, use the ``extraction-api-gpu``. .. note:: The number of objects and cameras depends on your implementation. .. note:: The :ref:`liveness detector ` is much slower on CPU than on GPU.