Architecture

Though you mostly interact with FindFace Multi through its web interface, be sure to take a minute to learn the FindFace Multi architecture. This knowledge is essential for the FindFace Multi deployment, integration, maintenance, and troubleshooting.

In this chapter:

Recognition Objects and Recognition Process

FindFace Multi can recognize the following objects and their attributes:

  • human faces

  • human bodies (silhouettes)

  • vehicles

FindFace Multi detects an object in the photo or video and prepares its image through normalization. The normalized image is then used for extracting the object’s feature vector (an n-dimensional vector of numerical features that represent the object). Object feature vectors are stored in the database and further used for verification and identification purposes.

Docker Platform

FindFace Multi is deployed in Docker, a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Each FindFace Multi service runs in a Docker container.

Architectural Elements

FindFace Multi consists of the following fundamental architectural elements:

  • FindFace core, a cutting-edge AI-based recognition technology that can be used as a separate product FindFace Enterprise Server.

  • FindFace Multi, which is a turnkey application module for FindFace Enterprise Server.

The FindFace core internal architecture allows for embedding Video Recorder, an additional functionality that records, stores, and plays back video data from cameras.

Architecture scheme

architecture_en

FindFace core

There is a separate product FindFace Enterprise Server inside the FindFace core. It includes the following components:

Component

Ports in use

Description

Vendor

findface-extraction-api

18666

Service that uses neural networks to detect an object in an image and extract its feature vector. It also recognizes object attributes (for example, gender, age, emotions, beard, glasses, face mask - for face objects). CPU- or GPU-acceleration.

NtechLab own deployment

findface-sf-api

18411

Service that implements the internal HTTP API for object detection and recognition.

findface-tarantool-server

32001, shard ports (default 330xx, 81xx)

Service that provides interaction between the findface-sf-api service and the feature vector database (the Tarantool-powered database that stores object feature vectors.

findface-upload

3333

NginX-based web server used as a storage for original images, thumbnails, and normalized object images. If Video Recorder is installed, findface-upload also stores video data from cameras.

findface-facerouter

18820

Service used to define processing directives for detected objects. In FindFace Multi, its functions are performed by findface-multi-legacy (see FindFace Multi Application Module).

findface-video-manager

18810, 18811

Service, part of the video object detection module, that is used for managing the video object detection functionality, configuring the video object detector settings and specifying the list of to-be-processed video streams.

findface-video-worker

18999

Service, part of the video object detection module, that recognizes an object in the video and posts its normalized image, full frame and metadata (such as detection time) to the findface-facerouter service for further processing according to given directives. If Video Recorder is enabled, findface-video-worker sends video over to findface-video-storage. Provides face liveness detection if enabled. CPU- or GPU-acceleration.

findface-ntls

443 (TCP), 3133, 3185

License server that interfaces with the NtechLab Global License Server, a USB dongle, or hardware fingerprint to verify the license of your FindFace Multi instance.

findface-counter

18300

Service used for event deduplication.

findface-liveness-api

18301

Besides the embedded functionality provided by findface-video-worker, face liveness detection can also be harnessed as a standalone service findface-liveness-api. The service takes a specific number of frames from a video chunk and returns the best quality face, and decimal liveness result averaged across the taken frames. The service is also involved in the course of user authentication by face.

Tarantool

Shard ports (default 330xx, 81xx)

Third-party software that implements the feature vector database that stores extracted object feature vectors and identification events. The system data, records, user accounts, and camera settings are stored in PostgreSQL (part of the FindFace Multi application module).

Tarantool

etcd

2379

Third-party software that implements a distributed key-value store for findface-video-manager. Used as a coordination service in the distributed system, providing the video object detector with fault tolerance.

etcd

NginX

80; SSL: 8002, 8003, 443, 80

Third-party software that implements the system web interfaces.

nginx

memcached

11211

Third-party software that implements a distributed memory caching system. Used by findface-sf-api as a temporary storage for extracted object feature vectors before they are written to the feature vector database powered by Tarantool.

memcached

FindFace Multi Application Module

The FindFace Multi application module includes the following components:

Component

Ports in use

Description

Vendor

findface-multi-legacy

Configurable

Service that serves as a gateway to the FindFace core. Provides interaction between the FindFace core and the web interface, the system functioning as a whole, HTTP and web socket, object monitoring, event notifications, episodes, webhooks, object clusterization, and counters.

NtechLab own deployment

findface-multi-pause

n/a

Internal services that assist findface-multi-legacy. The findface-multi-audit service is created for the future. It will become full-fledged in the upcoming versions. As for now, it partly duplicates the findface-multi-legacy functionality. Use findface-multi-legacy to work with it. The findface-multi-identity-provider is the service for authentication, user management, and management of role-based model of accesses.

findface-multi-audit

8012, 8013, 8014

findface-multi-identity-provider

8022, 8023, 8024

findface-onvif-discovery

n/a

findface-multi-legacy-singleton-services

n/a

Runs single-instance-services, such as synchronization of cameras with the findface-video-manager, clustering and other single-instance-services.

findface-multi-identity-provider-migrate

n/a

Services that run database migration, create galleries and make other settings.

findface-multi-legacy-migrate

n/a

findface-multi-healthcheck

n/a

Checks container’s health on startup. Expects the containers to be “healthy” before starting a UI.

findface-multi-cleaner

n/a

A part of the findface-multi-legacy functionality, detached into a separate service. It serves for the FindFace Multi data clean up (except for the VMS video archives clean up).

findface-multi-cleaner-vms

n/a

A part of the findface-multi-legacy functionality, detached into a separate service. It is responsible for the VMS video archives clean up.

findface-multi-file-mover

n/a

A file moving service that is used for tiered data storage. The service transfers images from the short-term storage to the long-term storage.

findface-multi-ui

Configurable

Main web interface used to interact with FindFace Multi. Based on the Django framework. Allows you to work with object recognition events, search for objects, manage cameras, users, records, watch lists, collect real-time statistics, and many more.

NATS

4222

Third-party software that implements a message broker in findface-multi-legacy.

NATS

etcd

2379

Third-party software that implements locks in the findface-multi-legacy service e.g. locks in license checker, reports, video processing, object clusterization, etc.

etcd

Pgbouncer

5439

Third-party software, a lightweight connection pooler for PostgreSQL. Optional, used to increase the database performance under high load.

PgBouncer

PostgreSQL

5432

Third-party software that implements the main system database. This database stores records of persons and vehicles and data for internal use, including user accounts and camera settings. The object feature vectors and object identification events are stored in Tarantool (part of the FindFace core).

PostgreSQL

Video Recorder

Video Recorder is an optional part of the FindFace core. It includes the following services:

Component

Ports in use

Description

Vendor

findface-video-storage

18611

Service that implements video chunk management. It takes video chunks from the findface-video-worker service, puts them into the storage (findface-upload), and writes their meta-information and whereabouts to the Video Recorder database (MongoDB). By request from findface-multi-legacy, it issues info about existing video chunks in the form of Websocket-links to the corresponding streams. These links are further used by findface-video-streamer-cpu to deliver the video to a user for viewing and downloading.

NtechLab own deployment

findface-video-streamer

9000

After receiving a findface-multi-ui request, this service uses Websocket to extract requested video chunks from findface-video-storage and findface-video-worker (only the last chunk if it’s not yet recorded to the storage). Then it merges the video chunks into a one-piece video and delivers it to a user for viewing and downloading.

MongoDB

27017

Third-party software that implements the Video Recorder database. The database stores meta-information of the video chunks, including their location. The video chunks themselves are stored in the findface-upload service.

MongoDB

See also

Single- and Multi-Host Deployment

You can deploy FindFace Multi on a single host or in a multi-host environment. If you opt for the latter, we offer you one of the following deployment schemes:

CPU- and GPU-acceleration

The findface-extraction-api and findface-video-worker services can be either CPU- or GPU-based. You will have an opportunity to choose the acceleration type you need during the installation.

Important

Refer to Requirements when choosing hardware configuration.

Important

If the resolution of a camera(s) in use is more than 1280x720px, it is strongly recommended to use the GPU-accelerated package findface-video-worker.

Note

The liveness detector is much slower on CPU than on GPU.