Deploy Step-by-Step from Repository¶
This section will guide you through the FindFace Security step-by-step deployment process. Follow the instructions below minding the sequence.
In this section:
Install APT Repository¶
First of all, install the FindFace apt repository as follows:
Download the installer file
findface-security-and-server-4.2.run
.Put the
.run
file into some directory on the designated host (for example,/home/username
).From this directory, make the
.run
file executable.chmod +x findface-security-and-server-4.2.run
Execute the
.run
file.sudo ./findface-security-and-server-4.2.run
The installer will ask you a few questions and perform several automated checks to ensure that the host meets the system requirements. Fill out the prompts appropriately once requested. The questions are the following:
Product to install:
FindFace Security
.Installation type:
repo: Don't install anything, just set up the APT repository
.Neural network models to install if necessary. To select a model(s), deselect all those on the list by entering
-*
in the command line first, then select the required model by entering its sequence number (keyword): for example,1 3
. Enter done to save your selection and proceed to another step.Important
At least one model for face biometry has to be installed.
After that, the FindFace apt repository will be automatically installed.
Prerequisites¶
FindFace Security requires such third-party software as PostgreSQL, Pgbouncer, Redis, etcd, and memcached. Do the following:
Install the prerequisite packages as such:
On Ubuntu 16.04:
sudo apt update sudo apt install -y postgresql-9.5 redis-server etcd memcached pgbouncer
On Ubuntu 18.04:
sudo apt update sudo apt install -y postgresql-10 redis-server etcd memcached pgbouncer
Open the
memcached
configuration file. Set the maximum memory to use for items in megabytes:-m 512
. Set the max item size:-I 16m
. If one or both of these parameters are absent, simply add them in the file.sudo vi /etc/memcached.conf -m 512 -I 16m
Give a strong password to the
ntech
user (9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3
in the example below). Output the credentials to thepgbouncer
user list.echo '"ntech" "9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3"' | sudo tee -a /etc/pgbouncer/userlist.txt
Configure
pgbouncer
. In/etc/pgbouncer/pgbouncer.ini
, add theffsecurity
database to thedatabases
section. Configure other parameters, as shown in the example below.sudo vi /etc/pgbouncer/pgbouncer.ini [databases] ffsecurity = dbname=ffsecurity host=localhost port=5432 user=ntech ... auth_type = plain pool_mode = transaction max_client_conn = 16384 syslog = 1 listen_port = 5439
Enable the prerequisite services autostart and launch the services:
On Ubuntu 16.04:
sudo systemctl enable postgresql@9.5-main.service redis-server etcd.service memcached.service pgbouncer.service sudo systemctl start postgresql@9.5-main.service redis-server etcd.service memcached.service pgbouncer.service
On Ubuntu 18.04:
sudo systemctl enable postgresql@10-main.service redis-server etcd.service memcached.service pgbouncer.service sudo systemctl start postgresql@10-main.service redis-server etcd.service memcached.service pgbouncer.service
Provide Licensing¶
See also
You receive a license file from your NTechLab manager. If you opt for the on-premise licensing, we will also send you a USB dongle.
The FindFace Security licensing is provided as follows:
Deploy
findface-ntls
, license server in the FindFace core.Important
There must be only one
findface-ntls
instance in eachFindFace Security
installation.Tip
In the
findface-ntls
configuration file, you can change the license folder and specify your proxy server IP address if necessary. You can also change thefindface-ntls
web interface remote access settings. See findface-ntls for details.sudo apt update sudo apt install -y findface-ntls sudo systemctl enable findface-ntls.service && sudo systemctl start findface-ntls.service
Upload the license file via the
findface-ntls
web interface in one of the following ways:Navigate to the
findface-ntls
web interfacehttp://<NTLS_IP_address>:3185/#/
. Upload the license file.Tip
Later on, use the FindFace Security main web interface to consult your license information, and upgrade or extend your license (
).Directly put the license file into the license folder (by default,
/opt/ntech/license
, can be changed in the/etc/findface-ntls.cfg
configuration file).
For the on-premise licensing, insert the USB dongle into a USB port.
If the licensable components are installed on remote hosts, specify the IP address of the
findface-ntls
host in their configuration files. See findface-extraction-api, findface-tarantool-server, Video face detection: findface-video-manager and findface-video-worker for details.
See also
Deploy Main Database¶
In FindFace Security, the main system database is based on PostgreSQL. To deploy the main database, do the following:
Open the
pgbouncer
list of users/etc/pgbouncer/userlist.txt
. Copy thentech
user’s password (9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3
in the example below).sudo cat /etc/pgbouncer/userlist.txt "ntech" "9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3"
Using the PostgreSQL console, create a new user
ntech
with the copied password, and databasesffsecurity
andffcounter
in PostgreSQL.sudo -u postgres psql postgres=# CREATE ROLE ntech WITH LOGIN PASSWORD '9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3'; postgres=# CREATE DATABASE ffsecurity WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE template0; postgres=# CREATE DATABASE ffcounter WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='C.UTF-8' LC_CTYPE='C.UTF-8' TEMPLATE template0;
Tip
To quit from the PostgreSQL console, type
\q
and press Enter.Allow authentication by UID of a socket client in PostgreSQL. Restart PostgreSQL.
On Ubuntu 16.04:
echo 'local all ntech peer' | sudo tee -a /etc/postgresql/9.5/main/pg_hba.conf sudo systemctl restart postgresql@9.5-main.service
On Ubuntu 18.04:
echo 'local all ntech peer' | sudo tee -a /etc/postgresql/10/main/pg_hba.conf sudo systemctl restart postgresql@10-main.service
Deploy FindFace Core¶
To deploy the FindFace core, do the following:
Tip
You can find the description of the FindFace core components and their configuration parameters in Architecture and Components in Depth.
Install the FindFace core components:
sudo apt update sudo apt install -y findface-tarantool-server findface-extraction-api findface-sf-api findface-upload findface-video-manager findface-video-worker-cpu
Note
To install the GPU-accelerated
findface-extraction-api
component, usefindface-extraction-api-gpu
instead offindface-extraction-api
in the command.Note
To install the GPU-accelerated
findface-video-worker
component, usefindface-video-worker-gpu
instead offindface-video-worker-cpu
in the command. If you have several video cards on your server, see Multiple Video Cards Usage.Important
Be sure to manually install neural network models on the host(s) with
findface-extraction-api
.In the
findface-sf-api
configuration file, enable theallow-return-facen
parameter.sudo vi /etc/findface-sf-api.ini ... limits: ... allow-return-facen: true ...
In the
findface-extraction-api
configuration file, enable recognition models for face features such as gender, age, emotions, glasses, beard, and face mask, subject to your needs. Be sure to choose the right acceleration type for each model, matching the acceleration type offindface-extraction-api
: CPU or GPU. Be aware thatfindface-extraction-api
on CPU can work only with CPU-models, whilefindface-extraction-api
on GPU supports both CPU- and GPU-models. See Face Features Recognition for details.sudo vi /etc/findface-extraction-api.ini models: age: faceattr/age.v1.cpu.fnk emotions: faceattr/emotions.v1.cpu.fnk face: face/ifruit_320.cpu.fnk gender: faceattr/gender.v2.cpu.fnk beard: faceattr/beard.v0.cpu.fnk glasses3: faceattr/glasses3.v0.cpu.fnk medmask3: faceattr/medmask3.v0.cpu.fnk
The following models are available:
Face feature Acceleration Configuration file parameter face (biometry) CPU face: face/ifruit_320.cpu.fnk
face: face/ifruit_160.cpu.fnk
GPU face: face/ifruit_320.gpu.fnk
face: face/ifruit_160.gpu.fnk
age CPU age: faceattr/age.v1.cpu.fnk
GPU age: faceattr/age.v1.gpu.fnk
gender CPU gender: faceattr/gender.v2.cpu.fnk
GPU gender: faceattr/gender.v2.gpu.fnk
emotions CPU emotions: faceattr/emotions.v1.cpu.fnk
GPU emotions: faceattr/emotions.v1.gpu.fnk
glasses CPU glasses3: faceattr/glasses3.v0.cpu.fnk
GPU glasses3: faceattr/glasses3.v0.gpu.fnk
beard CPU beard: faceattr/beard.v0.cpu.fnk
GPU beard: faceattr/beard.v0.gpu.fnk
face mask CPU medmask3: faceattr/medmask3.v0.cpu.fnk
GPU medmask3: faceattr/medmask3.v0.gpu.fnk
Tip
To disable a recognition model, simply pass an empty value to a relevant parameter. Do not remove the parameter itself as in this case the system will be searching for the default model.
models: gender: "" age: "" emotions: ""
Open the
/etc/findface-video-worker-cpu.ini
(/etc/findface-video-worker-gpu.ini
) configuration file. In themgr-static
parameter, specify thefindface-video-manager
host IP address, which providesfindface-video-worker
with settings and the video stream list. In thecapacity
parameter, specify the maximum number of video streams to be processed byfindface-video-worker
. In thestreamer
section, specify the IP address and port to access the video wall. The streamerport
must be set to18999
.sudo vi /etc/findface-video-worker-cpu.ini sudo vi /etc/findface-video-worker-gpu.ini mgr-static=127.0.0.1:18811 capacity=10 [streamer] #------------------------------ ## streamer/shots webserver port, 0=disabled ## type:number env:CFG_STREAMER_PORT longopt:--streamer-port port = 18999 ## streamer url - how to access this worker on streamer_port ## type:string env:CFG_STREAMER_URL longopt:--streamer-url url = 127.0.0.1:18999
Enable the FindFace core services autostart and launch the services.
sudo systemctl enable findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-cpu sudo systemctl start findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-cpu
Deploy FindFace Security Application Module and Biometric Database¶
To deploy the FindFace Security application module, do the following:
Install the
findface-security
,ffsecurity-ui
, andfindface-counter
components. Enable thefindface-counter
autostart and launch the service.sudo apt update sudo apt install -y ffsecurity ffsecurity-ui findface-counter sudo systemctl enable findface-counter && sudo systemctl start findface-counter
Migrate the database architecture from FindFace Security to PostgreSQL, create user groups with predefined rights and the first user with administrator rights (a.k.a. Super Administrator).
Important
Super Administrator cannot be deprived of its rights, whatever the role.
sudo findface-security migrate sudo findface-security create_groups sudo findface-security create_default_user
Create a structure of the Tarantool-based biometric database.
sudo findface-security make_tnt_schema | sudo tee /etc/ffsecurity/tnt_schema.lua
Open the
/etc/tarantool/instances.enabled/FindFace.lua
configuration file. Check whether it contains thedofile
command,meta_indexes
andmeta_scheme
definitions, as in the example below.sudo vi /etc/tarantool/instances.enabled/FindFace.lua dofile("/etc/ffsecurity/tnt_schema.lua") -- host:port to bind, HTTP API FindFace = require("FindFace") FindFace.start("127.0.0.1", 8001, { license_ntls_server="127.0.0.1:3133", meta_indexes=meta_indexes, meta_scheme = meta_scheme })
Enable the
findface-tarantool-server
service autostart and launch the service.sudo systemctl enable tarantool@FindFace.service && sudo systemctl start tarantool@FindFace.service
Open the
/etc/ffsecurity/config.py
configuration file. Specify the following parameters:SERVICE_EXTERNAL_ADDRESS
: FindFace Security IP address or URL prioritized for the Genetec integration and webhooks. Once this parameter not specified, the system usesEXTERNAL_ADDRESS
for these purposes. To use Genetec and webhooks, be sure to specify at least one of those parameters:SERVICE_EXTERNAL_ADDRESS
,EXTERNAL_ADDRESS
.EXTERNAL_ADDRESS
: (Optional) IP address or URL that can be used to access the FindFace Security web interface. Once this parameter not specified, the system auto-detects it as the external IP address. To access FindFace Security, you can use both the auto-detected and specified IP addresses.VIDEO_DETECTOR_TOKEN
: to authorize the video face detection module, come up with a token and specify it here.VIDEO_MANAGER_ADDRESS
: IP address of thefindface-video-manager
host.NTLS_HTTP_URL
: IP address of thefindface-ntls
host.ROUTER_URL
: IP address of thefindface-security
host that will receive detected faces from thefindface-video-worker
instance(s). Specify either external or internal IP address, subject to the network through whichfindface-video-worker
interacts withfindface-security
. Change the default port, subject to the redirect settings from HTTP to HTTPS, or omit it leaving only the IP address.SF_API_ADDRESS
: IP address of thefindface-sf-api
host.DATABASES
(section): fill it in as such:'PORT': 5439, 'USER': 'ntech', 'PASSWORD': '<password from /etc/pgbouncer/userlist.txt>'
(see Prerequisites).
Tip
If necessary, ensure data security by enabling SSL.
Tip
If necessary, set
’IGNORE_UNMATCHED’: True
to disable logging events for faces which have no match in the dossiers (negative verification result). Enable this option if the system has to process a large number of faces. The face similarity threshold for verification is defined by theCONFIDENCE_THRESHOLD
parameter.Tip
It is recommended to change the
MINIMUM_DOSSIER_QUALITY
default value. This parameter determines the minimum quality of a face in a dossier photo. Photos containing faces of worse quality will be rejected when uploading to a dossier. Upright faces in frontal position are considered the best quality. They result in values around 0, mostly negative (such as -0.00067401276, for example). Inverted faces and large face angles are estimated with negative values some -5 and less. By default,’MINIMUM_DOSSIER_QUALITY’: -2
which is the average quality.Important
If you enabled recognition models in the
findface-extraction-api
configuration file, add the following line in theFFSECURITY
section: ‘EVENTS_FEATURES’: [‘gender’, ‘age’, ‘emotions’, ‘beard’, ‘glasses’, ‘medmask’], subject to the list of enabled models. This line must be placed betweenSF_API_ADDRESS
andLIVENESS_THRESHOLD
as shown in the example below. See Face Features Recognition for details.sudo vi /etc/ffsecurity/config.py # ============================================================================== # FindFace Security configuration file # ============================================================================== # # This config file is written in Python's syntax and interpreted at FindFace Security # service startup. You have to restart the service in order to apply changes. # # If you have any questions or suggestions, please contact us at [email protected] # ============================================================================== # GENERAL SETTINGS # ============================================================================== # enables additional logs DEBUG = False # media files directory MEDIA_ROOT = "/var/lib/ffsecurity/uploads" # static files directory STATIC_ROOT = "/var/lib/ffsecurity/static" # language code LANGUAGE_CODE = 'en-us' # time zone TIME_ZONE = 'UTC' # Database is used by FindFace Security to store cameras, # camera groups, watchlists and so on. Only PostgreSQL is supported. DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'DISABLE_SERVER_SIDE_CURSORS': True, 'NAME': 'ffsecurity', 'PORT': 5439, 'USER': 'ntech', 'PASSWORD': '9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3' } } # Signature key for session encryption # Use pwgen -sncy 50 1|tr "'" "." to generate your own unique key SECRET_KEY = 'd1c22e3b368f2f8be4caa95b3540ce0c' # ============================================================================== # FINDFACE SECURITY SETTINGS # ============================================================================== # SERVICE_EXTERNAL_ADDRESS is prioritized for FFSecurity webhooks and Genetec plugin. # EXTERNAL_ADDRESS is used instead if SERVICE_EXTERNAL_ADDRESS is not provided. # You must provide either SERVICE_EXTERNAL_ADDRESS or EXTERNAL_ADDRESS in order # to be able to work with FFSecurity webhooks and Genetec plugin. SERVICE_EXTERNAL_ADDRESS = 'http://127.0.0.1' # EXTERNAL_ADDRESS is used to access objects created inside FFSecurity via external links. EXTERNAL_ADDRESS = '' # - Base FFSecurity settings - FFSECURITY = { # findface-video-worker authorization token 'VIDEO_DETECTOR_TOKEN': 'fdc0fea20cf0a734c5e2de1886fc07b4', # base face matching confidence threshold 'CONFIDENCE_THRESHOLD': 0.739, # episodes specific matching threshold that is used to join faces in an episode 'EPISODES_THRESHOLD': 0.689, # minimum face quality sufficient to add it to a dossier 'MINIMUM_DOSSIER_QUALITY': -2, # skip all unmatched faces 'IGNORE_UNMATCHED': False, # matched events older than EVENTS_MAX_MATCHED_AGE will be automatically # deleted (every night at 1:17 am by default) 'EVENTS_MAX_MATCHED_AGE': 30, # same as above but for unmatched events 'EVENTS_MAX_UNMATCHED_AGE': 30, # same as EVENTS_MAX_MATCHED_AGE but for matched full frame images only (thumbnails won't be deleted) 'EVENTS_MAX_FULLFRAME_UNMATCHED_AGE': 30, # same as above but for unmatched full frame images only (thumbnails won't be deleted) 'EVENTS_MAX_FULLFRAME_MATCHED_AGE': 30, # NTLS licence server url 'NTLS_HTTP_URL': 'http://127.0.0.1:3185', # findface-video-worker face posting address, # it must be set to either FFSecurity EXTERNAL_ADDRESS (by default) # or findface-facerouter url (in some specific cases) 'ROUTER_URL': 'http://127.0.0.1', # send serialized dossiers, dossier-lists, camera and camera groups in webhooks 'VERBOSE_WEBHOOKS': False, 'THUMBNAIL_JPEG_QUALITY': 75, # FFServer services urls 'VIDEO_MANAGER_ADDRESS': 'http://127.0.0.1:18810', 'SF_API_ADDRESS': 'http://127.0.0.1:18411', 'FFCOUNTER_ADDRESS': 'http://127.0.0.1:17300', # additional events features. # make sure that corresponding extractors # are licensed and enabled at findface-extraction-api config file. # available features are: gender, age, emotions, beard, glasses, medmask. 'EVENTS_FEATURES': [], # feature specific confidence thresholds 'LIVENESS_THRESHOLD': 0.75, 'EMOTIONS_THRESHOLD': 0.25, 'BEARD_THRESHOLD': 0.7, # -- Persons configuration -- # rrule (recurrence rule) for scheduling persons clusterization 'PERSONS_CLUSTERIZATION_SCHEDULE': 'RRULE:FREQ=DAILY;INTERVAL=1;WKST=MO;BYHOUR=0;BYMINUTE=0', # face to person matching confidence threshold 'PERSONS_CONFIDENCE_THRESHOLD': 0.739, # minimum face quality for handle face in person service 'PERSON_EVENT_MIN_QUALITY': -2, # counters full frame saving options: # `always` - save always # `detect` - save only if faces or silhouettes have been detected # `never` - never save full frames 'COUNTERS_SAVE_FULLFRAME': 'always', 'COUNTERS_FULLFRAME_JPEG_QUALITY': 75, 'COUNTERS_THUMBNAIL_JPEG_QUALITY': 75, # -- Optional parameters -- # Edit CUSTOM_FIELDS section to customize dossier content. # Below is an example for integration FindFace Security with Sigur. # 'CUSTOM_FIELDS': { # 'dossier_meta': { # 'items': [ # { # 'name': 'personid', # 'default': '', # 'label': 'PersonID', # 'display': ['list', 'form'], # 'description': 'Sigur person ID' # }, # { # 'name': 'firstname', # 'default': '', # 'label': 'First Name', # 'display': ['list', 'form'], # 'description': 'Sigur first name' # }, # { # 'name': 'lastname', # 'default': '', # 'label': 'Last Name', # 'display': ['list', 'form'], # 'description': 'Sigur last name' # }, # { # 'name': 'version', # 'default': '', # 'label': 'Version', # 'display': ['list', 'form'], # 'description': 'Sigur photo version' # } # ], # 'filters': [ # { # 'name': 'personid', # 'label': 'Sigur person ID filter', # 'field': 'personid' # } # ] # } # }, # maximum event age in seconds than could be added to an episode. # 'EPISODE_SEARCH_INTERVAL': 60, # If none of these events matched, new episode is created. # maximum episode duration (episode is closed after) # 'EPISODE_MAX_DURATION': 300, # if no new event added to an episode during this timeout, episode will be closed. # 'EPISODE_EVENT_TIMEOUT': 30, # maximum created thumbnail width # 'THUMBNAIL_MAX_WIDTH': 320, # social backend url. Contact support for additional information. # 'SOCIAL_BACKEND': None, # 'SOCIAL_HEADERS': {}, # unacknowledged events notification interval # 'UNACKNOWLEDGED_NOTIFY_INTERVAL': 1, } # - FindFace Security user interface configuration dictionary - FFSECURITY_UI_CONFIG = { "event": { "features": { "f_gender_class": ["male", "female"], "age": { "f_age_gte": "", "f_age_lte": "" }, "f_emotions_class": ["angry", "disgust", "fear", "happy", "sad", "surprise", "neutral"], "f_glasses_class": ["none", "eye", "sun"], "f_beard_class": ["none", "beard"], "f_liveness_class": ["real", "fake"], "f_medmask_class": ["none", "correct"], } } } # -- ASGI-server configuration -- # consult support before changing these settings. # per worker thread pool size. ASGI_THREADS = 32 UVICORN_SETTINGS = { # worker processes count, 'auto' sets it to logical cpu count 'workers': 'auto', 'host': 'localhost', 'port': 8002, # websocket worker processes count, # 'auto' sets it to logical cpu count, but not more than 8. 'ws-workers': 'auto', 'ws-host': 'localhost', 'ws-port': 8003, } # disable unused services to increase # overall system performance in some cases. SERVICES = { "ffsecurity": { "episodes": True, "webhooks": True, "persons": False, } } # ============================================================================== # FINDFACE SECURITY PLUGINS # ============================================================================== # Uncomment lines below to enable plugins. Please consult documentation for # a plugin specific settings. # =============== Axxon ================ # INSTALLED_APPS.append('ffsecurity_axxon') # FFSECURITY['AXXON'] = [ # { # 'name': 'server_name', # 'api': 'http://example.com/', # 'rtsp': 'rtsp://example.com:554/', # 'user': 'user', # 'password': 'password', # } # ] # FFSECURITY_UI_CONFIG['dossier'] = { # 'video': True, # } # =============== Genetec ================ # INSTALLED_APPS.append('ffsecurity_genetec') # ================ Sova ================== # INSTALLED_APPS.append('ffsecurity_sova') # ======= CryptoPRO authentication ======= # INSTALLED_APPS.append('ffsecurity_cproauth') # REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] = [ # 'ffsecurity.auth.TokenAuthentication', # 'ffsecurity_cproauth.auth.CryptoProOrTokenAuthentication' # ] # ========== DossierLists sync =========== # INSTALLED_APPS.append('ffsecurity_sync') # token must be identical on master and slave # use pwgen -s 64 1 # SYNC_TOKEN = 'change_me' # SYNC_TIME = { # # 24 hour format # 'hour': 3, # 'minute': 0, # }
Generate a signature key for the session encryption (used by Django) by executing:
pwgen -sncy 50 1|tr “’” “.”
. Specify this key asSECRET_KEY
.Start the services.
sudo systemctl enable findface-security sudo systemctl start findface-security
Disable the default nginx server and add the
findface-security
server to the list of enabled servers. Restart nginx.sudo rm /etc/nginx/sites-enabled/default sudo ln -s /etc/nginx/sites-available/ffsecurity-nginx.conf /etc/nginx/sites-enabled/ sudo nginx -s reload