Update to FindFace Multi 2.1.3
Tip
If you use our product FindFace Security deployed on Ubuntu 18.04, upgrade it to FindFace Multi 1.2 and then update it to FindFace Multi 2.1.3.
If you use FindFace Multi 2.1 or FindFace Multi 2.1.0.1, update it to FindFace Multi 2.1.3 by following the instruction.
The Incidents and VNS plugins, operable in FindFace CIBR, are not supported in FindFace Multi 2.1+. That’s why there’s no need to enable ffsecurity_incidents
, ffsecurity_vns
, and ffsecurity_puppeteer
plugins in the findface-multi-legacy.py
configuration file after product update.
Integration with Axxon Next is included into external VMS integration in FindFace Multi 2.1+ and configured through VMS integration plugin.
If you created person and car cards with custom fields in FindFace Multi 1.2 and want them to be displayed in the FindFace Multi 2.1.3 UI, then it is necessary to copy the CUSTOM_FIELDS
section from the old configuration file in step #2 and paste it to the new configuration file in step #8.
In this section:
Update FindFace Multi 1.2 to FindFace Multi 2.1.3
Important
Before you start the update, disable all cameras.
To update FindFace Multi 1.2 to FindFace Multi 2.1.3, do the following:
Create a backup copy of the old schema of the Tarantool-based feature vector database:
sudo cp /etc/findface-security/tnt_schema.lua /etc/findface-security/old_tnt_schema.lua
Starting from version 2.0, one of the most significant differences between FindFace Multi and earlier versions of the product is the structure of the Tarantool biometric database (so called meta-schema). The new structure is created as a set of spaces, while in previous versions of the product there was only one space by default in the structure of the Tarantool-based database.
Open the
/etc/findface-security/config.py
configuration file. Save the values of the following parameters for later use:EXTERNAL_ADDRESS
,SECRET_KEY
,VIDEO_DETECTOR_TOKEN
,ROUTER_URL
,CUSTOM_FIELDS
.sudo vi /etc/findface-security/config.py EXTERNAL_ADDRESS = "http://172.20.77.58" ... # use pwgen -sncy 50 1|tr "'" "." to generate your own unique key SECRET_KEY = 'c8b533847bbf7142102de1349d33a1f6' FFSECURITY = { 'VIDEO_DETECTOR_TOKEN': '381b0f4a20495227d04185ab02f5085f', ... 'ROUTER_URL': 'http://172.20.77.58', ... # -- Custom model fields -- # Edit CUSTOM_FIELDS -> `human_card` section to customize human card fields. # Edit CUSTOM_FIELDS -> `face_object` section to customize face object fields. # Below is an example with every field type possible. # 'CUSTOM_FIELDS': { # 'human_card': { # 'items': [ # { # 'name': 'personid', # 'default': '', # 'label': 'PersonID', # 'display': ['list', 'form'], # 'description': 'Sigur person ID', # 'editable': False # }, # { # 'name': 'firstname', # 'default': '', # 'label': 'First Name', # 'display': ['list', 'form'], # 'description': 'Sigur first name', # 'editable': False # }, # { # 'name': 'lastname', # 'default': '', # 'label': 'Last Name', # 'display': ['list', 'form'], # 'description': 'Sigur last name', # 'editable': False # }, # { # 'name': 'version', # 'default': '', # 'label': 'Version', # 'display': ['list', 'form'], # 'description': 'Sigur photo version', # 'editable': False # } # ], # 'filters': [ # { # 'name': 'personid', # 'label': 'Sigur person ID filter', # 'field': 'personid' # } # ] # }, # 'face_object': { # 'items': [ # { # "field_name": "tag_name_1", # "type": "string", # "default": "change_me" # }, # { # "field_name": "tag_name_2", # "type": "uint", # "default": 123 # }, # { # "field_name": "tag_name_3", # "type": "bool", # "default": True # }, # ] # } # }, }
Stop the
findface-security
service.sudo systemctl stop findface-security.service
Create a backup of the Tarantool-based feature vector database in any directory of your choice, for example,
/etc/findface_dump
.sudo mkdir -p /etc/findface_dump cd /etc/findface_dump sudo findface-storage-api-dump -config /etc/findface-sf-api.ini
To avoid port conflicts, stop and disable all services before installing a new version.
Note
There are eight shards in the example below. If it differs with the number of shards in your system, adjust the below command accordingly. E.g., for the system with sixteen shards, replace
tarantool@shard-00{1..8}.service
withtarantool@shard-0{01..16}.service
. Get the list of active shards with thels /etc/tarantool/instances.enabled/
command.You may stop and disable the services one by one:
sudo systemctl stop postgresql.service sudo systemctl stop postgresql@10-main sudo systemctl stop findface-*.service sudo systemctl stop pgbouncer.service sudo systemctl stop tarantool@shard-00{1..8}.service sudo systemctl stop nats-server.service sudo systemctl stop etcd.service sudo systemctl stop mongod.service sudo systemctl stop mongodb.service sudo systemctl stop memcached.service sudo systemctl stop nginx.service sudo systemctl disable postgresql.service sudo systemctl disable postgresql@10-main sudo systemctl disable pgbouncer.service sudo systemctl disable findface-extraction-api.service sudo systemctl disable findface-security.service sudo systemctl disable findface-security-onvif.service sudo systemctl disable findface-sf-api.service sudo systemctl disable findface-ntls.service sudo systemctl disable findface-video-manager.service sudo systemctl disable findface-video-worker-cpu.service sudo systemctl disable findface-video-worker-gpu.service sudo systemctl disable findface-counter.service sudo systemctl disable findface-liveness-api.service sudo systemctl disable findface-video-streamer-cpu.service sudo systemctl disable findface-video-streamer-gpu.service sudo systemctl disable findface-video-storage.service sudo systemctl disable tarantool@shard-00{1..8}.service sudo systemctl disable nats-server.service sudo systemctl disable etcd.service sudo systemctl disable mongod.service sudo systemctl disable mongodb.service sudo systemctl disable memcached.service sudo systemctl disable nginx.service
Or you may use the following compact commands instead:
sudo systemctl stop postgresql.service postgresql@10-main findface-*.service pgbouncer.service tarantool@shard-00{1..8}.service nats-server.service etcd.service mongod.service mongodb.service memcached.service nginx.service sudo systemctl disable postgresql.service postgresql@10-main pgbouncer.service findface-extraction-api.service findface-security.service findface-security-onvif.service findface-sf-api.service findface-ntls.service findface-video-manager.service findface-video-worker-cpu.service findface-video-worker-gpu.service findface-counter.service findface-liveness-api.service findface-video-streamer-cpu.service findface-video-streamer-gpu.service findface-video-storage.service tarantool@shard-00{1..8}.service nats-server.service etcd.service mongod.service mongodb.service memcached.service nginx.service
Install the FindFace Multi 2.1.3 instance. Don’t forget to prepare a server first:
See:
After FindFace Multi installation, stop all the containers from the
/opt/findface-multi/
directory.cd /opt/findface-multi/ sudo docker-compose stop
Open the
/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py
configuration file and paste saved on step #2 values for the parametersEXTERNAL_ADDRESS
,SECRET_KEY
,VIDEO_DETECTOR_TOKEN
,ROUTER_URL
, andCUSTOM_FIELDS
into it.sudo vi /opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py ... # Use pwgen -sncy 50 1|tr "'" "." to generate your own unique key SECRET_KEY = '002231ccb690586f4d33e98322c591bb' ... SERVICE_EXTERNAL_ADDRESS = 'http://172.20.77.58' # EXTERNAL_ADDRESS is used to access objects created inside FFSecurity via external links. EXTERNAL_ADDRESS = 'http://172.20.77.58' ... # findface-video-worker authorization token 'VIDEO_DETECTOR_TOKEN': '8977e1b0067d43f6c908d0bf60363255', ... # findface-video-worker face posting address, # it must be set to either FFSecurity EXTERNAL_ADDRESS (by default) # or findface-facerouter url (in some specific cases) 'ROUTER_URL': 'http://127.0.0.1:80',
Open the old version of the
findface-ntls
configuration file available at/etc/findface-ntls.cfg
and check it against the new version/opt/findface-multi/configs/findface-ntls/findface-ntls.yaml
. Move all the custom parameters from the old version to the new one. Do the same for other components, e.g., forfindface-extract-api
, check/etc/findface-extract-api.ini
against/opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml
, forfindface-sf-api
, check/etc/findface-sf-api.ini
against/opt/findface-multi/configs/findface-sf-api/findface-sf-api.yaml
, etc.sudo vi /etc/findface-ntls.cfg sudo vi /opt/findface-multi/configs/findface-ntls/findface-ntls.yaml sudo vi /etc/findface-extraction-api.ini sudo vi /opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml sudo vi /etc/findface-sf-api.ini sudo vi /opt/findface-multi/configs/findface-sf-api/findface-sf-api.yaml
Important
Follow these rules to transfer parameters from the old configuration file to the new one:
If there is a new neural network model in the new configuration file, replace it with the previous one (considering that the previous model is still included in FindFace Multi 2.1.3), and if the previous model is missing in FindFace Multi 2.1.3, then do not change anything. In this case, you will have to do migration to a different neural network model.
If a parameter had an empty value in the old configuration file, but has a certain value in the new configuration file, delete its value in the new configuration file.
Keep as is those parameters that were not included in the old configuration file, but are present in the new configuration file.
Modify the Tarantool database structure by applying the
tnt_schema.lua
schema from FindFace Multi 2.1.3.sudo docker run --rm --network host --volume '/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 make-tnt-schema | sudo tee /etc/findface-security/tnt_schema.lua
Purge data from all the directories relevant to active shards.
sudo rm /opt/ntech/var/lib/tarantool/shard-*/{index,snapshots,xlogs}/*
Copy the meta-schema of the
default
space from theold_tnt_schema.lua
configuration file to the newtnt_schema.lua
configuration file, so that the old meta-schema is still available. An easy way to do it is to follow these steps:12.1. In the
/etc/findface-security/old_tnt_schema.lua
file, rename the following fields:meta_scheme --> meta_scheme_default meta_indexes --> meta_indexes_default
12.2. In the new configuration file
/etc/findface-security/tnt_schema.lua
, replace the following lines at the beginning of the file:cfg_spaces = { default = { meta_scheme = { -- internal.normalized_id: { default = '', field_type = 'string', id = 1, name = 'normalized_id', }, -- internal.feat: { default = '', field_type = 'string', id = 2, name = 'feat', }, }, meta_indexes = {} },
with these ones:
dofile("/etc/findface-security/old_tnt_schema.lua") spaces = { default = { meta_scheme=meta_scheme_default, meta_indexes=meta_indexes_default },
Navigate to the directory with Tarantool configuration file(s)
/etc/tarantool/instances.available/
. Import new meta-schematnt_schema.lua
into each configuration fileshard-00*.lua
, as in the example below.sudo vi /etc/tarantool/instances.available/shard-00*.lua dofile("/etc/findface-security/tnt_schema.lua") FindFace = require("FindFace") FindFace.start("127.0.0.1", 8104, { license_ntls_server="127.0.0.1:3133", replication = replication_master, spaces = spaces })
Remove the default configuration file
FindFace.lua
, generated by thefindface-tarantool-server
package, as it will block the restart, required on the next step.sudo rm -rf /etc/tarantool/instances*/FindFace.lua
Restart the
findface-tarantool-server
shards.TNT=$(ls /etc/tarantool/instances.enabled/ | cut -c 7,8,9) for i in $TNT; do sudo systemctl restart tarantool@shard-$i.service ; done
Upon completion of the above steps, the shards will still keep the old galleries created within the default
space, but new spaces (e.g., ffsec_body_objects_space
, ffsec_face_clusters_space
, and so on) will also become available.
Restore old data from the backup. The data will be restored as it existed previously: all galleries will stay within the
default
space.sudo systemctl start findface-ntls.service cd /etc/findface_dump for x in *.json; do sudo findface-storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done
Migrate galleries from the
default
space to new spaces:sudo systemctl start findface-sf-api.service sudo systemctl start nginx.service sudo docker run --rm --network host --volume '/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 /opt/findface-security/bin/python3 /tigre_prototype/manage.py migrate_tnt_space
Perform PostgreSQL database migrations for FindFace Multi 2.1.3 compatibility. Do the following:
18.1. Navigate to the
/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py
configuration file. In theDATABASES -> default
section, temporarily replacePASSWORD
with the old one, used in the/etc/findface-security/config.py
configuration file.Important
Make sure to write down the password from the
/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py
configuration file. You will need it later.sudo vi /opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'DISABLE_SERVER_SIDE_CURSORS': True, 'NAME': 'ffsecurity', 'HOST': '127.0.0.1', 'PORT': 5439, 'USER': 'ntech', 'PASSWORD': 'XXXXXXXXXXXXXXXX' } }
18.2. In the
/etc/pgbouncer/pgbouncer.ini
file, add the following line to thedatabases
section:ffsecurity_session = dbname=ffsecurity host=localhost port=5432 user=ntech pool_mode=session pool_size=10
18.3. On the host system, perform the database migration:
sudo systemctl start postgresql.service sudo systemctl start pgbouncer.service sudo docker run --rm --network host --volume '/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 /opt/findface-security/bin/python3 /tigre_prototype/manage.py migrate
18.4. Back up an existing database with PostgreSQL, installed on the host system.
cd /opt/findface-multi/ sudo -u postgres pg_dump --verbose --disable-triggers ffsecurity | sudo tee dump_ffsecurity.sql sudo -u postgres pg_dump -t auth_group -t ffsecurity_adgroupguid -t ffsecurity_deviceblacklistrecord -t ffsecurity_ffsecauthsession -t ffsecurity_grouppermission -t ffsecurity_runtimesetting -t ffsecurity_user -t ffsecurity_user_groups -t ffsecurity_user_user_permissions -t ffsecurity_userkeyvalue -t knox_authtoken -t ffsecurity_watchlistpermission -t ffsecurity_cameragrouppermission --data-only --verbose --no-acl --no-owner --disable-triggers ffsecurity | sudo tee dump_identity_provider.sql
18.5. Back up role permissions.
sudo docker run --rm --network host --volume '/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 /opt/findface-security/bin/python3 /tigre_prototype/manage.py dump_permissions | sudo tee permissions.csv
18.6. Change back the password, replaced in step #18.1
18.7. Stop all the services.
sudo systemctl stop findface-sf-api.service nginx.service tarantool@shard-00{1..8}.service postgresql.service pgbouncer.service
18.8. Open the
/opt/findface-multi/docker-compose.yaml
file and copyPOSTGRES_PASSWORD
value to use it in further commands.18.9. Recreate the
ffsecurity
database to clean it up from the default data. Paste{POSTGRES_PASSWORD}
value that you previously copied in step #18.8 into the command below:sudo docker-compose up -d postgresql sudo docker exec -it -u postgres findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} dropdb ffsecurity" sudo docker exec -it -u postgres findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} createdb -O ntech --encoding=UTF-8 --lc-collate=C.UTF-8 --lc-ctype=C.UTF-8 --template=template0 ffsecurity"
18.10. Restore data into the recreated
ffsecurity
database. Paste{POSTGRES_PASSWORD}
value that you previously copied in step #18.8 into the command below:sudo docker exec -i findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} psql --username postgres ffsecurity" < dump_ffsecurity.sql
18.11. Recreate the
ffsecurity_identity_provider
database to clean it up from the default data. Paste{POSTGRES_PASSWORD}
value that you previously copied in step #18.8 into the command below:sudo docker exec -it -u postgres findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} dropdb ffsecurity_identity_provider" sudo docker exec -it -u postgres findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} createdb -O ntech --encoding=UTF-8 --lc-collate=C.UTF-8 --lc-ctype=C.UTF-8 --template=template0 ffsecurity_identity_provider"
18.12. Run migration.
sudo docker-compose up -d pgbouncer sudo docker run --rm --network host --volume '/opt/findface-multi/configs/findface-multi-identity-provider/findface-multi-identity-provider.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/identity-provider:ffmulti-2.1.3 /opt/findface-security/bin/python3 /tigre_prototype/manage.py migrate
18.13. Restore data into the recreated
ffsecurity_identity_provider
database. Paste{POSTGRES_PASSWORD}
value that you previously copied in step #18.8 into the command below:sudo docker exec -i findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} psql --username postgres ffsecurity_identity_provider" < dump_identity_provider.sql
18.14. Start all the services.
sudo docker-compose up -d
18.15. Run the command to create records in the outbox table for watch lists and camera groups. Paste
{POSTGRES_PASSWORD}
value that you previously copied in step #18.8 into the command below:sudo docker exec -i findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} pg_dump --username postgres -f cg_wl_permissions.sql -t ffsecurity_cameragrouppermission -t ffsecurity_watchlistpermission --data-only ffsecurity_identity_provider" sudo docker run --rm --network host --volume '/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 /opt/findface-security/bin/python3 /tigre_prototype/manage.py create_outbox sudo docker exec -i findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} psql --username postgres -c 'TRUNCATE ffsecurity_cameragrouppermission, ffsecurity_watchlistpermission RESTART IDENTITY;' ffsecurity_identity_provider" sudo docker exec -i findface-multi-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} psql --username postgres ffsecurity_identity_provider < cg_wl_permissions.sql"
18.16. Before you restore role permissions into the
identity_provider
service, examine the/opt/findface-multi/permissions.csv
file. Make sure to replace*_ffsecauthtoken
with*_authtoken
if any. This is mostly applicable to those cases when FindFace Multi 1.2 installation was an upgrade from earlier versions of the product.After that, restore role permissions into the
identity_provider
service.sudo docker run --rm --network host -v /opt/findface-multi/configs/findface-multi-identity-provider/findface-multi-identity-provider.py:/etc/findface-security/config.py:ro -v $(pwd)/permissions.csv:/var/permissions.csv:ro docker.int.ntl/ntech/multi/multi/identity-provider:ffmulti-2.1.3 /opt/findface-security/bin/python3 /tigre_prototype/manage.py load_permissions /var/permissions.csv
18.17. Copy full frame photos, normalized images, and the license file. Copy files from the
/opt/ntech/license/
folder into the/opt/findface-multi/data/findface-ntls/
folder, from the/var/lib/findface-security/uploads/
folder into the/opt/findface-multi/data/findface-multi-legacy/uploads/
folder, from the/var/lib/ffupload/uploads/
folder into the/opt/findface-multi/data/findface-upload/uploads/
folder.sudo cp -r /opt/ntech/license/* /opt/findface-multi/data/findface-ntls/ sudo cp -r /var/lib/findface-security/uploads/* /opt/findface-multi/data/findface-multi-legacy/uploads/ sudo cp -r /var/lib/ffupload/uploads/* /opt/findface-multi/data/findface-upload/uploads/ sudo chmod 777 -R /opt/findface-multi/data/findface-upload/uploads/ sudo chown www-data:www-data -R /opt/findface-multi/data/findface-upload/uploads/*
Alternatively, the above folders can be directly mounted into the relevant docker containers via the
docker-compose.yaml
file, like in the example below:sudo vi /opt/findface-multi/docker-compose.yaml ... findface-upload: image: docker.int.ntl/ntech/universe/upload:ffserver-9.230407.1 logging: {driver: journald} network_mode: service:pause restart: always volumes: ['./configs/findface-upload/40-ffupload.sh:/docker-entrypoint.d/40-ffupload.sh:ro', '/var/lib/ffupload:/var/lib/ffupload'] ... findface-multi-identity-provider: depends_on: findface-multi-identity-provider-migrate: {condition: service_completed_successfully} healthcheck: {interval: 5s, retries: 3, start_period: 60s, test: 'curl -f http://localhost:8022/healthcheck || exit 1', timeout: 3s} image: docker.int.ntl/ntech/multi/multi/identity-provider:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: always volumes: ['./configs/findface-multi-identity-provider/findface-multi-identity-provider.py:/etc/findface-security/config.py:ro', '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads'] ... findface-multi-identity-provider-migrate: command: [migrate] depends_on: [pgbouncer, nats, nats-jetstream, findface-sf-api, findface-liveness-api, etcd] environment: {ADMIN_PASSWORD: admin} image: docker.int.ntl/ntech/multi/multi/identity-provider:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: on-failure volumes: ['./configs/findface-multi-identity-provider/findface-multi-identity-provider.py:/etc/findface-security/config.py:ro', '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads'] ... findface-multi-legacy: depends_on: findface-multi-identity-provider: {condition: service_started} findface-multi-legacy-migrate: {condition: service_completed_successfully} healthcheck: {interval: 5s, retries: 3, start_period: 60s, test: 'curl -f http://localhost:8002/healthcheck || exit 1', timeout: 3s} image: docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: always volumes: ['./configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro', '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads'] ... findface-multi-legacy-migrate: command: [migrate] depends_on: [pgbouncer, nats, findface-sf-api, findface-counter, nats-jetstream] image: docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: on-failure volumes: ['./configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro', '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads'] ... findface-multi-legacy-singleton-services: command: [run-single-instance-services] depends_on: findface-multi-identity-provider: {condition: service_healthy} findface-multi-legacy-migrate: {condition: service_completed_successfully} healthcheck: {interval: 5s, retries: 3, start_period: 60s, test: 'curl -f http://localhost:9901/healthcheck || exit 1', timeout: 3s} image: docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: always volumes: ['./configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro', '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads'] ... cleaner: command: [run-cleaner-service] depends_on: [pgbouncer, findface-sf-api, findface-multi-legacy] image: docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: always volumes: ['./configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro', '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads'] ... findface-multi-ui: depends_on: [findface-multi-legacy] image: docker.int.ntl/ntech/multi/multi/ui:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: always volumes: ['./configs/findface-multi-ui/nginx-site.conf:/etc/nginx/conf.d/default.conf:ro', '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads'] ... findface-multi-file-mover: command: [python3, /app/service.py, --config=/etc/config.yml] depends_on: [findface-multi-legacy, nats-jetstream] image: docker.int.ntl/ntech/multi/multi/file-mover:ffmulti-2.1.3 logging: {driver: journald} network_mode: service:pause restart: always volumes: ['./configs/findface-multi-file-mover/findface-multi-file-mover.yaml:/etc/config.yml:ro', '/var/lib/findface-security/uploads:/mnt/slow_storage/uploads'] ...
18.18. To move Tarantool data, do the following:
Stop all FindFace Multi containers:
sudo docker-compose down
Start the old shards and the
findface-sf-api
service again:sudo systemctl start tarantool@shard-00{1..8}.service findface-sf-api.service
Create a new backup of the feature vector database:
sudo mkdir -p /etc/findface_dump_final sudo findface-storage-api-dump -output-dir=/etc/findface_dump_final -config /etc/findface-sf-api.ini
Stop the rest of the services, clear the
instances.enabled
directory, start the containers again, and perform the storage-api-restore operation:sudo systemctl stop tarantool@shard-00{1..8}.service findface-sf-api.service findface-ntls.service sudo rm /etc/tarantool/instances.enabled/* sudo docker-compose up -d sudo findface-storage-api-restore -config /opt/findface-multi/configs/findface-sf-api/findface-sf-api.yaml /etc/findface_dump_final/*.json
The update has been completed, but the new version includes new neural network models, so it is also necessary to migrate feature vectors to a different neural network model, or you can use old neural network models by moving them from the /usr/share/findface-data/models/
directory to the /opt/findface-multi/models/
directory, specifying them in the /opt/findface-multi/configs/findface-extraction-api/findface-extraction-api.yaml
file.
Note that object detection models in FindFace Multi 1.2 are stored in separate directories, i.e., face detection models are stored in the /usr/share/findface-data/models/facedet/
directory, car and body detection models are stored in the /usr/share/findface-data/models/cadet/
and /usr/share/findface-data/models/pedet/
directories respectively. In FindFace Multi 2.1.3 all object detection models can be found in the /opt/findface-multi/models/detector/
directory. When moving old neural network models from the /usr/share/findface-data/models/
directory to the /opt/findface-multi/models/
directory, make sure to place all object detection models (facedet
, cadet
, pedet
) to the /opt/findface-multi/models/detector/
directory.
See for reference:
$ ls -lash /usr/share/findface-data/models
total 52K
4.0K drwxr-xr-x 13 root root 4.0K Jul 15 14:48 .
4.0K drwxr-xr-x 3 root root 4.0K Jul 15 14:48 ..
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 cadet
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 carattr
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 carnorm
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 carrec
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 face
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 faceattr
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 facedet
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 facenorm
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 pedattr
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 pedet
4.0K drwxr-xr-x 2 root root 4.0K Jul 15 14:48 pedrec
$ ls -lash /opt/findface-multi/models/
total 44K
4.0K drwxr-xr-x 11 root root 4.0K Jul 17 13:37 .
4.0K drwxr-xr-x 6 root root 4.0K Jul 19 15:31 ..
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 13:37 carattr
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 13:37 carnorm
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 13:36 carrec
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 16:20 detector
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 16:24 face
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 16:24 faceattr
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 13:37 facenorm
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 13:37 pedattr
4.0K drwxr-xr-x 2 root root 4.0K Jul 17 13:37 pedrec
Important
We highly recommend disabling the Ubuntu automatic update to preserve the FindFace Multi compatibility with the installation environment. In this case, you will be able to update your OS manually, fully controlling which packages to update.
To disable the Ubuntu automatic update, execute the following commands:
sudo apt-get remove unattended-upgrades
sudo systemctl stop apt-daily.timer
sudo systemctl disable apt-daily.timer
sudo systemctl disable apt-daily.service
sudo systemctl daemon-reload
Update FindFace Multi 2.1 / 2.1.0.1 to FindFace Multi 2.1.3
No matter whether you are updating from FindFace Multi 2.1 or 2.1.0.1 to FindFace Multi 2.1.3, the update process will be the same. Do the following:
In a user home directory, create a folder for FindFace Multi 2.1 backup data.
mkdir ~/backup_data/
Back up FindFace Multi configuration files and the
docker-compose.yaml
file and switch to the/opt/findface-multi/
directory.sudo tar -cvzf ~/backup_data/configs.tar.gz -C /opt/findface-multi/ configs sudo cp /opt/findface-multi/docker-compose.yaml ~/backup_data/ cd /opt/findface-multi/
Back up PostgreSQL data.
sudo docker-compose cp postgresql:/bitnami/postgresql ~/backup_data/pg_bkp
Back up Tarantool data.
sudo docker exec -it findface-multi-findface-sf-api-1 bash -c "mkdir ffmulti_dump; cd ffmulti_dump && /storage-api-dump -config /etc/findface-sf-api.ini" sudo docker cp findface-multi-findface-sf-api-1:/ffmulti_dump ~/backup_data/
List content of the
backup_data/
folder, located in a user home directory. It will look like this.ls ~/backup_data/ configs.tar.gz docker-compose.yaml ffmulti_dump pg_bkp
Stop all FindFace Multi containers.
sudo docker-compose down
Remove the
/opt/findface-multi/data/findface-tarantool-server/
and/opt/findface-multi/data/postgresql/
folders.sudo rm -r /opt/findface-multi/data/findface-tarantool-server/ sudo rm -r /opt/findface-multi/data/postgresql/
Install the FindFace Multi 2.1.3 instance into the
/opt/findface-multi
directory.Restore Tarantool data from the backup.
Copy the
~/backup_data/ffmulti_dump/
folder to thefindface-multi-findface-sf-api-1
container.sudo docker cp ~/backup_data/ffmulti_dump/ findface-multi-findface-sf-api-1:/
Restore Tarantool data.
sudo docker exec -it findface-multi-findface-sf-api-1 bash -c 'cd ffmulti_dump && for x in *.json; do /storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done;'
In the new configuration files, replace the
NTECH USER
password with the one, used in FindFace Multi 2.1. The old password can be obtained from theconfigs/postgresql/40-init.sql
file in the backup archiveconfigs.tar.gz
. TheNTECH USER
password should be replaced in the following configuration files:findface-multi-legacy.py
findface-multi-line-crossing-analytics.yaml
findface-multi-identity-provider.py
findface-multi-audit.py
pgbouncer -> userlist.txt
postgresql -> 40-init.sql
findface-counter.yaml
In the new configuration file
/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py
, replace theSECRET_KEY
value in theGENERAL SETTINGS
section and'VIDEO_DETECTOR_TOKEN'
value in theFINDFACE SECURITY SETTINGS
section with the ones, used in FindFace Multi 2.1. The old parameter values can be obtained from theconfigs/findface-multi-legacy/findface-multi-legacy.py
file in the backup archiveconfigs.tar.gz
.Compare previous configuration files against the new ones. Move all custom parameters from the previous version to the new one. Compare configuration files for the services
findface-ntls
,findface-extract-api
,findface-sf-api
, etc. You may need to apply changes to thedocker-compose.yaml
file if you previously configured it to bring or exclude services or made any other changes.Important
Follow these rules to transfer parameters from the old configuration file to the new one:
If there is a new neural network model in the new configuration file, replace it with the previous one (considering that the previous model is still included in FindFace Multi 2.1.3), and if the previous model is missing in FindFace Multi 2.1.3, then do not change anything. In this case, you will have to do migration to a different neural network model.
If a parameter had an empty value in the old configuration file, but has a certain value in the new configuration file, delete its value in the new configuration file.
Keep as is those parameters that were not included in the old configuration file, but are present in the new configuration file.
Stop all FindFace Multi containers from the
/opt/findface-multi
directory.cd /opt/findface-multi sudo docker-compose down
Copy data from the
~/backup_data/pg_bkp/data/
directory to the/opt/findface-multi/data/postgresql
directory.sudo cp -r ~/backup_data/pg_bkp/data/ /opt/findface-multi/data/postgresql
Start all FindFace Multi containers.
sudo docker-compose up -d
Note
You can additionally deploy Video Recorder. See Deploy Video Recorder Step-by-Step.