Update FindFace Security to 4.2

Warning

Since version 4.1, FindFace Security utilizes a new version of Tarantool. The biometric database from previous FindFace Security versions (4.0 and earlier) is NOT COMPATIBLE with FindFace Security 4.2. Be sure to use the Backup/Restore functionality to update the product to 4.2 (see the full algorithm below).

To update FindFace Security from any previous version to 4.2, do the following:

  1. Open the findface-security configuration file. Save the values of the following parameters for later use: EXTERNAL_ADDRESS, SECRET_KEY, VIDEO_DETECTOR_TOKEN, ROUTER_URL.

    sudo vi /etc/ffsecurity/config.py
    
    EXTERNAL_ADDRESS = "http://172.20.77.58"
    
    ...
    # use pwgen -sncy 50 1|tr "'" "." to generate your own unique key
    SECRET_KEY = 'c8b533847bbf7142102de1349d33a1f6'
    
    FFSECURITY = {
       'VIDEO_DETECTOR_TOKEN': '381b0f4a20495227d04185ab02f5085f',
       ...
       'ROUTER_URL': 'http://172.20.77.58',
       ...
    }
    
  2. Stop the findface-security service.

    sudo systemctl stop findface-security*
    
  3. Create a backup of the Tarantool-based biometric database in any directory of your choice, for example, /tmp/dump.

    Tip

    See Backup and Restore Data Storages for details.

    mkdir -p /tmp/dump
    cd /tmp/dump
    sudo findface-storage-api-dump -config /etc/findface-sf-api.ini
    
  4. Install the apt repository with the new FindFace Security, using the console installer as described in this section.

  5. Install the pgbouncer package as such:

    sudo apt update
    sudo apt install -y pgbouncer
    
  6. Give a strong password to the ntech user (9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3 in the example below). Output the credentials to the pgbouncer user list.

    echo '"ntech" "9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3"' | sudo tee -a /etc/pgbouncer/userlist.txt
    
  7. Configure pgbouncer. In /etc/pgbouncer/pgbouncer.ini, add ffsecurity to the databases section. Configure other parameters, as shown in the example below.

    sudo vi /etc/pgbouncer/pgbouncer.ini
    [databases]
    ffsecurity = dbname=ffsecurity host=localhost port=5432 user=ntech
    ...
    ; ip address or * which means all ip-s
    listen_addr = 127.0.0.1
    listen_port = 5439
    ...
    ; any, trust, plain, crypt, md5
    auth_type = plain
    ...
    ; When server connection is released back to pool:
    ;   session      - after client disconnects
    ;   transaction  - after transaction finishes
    ;   statement    - after statement finishes
    pool_mode = transaction
    ...
    ; total number of clients that can connect
    max_client_conn = 16384
    ...
    ;; Syslog settings
    syslog = 1
    ;syslog_facility = daemon
    ;syslog_ident = pgbouncer
    
  8. Copy the password of the ntech user (9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3 in the example). In PostgreSQL, set the copied password for the ntech role. Open the PostgreSQL interactive terminal. You will see the line postgres=# appear. After the # sign, enter the following command: ALTER ROLE ntech PASSWORD '<copied password>'.

    sudo -u postgres psql
    
    postgres=# ALTER ROLE ntech PASSWORD '9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3';
    
  9. Using the PostgreSQL interactive terminal, create a database ffcounter in PostgreSQL.

    postgres=# CREATE DATABASE ffcounter WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='C.UTF-8' LC_CTYPE='C.UTF-8' TEMPLATE template0;
    
  10. Create and configure pgbouncer.service.

    sudo touch /etc/systemd/system/pgbouncer.service
    sudo vi /etc/systemd/system/pgbouncer.service
    

    Insert the following code:

    [Unit]
    Description=Pgbouncer service
    After=postgresql.service
    Before=findface-security.service
    [Service]
    User=postgres
    Group=postgres
    ExecStart=/usr/sbin/pgbouncer "/etc/pgbouncer/pgbouncer.ini"
    [Install]
    WantedBy=multi-user.target
    
  11. Enable the pgbouncer.service autostart and restart it:

    sudo systemctl enable pgbouncer.service
    sudo systemctl restart pgbouncer.service
    
  12. Install the new FindFace Security services from the repository, following your architecture outline.

    CPU-version:

    sudo apt update
    sudo apt install ffsecurity ffsecurity-ui findface-extraction-api findface-ntls findface-sf-api findface-tarantool-server findface-upload findface-video-manager findface-video-worker-cpu findface-counter
    

    GPU-version:

    sudo apt update
    sudo apt install ffsecurity ffsecurity-ui findface-extraction-api-gpu findface-ntls findface-sf-api findface-tarantool-server findface-upload findface-video-manager findface-video-worker-gpu findface-gpudetector-data findface-counter
    

    Important

    FindFace Security 4.2 on GPU requires the nvidia-440 and cuda 10.2 drivers. Make sure the right driver versions are installed.

    Important

    At some moment, you will be prompted to choose which version of the findface-security configuration file to keep. Opt for Install the packages maintainer’s version.

  13. Enable the findface-counter service autostart.

    sudo systemctl enable findface-counter
    
  14. Open the findface-security configuration file and paste the saved EXTERNAL_ADDRESS, SECRET_KEY, VIDEO_DETECTOR_TOKEN, and ROUTER_URL into it. Fill in the DATABASES section by analogy: 'PORT': 5439, 'USER': 'ntech', 'PASSWORD': '9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3' (password from /etc/pgbouncer/userlist.txt).

    sudo vi /etc/ffsecurity/config.py
    ...
    # Database is used by FindFace Security to store cameras,
    # camera groups, watchlists and so on. Only PostgreSQL is supported.
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.postgresql',
            'DISABLE_SERVER_SIDE_CURSORS': True,
            'NAME': 'ffsecurity',
            'PORT': 5439, 'USER': 'ntech', 'PASSWORD': '9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3',
        }
    }
    ...
    # Use pwgen -sncy 50 1|tr "'" "." to generate your own unique key
    SECRET_KEY = '002231ccb690586f4d33e98322c591bb'
    ...
    SERVICE_EXTERNAL_ADDRESS = 'http://172.20.77.58'
    # EXTERNAL_ADDRESS is used to access objects created inside FFSecurity via external links.
    EXTERNAL_ADDRESS = 'http://172.20.77.58'
    ...
        # findface-video-worker authorization token
        'VIDEO_DETECTOR_TOKEN': '8977e1b0067d43f6c908d0bf60363255',
    ...
        # findface-video-worker face posting address,
        # it must be set to either FFSecurity EXTERNAL_ADDRESS (by default)
        # or findface-facerouter url (in some specific cases)
        'ROUTER_URL': 'http://127.0.0.1:80',
    
  15. Modify the Tarantool database structure by applying the tnt_schema.lua file from the new version.

    sudo findface-security make_tnt_schema | sudo tee /etc/ffsecurity/tnt_schema.lua
    
  16. Stop the findface-tarantool-server shards. Remove the Tarantool database (default database or shards).

    sudo systemctl stop 'tarantool@*'
    
    sudo rm -R /opt/ntech/var/lib/tarantool/shard-00*/index/*
    sudo rm -R /opt/ntech/var/lib/tarantool/shard-00*/snapshots/*
    sudo rm -R /opt/ntech/var/lib/tarantool/shard-00*/xlogs/*
    
  17. Restart the findface-tarantool-server shards.

    TNT=$(ls /etc/tarantool/instances.enabled/ | wc -l)
    for i in $(seq 1 $TNT); do sudo systemctl start tarantool@shard-00$i.service ; done
    
  18. Restart the services.

    sudo systemctl restart findface-security.service
    sudo systemctl restart findface-ntls findface-extraction-api findface-video-worker* findface-video-manager findface-sf-api findface-counter
    
  19. Restore the Tarantool database from the backup.

    cd /tmp/dump
    
    for x in *.json; do sudo findface-storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done
    
  20. Migrate the main database architecture from FindFace Security to PostgreSQL, re-create user groups with predefined rights and the first user with administrator rights.

    sudo findface-security migrate
    sudo findface-security create_groups
    sudo findface-security create_default_user
    
  21. Restart PostgreSQL.

    On Ubuntu 16.04:

    sudo systemctl restart postgresql@9.5-main.service
    

    On Ubuntu 18.04:

    sudo systemctl restart postgresql@10-main.service