Update to FindFace CIBR 2.1.2.1

Update FindFace CIBR 1.3 to FindFace CIBR 2.1.2.1

To update FindFace CIBR 1.3 to FindFace CIBR 2.1.2.1, do the following:

  1. Create a backup copy of the old schema of the Tarantool-based feature vector database:

    sudo cp /etc/findface-security/tnt_schema.lua /etc/findface-security/old_tnt_schema.lua
    

    Starting from version 2.1.1, one of the most significant differences between FindFace CIBR and earlier versions of the product is the structure of the Tarantool biometric database (so called meta-schema). The new structure is created as a set of spaces, while in previous versions of the product there was only one space by default in the structure of the Tarantool-based database.

  2. Open the /etc/findface-security/config.py configuration file. Save the values of the following parameters for later use: EXTERNAL_ADDRESS, SECRET_KEY, VIDEO_DETECTOR_TOKEN, ROUTER_URL, CUSTOM_FIELDS.

    sudo vi /etc/findface-security/config.py
    
    EXTERNAL_ADDRESS = "http://172.20.77.58"
    ...
    # use pwgen -sncy 50 1|tr "'" "." to generate your own unique key
    SECRET_KEY = 'c8b533847bbf7142102de1349d33a1f6'
    FFSECURITY = {
    'VIDEO_DETECTOR_TOKEN': '381b0f4a20495227d04185ab02f5085f',
    ...
    'ROUTER_URL': 'http://172.20.77.58',
    ...
    # -- Custom model fields --
    # Edit CUSTOM_FIELDS -> `human_card` section to customize human card fields.
    # Edit CUSTOM_FIELDS -> `face_object` section to customize face object fields.
    # Below is an example with every field type possible.
    # 'CUSTOM_FIELDS': {
    #     'human_card': {
    #         'items': [
    #             {
    #                 'name': 'personid',
    #                 'default': '',
    #                 'label': 'PersonID',
    #                 'display': ['list', 'form'],
    #                 'description': 'Sigur person ID',
    #                 'editable': False
    #             },
    #             {
    #                 'name': 'firstname',
    #                 'default': '',
    #                 'label': 'First Name',
    #                 'display': ['list', 'form'],
    #                 'description': 'Sigur first name',
    #                 'editable': False
    #             },
    #             {
    #                 'name': 'lastname',
    #                 'default': '',
    #                 'label': 'Last Name',
    #                 'display': ['list', 'form'],
    #                 'description': 'Sigur last name',
    #                 'editable': False
    #             },
    #             {
    #                 'name': 'version',
    #                 'default': '',
    #                 'label': 'Version',
    #                 'display': ['list', 'form'],
    #                 'description': 'Sigur photo version',
    #                 'editable': False
    #             }
    #         ],
    #         'filters': [
    #             {
    #                 'name': 'personid',
    #                 'label': 'Sigur person ID filter',
    #                 'field': 'personid'
    #             }
    #         ]
    #     },
    #     'face_object': {
    #         'items': [
    #             {
    #                 "field_name": "tag_name_1",
    #                 "type": "string",
    #                 "default": "change_me"
    #             },
    #             {
    #                 "field_name": "tag_name_2",
    #                 "type": "uint",
    #                 "default": 123
    #             },
    #             {
    #                 "field_name": "tag_name_3",
    #                 "type": "bool",
    #                 "default": True
    #             },
    #         ]
    #     }
    # },
    }
    
  3. Stop the findface-security service.

    sudo systemctl stop findface-security.service
    
  4. Create a backup of the Tarantool-based feature vector database in any directory of your choice, for example, /etc/findface_dump.

    sudo mkdir -p /etc/findface_dump
    cd /etc/findface_dump
    sudo findface-storage-api-dump -config /etc/findface-sf-api.ini
    
  5. To avoid port conflicts, stop and disable all services before installing a new version.

    Note

    There are eight shards in the example below. If it differs with the number of shards in your system, adjust the below command accordingly. E.g., for the system with sixteen shards, replace tarantool@shard-00{1..8}.service with tarantool@shard-0{01..16}.service. Get the list of active shards with the ls /etc/tarantool/instances.enabled/ command.

    You may stop and disable the services one by one:

    sudo systemctl stop postgresql.service
    sudo systemctl stop postgresql@10-main
    sudo systemctl stop findface-*.service
    sudo systemctl stop pgbouncer.service
    sudo systemctl stop tarantool@shard-00{1..8}.service
    sudo systemctl stop nats-server.service
    sudo systemctl stop etcd.service
    sudo systemctl stop mongod.service
    sudo systemctl stop mongodb.service
    sudo systemctl stop memcached.service
    sudo systemctl stop nginx.service
    
    sudo systemctl disable postgresql.service
    sudo systemctl disable postgresql@10-main
    sudo systemctl disable pgbouncer.service
    sudo systemctl disable findface-extraction-api.service
    sudo systemctl disable findface-security.service
    sudo systemctl disable findface-security-onvif.service
    sudo systemctl disable findface-sf-api.service
    sudo systemctl disable findface-ntls.service
    sudo systemctl disable findface-video-manager.service
    sudo systemctl disable findface-video-worker-cpu.service
    sudo systemctl disable findface-video-worker-gpu.service
    sudo systemctl disable findface-counter.service
    sudo systemctl disable findface-liveness-api.service
    sudo systemctl disable findface-video-streamer-cpu.service
    sudo systemctl disable findface-video-streamer-gpu.service
    sudo systemctl disable findface-video-storage.service
    sudo systemctl disable tarantool@shard-00{1..8}.service
    sudo systemctl disable nats-server.service
    sudo systemctl disable etcd.service
    sudo systemctl disable mongod.service
    sudo systemctl disable mongodb.service
    sudo systemctl disable memcached.service
    sudo systemctl disable nginx.service
    

    Or you may use the following compact commands instead:

    sudo systemctl stop postgresql.service postgresql@10-main findface-*.service pgbouncer.service tarantool@shard-00{1..8}.service nats-server.service etcd.service mongod.service mongodb.service memcached.service nginx.service
    
    sudo systemctl disable postgresql.service postgresql@10-main pgbouncer.service findface-extraction-api.service findface-security.service findface-security-onvif.service findface-sf-api.service findface-ntls.service findface-video-manager.service findface-video-worker-cpu.service findface-video-worker-gpu.service findface-counter.service findface-liveness-api.service findface-video-streamer-cpu.service findface-video-streamer-gpu.service findface-video-storage.service tarantool@shard-00{1..8}.service nats-server.service etcd.service mongod.service mongodb.service memcached.service nginx.service
    
  6. Install the FindFace CIBR 2.1.2.1 instance. Don’t forget to prepare a server first:

    See:

  7. After FindFace CIBR installation, stop all containers from the /opt/findface-cibr/ directory.

    cd /opt/findface-cibr/
    
    sudo docker-compose stop
    
  8. Open the /opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py configuration file and paste saved on step #2 values for the parameters EXTERNAL_ADDRESS, SECRET_KEY, VIDEO_DETECTOR_TOKEN, ROUTER_URL, and CUSTOM_FIELDS into it.

    sudo vi /opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py
    ...
    # Use pwgen -sncy 50 1|tr "'" "." to generate your own unique key
    SECRET_KEY = '002231ccb690586f4d33e98322c591bb'
    ...
    SERVICE_EXTERNAL_ADDRESS = 'http://172.20.77.58'
    # EXTERNAL_ADDRESS is used to access objects created inside FFSecurity via external links.
    EXTERNAL_ADDRESS = 'http://172.20.77.58'
    ...
        # findface-video-worker authorization token
        'VIDEO_DETECTOR_TOKEN': '8977e1b0067d43f6c908d0bf60363255',
    ...
        # findface-video-worker face posting address,
        # it must be set to either FFSecurity EXTERNAL_ADDRESS (by default)
        # or findface-facerouter url (in some specific cases)
        'ROUTER_URL': 'http://127.0.0.1:80',
    
  9. Open the old version of the findface-ntls configuration file available at /etc/findface-ntls.cfg and check it against the new version /opt/findface-cibr/configs/findface-ntls/findface-ntls.yaml. Move all the custom parameters from the old version to the new one. Do the same for other components, e.g., for findface-extract-api, check /etc/findface-extract-api.ini against /opt/findface-cibr/configs/findface-extraction-api/findface-extraction-api.yaml, for findface-sf-api, check /etc/findface-sf-api.ini against /opt/findface-cibr/configs/findface-sf-api/findface-sf-api.yaml, etc.

    sudo vi /etc/findface-ntls.cfg
    sudo vi /opt/findface-cibr/configs/findface-ntls/findface-ntls.yaml
    sudo vi /etc/findface-extraction-api.ini
    sudo vi /opt/findface-cibr/configs/findface-extraction-api/findface-extraction-api.yaml
    sudo vi /etc/findface-sf-api.ini
    sudo vi /opt/findface-cibr/configs/findface-sf-api/findface-sf-api.yaml
    

    Important

    Follow these rules to transfer parameters from the old configuration file to the new one:

    • If a parameter had an empty value in the old configuration file, but has a certain value in the new configuration file, delete its value in the new configuration file.

    • Keep as is those parameters that were not included in the old configuration file, but are present in the new configuration file.

  10. Modify the Tarantool database structure by applying the tnt_schema.lua schema from FindFace CIBR 2.1.2.1.

    sudo docker run --rm --network host --volume '/opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2.1 make-tnt-schema | sudo tee /etc/findface-security/tnt_schema.lua
    
  11. Purge data from all the directories relevant to active shards.

    sudo rm /opt/ntech/var/lib/tarantool/shard-*/{index,snapshots,xlogs}/*
    
  12. Copy the meta-schema of the face_clusters_space space from the old_tnt_schema.lua configuration file to the new tnt_schema.lua configuration file. An easy way to do it is to follow these steps:

    12.1. In the new configuration file /etc/findface-security/tnt_schema.lua, replace the following line at the beginning of the file:

    cfg_spaces = {
    

    with this one:

    spaces = {
    

    12.2. Copy and add face_clusters_space space from the old_tnt_schema.lua old configuration file:

    face_clusters_space = {
        meta_scheme = {
          -- <class 'ffsecurity.entities.cluster.face.models.FaceCluster'>.normalized_id:
          {
              default = '',
              field_type = 'string',
              id = 1,
              name = 'normalized_id',
          },
          -- <class 'ffsecurity.entities.cluster.face.models.FaceCluster'>.feat:
          {
              default = '',
              field_type = 'string',
              id = 2,
              name = 'feat',
          },
          -- <class 'ffsecurity.entities.cluster.face.models.FaceCluster'>.m:nonnormalized_emben:
          {
              default = '',
              field_type = 'string',
              id = 3,
              name = 'm:nonnormalized_emben',
          },
          -- <class 'ffsecurity.entities.cluster.face.models.FaceCluster'>.m:case:
          {
              default = '0',
              field_type = 'unsigned',
              id = 4,
              name = 'm:case',
          },
        },
        meta_indexes = {'m:case'}
      },
    
  13. Remove the default configuration file FindFace.lua, generated by the findface-tarantool-server package, as it will block the restart, required on the next step.

    sudo rm -rf /etc/tarantool/instances*/FindFace.lua
    
  14. Restart the findface-tarantool-server shards.

    TNT=$(ls /etc/tarantool/instances.enabled/ | cut -c 7,8,9)
    for i in $TNT; do sudo systemctl restart tarantool@shard-$i.service ; done
    

    Upon completion of the above steps, the shards will still keep the old galleries created within the default space, but new spaces (e.g., ffsec_face_case_events, ffsec_face_case_clusters and so on) will also become available.

  15. Restore old data from the backup. The data will be restored as it existed previously: all galleries will stay within the default space.

    sudo systemctl start findface-ntls.service
    cd /etc/findface_dump
    for x in *.json; do sudo findface-storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done
    
  16. Start old services findface-sf-api.service, nginx.service, they will be required during PostgreSQL migration.

    sudo systemctl start findface-sf-api.service
    sudo systemctl start nginx.service
    
  17. Perform PostgreSQL database migrations for FindFace CIBR 2.1.2.1 compatibility. Do the following:

    17.1. Navigate to the /opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py configuration file. In the DATABASES -> default section, temporarily replace PASSWORD with the old one, used in the /etc/findface-security/config.py configuration file.

    Important

    Make sure to write down the password from the /opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py configuration file. You will need it later.

    sudo vi /opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py
    
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.postgresql',
            'DISABLE_SERVER_SIDE_CURSORS': True,
            'NAME': 'ffsecurity', 'HOST': '127.0.0.1', 'PORT': 5439, 'USER': 'ntech', 'PASSWORD': 'XXXXXXXXXXXXXXXX'
        }
    }
    

    17.2. On the host system, perform the database migration:

    sudo systemctl start postgresql.service
    sudo systemctl start pgbouncer.service
    sudo docker run --rm --network host --volume '/opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2.1 /opt/findface-security/bin/python3 /tigre_prototype/manage.py migrate
    

    17.3. Back up an existing database with PostgreSQL, installed on the host system.

    cd /opt/findface-cibr/
    sudo -u postgres pg_dump --verbose --disable-triggers ffsecurity | sudo tee dump_ffsecurity.sql
    sudo -u postgres pg_dump -t auth_group -t ffsecurity_adgroupguid -t ffsecurity_deviceblacklistrecord -t ffsecurity_ffsecauthsession -t ffsecurity_grouppermission -t ffsecurity_runtimesetting -t ffsecurity_user -t ffsecurity_user_groups -t ffsecurity_user_user_permissions -t ffsecurity_userkeyvalue -t knox_authtoken -t ffsecurity_watchlistpermission -t ffsecurity_cameragrouppermission -t ffsecurity_casepermission --data-only --verbose --no-acl --no-owner --disable-triggers ffsecurity | sudo tee dump_identity_provider.sql
    

    17.4. Back up role permissions.

    sudo docker run --rm --network host --volume '/opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2.1 /opt/findface-security/bin/python3 /tigre_prototype/manage.py dump_permissions | sudo tee permissions.csv
    

    17.5. Change back the password, replaced in step #17.1

    17.6. Stop all the services.

    sudo systemctl stop findface-sf-api.service nginx.service tarantool@shard-00{1..8}.service postgresql.service pgbouncer.service
    

    17.7. Open the /opt/findface-cibr/docker-compose.yaml file and copy a new POSTGRES_PASSWORD value to use it in further commands.

    cat docker-compose.yaml | grep POSTGRES_PASSWORD
    environment: {POSTGRESQL_ALLOW_REMOTE_CONNECTIONS: 'no', POSTGRES_PASSWORD: {NEW_POSTGRES_PASSWORD}}
    

    17.8. Remove PostgreSQL data. Then, start PostgreSQL by starting the containers and recreate all existing databases.

    sudo rm -rf data/postgresql
    sudo docker-compose up -d --force-recreate postgresql
    

    17.9. Restore data into the recreated ffsecurity database. Paste the {POSTGRES_PASSWORD} value that you previously copied in step #17.7 into the command below:

    sudo docker exec -i findface-cibr-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} psql --username postgres ffsecurity" < dump_ffsecurity.sql
    

    17.10. Run migration.

    sudo docker-compose up -d pgbouncer
    sudo docker run --rm --network host --volume '/opt/findface-cibr/configs/findface-multi-identity-provider/findface-multi-identity-provider.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/identity-provider:ffcibr-2.1.2.1 /opt/findface-security/bin/python3 /tigre_prototype/manage.py migrate
    

    17.11. Restore data into the recreated ffsecurity_identity_provider database. Paste the {POSTGRES_PASSWORD} value that you previously copied in step #17.7 into the command below:

    sudo docker exec -i findface-cibr-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} psql --username postgres ffsecurity_identity_provider" < dump_identity_provider.sql
    

    17.12. Start all the services.

    sudo docker-compose up -d
    

    17.13. Run the command to create records in the outbox table for watch lists, camera groups, and cases.

    sudo docker run --rm --network host --volume '/opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2.1 /opt/findface-security/bin/python3 /tigre_prototype/manage.py export_to_outbox --watchlists --cameragroups --cases
    

    17.14. Before you restore role permissions into the identity_provider service, examine the /opt/findface-cibr/permissions.csv file. Make sure to replace *_ffsecauthtoken with *_authtoken if any. This is mostly applicable to those cases when FindFace CIBR 1.3 installation was an upgrade from earlier versions of the product.

    After that, restore role permissions into the identity_provider service.

    sudo docker run --rm --network host -v /opt/findface-cibr/configs/findface-multi-identity-provider/findface-multi-identity-provider.py:/etc/findface-security/config.py:ro -v $(pwd)/permissions.csv:/var/permissions.csv:ro docker.int.ntl/ntech/multi/multi/identity-provider:ffcibr-2.1.2.1 /opt/findface-security/bin/python3 /tigre_prototype/manage.py load_permissions /var/permissions.csv
    

    17.15. Copy full frame photos, normalized images, and the license file. Copy files from the /opt/ntech/license/ folder into the /opt/findface-cibr/data/findface-ntls/ folder, from the /var/lib/findface-security/uploads/ folder into the /opt/findface-cibr/data/findface-multi-legacy/uploads/ folder, from the /var/lib/ffupload/uploads/ folder into the /opt/findface-cibr/data/findface-upload/uploads/ folder.

    sudo cp -r /opt/ntech/license/* /opt/findface-cibr/data/findface-ntls/
    sudo cp -r /var/lib/findface-security/uploads/* /opt/findface-cibr/data/findface-multi-legacy/uploads/
    sudo cp -r /var/lib/ffupload/uploads/* /opt/findface-cibr/data/findface-upload/uploads/
    sudo chmod 777 -R /opt/findface-cibr/data/findface-upload/uploads/
    sudo chown www-data:www-data -R /opt/findface-cibr/data/findface-upload/uploads/*
    

    Alternatively, the above folders can be directly mounted into the relevant docker containers via the docker-compose.yaml file, like in the example below:

    sudo vi /opt/findface-cibr/docker-compose.yaml
    
    ...
    findface-upload:
      image: docker.int.ntl/ntech/universe/upload:ffserver-9.230407.1
      logging: {driver: journald}
      network_mode: service:pause
      restart: always
      volumes: ['./configs/findface-upload/40-ffupload.sh:/docker-entrypoint.d/40-ffupload.sh:ro',
        '/var/lib/ffupload:/var/lib/ffupload']
    ...
    findface-multi-identity-provider:
      depends_on: [pgbouncer, nats, findface-sf-api, findface-liveness-api, etcd]
      environment: {ADMIN_PASSWORD: <ADMIN_PASSWORD>}
      image: docker.int.ntl/ntech/multi/multi/identity-provider:ffcibr-2.1.2.1
      logging: {driver: journald}
      network_mode: service:pause
      restart: always
      volumes: ['./configs/findface-multi-identity-provider/findface-multi-identity-provider.py:/etc/findface-security/config.py:ro',
        '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads']
    ...
    findface-multi-legacy:
      depends_on: [pgbouncer, nats, findface-sf-api, findface-counter, findface-liveness-api,
        etcd, findface-multi-identity-provider]
      environment: {ADMIN_PASSWORD: <ADMIN_PASSWORD>}
      image: docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2.1
      logging: {driver: journald}
      network_mode: service:pause
      restart: always
      volumes: ['./configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro',
        '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads']
    ...
    findface-multi-ui:
      depends_on: [findface-multi-legacy]
      image: docker.int.ntl/ntech/multi/multi/ui-cibr:ffcibr-2.1.2.1
      logging: {driver: journald}
      network_mode: service:pause
      restart: always
      volumes: ['./configs/findface-multi-ui/nginx-site.conf:/etc/nginx/conf.d/default.conf:ro',
        '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads']
    ...
    cleaner:
      command: [run-cleaner-service]
      depends_on: [pgbouncer, findface-sf-api, findface-multi-legacy]
      image: docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2.1
      logging: {driver: journald}
      network_mode: service:pause
      restart: always
      volumes: ['./configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro',
        '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads']
    ...
    findface-multi-legacy-singleton-services:
      command: [run-single-instance-services]
      depends_on: [findface-multi-identity-provider, findface-multi-legacy]
      image: docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2.1
      logging: {driver: journald}
      network_mode: service:pause
      restart: always
      volumes: ['./configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro',
        '/var/lib/findface-security/uploads:/var/lib/findface-security/uploads']
    

    17.16. To move Tarantool data, do the following:

    Stop all FindFace CIBR containers:

    sudo docker-compose down
    

    Start the old shards and the findface-sf-api service again:

    sudo systemctl start tarantool@shard-00{1..8}.service findface-sf-api.service
    

    Create a new backup of the feature vector database:

    sudo mkdir -p /etc/findface_dump_final
    sudo findface-storage-api-dump -output-dir=/etc/findface_dump_final -config /etc/findface-sf-api.ini
    

    Stop the rest of the services, clear the instances.enabled directory, start the containers again, and perform the storage-api-restore operation:

    sudo systemctl stop tarantool@shard-00{1..8}.service findface-sf-api.service findface-ntls.service
    sudo rm /etc/tarantool/instances.enabled/*
    sudo docker-compose up -d
    sudo findface-storage-api-restore -config /opt/findface-cibr/configs/findface-sf-api/findface-sf-api.yaml /etc/findface_dump_final/*.json
    

The update has been completed. The new version includes new neural network models, but the migration of feature vectors to a different neural network model is not required. You can use old neural network models by moving them from the /usr/share/findface-data/models/ directory to the /opt/findface-cibr/models/ directory, specifying them in the /opt/findface-cibr/configs/findface-extraction-api/findface-extraction-api.yaml file, if they are included in FindFace CIBR 2.1.2.1.

Note that face detection models in FindFace CIBR 1.3 are stored in the /usr/share/findface-data/models/facedet/ directory. In FindFace CIBR 2.1.2.1 all face detection models can be found in the /opt/findface-cibr/models/detector/ directory. When moving old neural network models from the /usr/share/findface-data/models/ directory to the /opt/findface-cibr/models/ directory, make sure to place all face detection models (facedet) to the /opt/findface-cibr/models/detector/ directory.

See for reference:

$ ls -lash /usr/share/findface-data/models
total 52K
4.0K drwxr-xr-x  2 root root 4.0K Jul 15 14:48 face
4.0K drwxr-xr-x  2 root root 4.0K Jul 15 14:48 faceattr
4.0K drwxr-xr-x  2 root root 4.0K Jul 15 14:48 facedet
4.0K drwxr-xr-x  2 root root 4.0K Jul 15 14:48 facenorm

$ ls -lash /opt/findface-cibr/models/
total 44K
4.0K drwxr-xr-x  2 root root 4.0K Jul 17 16:20 detector
4.0K drwxr-xr-x  2 root root 4.0K Jul 17 16:24 face
4.0K drwxr-xr-x  2 root root 4.0K Jul 17 16:24 faceattr
4.0K drwxr-xr-x  2 root root 4.0K Jul 17 13:37 facenorm

Important

We highly recommend disabling the Ubuntu automatic update to preserve the FindFace CIBR compatibility with the installation environment. In this case, you will be able to update your OS manually, fully controlling which packages to update.

To disable the Ubuntu automatic update, execute the following commands:

sudo apt-get remove unattended-upgrades
sudo systemctl stop apt-daily.timer
sudo systemctl disable apt-daily.timer
sudo systemctl disable apt-daily.service
sudo systemctl daemon-reload

Update FindFace CIBR 2.1.2 to FindFace CIBR 2.1.2.1

To update FindFace CIBR 2.1.2 to FindFace CIBR 2.1.2.1, do the following:

  1. In a user home directory, create a folder for FindFace CIBR 2.1.2 backup data.

    mkdir ~/backup_data/
    
  2. Back up FindFace CIBR configuration files and the docker-compose.yaml file and switch to the /opt/findface-cibr/ directory.

    sudo cp -r /opt/findface-cibr/configs/ ~/backup_data/configs/
    sudo cp /opt/findface-cibr/docker-compose.yaml ~/backup_data/
    
    cd /opt/findface-cibr/
    
  3. Back up PostgreSQL data.

    1. Open the /opt/findface-cibr/docker-compose.yaml file and copy POSTGRES_PASSWORD value to use it in further command.

      cat docker-compose.yaml | grep POSTGRES_PASSWORD
      environment: {POSTGRESQL_ALLOW_REMOTE_CONNECTIONS: 'no', POSTGRES_PASSWORD: {POSTGRES_PASSWORD}}
      
    2. Create a folder for PostgreSQL backup data.

      mkdir ~/backup_data/pg_bkp
      
    3. Create backups and paste the {POSTGRES_PASSWORD} value that you previously copied.

      sudo docker exec -i findface-cibr-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} pg_dump --username postgres ffsecurity" > ~/backup_data/pg_bkp/legacy.sql
      sudo docker exec -i findface-cibr-postgresql-1 /bin/bash -c "PGPASSWORD={POSTGRES_PASSWORD} pg_dump --username postgres ffsecurity_identity_provider" > ~/backup_data/pg_bkp/identity_provider.sql
      
  4. Back up Tarantool data.

    sudo docker exec -it findface-cibr-findface-sf-api-1 bash -c "mkdir ffmulti_dump; cd ffmulti_dump && /storage-api-dump -config /etc/findface-sf-api.ini"
    sudo docker cp findface-cibr-findface-sf-api-1:/ffmulti_dump ~/backup_data/
    
  5. List content of the backup_data/ folder, located in a user home directory. It will look like this.

    ls ~/backup_data/
    configs  docker-compose.yaml  ffmulti_dump  pg_bkp
    
  6. Stop all FindFace CIBR containers.

    sudo docker-compose down
    
  7. Remove the /opt/findface-cibr/data/findface-tarantool-server/ and /opt/findface-cibr/data/postgresql/ folders.

    sudo rm -r /opt/findface-cibr/data/findface-tarantool-server/
    
    sudo rm -r /opt/findface-cibr/data/postgresql/
    
  8. Install the FindFace CIBR 2.1.2.1 instance into the /opt/findface-cibr directory.

  9. Restore Tarantool data from the backup.

    1. Copy the ~/backup_data/ffmulti_dump/ folder to the findface-cibr-findface-sf-api-1 container.

      sudo docker cp  ~/backup_data/ffmulti_dump/ findface-cibr-findface-sf-api-1:/
      
    2. Restore Tarantool data.

    sudo docker exec -it findface-cibr-findface-sf-api-1 bash -c 'cd ffmulti_dump && for x in *.json; do /storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done;'
    
  10. In the new configuration file /opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py, replace the SECRET_KEY value in the GENERAL SETTINGS section and 'VIDEO_DETECTOR_TOKEN' value in the FINDFACE SECURITY SETTINGS section with the ones, used in FindFace CIBR 2.1.2. The old parameter values can be obtained from the findface-multi-legacy.py file from the ~/backup_data/configs/ directory.

  11. Compare previous configuration files against the new ones. Move all custom parameters from the previous version to the new one. Compare configuration files for the services findface-ntls, findface-extract-api, findface-sf-api, etc. You may need to apply changes to the docker-compose.yaml file if you previously configured it to bring or exclude services or made any other changes.

    Important

    Follow these rules to transfer parameters from the old configuration file to the new one:

    • If a parameter had an empty value in the old configuration file, but has a certain value in the new configuration file, delete its value in the new configuration file.

    • Keep as is those parameters that were not included in the old configuration file, but are present in the new configuration file.

  12. Stop all FindFace CIBR containers, remove findface-cibr-postgresql-1 container and postgresql data from the /opt/findface-cibr directory.

    cd /opt/findface-cibr
    sudo docker rm -f -v findface-cibr-postgresql-1
    sudo rm -rf data/postgresql
    sudo docker-compose down
    
  13. Restore data from backups.

    1. Recreate postgresql container.

      sudo docker-compose up -d --force-recreate postgresql
      
    2. Open the /opt/findface-cibr/docker-compose.yaml file and copy a new POSTGRES_PASSWORD value to use it in further step. Note that this password will be different from the previous one.

      cat docker-compose.yaml | grep POSTGRES_PASSWORD
      environment: {POSTGRESQL_ALLOW_REMOTE_CONNECTIONS: 'no', POSTGRES_PASSWORD: {NEW_POSTGRES_PASSWORD}}
      
    3. Restore data from backups. Paste the {NEW_POSTGRES_PASSWORD} value that you previously copied.

      sudo docker exec -i  findface-cibr-postgresql-1  bash -c "PGPASSWORD={NEW_POSTGRES_PASSWORD} psql --username postgres ffsecurity" < <(cat ~/backup_data/pg_bkp/legacy.sql)
      sudo docker exec -i  findface-cibr-postgresql-1  bash -c "PGPASSWORD={NEW_POSTGRES_PASSWORD} psql --username postgres ffsecurity_identity_provider" < <(cat ~/backup_data/pg_bkp/identity_provider.sql)
      
  14. Start all FindFace CIBR containers.

    sudo docker-compose up -d