Step-by-Step Deployment from Repository

This section will guide you through the FindFace step-by-step deployment process. Follow the instructions below minding the sequence.

Tip

Be sure to learn the FindFace architecture first.

In this section:

Install APT Repository

First of all, install the FindFace apt repository as follows:

  1. Download the installer file findface-*.run.

  2. Put the .run file into some directory on the designated host (for example, /home/username).

  3. From this directory, make the .run file executable.

    Note

    Be sure to specify the actual file name instead of findface-*.

    chmod +x findface-*.run
    
  4. Execute the .run file.

    sudo ./findface-*.run
    

    The installer will ask you a few questions and perform several automated checks to ensure that the host meets the system requirements. After filling out each prompt, press Enter. The questions and answers are the following:

    1. Q: Which product should be installed?

      A: 1

      Which product should be installed?
      
        1. [security]  FindFace Multi
        2. [server  ]  FindFace Server
        3. [video-worker]  FindFace Video Worker
        4. [nvidia-drivers]  NVIDIA CUDA drivers (installed automatically when you install gpu-variant of the products above)
      
      (default: security)
      product> 1
      
    2. Q: Please choose installation type:

      A: 3

      Please choose installation type:
      
      -  1 [stand-alone ]  Single Server
      -  2 [multi-worker]  Single Server, Multiple video workers
      -  3 [repo        ]  Don't install anything, just set up the APT repository
      -  4 [custom      ]  Fully customized installation
      
      (default: stand-alone)
      type> 3
      
    3. Q: APT repository doesn't include recognition models. Do you want to install them now?(y/n)

      A: y or n, subject to your deployment plan. Whatever the case, keep in mind to install necessary neural network models along with the findface-extraction-api component. To install them later, refer to Installation of Neural Network Models.

      APT repository doesn't include recognition models. Do you want to install them now?(y/n)
      (default: yes)
      repo_data> y
      
    4. Q: Select models to install (if you entered y on the previous step)

      A: By default, all neural network models are subject to installation. You can leave it as is by entering done, or select specific models. To do so, deselect all those on the list by entering -* in the command line, then select the required models by entering their sequence numbers (keyword): for example, 1 3 4. Enter done to save your selection and proceed to another step.

      Select models to install.
      Note that you will need to accordingly edit extraction-api and tntapi configuration files.
      At least one of recognition models has to be enabled.
      
      -  1 [v] ./findface-data-age.v2-cpu_3.0.0_all.deb
      ...
      ...
      - 68 [v] ./findface-data-quality.v1-gpu_3.0.0_all.deb
      
      
      Enter keyword to select matching choices or -keyword to clear selection.
      Enter "done" to save your selection and proceed to another step.
      findface-data.models> done
      

    After that, the FindFace apt repository will be automatically installed.

Prerequisites

FindFace requires such third-party software as PostgreSQL, Pgbouncer, NATS, etcd, and memcached. Do the following:

  1. Install the prerequisite packages as such:

    sudo apt update
    sudo apt install -y postgresql-10 nats-server etcd memcached pgbouncer
    
  2. Open the /etc/memcached.conf configuration file. Set the maximum memory in megabytes to use for memcached items: -m 1024. Set the maximum item size: -I 16m. If one or both of these parameters are absent, add them to the file.

    sudo vi /etc/memcached.conf
    
    -m 1024
    -I 16m
    
  3. Give a strong password to the ntech user (9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3 in the example below). Output the credentials to the pgbouncer user list.

    echo '"ntech" "9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3"' | sudo tee -a /etc/pgbouncer/userlist.txt
    
  4. Configure pgbouncer. In /etc/pgbouncer/pgbouncer.ini, paste the following content instead of the existing one, as shown in the example below.

    sudo vi /etc/pgbouncer/pgbouncer.ini
    
    [databases]
    ffsecurity = dbname=ffsecurity host=localhost port=5432 user=ntech
    ffsecurity_session = dbname=ffsecurity host=localhost port=5432 user=ntech pool_mode=session pool_size=10
    [pgbouncer]
    pidfile = /var/run/postgresql/pgbouncer.pid
    listen_addr = 127.0.0.1
    listen_port = 5439
    unix_socket_dir = /var/run/postgresql
    auth_type = plain
    auth_file = /etc/pgbouncer/userlist.txt
    pool_mode = transaction
    server_reset_query = DISCARD ALL
    max_client_conn = 16384
    default_pool_size = 70
    syslog = 1
    log_connections = 0
    log_disconnections = 0
    stats_period = 300
    
  5. Enable the prerequisite services autostart on boot and re-launch the services:

    sudo systemctl enable postgresql@10-main.service nats-server etcd.service memcached.service pgbouncer.service
    sudo systemctl restart postgresql@10-main.service nats-server etcd.service memcached.service pgbouncer.service
    

Deploy License Server

Important

See Licensing to learn about the NtechLab licensing policy.

To provide the FindFace licensing, deploy findface-ntls, license server in the FindFace core.

Important

There must be only one findface-ntls instance in each FindFace installation.

sudo apt update
sudo apt install -y findface-ntls
sudo systemctl enable findface-ntls.service && sudo systemctl start findface-ntls.service

Deploy Main Database

In FindFace, the main system database is based on PostgreSQL. To deploy the main database, do the following:

  1. Open the pgbouncer list of users /etc/pgbouncer/userlist.txt. Copy the ntech user’s password (9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3 in the example below).

    sudo cat /etc/pgbouncer/userlist.txt
    
    "ntech" "9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3"
    
  2. Using the PostgreSQL console, create a new user ntech with the copied password, and databases ffsecurity and ffcounter in PostgreSQL.

    sudo -u postgres psql
    
    postgres=# CREATE ROLE ntech WITH LOGIN PASSWORD '9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3';
    
    postgres=# CREATE DATABASE ffsecurity WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE template0;
    
    postgres=# CREATE DATABASE ffcounter WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='C.UTF-8' LC_CTYPE='C.UTF-8' TEMPLATE template0;
    

    Tip

    To quit from the PostgreSQL console, type \q and press Enter.

  3. Allow authentication by UID of a socket client in PostgreSQL. Restart PostgreSQL.

    echo 'local all ntech peer' | sudo tee -a /etc/postgresql/10/main/pg_hba.conf
    
    sudo systemctl restart postgresql@10-main.service
    

Deploy FindFace Core

To deploy the FindFace core, do the following:

Tip

You can find the description of the FindFace core components in Architecture.

  1. For FindFace on GPU, install NVIDIA drivers.

    Important

    Be sure the restart the server after the NVIDIA drivers installation is complete. Otherwise, the subsequent installation of the GPU-based components will experience a failure.

  2. Install the FindFace core components:

    On CPU:

    sudo apt update
    sudo apt install -y findface-tarantool-server findface-extraction-api findface-sf-api findface-upload findface-video-manager findface-video-worker-cpu
    

    On GPU:

    sudo apt update
    sudo apt install -y findface-tarantool-server findface-extraction-api-gpu findface-sf-api findface-upload findface-video-manager findface-video-worker-gpu
    

    Note

    If you have several video cards on your server, see Multiple Video Cards Usage.

    Important

    Keep in mind to manually install neural network models on the host(s) with findface-extraction-api later.

  3. In the /etc/findface-extraction-api.ini configuration file, switch the neural network model for face recognition to mango_320.

    On CPU:

    sudo vi /etc/findface-extraction-api.ini
    
    extractors:
        ...
      models:
        ...
        face_emben: face/mango_320.cpu.fnk
        ...
    

    On GPU:

    sudo vi /etc/findface-extraction-api.ini
    
    extractors:
        ...
      models:
        ...
        face_emben: face/mango_320.gpu.fnk
        ...
    
  4. In the /etc/findface-sf-api.ini configuration file, enable the allow-return-facen parameter.

    sudo vi /etc/findface-sf-api.ini
    
    ...
    limits:
      ...
      allow-return-facen: true
    ...
    
  5. Open the /etc/findface-video-worker-cpu.ini (/etc/findface-video-worker-gpu.ini) configuration file. Specify the following parameters:

    • In the mgr-static parameter, specify the findface-video-manager host IP address, which provides findface-video-worker with settings and the video stream list.

    • In the capacity parameter, specify the maximum number of video streams to be processed by findface-video-worker.

    sudo vi /etc/findface-video-worker-cpu.ini
    sudo vi /etc/findface-video-worker-gpu.ini
    
    mgr-static=127.0.0.1:18811
    
    capacity=10
    
    [streamer]
    #------------------------------
    ## streamer/shots webserver port, 0=disabled
    ## type:number env:CFG_STREAMER_PORT longopt:--streamer-port
    port = 18999
    
    ## streamer url - how to access this worker on streamer_port
    ## type:string env:CFG_STREAMER_URL longopt:--streamer-url
    url = 127.0.0.1:18999
    
  6. Enable the FindFace core services autostart and launch the services.

    On CPU:

    sudo systemctl enable findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-cpu
    sudo systemctl start findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-cpu
    

    On GPU:

    sudo systemctl enable findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-gpu
    sudo systemctl start findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-gpu
    

Deploy FindFace Application Module and Feature Vector Database

To deploy the FindFace application module, do the following:

  1. Install the findface-security, findface-security-ui, and findface-counter components. Enable the findface-counter autostart and launch the service.

    sudo apt update
    sudo apt install -y findface-security findface-security-ui findface-counter
    sudo systemctl enable findface-counter && sudo systemctl start findface-counter
    
  2. Create a structure of the Tarantool-based feature vector database by executing the command below.

    sudo findface-security make_tnt_schema | sudo tee /etc/findface-security/tnt_schema.lua
    
  3. Open the /etc/tarantool/instances.available/FindFace.lua configuration file. Check whether it contains the dofile command and the spaces definition, as in the example below.

    sudo vi /etc/tarantool/instances.available/FindFace.lua
    
    dofile("/etc/findface-security/tnt_schema.lua")
    
    
    -- host:port to bind, HTTP API
    FindFace = require("FindFace")
    FindFace.start("127.0.0.1", 8101, {
    license_ntls_server="127.0.0.1:3133",
    replication = replication_master,
    spaces = spaces
    })
    

    Important

    The IP address and port number specified in the shards section of the /etc/findface-sf-api.ini configuration file must be identical to those in the FindFace.start section.

    sudo vi /etc/tarantool/instances.available/FindFace.lua
    
    ...
    
    FindFace.start("127.0.0.1", 8101...)
    
    sudo vi /etc/findface-sf-api.ini
    
    storage-api:
      ...
      shards:
      - master: http://127.0.0.1:8101/v2/
      ...
    

    Important

    If you change the /etc/findface-sf-api.ini configuration file, be sure to restart the findface-sf-api service:

    sudo systemctl restart findface-sf-api.service
    
  4. Enable the findface-tarantool-server service autostart and launch the service.

    sudo systemctl enable tarantool@FindFace.service && sudo systemctl start tarantool@FindFace.service
    
  5. Open the /etc/findface-security/config.py configuration file. Specify the following parameters:

    Tip

    You can find the /etc/findface-security/config.py default version here.

    • EXTERNAL_ADDRESS: (Optional) IP address or URL used to access the FindFace web interface. If this parameter is not manually set, the system auto-detects it as the external IP address of the host.

      Note

      To access FindFace, you can use both the auto-detected and manually set IP addresses.

    • VIDEO_DETECTOR_TOKEN: to authorize the video object detection module, come up with a token and specify it here.

      Tip

      It’s a good idea to generate a token by executing:

      pwgen -sncy 50 1|tr "'" "."
      
    • VIDEO_MANAGER_ADDRESS: IP address of the findface-video-manager host.

    • NTLS_HTTP_URL: IP address of the findface-ntls host.

    • ROUTER_URL: IP address of the findface-security host that will receive detected objects from the findface-video-worker instance(s). Specify either external or internal IP address, subject to the network through which findface-video-worker interacts with findface-security. Change the default port, subject to the redirect settings from HTTP to HTTPS, or omit it leaving only the IP address.

    • SF_API_ADDRESS: IP address of the findface-sf-api host.

    • DATABASES (section): fill it in as such: 'PORT': 5439, 'USER': 'ntech', 'PASSWORD': '<password from /etc/pgbouncer/userlist.txt>' (see Prerequisites).

    Tip

    If necessary, ensure data security by enabling SSL.

  6. Generate a signature key for the session encryption (used by Django) by executing the command below. Specify this key as SECRET_KEY.

    pwgen -sncy 50 1|tr "'" "."
    
  7. Migrate the database architecture from FindFace to PostgreSQL, create predefined user roles and the first administrator (a.k.a. Super Administrator).

    Important

    The Super Administrator cannot be deprived of its rights, whatever the role.

    sudo findface-security migrate
    sudo findface-security create_groups
    sudo findface-security create_default_user
    
  8. Start the services.

    sudo systemctl enable findface-security
    sudo systemctl start findface-security
    
  9. Disable the default nginx server and add the findface-security server to the list of enabled servers. Restart nginx.

    sudo rm /etc/nginx/sites-enabled/default
    
    sudo ln -s /etc/nginx/sites-available/ffsecurity-nginx.conf /etc/nginx/sites-enabled/
    
    sudo nginx -s reload
    
  10. Provide licensing:

    Important

    To log in for the first time, use the default Super Administrator account admin:admin.

Note

To create more users or change the Super Administrator password, refer to Role and User Management.

Important

To preserve the FindFace compatibility with the installation environment, we highly recommend you to disable the Ubuntu automatic update. In this case, you will be able to update your OS manually, fully controlling which packages to update.

To disable the Ubuntu automatic update, execute the following commands:

sudo apt-get remove unattended-upgrades
sudo systemctl stop apt-daily.timer
sudo systemctl disable apt-daily.timer
sudo systemctl disable apt-daily.service
sudo systemctl daemon-reload

Important

The FindFace services log a large amount of data, which can eventually lead to disc overload. To prevent this from happening, we advise you to disable rsyslog due to its suboptimal log rotation scheme and use the appropriately configured systemd-journal service instead. See Service Logs for the step-by-step instructions.