.. _step-by-step: Deploy Step-by-Step from Repository --------------------------------------------------------------- This section will guide you through the FindFace Multi step-by-step deployment process. Follow the instructions below minding the sequence. .. rubric:: In this section: .. contents:: :local: .. _install-repo: Install APT Repository ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ First of all, install the FindFace apt repository as follows: #. Download the installer file ``findface-multi-1.1-and-server-5.1.run``. #. Put the ``.run`` file into some directory on the designated host (for example, ``/home/username``). #. From this directory, make the ``.run`` file executable. .. code:: chmod +x findface-multi-1.1-and-server-5.1.run #. Execute the ``.run`` file. .. code:: sudo ./findface-multi-1.1-and-server-5.1.run The installer will ask you a few questions and perform several automated checks to ensure that the host meets the system requirements. Fill out the prompts appropriately once requested. The questions and answers are the following: #. Product to install: ``FindFace Multi``. #. Installation type: ``repo: Don't install anything, just set up the APT repository``. #. Neural network models to install if necessary. To select a model(s), deselect all those on the list by entering ``-*`` in the command line, then select the required models by entering their sequence numbers (keyword): for example, ``1 3 4``. Enter done to save your selection and proceed to another step. .. important:: You must install at least one face biometry model. After that, the FindFace apt repository will be automatically installed. .. _prerequisites: Prerequisites ^^^^^^^^^^^^^^^^^^^^^ FindFace Multi requires such third-party software as PostgreSQL, Pgbouncer, NATS, etcd, and memcached. Do the following: #. Install the prerequisite packages as such: .. code:: sudo apt update sudo apt install -y postgresql-10 nats-server etcd memcached pgbouncer #. Open the ``/etc/memcached.conf`` configuration file. Set the maximum memory in megabytes to use for ``memcached`` items: ``-m 1024``. Set the maximum item size: ``-I 16m``. If one or both of these parameters are absent, add them to the file. .. code:: sudo vi /etc/memcached.conf -m 1024 -I 16m #. Give a strong password to the ``ntech`` user (``9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3`` in the example below). Output the credentials to the ``pgbouncer`` user list. .. code:: echo '"ntech" "9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3"' | sudo tee -a /etc/pgbouncer/userlist.txt #. Configure ``pgbouncer``. In ``/etc/pgbouncer/pgbouncer.ini``, add the ``ffsecurity`` database to the ``databases`` section. Configure named parameters, as shown in the example below. Parameters other than those must be commented out. .. code:: sudo vi /etc/pgbouncer/pgbouncer.ini [databases] ffsecurity = dbname=ffsecurity host=localhost port=5432 user=ntech [pgbouncer] pidfile = /var/run/postgresql/pgbouncer.pid listen_addr = 127.0.0.1 listen_port = 5439 unix_socket_dir = /var/run/postgresql auth_type = plain auth_file = /etc/pgbouncer/userlist.txt pool_mode = transaction server_reset_query = DISCARD ALL max_client_conn = 16384 default_pool_size = 20 syslog = 1 #. Enable the prerequisite services autostart and launch the services: .. code:: sudo systemctl enable postgresql@10-main.service nats-server etcd.service memcached.service pgbouncer.service sudo systemctl restart postgresql@10-main.service nats-server etcd.service memcached.service pgbouncer.service .. _licensing: Provide Licensing ^^^^^^^^^^^^^^^^^^^^^^^^ .. important:: See :ref:`licensing-principles` to learn about the NtechLab licensing policy. To provide the FindFace Multi licensing, deploy ``findface-ntls``, license server in the FindFace core. .. important:: There must be only one ``findface-ntls`` instance in each ``FindFace Multi`` installation. .. code:: sudo apt update sudo apt install -y findface-ntls sudo systemctl enable findface-ntls.service && sudo systemctl start findface-ntls.service Deploy Main Database ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In FindFace Multi, the main system database is based on PostgreSQL. To deploy the main database, do the following: #. Open the ``pgbouncer`` list of users ``/etc/pgbouncer/userlist.txt``. Copy the ``ntech`` user's password (``9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3`` in the example below). .. code:: sudo cat /etc/pgbouncer/userlist.txt "ntech" "9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3" #. Using the :program:`PostgreSQL` console, create a new user ``ntech`` with the copied password, and databases ``ffsecurity`` and ``ffcounter`` in PostgreSQL. .. code:: sudo -u postgres psql postgres=# CREATE ROLE ntech WITH LOGIN PASSWORD '9T3g1nXy9yx3y8MIGm9fbef3dia8UTc3'; postgres=# CREATE DATABASE ffsecurity WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE template0; postgres=# CREATE DATABASE ffcounter WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='C.UTF-8' LC_CTYPE='C.UTF-8' TEMPLATE template0; .. tip:: To quit from the :program:`PostgreSQL` console, type ``\q`` and press :kbd:`Enter`. #. Allow authentication by UID of a socket client in :program:`PostgreSQL`. Restart :program:`PostgreSQL`. .. code:: echo 'local all ntech peer' | sudo tee -a /etc/postgresql/10/main/pg_hba.conf sudo systemctl restart postgresql@10-main.service .. _findface-extraction-api: Deploy FindFace Core ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To deploy the FindFace core, do the following: .. tip:: You can find the description of the FindFace core components and their configuration parameters in :ref:`architecture` and :ref:`components`. #. For FindFace Multi on GPU, :ref:`install NVIDIA drivers `. .. important:: Be sure the restart the server after the NVIDIA drivers installation is complete. Otherwise, the subsequent installation of the GPU-based components experiences a failure. #. Install the FindFace core components: .. code:: sudo apt update sudo apt install -y findface-tarantool-server findface-extraction-api findface-sf-api findface-upload findface-video-manager findface-video-worker-cpu findface-liveness-api .. note:: To install the GPU-accelerated ``findface-extraction-api`` component, use ``findface-extraction-api-gpu`` instead of ``findface-extraction-api`` in the command. .. note:: To install the GPU-accelerated ``findface-video-worker`` component, use ``findface-video-worker-gpu`` instead of ``findface-video-worker-cpu`` in the command. If you have several video cards on your server, see :ref:`video-cards`. .. important:: Be sure to :ref:`manually install ` neural network models on the host(s) with ``findface-extraction-api``. #. In the ``/etc/findface-sf-api.ini`` configuration file, enable the ``allow-return-facen`` parameter. .. code:: sudo vi /etc/findface-sf-api.ini ... limits: ... allow-return-facen: true ... #. Open the ``/etc/findface-video-worker-cpu.ini`` (``/etc/findface-video-worker-gpu.ini``) configuration file. Specify the following parameters: * In the ``mgr-static`` parameter, specify the ``findface-video-manager`` host IP address, which provides ``findface-video-worker`` with settings and the video stream list. * In the ``capacity`` parameter, specify the maximum number of video streams to be processed by ``findface-video-worker``. * In the ``streamer`` section, specify the IP address and port to access the :ref:`Video Wall `. The streamer ``port`` must be set to ``18999``. Set ``tracks = true`` to improve how the object bboxes are displayed on the Video Wall. .. code:: sudo vi /etc/findface-video-worker-cpu.ini sudo vi /etc/findface-video-worker-gpu.ini mgr-static=127.0.0.1:18811 capacity=10 [streamer] #------------------------------ ## streamer/shots webserver port, 0=disabled ## type:number env:CFG_STREAMER_PORT longopt:--streamer-port port = 18999 ## streamer url - how to access this worker on streamer_port ## type:string env:CFG_STREAMER_URL longopt:--streamer-url url = 127.0.0.1:18999 ## use tracks instead detects for streamer ## type:bool env:CFG_STREAMER_TRACKS longopt:--streamer-tracks tracks = true #. Enable the FindFace core services autostart and launch the services. .. code:: sudo systemctl enable findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-cpu findface-liveness-api sudo systemctl start findface-extraction-api findface-sf-api findface-video-manager findface-video-worker-cpu findface-liveness-api .. note: The same commands for ``findface-video-worker-gpu`` are as such: .. code:: sudo systemctl enable findface-video-worker-gpu sudo systemctl start findface-video-worker-gpu Deploy FindFace Multi Application Module and Feature Vector Database ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To deploy the FindFace Multi application module, do the following: #. Install the ``findface-security``, ``findface-security-ui``, and ``findface-counter`` components. Enable the ``findface-counter`` autostart and launch the service. .. code:: sudo apt update sudo apt install -y findface-security findface-security-ui findface-counter sudo systemctl enable findface-counter && sudo systemctl start findface-counter #. Migrate the database architecture from FindFace Multi to :program:`PostgreSQL`, create user groups with :ref:`predefined ` rights and the first user with administrator rights (a.k.a. Super Administrator). .. important:: Super Administrator cannot be deprived of its rights, whatever the role. .. code:: sudo findface-security migrate sudo findface-security create_groups sudo findface-security create_default_user #. Create a structure of the Tarantool-based feature vector database by executing the command below. .. code:: sudo findface-security make_tnt_schema | sudo tee /etc/findface-security/tnt_schema.lua #. Open the ``/etc/tarantool/instances.available/FindFace.lua`` configuration file. Check whether it contains the ``dofile`` command, ``meta_indexes`` and ``meta_scheme`` definitions, as in the example below. .. code:: sudo vi /etc/tarantool/instances.available/FindFace.lua dofile("/etc/findface-security/tnt_schema.lua") -- host:port to bind, HTTP API FindFace = require("FindFace") FindFace.start("127.0.0.1", 8101, { license_ntls_server="127.0.0.1:3133", meta_indexes=meta_indexes, meta_scheme = meta_scheme }) .. important:: The IP address and port number specified in the ``shards`` section of the ``/etc/findface-sf-api.ini`` configuration file must be identical to those in the ``FindFace.start`` section. .. code:: sudo vi /etc/tarantool/instances.available/FindFace.lua ... FindFace.start("127.0.0.1", 8101...) .. code:: sudo vi /etc/findface-sf-api.ini storage-api: ... shards: - master: http://127.0.0.1:8101/v2/ ... .. important:: If you change the ``/etc/findface-sf-api.ini`` configuration file, be sure to restart the ``findface-sf-api`` service: .. code:: sudo systemctl restart findface-sf-api.service #. Enable the ``findface-tarantool-server`` service autostart and launch the service. .. code:: sudo systemctl enable tarantool@FindFace.service && sudo systemctl start tarantool@FindFace.service #. Open the ``/etc/findface-security/config.py`` configuration file. Specify the following parameters: .. tip:: You can find the ``/etc/findface-security/config.py`` default version :download:`here <_scripts/findface-security.config.py.txt>`. * ``SERVICE_EXTERNAL_ADDRESS``: FindFace Multi IP address or URL prioritized for the Genetec integration and webhooks. If this parameter is not specified, the system will be using ``EXTERNAL_ADDRESS`` for these purposes. .. important:: To use Genetec and webhooks, be sure to specify at least one of these parameters: ``SERVICE_EXTERNAL_ADDRESS``/``EXTERNAL_ADDRESS``. * ``EXTERNAL_ADDRESS``: (Optional) IP address or URL used to access the FindFace Multi web interface. If this parameter is not manually set, the system auto-detects it as the external IP address of the host. .. note:: To access FindFace Multi, you can use both the auto-detected and manually set IP addresses. * ``VIDEO_DETECTOR_TOKEN``: to authorize the video object detection module, come up with a token and specify it here. * ``VIDEO_MANAGER_ADDRESS``: IP address of the ``findface-video-manager`` host. * ``NTLS_HTTP_URL``: IP address of the ``findface-ntls`` host. * ``ROUTER_URL``: IP address of the ``findface-security`` host that will receive detected objects from the ``findface-video-worker`` instance(s). Specify either external or internal IP address, subject to the network through which ``findface-video-worker`` interacts with ``findface-security``. Change the default port, subject to the :ref:`redirect settings ` from HTTP to HTTPS, or omit it leaving only the IP address. * ``SF_API_ADDRESS``: IP address of the ``findface-sf-api`` host. * ``DATABASES`` (section): fill it in as such: ``'PORT': 5439, 'USER': 'ntech', 'PASSWORD': ''`` (see :ref:`prerequisites`). .. tip:: If necessary, ensure data security by enabling :ref:`SSL `. .. tip:: If necessary, set ``’IGNORE_UNMATCHED’: True`` to disable logging events for the objects that have no match with the dossiers (negative verification result). Enable this option if the system has to process a large number of objects. #. Generate a signature key for the session encryption (used by Django) by executing the command below. Specify this key as ``SECRET_KEY``. .. code:: pwgen -sncy 50 1|tr "'" "." #. Start the services. .. code:: sudo systemctl enable findface-security sudo systemctl start findface-security #. Disable the default nginx server and add the ``findface-security`` server to the list of enabled servers. Restart nginx. .. code:: sudo rm /etc/nginx/sites-enabled/default sudo ln -s /etc/nginx/sites-available/ffsecurity-nginx.conf /etc/nginx/sites-enabled/ sudo nginx -s reload #. Provide licensing: * Use the FindFace Multi main web interface to :ref:`upload the license file ` you have prior received from your manager (:menuselection:`Preferences -> License`). * For the on-premise licensing via a USB dongle, insert it into a USB port. * For the on-premise licensing via hardware fingerprint, refer to :ref:`sentinel`. .. important:: To log in for the first time, use the default Super Administrator account ``admin:admin``. .. note:: To create more users or change the Super Administrator password, refer to :ref:`users`. .. important:: To preserve the FindFace Multi compatibility with the installation environment, we highly recommend you to disable the Ubuntu automatic update. In this case, you will be able to update your OS manually, fully controlling which packages to update. To disable the Ubuntu automatic update, execute the following commands: .. code:: sudo apt-get remove unattended-upgrades sudo systemctl stop apt-daily.timer sudo systemctl disable apt-daily.timer sudo systemctl disable apt-daily.service sudo systemctl daemon-reload .. important:: The FindFace Multi services log a large amount of data, which can eventually lead to disc overload. To prevent this from happening, we advise you to disable ``rsyslog`` due to its suboptimal log rotation scheme and use the appropriately configured ``systemd-journal`` service instead. See :ref:`logs` for the step-by-step instructions.