.. _backup-restore:


Back Up and Recover Data Storages 
========================================

This section is all about the backup and recovery of the FindFace Multi data storages, which are the following:

* Tarantool-based feature vector database that stores object feature vectors and identification events.
* Main system database ``ffsecurity`` based on PostgreSQL, that stores internal system data, dossiers, user accounts, and camera settings. 
* Directory ``/var/lib/findface-security/uploads`` that stores uploaded dossier photos, video files, full frames of events and counters, and object thumbnails.
* Directory ``/var/lib/ffupload/`` that stores only such event artifacts as normalized object images. 

.. rubric:: In this section:

.. contents::
   :local:


Feature Vector Database Backup and Recovery
---------------------------------------------

There are the following galleries in the Tarantool-based feature vector database:

* ``ffsec_body_events``: feature vectors extracted from bodies detected in the video.
* ``ffsec_body_objects``: feature vectors extracted from bodies in dossier photos.
* ``ffsec_car_events``: feature vectors extracted from cars detected in the video.
* ``ffsec_car_objects``: feature vectors extracted from cars in dossier photos.
* ``ffsec_face_events``: feature vectors extracted from faces detected in the video.
* ``ffsec_face_objects``: feature vectors extracted from faces in dossier photos.
* ``ffsec_user_face``: feature vectors extracted from photos of FindFace Multi users, used for face-based authentication.
* ``ffsec_persons``: centroids of persons (virtual feature vectors averaged across all person's faces) and metadata.


The database backup/recovery functionality allows you to fully restore all the galleries when needed. 

To avoid data loss, we recommend you back up a feature vector database at least once a week. Overall, the backups' frequency depends on the number of dossiers and object recognition events, and available disk space.

Be sure to back up the database before :ref:`migrating <migrate>` your system to another neural network model.


Utilities
^^^^^^^^^^^^^^^^^

To back up and recover the FindFace Multi feature vector database, the following utilities are needed:

#. backup: ``findface-storage-api-dump``,
#. recovery: ``findface-storage-api-restore``.

These utilities are automatically installed along with ``findface-sf-api``.


Back Up Database
^^^^^^^^^^^^^^^^^^^^^^^^

To back up the feature vector database, use the ``findface-storage-api-dump`` utility as follows:

.. important::
   The following services must be active: ``findface-tarantool-server``, ``findface-sf-api``.

.. note::
   The backup functionality can be applied to a distributed database. In this case, the ``findface-storage-api-dump`` utility will back up galleries on all the shards specified in ``/etc/findface-sf-api.ini``.   

#. On the server with ``findface-sf-api``, create a directory to store the backup files (``/etc/findface_dump`` in the example below).
#. Launch the ``findface-storage-api-dump`` utility by executing: 

   .. code::

      sudo findface-storage-api-dump -output-dir=/etc/findface_dump -config /etc/findface-sf-api.ini

   The utility will back up at once all the galleries into the files with corresponding names (``ffsec_body_events.json``, ``ffsec_face_events``, etc.) and save them into the directory. These files contain all the data needed to restore the entire database. 

Recover Database
^^^^^^^^^^^^^^^^^^^^^^^^^^

To recover the feature vector database from the backup, launch the ``findface-storage-api-restore`` utility for all the files in the backup folder: 

.. code::
 
   sudo findface-storage-api-restore -config /etc/findface-sf-api.ini /etc/findface_dump/*.json

The recovery process can be interrupted and resumed whenever necessary. To resume the process after the interruption, launch the ``findface-storage-api-restore`` utility again.



.. seealso::

   * :ref:`backup-config`
   * :ref:`restore-config`


Main Database Backup and Recovery
----------------------------------------

To back up the main database ``ffsecurity`` based on PostgreSQL, execute:

.. code::

   sudo -u postgres pg_dump ffsecurity > ffsecurity_postgres_backup.sql

To recover the main database, do the following:

#. Stop the ``findface-security`` service.

   .. code::

      sudo systemctl stop findface-security.service

#. Stop the ``pgbouncer`` service to delete its active sessions with the ``ffsecurity`` database.

   .. code::

      sudo systemctl stop pgbouncer.service 

#. Open the PostgreSQL interactive terminal.

   .. code::

      sudo -u postgres psql

#. Remove the old ``ffsecurity`` database.

   .. code::

      DROP DATABASE ffsecurity;

#. Create a new ``ffsecurity`` database. Leave the PostgreSQL interactive terminal.

   .. code::

      CREATE DATABASE ffsecurity WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='C.UTF-8' LC_CTYPE='C.UTF-8' TEMPLATE template0;

#. Start the ``pgbouncer`` service.

   .. code::

      sudo systemctl start pgbouncer.service 

#. Recover the database content from the backup.

   .. code::

      sudo -u postgres psql -d ffsecurity -f ffsecurity_postgres_backup.sql

#. Migrate the database architecture from FindFace Multi to :program:`PostgreSQL`, re-create user groups with :ref:`predefined <users>` rights and the first user with administrator rights. 

   .. code::

      sudo findface-security migrate
      sudo findface-security create_groups
      sudo findface-security create_default_user


#. Start the ``findface-security`` service.

   .. code::

      sudo systemctl start findface-security.service




Artifacts Backup and Recovery
----------------------------------

The FindFace Multi artifacts, such as uploaded dossier photos, video files, and such event artifacts as full frames, object thumbnails, and normalized object images, are stored in the following directories:

* ``/var/lib/findface-security/uploads``
* ``/var/lib/ffupload/``

To back up the artifacts, execute:

.. code::

   sudo tar -cvzf /home/some_directory/var_lib_ffsecurity_uploads.tar.gz /var/lib/findface-security/uploads/
   sudo tar -cvzf /home/some_directory/var_lib_ffupload.tar.gz /var/lib/ffupload/

To recover the artifacts, execute the following commands from the root directory:

.. code::

   cd / 
   sudo tar -xvf /home/some_directory/var_lib_ffsecurity_uploads.tar.gz
   sudo tar -xvf /home/some_directory/var_lib_ffupload.tar.gz