.. _backup-restore:


Backup and Restore Data Storages 
========================================

This section is all about backup and restore of the FindFace Security data storages, which are the following:

* Tarantool-based biometric database that stores biometric samples (feature vectors) and face identification events.
* Main system database based on PostgreSQL, that stores internal system data, dossiers, user accounts, and camera settings. 
* Directory ``/var/lib/findface-security/uploads`` that stores uploaded dossier photos, video files, and such event artifacts as full frames, face thumbnails, and normalized face images.
* Directory ``/var/lib/ffupload/`` that stores only such event artifacts as face thumbnails. 

.. rubric:: In this section:

.. contents::
   :local:


Biometric Database Backup and Restore
----------------------------------------

There are 3 galleries in the Tarantool-based biometric database:

* ``ffsec_dossier_face``: biometric samples extracted from dossier photos.
* ``ffsec_events``: biometric samples extracted from faces detected in the video.
* ``ffsec_persons``: centroids of persons (virtual biometric samples averaged across all person's faces) and metadata.

The database backup/restore functionality allows you to fully restore all the galleries when needed. 

To avoid data loss, we recommend you to create a biometric database backup at least once a week. Overall, the frequency of backups depends on the number of dossiers and face recognition events, as well as available disk space.

Be sure to backup the database before :ref:`migrating <migrate>` your system to another biometric model.


Utilities
^^^^^^^^^^^^^^^^^

To backup and restore the FindFace Security biometric database, the following utilities are needed:

#. backup: ``findface-storage-api-dump``,
#. restore: ``findface-storage-api-restore``.

These utilities are automatically installed along with ``findface-sf-api``.


Backup Database
^^^^^^^^^^^^^^^^^^^^^^^^

To backup the biometric database, use the ``findface-storage-api-dump`` utility as follows:

.. important::
   The following services have to be active: ``findface-tarantool-server``, ``findface-sf-api``.

.. note::
   The backup functionality can be applied to a distributed database. In this case, the ``findface-storage-api-dump`` utility will backup galleries on all the shards specified in ``/etc/findface-sf-api.ini``.   

#. Backup Tarantool.

   .. code::

      sudo cp -r /etc/tarantool /home/<some_directory>
      sudo cp -r /opt/ntech/var/lib/tarantool/ /home/<some_directory>/shardsbackup

#. On the server with ``findface-sf-api``, create a directory to store the backup files (``/etc/findface_dump`` in the example below).
#. Launch the ``findface-storage-api-dump`` utility by executing: 

   .. code::

      sudo findface-storage-api-dump -output-dir=/etc/findface_dump -config /etc/findface-sf-api.ini

   The utility will backup at once all the galleries into the files with corresponding names ``ffsec_dossier_face.json``, ``ffsec_events.json``, ``ffsec_persons.json``, and save them into the directory. These files contain all the data needed to restore the entire database. 

Restore Database
^^^^^^^^^^^^^^^^^^^^^^^^^^

To restore the biometric database from the backup, do the following:

#. Use HTTP API to create initial galleries in the database: ``ffsec_dossier_face``, ``ffsec_events``, ``ffsec_persons``.

   .. tip::
      See :ref:`api`.

   .. code::

      curl -D - -X POST -s 'http://localhost:8101/v2/galleries/add/ffsec_dossier_face'
      curl -D - -X POST -s 'http://localhost:8101/v2/galleries/add/ffsec_events'
      curl -D - -X POST -s 'http://localhost:8101/v2/galleries/add/ffsec_persons'

#. Launch the ``findface-storage-api-restore`` utility for all the files in the backup folder: 

   .. code::
 
      sudo findface-storage-api-restore -config /etc/findface-sf-api.ini /etc/findface_dump/*.json

   The restore process can be interrupted and resumed when necessary. To resume the process after interruption, simply launch the ``findface-storage-api-restore`` utility again.



.. seealso::

   * :ref:`backup-config`
   * :ref:`restore-config`


Main Database Backup and Restore
----------------------------------------

To backup the PostgreSQL database, execute:

.. code::

   sudo -u postgres pg_dump ffsecurity > ffsecurity_postgres_backup.sql

To restore the PostgreSQL database, execute:

.. important::
   For successful recovery, make sure to have at hand an empty database ``ffsecurity``.

.. code::

   sudo -u postgres psql -d ffsecurity -f ffsecurity_postgres_backup.sql





Artifacts Backup
-----------------------------

The FindFace Security artifacts, such as uploaded dossier photos, video files, and such event artifacts as full frames, face thumbnails, and normalized face images, are stored in the following directories:

* ``/var/lib/findface-security/uploads``
* ``/var/lib/ffupload/``

To backup the artifacts, execute:

.. code::

   tar -cvzf var_lib_ffsecurity_uploads.tar.gz /var/lib/findface-security/uploads
   tar -cvzf var_lib_ffupload.tar.gz /var/lib/ffupload/