Back Up and Recover Data Storages

This section is all about the backup and recovery of the FindFace data storages, which are the following:

Tarantool-based biometric database that stores biometric samples (feature vectors) and face identification events.
Main system database ffsecurity based on PostgreSQL, that stores internal system data, dossiers, user accounts, and camera settings.
Directory /var/lib/findface-security/uploads that stores uploaded dossier photos, video files, and such event artifacts as full frames, face thumbnails, and normalized face images.
Directory /var/lib/ffupload/ that stores only such event artifacts as face thumbnails.

In this section:

Biometric Database Backup and Recovery
Main Database Backup and Recovery
Artifacts Backup and Recovery

Biometric Database Backup and Recovery 

There are 3 galleries in the Tarantool-based biometric database:

ffsec_dossier_face: biometric samples extracted from dossier photos.
ffsec_events: biometric samples extracted from faces detected in the video.
ffsec_persons: centroids of persons (virtual biometric samples averaged across all person’s faces) and metadata.

The database backup/recovery functionality allows you to fully restore all the galleries when needed.

To avoid data loss, we recommend you back up a biometric database at least once a week. Overall, the backups’ frequency depends on the number of dossiers and face recognition events, and available disk space.

Be sure to back up the database before migrating your system to another biometric model.

Utilities 

To back up and recover the FindFace biometric database, the following utilities are needed:

backup: findface-storage-api-dump,
recovery: findface-storage-api-restore.

These utilities are automatically installed along with findface-sf-api.

Back Up Database 

To back up the biometric database, use the findface-storage-api-dump utility as follows:

Important

The following services must be active: findface-tarantool-server, findface-sf-api.

Note

The backup functionality can be applied to a distributed database. In this case, the findface-storage-api-dump utility will back up galleries on all the shards specified in /etc/findface-sf-api.ini.

On the server with findface-sf-api, create a directory to store the backup files (/etc/findface_dump in the example below).
Launch the findface-storage-api-dump utility by executing:
```
sudo findface-storage-api-dump -output-dir=/etc/findface_dump -config /etc/findface-sf-api.ini
```
The utility will back up at once all the galleries into the files with corresponding names ffsec_dossier_face.json, ffsec_events.json, ffsec_persons.json, and save them into the directory. These files contain all the data needed to restore the entire database.

Recover Database 

To recover the biometric database from the backup, launch the findface-storage-api-restore utility for all the files in the backup folder:

sudo findface-storage-api-restore -config /etc/findface-sf-api.ini /etc/findface_dump/*.json

The recovery process can be interrupted and resumed whenever necessary. To resume the process after the interruption, launch the findface-storage-api-restore utility again.

Main Database Backup and Recovery 

To back up the main database ffsecurity based on PostgreSQL, execute:

sudo -u postgres pg_dump ffsecurity > ffsecurity_postgres_backup.sql

To recover the main database, do the following:

Stop the findface-security service.

sudo systemctl stop findface-security.service

Stop the pgbouncer service to delete its active sessions with the ffsecurity database.
```
sudo systemctl stop pgbouncer.service
```
Open the PostgreSQL interactive terminal.
```
sudo -u postgres psql
```
Remove the old ffsecurity database.
```
DROP DATABASE ffsecurity;
```

Create a new ffsecurity database. Leave the PostgreSQL interactive terminal.

CREATE DATABASE ffsecurity WITH OWNER ntech ENCODING 'UTF-8' LC_COLLATE='C.UTF-8' LC_CTYPE='C.UTF-8' TEMPLATE template0;

Start the pgbouncer service.
```
sudo systemctl start pgbouncer.service
```

Recover the database content from the backup.

sudo -u postgres psql -d ffsecurity -f ffsecurity_postgres_backup.sql

Migrate the database architecture from FindFace to PostgreSQL, re-create user groups with predefined rights and the first user with administrator rights.
```
sudo findface-security migrate
sudo findface-security create_groups
sudo findface-security create_default_user
```

Start the findface-security service.

sudo systemctl start findface-security.service

Artifacts Backup and Recovery 

The FindFace artifacts, such as uploaded dossier photos, video files, and such event artifacts as full frames, face thumbnails, and normalized face images, are stored in the following directories:

/var/lib/findface-security/uploads
/var/lib/ffupload/

To back up the artifacts, execute:

sudo tar -cvzf /home/some_directory/var_lib_ffsecurity_uploads.tar.gz /var/lib/findface-security/uploads/
sudo tar -cvzf /home/some_directory/var_lib_ffupload.tar.gz /var/lib/ffupload/

To recover the artifacts, execute the following commands from the root directory:

cd /
sudo tar -xvf /home/some_directory/var_lib_ffsecurity_uploads.tar.gz
sudo tar -xvf /home/some_directory/var_lib_ffupload.tar.gz

Back Up and Recover Data Storages

Biometric Database Backup and Recovery

Utilities

Back Up Database

Recover Database

Main Database Backup and Recovery

Artifacts Backup and Recovery