Distributed Dossier Database

In a distributed architecture, it is often necessary to have the dossier database distributed among several hosts.

In the current implementation, the dossier database is available for editing only on the principal server known as master. It is in sync with several additional FindFace instances that serve as slaves. On the slaves, the dossier database is available only for reading and monitoring.

Important

You will be able to delete dossiers on the slaves if the master is unavailable.

Important

If a watch list on the future slave already contains dossiers, they will be replaced with those from the master during the first synchronization. All information in the previous dossiers will be lost.

Warning

Neural networks on the master and slaves must be identical.

In this section:

Configure Master/Slave Synchronization

To configure master/slave synchronization, do the following:

  1. On the master, open the /etc/findface-security/config.py configuration file. Come up with a synchronization token and specify it in the SYNC_TOKEN parameter (be sure to uncomment it prior).

    sudo vi /etc/findface-security/config.py

...

# ========== DossierLists sync ===========
...
# token must be identical on master and slave
# use pwgen -s 64 1
SYNC_TOKEN = 'ABC_123456789'
...

  2. On the slave(s), uncomment the SYNC_TOKEN parameter in the /etc/findface-security/config.py configuration file and paste the created synchronization token into it. The tokens on the master and slaves must be identical.

The master/slave sync is now set and will be enabled once you configure a watch list replication from the master to slave(s).

Replicate Watch List from Master to Slaves

To replicate a watch list from the master to slave instances, send a POST request to the slave with the following parameters in the body:

  • remote_dossier_list: id of the original watch list on the master

  • remote_url: master URL

  • slave_dossier_list: id of the watch list on the slave, which is to be a replica of the original watch list

POST /sync/dossier-lists/
{remote_dossier_list: 1,
remote_url: "http://172.17.46.14",
slave_dossier_list: 3}

Set Synchronization Time

To schedule dossier synchronization, do the following:

  1. Open the /etc/findface-security/config.py configuration file on the master.

    sudo vi /etc/findface-security/config.py

  2. Uncomment and set the following parameters:

    • SYNC_SCHEDULE: recurrence rule (RRULE) that defines the sync schedule.

      Tip

      See the RRULE calculator here.

    • SYNC_AT_STARTUP: if True, synchronization occurs on the FindFace startup and restart.

    • SYNC_AT_CREATION: if True, synchronization immediately occurs after you set up synchronization for a watch list.

    ...

# ========== DossierLists sync ===========
...

# rrule that defines sync schedule
SYNC_SCHEDULE = 'RRULE:FREQ=DAILY;WKST=MO;BYHOUR=11;BYMINUTE=0'
# if True synchronization will occur on FindFace startup and restart
SYNC_AT_STARTUP = True
# if True synchronization will occur immediately after creating synchronization for dossier list
SYNC_AT_CREATION = True

  3. Uncomment the mentioned above parameters on each slave. The parameter values can be arbitrary.

Cancel Watch List Replication and Synchronization

To cancel a watch list replication and synchronization, send the following API request to the slave with the {id} of the watch list on the slave:

DELETE /sync/dossier-lists/{id}/

Duplicate Functionality to Web Interface

By default, you can enable and disable watch list replication only via API. To make the functionality available in the web interface as well, do the following:

  1. Open the /etc/findface-security/config.py configuration file on the master.

    sudo vi /etc/findface-security/config.py

  2. Enable the ffsecurity_sync plugin by uncommenting the line INSTALLED_APPS.append('ffsecurity_sync') into the plugins section:

    ...

# ========== DossierLists sync ===========
INSTALLED_APPS.append('ffsecurity_sync')
...

  3. Do the same on each slave.

  4. On each host, migrate the main database architecture from FindFace to PostgreSQL. Re-create user groups in the main database. Restart the findface-security service.

    sudo findface-security migrate
sudo findface-security create_groups
sudo systemctl restart findface-security.service