.. _distributed-dossiers:

Distributed Dossier Database
======================================================

In a distributed architecture, it is often necessary to have the dossier database distributed among several hosts. 

In the current implementation of FindFace Multi, the distributed dossier database is implemented as follows. The dossier database is available for editing only on the principal server known as Leader. It is in sync with several additional FindFace Multi instances that serve as Subscribers. On the Subscribers, the dossier database is available only for reading and monitoring.

.. important::
   You will be able to delete dossiers on the Subscribers if the Leader is unavailable.  

.. important::
   If a watch list on the future Subscriber already contains dossiers, synchronization will be canceled. Make sure that the watch list is empty. 

.. warning::
   Neural networks on the Leader and Subscribers must be identical. 



.. rubric:: In this section:

.. contents::
   :local:
 
Configure Leader/Subscribers Synchronization
-----------------------------------------------
 
To configure Leader/Subscribers synchronization, do the following:

#. On the Leader, open the ``/etc/findface-security/config.py`` configuration file. Come up with a synchronization token and specify it in the ``SYNC_TOKEN`` parameter (be sure to uncomment it prior). 

   .. code::

      sudo vi /etc/findface-security/config.py
      
      ...

      # ========== DossierLists sync ===========
      ...
      # token must be identical on master and slave
      # use pwgen -s 64 1
      SYNC_TOKEN = 'ABC_123456789'
      ...

#. On the Subscriber(s), uncomment the ``SYNC_TOKEN`` parameter in the ``/etc/findface-security/config.py`` configuration file and paste the created synchronization token into it. The tokens on the Leader and Subscribers must be identical.


The Leader/Subscriber sync is now set and will be enabled once you configure a watch list replication from the Leader to Subscriber(s).


Replicate Watch List from Leader to Subscribers
-------------------------------------------------------

To replicate a watch list from the Leader to a Subscriber, send a POST request to the Subscriber with the following parameters in the body:

* ``remote_dossier_list``: id of the original watch list on the Leader
* ``remote_url``: Leader URL
* ``slave_dossier_list``: id of the watch list on the Subscriber, which is to be a replica of the original watch list

.. code::

   POST /sync/dossier-lists/
   {remote_dossier_list: 1,
   remote_url: "http://172.17.46.14",
   slave_dossier_list: 3}

  
Set Synchronization Time
-------------------------------

To schedule dossier synchronization, do the following:

#. Open the ``/etc/findface-security/config.py`` configuration file on the Leader.

   .. code::

      sudo vi /etc/findface-security/config.py

#. Uncomment and set the following parameters: 

   * ``SYNC_SCHEDULE``: recurrence rule (RRULE) that defines the sync schedule.

     .. tip::

        See the RRULE calculator `here <https://jkbrzt.github.io/rrule/>`_.
 
   * ``SYNC_AT_STARTUP``: if ``True``, synchronization occurs on the FindFace Multi startup and restart.
   * ``SYNC_AT_CREATION``: if ``True``, synchronization immediately occurs after you set up synchronization for a watch list.

   .. code::

      ...

      # ========== DossierLists sync ===========
      ...

      # rrule that defines sync schedule
      SYNC_SCHEDULE = 'RRULE:FREQ=DAILY;WKST=MO;BYHOUR=11;BYMINUTE=0'
      # if True synchronization will occur on FindFace Security startup and restart
      SYNC_AT_STARTUP = True
      # if True synchronization will occur immediately after creating synchronization for dossier list
      SYNC_AT_CREATION = True

#. Uncomment the mentioned above parameters on each Subscriber. The parameter values can be arbitrary.



Cancel Watch List Replication and Synchronization
-----------------------------------------------------

To cancel a watch list replication and synchronization, send the following API request to the Subscriber containing the ``{id}`` of the watch list on the Subscriber:

.. code::

   DELETE /sync/dossier-lists/{id}/


Duplicate Functionality to Web Interface
----------------------------------------------

By default, you can enable and disable watch list replication only via API. To make the functionality available in the web interface as well, do the following:

#. Open the ``/etc/findface-security/config.py`` configuration file on the Leader.

   .. code::

      sudo vi /etc/findface-security/config.py

#. Enable the ``ffsecurity_sync`` plugin by uncommenting the line ``INSTALLED_APPS.append('ffsecurity_sync')`` into the plugins section:

   .. code::

      ...

      # ========== DossierLists sync ===========
      INSTALLED_APPS.append('ffsecurity_sync')
      ...

#. Do the same on each Subscriber.
#. On each host, migrate the main database architecture from FindFace Multi to :program:`PostgreSQL`. Re-create user groups in the main database. Restart the ``findface-security`` service. 

   .. code::

      sudo findface-security migrate
      sudo findface-security create_groups
      sudo systemctl restart findface-security.service