.. _migrate:

Migrate Feature Vectors to Different Neural Network Model
==============================================================

.. tip::
   Do not hesitate to contact our experts on migration by support@ntechlab.com.

.. important::
   In the case, you are doing migration as part of the system update to a newer version, :ref:`complete the update <update>` first. Only after that can you proceed to the migration. 

This section is about how to migrate object feature vectors to another neural network model.

Do the following:

#. Create a backup of the Tarantool-based feature vector database in any directory of your choice, for example, ``/etc/findface_dump``. 

   .. tip::
      See :ref:`backup-restore` for details.
 
   .. code::

      sudo mkdir -p /etc/findface_dump
      sudo cd /etc/findface_dump
      sudo findface-storage-api-dump -config /etc/findface-sf-api.ini


#. Stop the ``findface-sf-api`` service.

   .. code::

      sudo systemctl stop findface-sf-api.service

#. Create new shards that will host regenerated feature vectors. 

   #. Open the ``/etc/tarantool/instances.available/`` directory and find out the number of shards by counting the number of configuration files ``shard-*.lua``. 

      .. note::
         There are four shards in the example below. 
    
      .. code::

         cd /etc/tarantool/instances.available/

         ls -l

         shard-001.lua
         shard-002.lua
         shard-003.lua
         shard-004.lua

   #. Create the same number of new shards by copying the configuration files ``shard-*.lua``. 

      .. note::
         For convenience, the second digit in the new names is ``1``: ``shard-01*.lua``.

      .. code::

         sudo cp shard-001.lua shard-011.lua
         sudo cp shard-002.lua shard-012.lua
         sudo cp shard-003.lua shard-013.lua
         sudo cp shard-004.lua shard-014.lua 

   #. Modify the following lines in each new shard's configuration file, subject to its name (``shard-011``, ``shard-012``, etc., in our example):

         +----------------------------------------------------------------+----------------------------------------------------------------+
         | Old value                                                      | New value                                                      |
         +================================================================+================================================================+
         | listen = '127.0.0.1:32001'                                     | Listen = '127.0.0.1:32011'                                     |
         +----------------------------------------------------------------+----------------------------------------------------------------+
         | vinyl_dir = '/opt/ntech/var/lib/tarantool/shard-001'           | vinyl_dir = '/opt/ntech/var/lib/tarantool/shard-011'           |
         +----------------------------------------------------------------+----------------------------------------------------------------+        
         | work_dir = '/opt/ntech/var/lib/tarantool/shard-001'            | work_dir = '/opt/ntech/var/lib/tarantool/shard-011'            |
         +----------------------------------------------------------------+----------------------------------------------------------------+
         | memtx_dir = '/opt/ntech/var/lib/tarantool/shard-001/snapshots' | memtx_dir = '/opt/ntech/var/lib/tarantool/shard-011/snapshots' |
         +----------------------------------------------------------------+----------------------------------------------------------------+
         | wal_dir = '/opt/ntech/var/lib/tarantool/shard-001/xlogs'       | wal_dir = '/opt/ntech/var/lib/tarantool/shard-011/xlogs'       |
         +----------------------------------------------------------------+----------------------------------------------------------------+
         | FindFace.start("127.0.0.1", 8101, {                            | FindFace.start("127.0.0.1", 8111, {                            |
         +----------------------------------------------------------------+----------------------------------------------------------------+

   #. Create symbolic links to the new shards.

      .. code::

         cd /etc/tarantool/instances.enabled/

         sudo ln -s /etc/tarantool/instances.available/shard-01*.lua /etc/tarantool/instances.enabled/

   #. Create directories that will host files of the new shards. Assign permissions for the created directories.

      .. code::

         cd /opt/ntech/var/lib/tarantool/

         mkdir -p shard-01{1..4}/{index,snapshots,xlogs}

         chown tarantool:tarantool shard-01* shard-01*/*

#. Open the ``/etc/findface-extraction-api`` configuration file and replace the extraction models with the new ones in the ``body_emben``, ``car_emben``, and ``face_emben`` parameters, depending on the object types to migrate.

   .. code::

      sudo vi /etc/findface-extraction-api.ini 

      extractors:
        ...
        models:
          ...
          body_emben: pedrec/<new_model_body>.gpu.fnk
          ...
          car_emben: carrec/<new_model_car>.gpu.fnk
          ...
          face_emben: face/<new_model_face>.gpu.fnk 

   Restart the ``findface-extraction-api`` service.

   .. code::

      sudo systemctl restart findface-extraction-api.service

#. Start the new shards.

   .. code::

      for i in {11..14}; do sudo systemctl start tarantool@shard-0$i; done

#. Create a configuration file with migration settings ``<migration.ini>`` based on the example below. 

   .. code::

      extraction-api:
        timeouts:
          connect: 5s
          response_header: 30s
          overall: 35s
          idle_connection: 0s
        extraction-api: http://127.0.0.1:18666
      storage-api-from: # current location of the gallery
        timeouts:
          connect: 5s
          response_header: 30s
          overall: 35s
          idle_connection: 10s
        max-idle-conns-per-host: 20
        shards:
          - master: http://127.0.0.1:8101/v2/
            slave: ""
          - master: http://127.0.0.1:8102/v2/
            slave: ""
          - master: http://127.0.0.1:8103/v2/
            slave: ""
          - master: http://127.0.0.1:8104/v2/
            slave: ""
      storage-api-to:
        timeouts:
          connect: 5s
          response_header: 30s
          overall: 35s
          idle_connection: 10s
        max-idle-conns-per-host: 20
        shards:
          - master: http://127.0.0.1:8111/v2/
            slave: ""
          - master: http://127.0.0.1:8112/v2/
            slave: ""
          - master: http://127.0.0.1:8113/v2/
            slave: ""
          - master: http://127.0.0.1:8114/v2/
            slave: ""
      workers_num: 3 
      faces_limit: 100 
      extraction_batch_size: 8
      normalized_storage:
        type: webdav
        enabled: True
        webdav:
          upload-url: http://127.0.0.1:3333/uploads/
        s3:
          endpoint: ""
          bucket-name: ""
          access-key: ""
          secret-access-key: ""
          secure: False
          region: ""
          public-url: ""
          operation-timeout: 30


   In the ``storage-api-from`` section, specify the old shards to migrate the data from. 

   .. code::

      storage-api-from: # current location of the gallery
        ...
        shards:
          - master: http://127.0.0.1:8101/v2/
            slave: ""
          - master: http://127.0.0.1:8102/v2/
            slave: ""
          - master: http://127.0.0.1:8103/v2/
            slave: ""
           - master: http://127.0.0.1:8104/v2/
        ...

   In the ``storage-api-to`` section, specify the new shards that will host migrated data.   

   .. code::

      storage-api-to:
        ...
        shards:
          - master: http://127.0.0.1:8111/v2/
            slave: ""
          - master: http://127.0.0.1:8112/v2/
            slave: ""
          - master: http://127.0.0.1:8113/v2/
            slave: ""
          - master: http://127.0.0.1:8114/v2/
            slave: ""
        ...

#. Launch the ``findface-sf-api-migrate`` utility with the ``-config`` option and provide the ``<migration.ini>`` configuration file.

   .. code::

      findface-sf-api-migrate -config migration.ini
 
   .. note::
      The migration process can take up a significant amount of time if there are many events and cards in the system.


#. After the migration is complete, stop the old shards and disable their autostart in OS (do not remove them). 

   .. code::

      for i in {01..04}; do sudo systemctl stop tarantool@shard-0$i.service ; done

      for i in {01..04}; do sudo systemctl disable tarantool@shard-0$i.service ; done

#. Open the ``/etc/findface-sf-api.ini`` configuration file and adjust the shards ports, subject to the new shards settings. Restart the ``findface-sf-api`` service.

   .. code::

      sudo vi /etc/findface-sf-api.ini

      shards:
        - master: http://127.0.0.1:8111/v2/
          slave: ""
        - master: http://127.0.0.1:8112/v2/
          slave: ""
        - master: http://127.0.0.1:8113/v2/
          slave: ""
        - master: http://127.0.0.1:8114/v2/
          slave: ""

      sudo systemctl start findface-sf-api.service

#. Import the new database structure from the ``tnt_schema.lua`` file.

   .. code::

      sudo findface-security make_tnt_schema | sudo tee /etc/findface-security/tnt_schema.lua

   .. seealso::
      :ref:`modify-schema`.

#. Migrate the main database architecture from FindFace Multi to :program:`PostgreSQL`, re-create :ref:`predefined <users>` user roles and the first administrator.

   .. code::

      sudo findface-security migrate
      sudo findface-security create_groups
      sudo findface-security create_default_user


#. Restart the services.

   .. code::

      sudo systemctl restart findface-security.service
      sudo systemctl restart findface-extraction-api findface-video-worker* findface-video-manager findface-sf-api

#. Migrate clusters as well if :ref:`this functionality <clusters>` is enabled in your system. To do so, execute the following command:

   .. note::
      List the object types to migrate as the command options: ``--face``, ``--body``, ``--car``.

   .. code::

      sudo findface-security migrate_clusters --face --body --car --use-best-event --use-thumbnail --force-clusterin

   As a result, the system will regenerate feature vectors for the existing cluster events and automatically launch the scheduled clustering to rebuild clusters.