Modify Feature Vector Database Structure

Sometimes it may be necessary to apply a new structural schema to your Tarantool-based feature vector database, for example, when updating to the latest version of the product, or when you want to enhance the default database structure with additional parameters, advanced face metadata, and so on.

In this section:

About Database Structure

In FindFace Multi, the database structure is set via the /opt/findface-multi/configs/findface-tarantool-server/tnt-schema.lua file.

The structure is created as a set of spaces and fields. Each field is described with the following parameters:

• id: field id;

• name: field name, must be the same as the name of a relevant object parameter;

• field_type: data type (unsigned|string|set[string]|set[unsigned]);

• default: field default value. If a default value exceeds 1e14 – 1, use a string data type to specify it, for example, "123123.." instead of 123123...

You can find the default tnt_schema.lua file here.

Structure Modification

To modify the database structure, do the following:

1. Create a backup of the Tarantool-based feature vector database in any directory of your choice, for example, /etc/ffmulti_dump.

   sudo docker exec -it findface-multi-findface-sf-api-1 bash -c "mkdir ffmulti_dump; cd ffmulti_dump && /storage-api-dump -config /etc/findface-sf-api.ini"
   sudo docker cp findface-multi-findface-sf-api-1:/ffmulti_dump /etc
   

2. Modify the database structure by applying the new schema to the tnt_schema.lua file.

   sudo docker run --rm --network host --volume '/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffmulti-2.0.0 make-tnt-schema | sudo tee /opt/findface-multi/configs/findface-tarantool-server/tnt-schema.lua
   

3. Navigate to the directory with Tarantool configuration file(s) /opt/findface-multi/configs/findface-tarantool-server/. For each shard shard-00*.lua, check whether it contains the dofile command. Make sure the replication and spaces are defined within the FindFace.start parameter, as in the example below.

   sudo vi /opt/findface-multi/configs/findface-tarantool-server/shard-00*.lua
   
   dofile("/tnt_schema.lua")
   
   
   -- host:port to bind, HTTP API
   FindFace = require("FindFace")
   FindFace.start("127.0.0.1", 8101, {
       license_ntls_server="127.0.0.1:3133",
       replication = replication_master,
       spaces = spaces
   })
   

4. Purge data from all the directories relevant to active shards.

   sudo rm /opt/findface-multi/data/findface-tarantool-server/shard-*/{index,snapshots,xlogs}/*
   

5. Restart the findface-tarantool-server shards.

   docker restart findface-multi-findface-tarantool-server-shard-001-1 findface-multi-findface-tarantool-server-shard-002-1 findface-multi-findface-tarantool-server-shard-003-1 findface-multi-findface-tarantool-server-shard-004-1 findface-multi-findface-tarantool-server-shard-005-1 findface-multi-findface-tarantool-server-shard-006-1 findface-multi-findface-tarantool-server-shard-007-1 findface-multi-findface-tarantool-server-shard-008-1
   

6. Restore the Tarantool database from the backup.

   sudo docker exec -it findface-multi-findface-sf-api-1 bash -c 'cd ffmulti_dump && for x in *.json; do /storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done;'
   

   Important

   If some fields were removed from the new database structure, you have to first manually delete the corresponding data from the backup copy.

See also

Custom Metadata in Tarantool