Modify Feature Vector Database Structure

Sometimes it may be necessary to apply a new structural schema to your Tarantool-based feature vector database, for example, when updating to the latest version of the product, or when you want to enhance the default database structure with additional parameters, advanced face metadata, and so on.

In this section:

About Database Structure

In FindFace, the database structure is set via the /etc/findface-security/tnt_schema.lua file.

The structure is created as a set of spaces and fields. Each field is described with the following parameters:

• id: field id;

• name: field name, must be the same as the name of a relevant object parameter;

• field_type: data type;

• default: field default value. If a default value exceeds ‘1e14 – 1’, use a string data type to specify it, for example, "123123.." instead of 123123...

You can find the default tnt_schema.lua file here.

Structure Modification

To modify the database structure, do the following:

1. Stop the findface-security service.

   sudo systemctl stop findface-security.service
   

2. Create a backup of the Tarantool-based feature vector database in any directory of your choice, for example, /etc/findface_dump.

   Tip

   See Back Up and Recover FindFace and Data for details.

   mkdir -p /etc/findface_dump
   cd /etc/findface_dump
   sudo findface-storage-api-dump -config /etc/findface-sf-api.ini
   

3. Prepare the tnt_schema.lua file containing the new database structure.

4. Modify the database structure by applying the new tnt_schema.lua file.

   sudo findface-security make_tnt_schema | sudo tee /etc/findface-security/tnt_schema.lua
   

5. Navigate to the directory with Tarantool configuration file(s) /etc/tarantool/instances.available/. For each shard, check whether it contains the dofile command and the spaces definition, as in the example below.

   sudo vi /etc/tarantool/instances.available/<shard_00N>.lua
   
   dofile("/etc/findface-security/tnt_schema.lua")
   
   -- host:port to bind, HTTP API
   FindFace = require("FindFace")
   FindFace.start("127.0.0.1", 8101, {
   license_ntls_server="127.0.0.1:3133",
   replication = replication_master,
   spaces = spaces
   })
   

6. Stop the findface-tarantool-server shards. Purge data from all the directories relevant to active shards.

   sudo systemctl stop 'tarantool@*'
   
   sudo rm /opt/ntech/var/lib/tarantool/shard-*/{index,snapshots,xlogs}/*
   

7. Restart the findface-tarantool-server shards.

   TNT=$(ls /etc/tarantool/instances.enabled/ | cut -c 7,8,9)
   for i in $TNT; do sudo systemctl restart tarantool@shard-$i.service ; done
   

8. Restore the Tarantool database from the backup.

   Important

   If some fields were removed from the new database structure, you have to first manually delete the corresponding data from the backup copy.

   cd /etc/findface_dump
   for x in *.json; do curl -X POST "http://127.0.0.1:18411/v2/galleries/${x%%.json}"; done
   for x in *.json; do sudo findface-storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done
   

9. Start the findface-security service.

   sudo systemctl start findface-security.service
   

See also

Custom Metadata in Tarantool