Modify Feature Vector Database Structure

Sometimes it may be necessary to apply a new structural schema to your Tarantool-based feature vector database, for example, when updating to the latest version of the product, or when you want to enhance the default database structure with additional parameters, advanced face metadata, and so on.

In this section:

About Database Structure

In FindFace, the database structure is set via the /etc/findface-security/tnt_schema.lua file.

The structure is created as a set of spaces and fields. Each field is described with the following parameters:

  • id: field id;

  • name: field name, must be the same as the name of a relevant object parameter;

  • field_type: data type;

  • default: field default value. If a default value exceeds ‘1e14 – 1’, use a string data type to specify it, for example, "123123.." instead of 123123...

You can find the default tnt_schema.lua file here.

Structure Modification

To modify the database structure, do the following:

  1. Stop the findface-security service.

    sudo systemctl stop findface-security.service
    
  2. Create a backup of the Tarantool-based feature vector database in any directory of your choice, for example, /etc/findface_dump.

    mkdir -p /etc/findface_dump
    cd /etc/findface_dump
    sudo findface-storage-api-dump -config /etc/findface-sf-api.ini
    
  3. Prepare the tnt_schema.lua file containing the new database structure.

  4. Modify the database structure by applying the new tnt_schema.lua file.

    sudo findface-security make_tnt_schema | sudo tee /etc/findface-security/tnt_schema.lua
    
  5. Navigate to the directory with Tarantool configuration file(s) /etc/tarantool/instances.available/. For each shard, check whether it contains the dofile command and the spaces definition, as in the example below.

    sudo vi /etc/tarantool/instances.available/<shard_00N>.lua
    
    dofile("/etc/findface-security/tnt_schema.lua")
    
    -- host:port to bind, HTTP API
    FindFace = require("FindFace")
    FindFace.start("127.0.0.1", 8101, {
    license_ntls_server="127.0.0.1:3133",
    replication = replication_master,
    spaces = spaces
    })
    
  6. Stop the findface-tarantool-server shards. Purge data from all the directories relevant to active shards.

    sudo systemctl stop '[email protected]*'
    
    sudo rm /opt/ntech/var/lib/tarantool/shard-*/{index,snapshots,xlogs}/*
    
  7. Restart the findface-tarantool-server shards.

    TNT=$(ls /etc/tarantool/instances.enabled/ | cut -c 7,8,9)
    for i in $TNT; do sudo systemctl restart [email protected]$i.service ; done
    
  8. Restore the Tarantool database from the backup.

    Important

    If some fields were removed from the new database structure, you have to first manually delete the corresponding data from the backup copy.

    cd /etc/findface_dump
    for x in *.json; do curl -X POST "http://127.0.0.1:18411/v2/galleries/${x%%.json}"; done
    for x in *.json; do sudo findface-storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done
    
  9. Start the findface-security service.

    sudo systemctl start findface-security.service