Modify Feature Vector Database Structure

Sometimes it may be necessary to apply a new structural schema to your Tarantool-based feature vector database, for example, when updating to the latest version of the product, or when you want to enhance the default database structure with additional parameters, advanced face metadata, and so on.

In this section:

About Database Structure

In FindFace CIBR, the database structure is set via the /opt/findface-cibr/configs/findface-tarantool-server/tnt-schema.lua file.

The structure is created as a set of spaces and fields. Each field is described with the following parameters:

  • id: field id;

  • name: field name, must be the same as the name of a relevant object parameter;

  • field_type: data type (unsigned|string|set[string]|set[unsigned]);

  • default: field default value. If a default value exceeds 1e14 – 1, use a string data type to specify it, for example, "123123.." instead of 123123...

You can find the default tnt-schema.lua file here.

Structure Modification

To modify the database structure, do the following:

  1. Create a backup of the Tarantool-based feature vector database in any directory of your choice, for example, /etc/ffcibr_dump.

    sudo docker exec -it findface-cibr-findface-sf-api-1 bash -c "mkdir ffcibr_dump; cd ffcibr_dump && /storage-api-dump -config /etc/findface-sf-api.ini"
sudo docker cp findface-cibr-findface-sf-api-1:/ffcibr_dump /etc

  2. Modify the database structure by applying the new schema to the tnt-schema.lua file.

    sudo docker run --rm --network host --volume '/opt/findface-cibr/configs/findface-multi-legacy/findface-multi-legacy.py:/etc/findface-security/config.py:ro' docker.int.ntl/ntech/multi/multi/legacy:ffcibr-2.1.2 make-tnt-schema | sudo tee /opt/findface-cibr/configs/findface-tarantool-server/tnt-schema.lua

  3. Open the docker-compose.yaml file. Make sure each Tarantool shard service contains defined variable CFG_EXTRA_LUA: loadfile("/tnt_schema.lua") in the environment section.

    sudo vi /opt/findface-cibr/docker-compose.yaml

...
findface-tarantool-server-shard-001:
    depends_on: [findface-ntls]
    environment: {CFG_EXTRA_LUA: loadfile("/tnt_schema.lua")()...
...

  4. Purge data from all the directories relevant to active shards.

    sudo rm /opt/findface-cibr/data/findface-tarantool-server/shard-*/{index,snapshots,xlogs}/*

  5. Restart the findface-tarantool-server shards.

    docker restart findface-cibr-findface-tarantool-server-shard-001-1 findface-cibr-findface-tarantool-server-shard-002-1 findface-cibr-findface-tarantool-server-shard-003-1 findface-cibr-findface-tarantool-server-shard-004-1 findface-cibr-findface-tarantool-server-shard-005-1 findface-cibr-findface-tarantool-server-shard-006-1 findface-cibr-findface-tarantool-server-shard-007-1 findface-cibr-findface-tarantool-server-shard-008-1

  6. Restore the Tarantool database from the backup.

    sudo docker exec -it findface-cibr-findface-sf-api-1 bash -c 'cd ffcibr_dump && for x in *.json; do /storage-api-restore -config /etc/findface-sf-api.ini < "$x"; done;'

    Important

    If some fields were removed from the new database structure, you have to first manually delete the corresponding data from the backup copy.

See also

Custom Metadata in Tarantool