findface-tarantool-server

The findface-tarantool-server service provides interaction between the findface-sf-api service and the Tarantool-based biometric database in the following way:

Tip

See Tarantool official documentation for details.

• From findface-sf-api, findface-tarantool-server receives data, such as information of detected in video faces, to write into the biometric database.
• By request from findface-sf-api, findface-tarantool-server performs database searches and returns search results.

To increase search speed, multiple findface-tarantool-server shards can be created on each Tarantool host. Their running concurrently leads to a remarkable increase in performance (70x-100x).

Functionality:

• saving face data to the biometric database,
• database search,
• implementation of direct API requests to the database (see Direct API Requests to Tarantool).

The findface-tarantool-server configuration is done through the /etc/tarantool/instances.enabled/<shard-*>.lua configuration file. In a cluster environment, configuration has to be done for each shard.

--
-- Please, read the tarantool cfg doc:
-- https://tarantool.org/doc/reference/configuration/index.html#box-cfg-params
--

box.cfg{
    --port to listen, direct tarantool access
    --Only need for admin operations
    --THIS IS NOT PORT YOU NEED FOR facenapi/sf-api
    listen = '127.0.0.1:33001',

    --Directory to store data
    vinyl_dir = '/opt/ntech/var/lib/tarantool/shard-001',
    work_dir = '/opt/ntech/var/lib/tarantool/shard-001',
    memtx_dir = '/opt/ntech/var/lib/tarantool/shard-001/snapshots',
    wal_dir = '/opt/ntech/var/lib/tarantool/shard-001/xlogs',

    --Maximum mem usage in bytes
    memtx_memory = 200 * 1024 * 1024,

    checkpoint_interval = 3600*4,
    checkpoint_count = 3,

    --uncomment only if you know what you are doing!!! and don't forget box.snapshot()
    -- wal_mode = 'none',

    --if true, tarantool tries to continue if there is an error while reading a snapshot/xlog files:  skips invalid records, reads as much data as possible and re-builds the file
    -- force_recovery = true,
}

pcall(function() box.schema.user.grant('guest', 'execute,read,write', 'universe') end)

dofile("/etc/ffsecurity/tnt_schema.lua")

-- host,port to bind for http server
-- this is what you need for facenapi
FindFace = require("FindFace")
FindFace.start("127.0.0.1", 8101, {
    license_ntls_server="127.0.0.1:3133",
    facen_size=576,
    meta_scheme = meta_scheme
})

When configuring findface-tarantool-server, refer to the following parameters:

Parameter	Description
memtx_memory	Maximum RAM that can be used by a Tarantool shard. Set in bytes, depending on the number of faces the shard handles. Consult our experts by support@ntechlab.com before setting this parameter.
force_recovery	Enables automatic database recovery. In this case, each time an error occurs while reading a snapshot or xlog file, Tarantool will skip invalid records, read as much data as possible, and re-build the file.
license_ntls_server	IP address and port of the findface-ntls license server.
facen_size	Feature vector size. Before editing this parameter, be sure to consult NTechLab experts.
meta_scheme	A database structure to store the face recognition results. The structure is created as a set of fields. Describe each field with the following parameters: id: field id; name: field name, must be the same as the name of a relevant face parameter; field_type: data type; default: field default value, if a default value exceeds ‘1e14 – 1’, use a string data type to specify it, for example, "123123.." instead of 123123...

Default database structure is passed from /etc/ffsecurity/tnt_schema.lua to the meta_scheme parameter.