tntapi

The tntapi service provides interaction between the sf-api service and the Tarantool-based feature vector database in the following way:

Tip

See Tarantool official documentation for details.

  • From sf-api, tntapi receives data, such as information of detected objects, to write into the feature vector database.

  • By request from sf-api, tntapi performs database searches and returns search results.

Multiple tntapi shards can be created on each Tarantool host to increase search speed. Their running concurrently leads to a remarkable increase in performance (70x-100x).

Functionality:

  • saving object data to the feature vector database,

  • database search,

  • implementation of direct API requests to the database (see Direct API requests to tntapi).

Important

In a multi-shard environment, the configuration has to be done for each shard.

When configuring tntapi, refer to the following box.cfg parameters:

Parameter

Description

  • listen

  • env TT_LISTEN

Binary host/port, used for admin operations and replication, e.g. 127.0.0.1:32001.

  • work_dir

  • env TT_WORK_DIR

Base working directory.

  • memtx_dir

  • env TT_MEMTX_DIR

Directory for snapshots, can be relative to work_dir.

  • wal_dir

  • env TT_WAL_DIR

Directory for xlogs, can be relative to work_dir.

  • memtx_memory

  • env TT_MEMTX_MEMORY

Maximum RAM that can be used by a Tarantool shard. Set in bytes, depending on the number of objects the hard handles. Consult our experts by support@ntechlab.com before setting this parameter.

  • checkpoint_interval

  • env TT_CHECKPOINT_INTERVAL

The interval between snapshots(), in seconds.

  • checkpoint_count

  • env TT_CHECKPOINT_COUNT

Maximum number of snapshots that may exist on the memtx_dir.

  • force_recovery

  • env TT_FORCE_RECOVERY

If force_recovery equals true, Tarantool tries to continue if there is an error while reading a snapshot file (at server instance start) or a write-ahead log file (at server instance start or when applying an update at a replica): skips invalid records, reads as much data as possible and lets the process finish with a warning.

See box.cfg for details.

Refer to the following parameters:

Parameter

Description

CFG_LISTEN_HOST

Default = 127.0.0.1. Host to public HTTP API.

CFG_LISTEN_PORT

Default = 8001. Port to public HTTP API.

CFG_NTLS

Default = "". IP address and port of the ntls license server.

CFG_LOG_REQUESTS

Default = true. Log requests.

CFG_FASTIDX_SEARCH_THREADS

Default = 1. Number of threads to search in the quick index.

CFG_ALLOW_META_SCHEME_MIGRATION

Default = false. Enable meta_scheme migration. Possible only when adding new fields to the schema, if the existing ones completely match.

CFG_META_SCHEME_MIGRATION_SINGLE_TRANSACTION

Default = true. If CFG_ALLOW_META_SCHEME_MIGRATION is enabled:

  • CFG_META_SCHEME_MIGRATION_SINGLE_TRANSACTION=true: migration of one gallery occurs in one transaction until all data has migrated to the new schema, there is no entry in xlog. Galleries with a large number of faces may require a significant amount of memory.

  • CFG_META_SCHEME_MIGRATION_SINGLE_TRANSACTION=false: before migration, a box.snapshot() is taken, a flag file is created in the wal_dir directory. In case of a crash during migration, it is necessary to delete all the contents of the wal_dir directory, the data will be downloaded from snapshot.

CFG_BATCH_SEARCH_MAX_SIZE

Default = 1. Maximum size of search batches. If you specify a value > 1, facen search for queries with the same other filters will be performed by batches of exactly this maximum size.

CFG_BATCH_SEARCH_MAX_DELAY_SECONDS

Default = 0.3. Maximum query timeout when batching a search (batch_search_max_size > 1). If exceeded, the search will be performed regardless of the simultaneous presence of batch_search_max_size of similar search queries.

CFG_EXTRA_LUA

Default = "". Additional LUA code that will be executed during initialization.

CFG_METRICS_HTTP_BUCKETS

Buckets for the histogram of query times in metrics, for example: CFG_METRICS_HTTP_BUCKETS="[0.2, 0.5, 1, 5]"

The database structure is set via the /opt/ffserver/configs/tnt-schema.lua file. You will have to manually set it via the configuration file.