Fast IndexΒΆ
To speed up the search, create a fast index for each gallery, using the findface-tarantool-build-index utility delivered with the console installer. The utility is independent of the findface-tarantool-server component and can be installed either on the localhost or on a remote host with access to Tarantool.
To create a fast index, do the following:
If you have installed the FindFace core step-by-step, install the
findface-tarantool-build-indexutility.sudo apt install findface-tarantool-build-index
Create a fast index for your gallery (
testgalin the case-study). First, connect to the Tarantool console.Important
The gallery must not be empty. See Direct API Requests to Tarantool to fill the gallery with faces.
Note
You have to repeat the fast index creation on each
findface-tarantool-servershard.tarantoolctl connect 127.0.0.1:33001
Run
prepare_preindex. Each element of the gallery will be moved from thelinearspace topreindex:127.0.0.1:33001> FindFace.Gallery.new("testgal"):prepare_preindex() --- ...
Prepare a file for generating the index:
127.0.0.1:33001> FindFace.Gallery.new("testgal"):save_preindex("/tmp/preindex.bin") --- ...
Launch index generation with the
findface-tarantool-build-indexutility (see--helpfor additional options). Depending on the number of elements, this process can take up to several hours and can be done on a separate, more powerful machine (for huge galleries we recommend c4.8xlarge amazon, for example, spot-instance).findface-build-index --input /tmp/preindex.bin --output /opt/ntech/var/lib/tarantool/default/index/testgal.idx --facen_size 320 Config values: .input = /tmp/preindex.bin .output = /opt/ntech/var/lib/tarantool/default/index/testgal.idx .facen_size = 320 .param_m = 12 .param_ef = 500 .limit = 4294967295 Building index: [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] 100% ; 3 / 3 Index saved at /opt/ntech/var/lib/tarantool/default/index/testgal.idx
Delete the
preindex.binfile.sudo rm /tmp/preindex.bin
Enable the fast index for the gallery.
Note
If Tarantool works as a replica set, copy the index file (
.idx) from the master instance to the same path on the replica before enabling the fast index for the master instance (:use_index).Tip
Do not forget to remove obsolete index files on the replica in order to avoid unnecessary index transitions, should the master instance and replica be heavily out of sync.
127.0.0.1:33001> FindFace.Gallery.new("testgal"):preindex_to_index() --- ... 127.0.0.1:33001> sudo FindFace.Gallery.new("testgal"):use_index("/opt/ntech/var/lib/tarantool/default/index/testgal.idx") --- ...
Search through the gallery should now be significantly faster. Information about the index remains in the service space, so when you restart Tarantool, the index will also be uploaded.
Warning
Do not move the index file to another location!