Multiple Video Cards Usage

Should you have several video cards installed on a physical server, you can create additional findface-extraction-api-gpu or findface-video-worker-gpu instances and distribute them across the video cards, one instance per card.

In this section:

Distribute findface-extraction-api-gpu Instances Across Several Video Cards

To distribute the findface-extraction-api-gpu instances across several video cards, do the following:

  1. Prepare the initial configuration file of findface-extraction-api-gpu for future copying. Open /etc/findface-extraction-api.ini. Limit the number of the findface-extraction-api-gpu instances down to 1 per video card.

    sudo vi /etc/findface-extraction-api.ini

...
extractors:
    instances: 1

  2. Create several copies of the /etc/findface-extraction-api.ini configuration file, subject to how many video cards you are going to use for feature vector extraction. Append numbers of the GPU devices that will be running the instances to the new names (GPU devices #0 and #7 in the example below).

    Note

    By default, GPU device numeration in a system starts from #0. To list the video cards in use, execute:

    nvidia-smi

    cp findface-extraction-api.ini findface-extraction-api-0.ini
cp findface-extraction-api.ini findface-extraction-api-7.ini

  3. Open the new configuration files. Specify the relevant GPU device numbers and adjust the listening ports. Be sure to associate each instance to a unique port.

    sudo vi /etc/findface-extraction-api-0.ini

listen: 127.0.0.1:18667
...

gpu_device: 0
...

    sudo vi /etc/findface-extraction-api-7.ini

listen: 127.0.0.1:18668
...

gpu_device: 7
...

  4. For the findface-extraction-api-gpu instances to work within one system, bind them via a load balancer, e.g., nginx. For simplicity, we recommend enabling nginx on the standard findface-extraction-api port 18666 (see details below).

    To set up load balancing, do the following:

    1. Create a new configuration file for nginx.

      sudo vi /etc/nginx/sites-available/lb_extractions

    2. Insert the following entry into the just created file. Be sure to specify the actual listening ports of the findface-extraction-api-gpu instances in the server directive (18667, 18668 in our example).

      upstream extractions {
    server 127.0.0.1:18667 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:18668 max_fails=3 fail_timeout=30s;
    }

    server {
    listen 18666 default_server;

    server_name _;

    location / {
        client_max_body_size 100m;
        proxy_pass http://extractions;
    }
    }

    3. Start the load balancer.

      ln -s /etc/nginx/sites-available/lb_extractions /etc/nginx/sites-enabled/lb_extractions
nginx -t
nginx -s reload

  5. Stop the initial findface-extraction-api-gpu service and disable its starting on boot.

    systemctl stop findface-extraction-api.service
systemctl disable findface-extraction-api.service

  6. Create a new service for the findface-extraction-api-gpu instances.

    /etc/systemd/system/findface-extraction-api@.service

    Insert the following entry:

    [Unit]
Description=Findface Extraction API service %i instance

[Service]
User=ntech
Restart=always
RestartSec=5
Type=notify
ExecStart=/usr/bin/findface-extraction-api -config="/etc/findface-extraction-api-%i.ini"
TimeoutStartSec=60m

[Install]
WantedBy=multi-user.target

  7. Start the findface-extraction-api-gpu instances. The first start might take up to 10 minutes.

    systemctl daemon-reload
systemctl enable findface-extraction-api@{0,7}
systemctl start findface-extraction-api@{0,7}

    Tip

    To check the status, use the following command:

    sudo systemctl list-units 'findface*' -a

Allocate findface-video-worker-gpu to Additional Video Card

To create an additional findface-video-worker-gpu instance and allocate it to a different video card, do the following:

  1. Display the status of the findface-video-worker-gpu primary service by executing:

    sudo systemctl status findface-video-worker-gpu.service

  2. Find the full path to the service in the following line:

    Loaded: loaded (/usr/lib/systemd/system/findface-video-worker-gpu.service); enabled; vendor preset: enabled

    It is findface-video-worker-gpu.service in our example (name may vary). Create a copy of the service under a new name.

    sudo cp /usr/lib/systemd/system/findface-video-worker-gpu.service /usr/lib/systemd/system/findface-video-worker-gpu2.service`

  3. In the same manner, create a copy of the primary service configuration file under a new name.

    sudo cp /etc/findface-video-worker-gpu.ini /etc/findface-video-worker-gpu2.ini

  4. Open the just created configuration file and actualize the GPU device number to use. Modify the streamer port number by the following formula: 18999 (port number for GPU #0) - GPU device number, i.e. for the GPU #1, port = 18998, for the GPU #2, port = 18997, and so on.

    sudo vi /etc/findface-video-worker-gpu2.ini

## cuda device number
device_number = 1

...

#------------------------------
[streamer]
#------------------------------
## streamer/shots webserver port, 0=disabled
## type:number env:CFG_STREAMER_PORT longopt:--streamer-port
port = 18999
...

  5. Open the new service and specify the just created configuration file.

    sudo vi /usr/lib/systemd/system/findface-video-worker-gpu2.service

ExecStart=/usr/bin/findface-video-worker-gpu --config /etc/findface-video-worker-gpu2.ini

  6. Reload the systemd daemon to apply the changes.

    sudo systemctl daemon-reload

  7. Enable the new service autostart.

    sudo systemctl enable findface-video-worker-gpu2.service

Created symlink from /etc/systemd/system/multi-user.target.wants/findface-video-worker-gpu2.service to /usr/lib/systemd/system/findface-video-worker-gpu2.service

  8. Launch the new service.

    sudo systemctl start findface-video-worker-gpu2.service

  9. Check the both findface-video-worker-gpu services status.

    sudo systemctl status findface-video-worker-* | grep -i 'Active:' -B 3

● findface-video-worker-gpu2.service - findface-video-worker-gpu daemon
  Loaded: loaded (/usr/lib/systemd/system/findface-video-worker-gpu2.service; enabled; vendor preset: enabled)
  Active: active (running) since Thu 2019-07-18 10:32:02 MSK; 1min 11s ago

...

● findface-video-worker-gpu.service - findface-video-worker-gpu daemon
  Loaded: loaded (/usr/lib/systemd/system/findface-video-worker-gpu.service; enabled; vendor preset: enabled)
  Active: active (running) since Mon 2019-07-15 15:18:33 MSK; 2 days ago