.. _video-cards:

Multiple Video Cards Usage
========================================


Should you have several video cards installed on a physical server, you can create additional ``findface-extraction-api-gpu`` or ``findface-video-worker-gpu`` instances and distribute them across the video cards, one instance per card.  

.. rubric:: In this section:

.. contents::
   :local:

Distribute ``findface-extraction-api-gpu`` Instances Across Several Video Cards 
-----------------------------------------------------------------------------------

To distribute the ``findface-extraction-api-gpu`` instances across several video cards, do the following:

#. Prepare the initial configuration file of ``findface-extraction-api-gpu`` for future copying. Open ``/etc/findface-extraction-api.ini``. Limit the number of the ``findface-extraction-api-gpu`` instances down to ``1`` per video card. 

   .. code::
      
      sudo vi /etc/findface-extraction-api.ini

      ...
      extractors:
          instances: 1

   .. tip:: 
      If necessary, make other adjustments. For example, you can enable other attribute recognition. See :ref:`face-features`, :ref:`cars`, :ref:`bodies`.

#. Create several copies of the ``/etc/findface-extraction-api.ini`` configuration file, subject to how many video cards you are going to use for feature vector extraction. Append numbers of the GPU devices that will be running the instances to the new names (GPU devices #0 and #7 in the example below). 

   .. note:: 
      By default, GPU device numeration in a system starts from #0. To list the video cards in use, execute:

      .. code::

         nvidia-smi

   .. code::

      cp findface-extraction-api.ini findface-extraction-api-0.ini
      cp findface-extraction-api.ini findface-extraction-api-7.ini
 
#. Open the new configuration files. Specify the relevant GPU device numbers and adjust the listening ports. Be sure to associate each instance to a unique port. 

   .. code::

      sudo vi /etc/findface-extraction-api-0.ini

      listen: 127.0.0.1:18667
      ...

      gpu_device: 0
      ...


   .. code::

      sudo vi /etc/findface-extraction-api-7.ini

      listen: 127.0.0.1:18668
      ...

      gpu_device: 7
      ...

#. For the ``findface-extraction-api-gpu`` instances to work within one system, bind them via a load balancer, e.g., ``nginx``. For simplicity, we recommend enabling ``nginx`` on the standard ``findface-extraction-api`` port ``18666`` (see details below). 

   To set up load balancing, do the following:

   #. Create a new configuration file for ``nginx``.

      .. code::
    
         sudo vi /etc/nginx/sites-available/lb_extractions

   #. Insert the following entry into the just created file. Be sure to specify the actual listening ports of the ``findface-extraction-api-gpu`` instances in the ``server`` directive (``18667``, ``18668`` in our example). 

      .. code::

         upstream extractions {
             server 127.0.0.1:18667 max_fails=3 fail_timeout=30s;
             server 127.0.0.1:18668 max_fails=3 fail_timeout=30s;
             }
      
             server {
             listen 18666 default_server;
      
             server_name _;
      
             location / {
                 client_max_body_size 100m;
                 proxy_pass http://extractions;
             }
             }

   #. Start the load balancer.

      .. code::

         ln -s /etc/nginx/sites-available/lb_extractions /etc/nginx/sites-enabled/lb_extractions
         nginx -t
         nginx -s reload

#. Stop the initial ``findface-extraction-api-gpu`` service and disable its starting on boot.

   .. code::

      systemctl stop findface-extraction-api.service
      systemctl disable findface-extraction-api.service

#. Create a new service for the ``findface-extraction-api-gpu`` instances.

   .. code::

      /etc/systemd/system/findface-extraction-api@.service

   Insert the following entry:

   .. code::

      [Unit]
      Description=Findface Extraction API service %i instance
 
      [Service]
      User=ntech
      Restart=always
      RestartSec=5
      Type=notify
      ExecStart=/usr/bin/findface-extraction-api -config="/etc/findface-extraction-api-%i.ini"
      TimeoutStartSec=60m
 
      [Install]
      WantedBy=multi-user.target

#. Start the ``findface-extraction-api-gpu`` instances. The first start might take up to 10 minutes.

   .. code::

      systemctl daemon-reload
      systemctl enable findface-extraction-api@{0,7}
      systemctl start findface-extraction-api@{0,7}

   .. tip::
      To check the status, use the following command:

      .. code::

         sudo systemctl list-units 'findface*' -a



Allocate ``findface-video-worker-gpu`` to Additional Video Card
-----------------------------------------------------------------------------

To create an additional ``findface-video-worker-gpu`` instance and allocate it to a different video card, do the following:
 

#. Display the status of the ``findface-video-worker-gpu`` primary service by executing:

   .. code::

      sudo systemctl status findface-video-worker-gpu.service
      
#. Find the full path to the service in the following line:

   .. code::

      Loaded: loaded (/usr/lib/systemd/system/findface-video-worker-gpu.service); enabled; vendor preset: enabled

   It is ``findface-video-worker-gpu.service`` in our example (name may vary). Create a copy of the service under a new name. 

   .. code::

      sudo cp /usr/lib/systemd/system/findface-video-worker-gpu.service /usr/lib/systemd/system/findface-video-worker-gpu2.service`

#. In the same manner, create a copy of the primary service configuration file under a new name.

   .. code::

      sudo cp /etc/findface-video-worker-gpu.ini /etc/findface-video-worker-gpu2.ini

#. Open the just created configuration file and actualize the GPU device number to use. Modify the streamer port number by the following formula: ``18999 (port number for GPU #0) - GPU device number``, i.e. for the GPU #1, ``port = 18998``, for the GPU #2, ``port = 18997``, and so on.  

   .. code::

      sudo vi /etc/findface-video-worker-gpu2.ini

      ## cuda device number
      device_number = 1

      ...

      #------------------------------      
      [streamer]
      #------------------------------
      ## streamer/shots webserver port, 0=disabled
      ## type:number env:CFG_STREAMER_PORT longopt:--streamer-port
      port = 18999
      ...

#. Open the new service and specify the just created configuration file. 

   .. code::

      sudo vi /usr/lib/systemd/system/findface-video-worker-gpu2.service

      ExecStart=/usr/bin/findface-video-worker-gpu --config /etc/findface-video-worker-gpu2.ini

#. Reload the ``systemd`` daemon to apply the changes. 
 
   .. code::

      sudo systemctl daemon-reload 

#. Enable the new service autostart.

   .. code::

      sudo systemctl enable findface-video-worker-gpu2.service

      Created symlink from /etc/systemd/system/multi-user.target.wants/findface-video-worker-gpu2.service to /usr/lib/systemd/system/findface-video-worker-gpu2.service

#. Launch the new service. 

   .. code::

      sudo systemctl start findface-video-worker-gpu2.service

#. Check the both ``findface-video-worker-gpu`` services status.

   .. code::

      sudo systemctl status findface-video-worker-* | grep -i 'Active:' -B 3 

      ● findface-video-worker-gpu2.service - findface-video-worker-gpu daemon
        Loaded: loaded (/usr/lib/systemd/system/findface-video-worker-gpu2.service; enabled; vendor preset: enabled)
        Active: active (running) since Thu 2019-07-18 10:32:02 MSK; 1min 11s ago

      ...

      ● findface-video-worker-gpu.service - findface-video-worker-gpu daemon
        Loaded: loaded (/usr/lib/systemd/system/findface-video-worker-gpu.service; enabled; vendor preset: enabled)
        Active: active (running) since Mon 2019-07-15 15:18:33 MSK; 2 days ago