.. _s3_storage:

S3-Compatible Storage Configuration
======================================

S3 (Simple Storage Service) provides reliable and long-term storage of an unlimited number of files and data. It gives a way to avoid the limitations of the file system when it comes to large amounts of data.

.. important::
   The ``minio:latest`` release no longer supports the required functionality. The latest version that is guaranteed to work is ``minio:RELEASE.2025-04-22T22-12-26Z``.

Configure S3-Compatible Storage
-------------------------------------

In this section, we will describe how to configure S3-compatible `MinIO <https://min.io/>`_ storage for operation with FindFace Multi. If you prefer another S3-compatible storage, you are responsible for configuring it.

#. :ref:`Prepare a server on Ubuntu <deploy-prepare-server>`.
   
   .. note::
      MinIO operation has been tested on Ubuntu only. Refer to the official `MinIO documentation <https://docs.min.io/>`_ to find out whether other OS are supported.

#. To start a container with MinIO, execute the following command, specifying the ``MINIO_ROOT_USER`` and the ``MINIO_ROOT_PASSWORD`` login credentials: 

   .. code::

      docker run -d \
        -p 9003:9000 \
        -p 9004:9001 \
        -e "MINIO_ROOT_USER=admin" \
        -e "MINIO_ROOT_PASSWORD=admin_admin" \
        -v /path/to/storage_directory:/data \
        quay.io/minio/minio:RELEASE.2025-04-22T22-12-26Z server /data --console-address ":9001"

   where ``/path/to/storage_directory`` is a directory allocated for the S3 storage.

#. MinIO web interface address will depend on the interface that you have selected during FindFace Multi :ref:`installation <installer>` in step #4.5. The port in use is ``9004``, e.g., ``192.168.112.254:9004``. Log in to the MinIO web interface with ``MINIO_ROOT_USER``, ``MINIO_ROOT_PASSWORD`` login credentials that you have specified in the previous step. 

#. In the MinIO UI, create a bucket (:guilabel:`Buckets` → :guilabel:`Create Bucket`). Assign a default name ``ffsec.storage`` to the bucket.

   .. note::
      We advise keeping the bucket name ``ffsec.storage`` as suggested by default, because it corresponds to the ``'STORAGE_BUCKET_NAME'`` parameter in the ``/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py`` and ``/opt/findface-multi/configs/findface-multi-legacy/findface-multi-identity-provider.py`` configuration files. If you use another name for a bucket, make sure to adjust it in the above-mentioned configuration files accordingly.


   |create_bucket_en|

    .. |create_bucket_en| image:: /_static/create_bucket_en.png
        :scale: 60%


#. Click on the bucket to open its settings.


   |bucket_settings_en|

    .. |bucket_settings_en| image:: /_static/bucket_settings_en.png
        :scale: 60%


#. Change the access policy for the bucket to ``public`` for the correct processing of video archives.


   |access_policy_en|

    .. |access_policy_en| image:: /_static/access_policy_en.png
        :scale: 60%


#. In the :guilabel:`User` → :guilabel:`Access Keys` section, create an access key. **Write down the** ``Access key`` **and** ``Secret key`` **values as you will need them later for FindFace Multi configuration.**


   |access_key_en|

    .. |access_key_en| image:: /_static/access_key_en.png
        :scale: 55%


   .. note::
      MinIO UI may differ. In the earlier versions, set up an access key in the :guilabel:`Identity` → :guilabel:`Service Accounts` → :guilabel:`Create service account` section.

.. _employ_s3_storage:

Configure FindFace Multi to Employ S3-Compatible Storage
------------------------------------------------------------

#. In the ``/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py`` configuration file, locate the ``EXTERNAL_STORAGE_CONFIG`` section. 

   .. code::

      sudo vi /opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py

      EXTERNAL_STORAGE_CONFIG = {
          'STORAGE_TYPE': None,
          'STORAGE_BUCKET_NAME': 'ffsec.storage',
          'ACCESS_KEY_ID': 'access-key',
          'SECRET_ACCESS_KEY': 'secret-key',
          'REGION_NAME': 'eu-west-3',
          'LOCAL_S3_SETTINGS': {
              'CONNECTION_ADDRESS': 'localhost:9003',
          },
      }

   Configure the ``EXTERNAL_STORAGE_CONFIG`` parameters:

   #. The ``'STORAGE_TYPE'`` parameter may take values ``None``, ``'LOCAL'``, ``'AWS'``:

      * ``None`` – local file system (default)

      * ``'LOCAL'`` – any S3-compatible storage (e.g., MinIO)

      * ``'AWS'`` – `Amazon S3 <https://aws.amazon.com/s3/>`_ storage. 

      Change the ``'STORAGE_TYPE'`` parameter to ``'LOCAL'`` to enable S3-compatible storage.

   #. The values of the ``'STORAGE_BUCKET_NAME'``, ``'ACCESS_KEY_ID'``, and ``'SECRET_ACCESS_KEY'`` parameters should correspond to those that were specified during MinIO initialization.

   #. ``'REGION_NAME'`` – the default value is ``eu-west-3``. It can be changed in MinIO settings.

   #. In the ``'LOCAL_S3_SETTINGS'`` → ``'CONNECTION_ADDRESS'``, replace the ``localhost`` with MinIO IP host address, e.g., ``192.168.112.254``.

#. Configure the ``/opt/findface-multi/configs/findface-multi-identity-provider/findface-multi-identity-provider.py`` file in the same way. Locate the ``EXTERNAL_STORAGE_CONFIG`` section and apply changes analogous to those made in the ``/opt/findface-multi/configs/findface-multi-legacy/findface-multi-legacy.py`` file in the previous step.

#. For the UI to load content (thumbnails, full frame images, reports, and so on), configure the ``/opt/findface-multi/configs/findface-multi-ui/nginx-site.conf`` file. In the ``server`` section, locate the ``set $csp_policy`` and specify the host and port of the S3 storage ``{ip_host_s3}:9003`` in the ``default-src``, ``img-src``, ``media-src``, and ``connect-src`` directives, e.g.:

   .. code::

      sudo vi /opt/findface-multi/configs/findface-multi-ui/nginx-site.conf

      server {
              ...

              set $csp_policy "default-src 'self' 192.168.112.254:9003; img-src 'self' 192.168.112.254:9003 *.tile.openstreetmap.org blob: data:; media-src 'self' 192.168.112.254:9003 blob:; connect-src 'self' 192.168.112.254:9003 data:; style-src 'self' 'unsafe-inline'; frame-src 'self';  object-src 'self'; frame-ancestors 'self'";
              ...
      }

#. Restart all FindFace Multi containers.

   .. code::

      cd /opt/findface-multi

      sudo docker-compose down
      sudo docker-compose up -d


Data Transfer Command
---------------------------

To transfer data to either remote or local storage, use the following command:

.. code::

   sudo docker exec -it findface-multi-findface-multi-legacy-1 /opt/findface-security/bin/python3 /tigre_prototype/manage.py transfer_data --to_local / --to_remote


* The argument ``--to_remote`` transfers data to remote storage.
* The argument ``--to_local``  transfers data to the local file system.


Before executing the above command, make sure that you have :ref:`configured FindFace Multi <employ_s3_storage>` to use S3-compatible storage.