Multi-Object API Methods v3

In this section:

Detect Methods

Detect Objects on the Photo

POST /v3/detect

This method detects an object in a provided image and returns coordinates of the rectangle around the object (a.k.a. bbox) and the object orientation.

Note

Object detection is done by the extraction-api component, so the sf-api component formats your initial request and forwards it to extraction-api.

Query string parameters:

  • detector: string, object detector to be applied to the image (see objects in the extraction-api configuration file).

  • cache: boolean, if true, the formatted request to extraction-api will include such feature vectors as face_emben, body_emben, car_emben and need_normalized (obtain a normalized object image). The objects in extraction-api response will be saved in memcached under a ID. You can find this ID in the id field of the response. If not specified, the sf-api response will not contain feature vectors.

  • autorotate: boolean, auto-rotates an original image to 4 different orientations and returns objects detected in each orientation.

  • roi: string, a region of interest in the image, takes an array of integers in the format roi=top,left,right,bottom, (e.g. {“left”: 0, “right”: 1000, “top”: 0, “bottom”: 1000}).

  • return_emben: boolean, returns a object feature vector in the response. Requires the enabled allow-return-facen flag in the sf-api configuration file. Requires the corresponding object in the sf-api configuration file in the objects section.

  • quality_estimator: boolean, enables object quality estimation.

  • attribute: array of strings in the format face_gender, face_age, face_emotions, face_beard, face_glasses3, enables recognition of the object features passed in the array.

Tip

To enable a boolean parameter (cache, etc.), use any of the following strings: 1, yes, true, or on, in any letter case.

Parameters in request body:

Image as a file of the image/jpeg, image/png, or image/webp MIME-type, or as a text/x-url link to a relevant public URL.

Returns:

Returns a temporary object that can be stored in the cashe memcache, redis, or sf-api memory. It contains:

  • temporary ID of the detection result (id, if cache enabled);

    Important

    When you form a request, be sure to check the relevance of the temporary ID before you refer to it as it tends to become irrelevant with time. If so, re-detect the object.

  • bbox: list of coordinates of the rectangles around the detected objects;

  • score: algorithm confidence in the result.

  • attributes (if passed): feature vector (if return_emben enabled) and attributes specified in the request parameters with extractor, model and result:

    • age (if face_age passed): number of years;

    • emotions (if face_emotions passed): 6 basic emotions + neutral (angry, disgust, fear, happy, sad, surprise, neutral) with algorithm confidence in each emotion expression;

    • gender (if face_gender passed): male or female, with model and algorithm confidence in the result ("confidence");

    • other attributes (if passed): beard (beard), glasses (sun, eye, or none), along with algorithm confidence in the result;

  • orientation: EXIF orientation of the photo.

Examples

Request

curl -i -X POST 'http://127.0.0.1:18411/v3/detect?cache=on&attribute=face_gender&attribute=face_age&attribute=face_emotions&attribute=face_glasses3' -H 'Content-Type: image/jpeg' --data-binary @sample.jpg

Response

HTTP/1.1 200 OK
Content-Type: application/json
X-Request-Id: SF:BpLnfgDs
Date: Thu, 23 May 2019 12:00:22 GMT
Content-Length: 713

{
    "objects": {
        "face": [
            {
                "id": "bjj8mlhjisgjrk6hj1v0",
                "bbox": {"left": 595, "top": 127, "right": 812, "bottom": 344},
                "score": 0.9171224,
                "attributes": {
                    "face_age": {"extractor": "face_age", "model": "age.v1", "result": 24},
                    "face_emotions": {
                        "extractor": "face_emotions",
                        "model": "emotions.v1",
                        "result": [
                            {"confidence": 0.9998919, "name": "neutral"},
                            {"confidence": 9.195882e-05, "name": "sad"},
                            {"confidence": 1.4304824e-05, "name": "happy"},
                            {"confidence": 1.8077538e-06, "name": "surprise"},
                            {"confidence": 4.2787672e-08, "name": "angry"},
                            {"confidence": 2.0801991e-08, "name": "disgust"},
                            {"confidence": 1.9873138e-11, "name": "fear"}
                        ]
                    },
                    "face_gender": {
                        "extractor": "face_gender",
                        "model": "gender.v2",
                        "result": [
                            {"confidence": 0.9999119, "name": "female"},
                            {"confidence": 8.809325e-05, "name": "male"}
                        ]
                    },
                    "face_glasses3": {
                        "attribute": "face_glasses3",
                        "model": "glasses3.v0",
                        "result": [
                            {"confidence": 0.99958307, "name": "none"},
                            {"confidence": 0.00033243417, "name": "eye"},
                            {"confidence": 8.4465064e-05, "name": "sun"}
                        ]
                    }
                }
            }
        ]
    },
    "orientation": 1
}

Get Detection Result from the Cache by ID

GET /v3/detect/:id

This method retrieves the detection results from the cache by their temporary ID (including feature vectors of the detected objects).

Parameters in path segments:

  • :id: string, the detection result temporary ID in the cache.

Returns:

JSON representation of the detection result.

Example

Request

curl -i -X GET 'http://127.0.0.1:18411/v3/detect/bg2gu31jisghl6pee09g'

Response:

HTTP/1.1 200 OK
Content-Type: application/json
X-Request-Id: SF:HlQBSnTu
Date: Wed, 05 Dec 2018 07:51:04 GMT
Content-Length: 477

{
    "face": {
        "id": "bjj8mlhjisgjrk6hj1v0",
        "bbox": {"left": 595, "top": 127, "right": 812, "bottom": 344},
        "score": 0.9171224,
        "attributes": {
            "face_age": {"extractor": "face_age", "model": "age.v1", "result": 24},
            "face_emotions": {
                "extractor": "face_emotions",
                "model": "emotions.v1",
                "result": [
                    {"confidence": 0.9998919, "name": "neutral"},
                    {"confidence": 9.195882e-05, "name": "sad"},
                    {"confidence": 1.4304824e-05, "name": "happy"},
                    {"confidence": 1.8077538e-06, "name": "surprise"},
                    {"confidence": 4.2787672e-08, "name": "angry"},
                    {"confidence": 2.0801991e-08, "name": "disgust"},
                    {"confidence": 1.9873138e-11, "name": "fear"}
                ]
            },
            "face_gender": {
                "extractor": "face_gender",
                "model": "gender.v2",
                "result": [
                    {"confidence": 0.9999119, "name": "female"},
                    {"confidence": 8.809325e-05, "name": "male"}
                ]
            },
            "face_glasses3": {
                "attribute": "face_glasses3",
                "model": "glasses3.v0",
                "result": [
                    {"confidence": 0.99958307, "name": "none"},
                    {"confidence": 0.00033243417, "name": "eye"},
                    {"confidence": 8.4465064e-05, "name": "sun"}
                ]
            }
        }
    }
}

Create Detection Result out of extraction-api Response

POST /v3/detect/premade

This method creates a detection result out of a extraction-api response. Requires allow-return-facen: true in sf-api configuration file.

Parameters in path segments:

The method doesn’t accept any parameters.

Parameters in request body:

Request body contains the object in a format similar to the response GET /v3/detect/:id: { "object type": object from the extraction-api response }

Returns:

JSON with "id" on success.

Example

Request

curl -i -X POST 'http://127.0.0.1:18411/v3/detect/premade' -H 'Content-Type: application/json' --data-binary '@extapi-face.json'

Response:

HTTP/1.1 200 OK
Content-Type: application/json
X-Request-Id: SF:jFSBuSPm
Date: Wed, 05 Dec 2018 08:08:56 GMT
Content-Length: 2

{"id": "bg2gu31jisghl6peea9g"}

Get Normalized Detect by ID

GET /v3/detect/:id/normalized

This method retrieves normalized detect by ID.

Parameters in path segments:

  • :id: ID.

Returns:

Normalized object images on success.

Example

Request

curl -i 'http://127.0.0.1:18411/v3/detect/bg2gu31jisghl6pee09g/normalized'

Response:

HTTP/1.1 200 OK
Content-Type: image/png
X-Request-Id: SF:mjv3Lyvf
Date: Thu, 09 Apr 2020 15:59:48 GMT
Transfer-Encoding: chunked

Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.

Compare Two Objects

GET /v3/verify

This method compares a pair of objects (faces, bodies) and returns a probability of their belonging to the same person (a.k.a. similarity, or confidence).

Query string parameters:

  • "obj1": the first object, either a detection result (ID of the result from the POST /v3/detect method), or one from the biometric database (in the format persistent:<objtype>/<gallery>/<id> or emben:[model:]<base64 emben> if allow-return-facen:true ).

  • "obj2": the second object, from the same possible sources as the first object.

Returns:

Algorithm confidence that the objects match.

Example

Request #1. Compare 2 detection results

curl -s 'http://127.0.0.1:18411/v3/verify?obj1=detection%3Acp1nh3k8kuic72tu0ljg%22&obj2=detection%3Acp1nh3k8kuic72tu0ljg%22' | jq

Response

{
 "confidence": 0.92764723
}

Request #2. Compare a detection result and a face from a gallery

curl -i 'http://127.0.0.1:18411/v3/verify?obj1=detection:bg2gu31jisghl6pee09g&obj2=persistent:face/history/4141715733352439863' | jq

Response

{
 "confidence": 0.999996
}

Get Models Information

GET /v3/models-info

Parameters in path segments::

The method doesn’t accept any parameters.

Returns:

Returns information about available (i.e. enabled) detectors, normalizers, extractors and objects in extraction-api.

Example

Request

curl -s http://localhost:18411/v3/models-info | jq

Response

{
  "detectors": {
    "body": {
      "object_types": [
        "body"
      ]
    },
    "car": {
      "object_types": [
        "car"
      ]
    },
    "gustav_body": {
      "object_types": [
        "body"
      ]
    },
    "gustav_car": {
      "object_types": [
        "car"
      ]
    },
    "headbodyface": {
      "object_types": [
        "head",
        "body",
        "face"
      ]
    },
    "license_plate": {
      "object_types": [
        "license_plate"
      ]
    },
    "license_plate_gustav_accurate": {
      "object_types": [
        "license_plate"
      ]
    },
    "shiloette": {
      "object_types": [
        "body"
      ]
    }
  },
  "normalizers": {
    "carlicplate": {
      "normalization_type": "carlicplate"
    },
    "cropbbox": {
      "normalization_type": "cropbbox"
    },
    "norm200": {
      "normalization_type": "norm200"
    }
  },
  "extractors": {
    "car_color": {
      "normalization": "crop1x",
      "model_name": "carattr_color.v0"
    },
    "car_quality": {
      "normalization": "cropbbox",
      "model_name": "carattr.quality.v0"
    },
    "face_emben": {
      "normalization": "norm200",
      "model_name": "kiwi_160"
    },
    "face_quality": {
      "normalization": "crop1x",
      "model_name": "quality_fast.v1"
    },
    "license_plate_quality": {
      "normalization": "cropbbox",
      "model_name": "carlicplateattr.quality.v0"
    }
  },
  "objects": {
    "car": {
      "quality_attribute": "car_quality",
      "base_normalizer": "cropbbox"
    },
    "face": {
      "quality_attribute": "face_quality",
      "base_normalizer": "crop2x"
    },
    "license_plate": {
      "quality_attribute": "license_plate_quality",
      "base_normalizer": "carlicplate"
    }
  }
}