Object Tracking Query

Object Tracking Query API

Send a single query frame to detect and localize one of your tracked objects. The request returns a 6-DoF pose for the object that is visible in the frame.

The request takes an objectCode (a single string or an array of up to 10 codes), which is what enables multi-object tracking in a single session.

Tracking multiple objects in one query

objectCode can be an array — pass 1 to 10 object codes in a single query when your application is set up to track several different objects but only one is expected to be in the camera frame at any moment.

This is useful when:

• Your scene contains a small set of known trackable objects (e.g. five different products on a shelf, or three machines in a maintenance bay) and you want MultiSet to figure out which one the user is currently looking at.

• You don't want to run separate per-object queries every frame and pick the best result client-side — let the cloud do the matching across the candidate objects in one round trip.

How it works: if you pass, say, 5 object codes, the cloud searches the query frame against the tracking maps for all 5 candidates and detects whichever object is visible in the frame. If multiple candidates are in view, the system returns the best match.

Only one tracking pose is returned per query. Even when you send multiple object codes, the response contains the pose for the single object the system detected in that frame. The detected object is identified in the objectCodes field of the response (an array containing the matched code).

To track two or more objects simultaneously in the same frame, send separate queries — multi-object pose return in a single response is not currently supported.

All object codes in a single query must belong to your account and be in active status. Unknown, inactive, or cross-account codes are rejected. Maximum 10 candidates per query.

Request body fields

Field	Type	Required	Notes
queryImage	file (multipart)	yes	Single RGB frame from the device camera.
objectCode	string | string[] (1–10)	yes	Object code(s) to match against.
fx, fy	number	yes	Camera focal length in pixels.
px, py	number	yes	Camera principal point in pixels.
width, height	number	yes	Query image resolution in pixels.
isRightHanded	boolean	no	Default false (LHS / Unity). Set true for RHS coordinate output.

ModelSet Query API

post

Query a VPS object using an uploaded image and camera parameters. Accepts objectCode to identify the target object, and returns the query result from the external API.

Authorizations

bearerAuth

AuthorizationstringRequired

Standard Authorization header using the Bearer scheme. Example: "Authorization: Bearer {token}"

Body

multipart/form-data

objectCodestringRequired

The code of the VPS object

Example: OBJ-001

fxstringRequired

Camera intrinsic parameter fx (focal length x-axis)

Example: 1000.0

fystringRequired

Camera intrinsic parameter fy (focal length y-axis)

Example: 1000.0

pxstringRequired

Camera intrinsic parameter px (principal point x-coordinate)

Example: 640.0

pystringRequired

Camera intrinsic parameter py (principal point y-coordinate)

Example: 480.0

isRightHandedstring · enumRequired

Specifies whether the coordinate system is right-handed. Because this is form-data, it's sent as a string ("true" or "false").

Example: truePossible values:

widthstringRequired

The width (in pixels) of the input image

Example: 1280

heightstringRequired

The height (in pixels) of the input image

Example: 720

queryImagestring · binaryRequired

The image file to query against the object

Responses

200

Query successful

application/json

successbooleanRequired

Whether the query was successful

Example: true

messagestringRequired

A descriptive message about the outcome of the query.

Example: Query successful.

400

Bad request - Invalid parameters or missing required fields

application/json

401

Unauthorized - Missing or invalid token

application/json

403

Forbidden - Access denied

application/json

404

Object not found

application/json

500

Internal server error

application/json

post

/vps/object/query

HTTPcURLJavaScriptPython

POST /v1/vps/object/query HTTP/1.1
Host: api.multiset.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: multipart/form-data
Accept: */*
Content-Length: 153

{
  "objectCode": "OBJ-001",
  "fx": "1000.0",
  "fy": "1000.0",
  "px": "640.0",
  "py": "480.0",
  "isRightHanded": "true",
  "width": "1280",
  "height": "720",
  "queryImage": "binary"
}

{
  "success": true,
  "result": {
    "position": {
      "x": 2.1234,
      "y": 1.5678,
      "z": 0.9876
    },
    "rotation": {
      "qx": 0.0123,
      "qy": 0.4567,
      "qz": 0.0789,
      "qw": 0.8901
    },
    "confidence": 0.92
  },
  "message": "Query successful."
}

Sample Response — pose found

When the cloud detects one of the candidate objects in the query frame:

{
    "poseFound": true,
    "position": {
        "x": 1.4085917235274734,
        "y": 0.4753337271392171,
        "z": 0.6372062117019925
    },
    "rotation": {
        "x": -0.10826634872129595,
        "y": 0.48317258532326446,
        "z": 0.07714477353215363,
        "w": 0.8653735230773266
    },
    "confidence": 0.46301103179753406,
    "objectCodes": [
        "OBJ_CL4JCEKIHWAJ"
    ]
}

• position / rotation — the detected object's 6-DoF pose in the camera coordinate frame (LHS / Unity by default; RHS when isRightHanded=true).

• objectCodes — array containing the single matched object code, even if multiple candidates were sent in the request. Use this field to tell which object was detected.

• confidence — match confidence in [0, 1].

Sample Response — pose not found

When none of the candidate objects could be detected in the frame:

{
    "poseFound": false,
    "errorCode": "001",
    "errorMessage": "Pose not found"
}

This is the expected response when the user hasn't pointed the camera at any of the tracked objects yet, or when the view is too occluded / motion-blurred. Continue querying with new frames until poseFound is true.

List Objects API

Returns the object codes available in your account — useful for building the candidate list you'll send to /vps/object/query.

Get a paginated list of VPS objects

get

Retrieves a paginated list of VPS objects. You can use query parameters to filter or search objects by name, and control pagination settings.

Authorizations

bearerAuth

AuthorizationstringRequired

Standard Authorization header using the Bearer scheme. Example: "Authorization: Bearer {token}"

Query parameters

pageinteger · min: 1Optional

The page number for pagination (starting from 1)

Default: 1Example: 1

limitinteger · min: 1 · max: 100Optional

Number of records per page (maximum 100)

Default: 10Example: 10

querystring · max: 255Optional

Filter or search text for partial match on object names

Example: Multiset

Responses

200

A paginated list of VPS objects retrieved successfully

application/json

currentPageinteger · min: 1Required

Current page number

Example: 1

totalPagesintegerRequired

Total number of pages available

Example: 5

totalCountintegerRequired

Total number of objects across all pages

Example: 42

400

Bad request - Invalid query parameters

application/json

401

Unauthorized - Missing or invalid token

application/json

500

Internal server error

application/json

get

/vps/object

HTTPcURLJavaScriptPython

GET /v1/vps/object HTTP/1.1
Host: api.multiset.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "data": [
    {
      "_id": "67a5b08736cb3f76aaf3a416",
      "objectName": "Multiset Object 1",
      "trackingType": "side",
      "source": {
        "provider": "web",
        "fileType": "glb",
        "coordinateSystem": "RHS"
      }
    },
    {
      "_id": "67a5b08736cb3f76aaf3a417",
      "objectName": "Multiset Object 2",
      "trackingType": "top",
      "source": {
        "provider": "mobile",
        "fileType": "obj",
        "coordinateSystem": "LHS"
      }
    }
  ],
  "currentPage": 1,
  "totalPages": 5,
  "totalCount": 42
}