Object Tracking Query

Object Tracking Query API

Send a single query frame to detect and localize one of your tracked objects. The request returns a 6-DoF pose for the object that is visible in the frame.

The request takes an objectCode (a single string or an array of up to 10 codes), which is what enables multi-object tracking in a single session.

Tracking multiple objects in one query

objectCode can be an array — pass 1 to 10 object codes in a single query when your application is set up to track several different objects but only one is expected to be in the camera frame at any moment.

This is useful when:

• Your scene contains a small set of known trackable objects (e.g. five different products on a shelf, or three machines in a maintenance bay) and you want MultiSet to figure out which one the user is currently looking at.

• You don't want to run separate per-object queries every frame and pick the best result client-side — let the cloud do the matching across the candidate objects in one round trip.

How it works: if you pass, say, 5 object codes, the cloud searches the query frame against the tracking maps for all 5 candidates and detects whichever object is visible in the frame. If multiple candidates are in view, the system returns the best match.

Only one tracking pose is returned per query. Even when you send multiple object codes, the response contains the pose for the single object the system detected in that frame. The detected object is identified in the objectCodes field of the response (an array containing the matched code).

To track two or more objects simultaneously in the same frame, send separate queries — multi-object pose return in a single response is not currently supported.

All object codes in a single query must belong to your account and be in active status. Unknown, inactive, or cross-account codes are rejected. Maximum 10 candidates per query.

Request body fields

Field	Type	Required	Notes
queryImage	file (multipart)	yes	Single RGB frame from the device camera.
objectCode	string | string[] (1–10)	yes	Object code(s) to match against.
fx, fy	number	yes	Camera focal length in pixels.
px, py	number	yes	Camera principal point in pixels.
width, height	number	yes	Query image resolution in pixels.
isRightHanded	boolean	no	Default false (LHS / Unity). Set true for RHS coordinate output.

Object Tracking Query

post

Localize an object from a single image. Provide the image plus the camera intrinsics (fx, fy, px, py) and the image width/height. Returns the object's pose when found.

Authorizations

AuthorizationstringRequired

JWT token obtained from the authentication endpoint.

Body

queryImagestring · binaryRequired

The query image.

objectCodeone ofRequired

One object code, or an array of 1–10 object codes to match against.

stringOptional

or

string[] · min: 1 · max: 10Optional

fxnumberRequired

Focal length x (pixels).

fynumberRequired

Focal length y (pixels).

pxnumberRequired

Principal point x (pixels).

pynumberRequired

Principal point y (pixels).

widthnumberRequired

Image width (pixels).

heightnumberRequired

Image height (pixels).

isRightHandedbooleanOptional

Whether the returned pose should be in a right-handed coordinate system.

Responses

200

Query processed. Check poseFound.

application/json

or

400

Bad Request. Missing/invalid fields.

401

Unauthorized. Invalid or expired token.

403

Forbidden.

404

Object not found.

500

Internal Server Error.

post/vps/object/query

POST /v1/vps/object/query HTTP/1.1
Host: api.multiset.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: multipart/form-data
Accept: */*
Content-Length: 113

{
  "queryImage": "binary",
  "objectCode": "text",
  "fx": 1,
  "fy": 1,
  "px": 1,
  "py": 1,
  "width": 1,
  "height": 1,
  "isRightHanded": true
}

{
  "poseFound": true,
  "position": {
    "x": 1,
    "y": 1,
    "z": 1
  },
  "rotation": {
    "x": 1,
    "y": 1,
    "z": 1,
    "w": 1
  },
  "confidence": 1,
  "objectCodes": [
    "text"
  ]
}

Sample Response — pose found

When the cloud detects one of the candidate objects in the query frame:

{
    "poseFound": true,
    "position": {
        "x": 1.4085917235274734,
        "y": 0.4753337271392171,
        "z": 0.6372062117019925
    },
    "rotation": {
        "x": -0.10826634872129595,
        "y": 0.48317258532326446,
        "z": 0.07714477353215363,
        "w": 0.8653735230773266
    },
    "confidence": 0.46301103179753406,
    "objectCodes": [
        "OBJ_CL4JCEKIHWAJ"
    ]
}

• position / rotation — the detected object's 6-DoF pose in the camera coordinate frame (LHS / Unity by default; RHS when isRightHanded=true).

• objectCodes — array containing the single matched object code, even if multiple candidates were sent in the request. Use this field to tell which object was detected.

• confidence — match confidence in [0, 1].

Sample Response — pose not found

When none of the candidate objects could be detected in the frame:

{
    "poseFound": false,
    "errorCode": "001",
    "errorMessage": "Pose not found"
}

This is the expected response when the user hasn't pointed the camera at any of the tracked objects yet, or when the view is too occluded / motion-blurred. Continue querying with new frames until poseFound is true.

List Objects API

Returns the object codes available in your account — useful for building the candidate list you'll send to /vps/object/query.

List Objects

get

List the tracked objects in your account, with pagination and optional search.

Authorizations

AuthorizationstringRequired

JWT token obtained from the authentication endpoint.

Query parameters

pageintegerOptional

Page number.

limitintegerOptional

Page size.

querystringOptional

Optional search string.

Responses

200

Paginated list of objects.

application/json

dataobject[]Optional

currentPageintegerOptional

totalPagesintegerOptional

totalCountintegerOptional

400

Bad Request.

401

Unauthorized. Invalid or expired token.

500

Internal Server Error.

get/vps/object

GET /v1/vps/object HTTP/1.1
Host: api.multiset.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "data": [
    {}
  ],
  "currentPage": 1,
  "totalPages": 1,
  "totalCount": 1
}