Localization

What is Localization?

Localization is the process of determining a device's precise position and orientation (6-DoF pose) within a pre-scanned map. When a query image is sent to the VPS API, the system searches the map's database of reference frames for visually similar candidates, then computes an exact pose using feature matching and geometry.

By default the search covers the entire map. In large maps or multi-floor buildings this can be slow and prone to confusion between visually similar areas. Localization parameters let you provide prior knowledge (floor level, known position, GPS coordinates, map scope) to narrow the search before it begins, improving both speed and accuracy.

Query Types

MultiSet VPS offers two ways to localize a device against a map: a single frame query and a multi frame query. Both send camera imagery to the same localization engine and return a 6-DoF pose; they differ in how many frames are sent, how long the request takes, and how robust the result is.

Single Frame Query

A single frame query sends one camera frame (a base64-encoded image plus its metadata) to the VPS. It is the fastest option, with a typical response time of around 2 seconds, and is ideal when you need quick, frequent pose updates or are localizing in well-textured, stable environments.

• Endpoints: POST /vps/map/query (JSON body) and POST /vps/map/query-form (multipart form-data)

• Input: 1 query image (max 1280 px on the longer side) + metadata

• Response time: ~2 seconds

• Best for: fast, repeated localization in stable, feature-rich scenes

Multi Frame Query

A multi frame query sends 4 to 6 camera frames together with the local SLAM tracking pose (position and rotation from ARKit or ARCore) captured at the moment of each image. Combining multiple viewpoints makes the result more robust and accurate, especially in challenging conditions such as repetitive textures, partial occlusion, or minor scene changes. The trade-off is higher latency, with a typical response time of up to around 5 seconds.

• Endpoint: POST /vps/map/multi-image-query (multipart form-data)

• Input: 4 to 6 query images + per-frame SLAM tracking pose

• Response time: up to ~5 seconds

• Best for: robust localization in difficult or visually repetitive environments

Choosing a Query Type

	Single Frame Query	Multi Frame Query
Images per request	1	4 to 6
Endpoint	/vps/map/query, /vps/map/query-form	/vps/map/multi-image-query
Typical response time	~2 seconds	up to ~5 seconds
Robustness	Good in stable scenes	Higher, handles challenging scenes
Best for	Fast, frequent updates	Accuracy in difficult conditions

The localization parameters described below apply to both query types. See Map Query for full request and response details.

Localization Parameters

hintPosition

A 3D coordinate [x, y, z] in map-local space (left-handed / Unity convention) that tells the server approximately where the device is. The search is limited to frames within a radius of that point (default 25 m, configurable via hintRadius).

Format	Example
Array of 3 numbers (JSON body)	"hintPosition": [2.5, 0.1, 8.0]
JSON-encoded string (form-data)	"hintPosition": "[2.5, 0.1, 8.0]"

Coordinates use the same LHS coordinate system as the localization response (when isRightHanded: false, the default). You can pass a previous localization result's position directly as the hint for the next query.

Use when you have approximate position from BLE, UWB, WiFi fingerprinting, or a previous localization result. For GPS-georeferenced maps, use geoHint instead.

hintMapCodes (MapSet only)

A list of map codes that limits the search to specific maps within a MapSet. Useful in multi-floor buildings where floor-level context is already known (e.g. from a floor selector, QR code scan, or BLE beacon).

Format	Example
Array of strings (JSON body)	"hintMapCodes": ["MAP_RJFKKWQ1787J", "MAP_XPQR3456ABCD"]
JSON-encoded string (form-data)	"hintMapCodes": "[\"MAP_RJFKKWQ1787J\"]"

Only valid when querying against a mapSetCode. Ignored for single-map queries.

hintFloorHeight

A vertical band [y_min, y_max] in map-local Y coordinates. Only frames whose camera position falls within this Y range are considered. Useful for floor-level filtering in multi-floor spaces without needing per-floor map codes.

Format	Example
Array of 2 numbers (JSON body)	"hintFloorHeight": [3.0, 6.0]
JSON-encoded string (form-data)	"hintFloorHeight": "[3.0, 6.0]"

Order does not matter — [-3, -1] and [-1, -3] both select y ∈ [-3, -1]. See HintFloorHeight for full details.

geoHint

The device's GPS coordinates [latitude, longitude, altitude] passed as a comma-separated string. For georeferenced maps, the server converts these to map-local coordinates and applies the same spatial radius filtering as hintPosition. Requires the map to be georeferenced in the MultiSet Developer Portal.

Format	Example
Comma-separated string	"geoHint": "37.4219983,-122.084,10.5"

Works for both single Maps and MapSets. Combine with convertToGeoCoordinates: "true" to receive the pose result back in GPS coordinates. See GeoHint in Localization for full details.

hintRadius

Controls the search radius around hintPosition or geoHint. Default is 25 m. Smaller values are faster but require a more accurate prior; larger values are more forgiving.

Format	Range	Default
Number (JSON body)	1–100 m	25 m
Numeric string (form-data)	"25"	25 m

Only applies when hintPosition or geoHint is provided. Has no effect otherwise.

use2DFiltering

When true, the spatial radius filter ignores the Y-axis (altitude) and uses only horizontal distance (X and Z). Useful for tall buildings or outdoor environments where GPS altitude is unreliable.

Format	Default
Boolean (JSON body)	false
String "true" / "false" (form-data)	"false"

Only applies when geoHint is provided. Works for both Maps and MapSets. Has no effect with hintPosition alone.

How Parameters Combine

Parameters are applied in sequence — each one narrows the candidate set further:

All map frames
  → hintMapCodes      (keep only frames from these maps)
  → hintPosition / geoHint + hintRadius  (keep frames within radius)
  → use2DFiltering    (if set, ignore Y in radius check)
  → hintFloorHeight   (keep frames within vertical band)
  → visual matching   (match the query image against the remaining candidates)

Using more hints together reduces the search space more aggressively and is particularly effective in large, multi-floor MapSets.

---

Quick Reference

Parameter	Type	Applies to	Purpose
hintPosition	[x, y, z]	Map / MapSet	Spatial radius filter from a known point
hintMapCodes	[string, ...]	MapSet only	Limit search to specific maps
hintFloorHeight	[y_min, y_max]	Map / MapSet	Vertical band filter
geoHint	"lat,lon,alt"	Map / MapSet	GPS-based spatial filter
hintRadius	number (1–100)	Map / MapSet	Radius for hintPosition / geoHint
use2DFiltering	boolean	Map / MapSet	Ignore Y-axis in geoHint radius check