# MultiSet Developer Docs
### Introduction
In this guide, you'll embark on an exciting journey to build your first application using the innovative MultiSet feature, enhanced with 3D mapping and a Visual Positioning System (VPS). By leveraging these advanced technologies, your app can deliver a more immersive and interactive user experience.
### Getting Started with MultiSet
The MultiSet developer documentation is your essential resource for creating and integrating MultiSet features into your application. This comprehensive guide will walk you through the processes of utilizing our robust APIs, SDKs, and AI Plugins. By following this documentation, you'll understand how to effectively harness these tools, paving the way for a seamless integration of cutting-edge functionalities.
The MultiSet Unity SDK is designed for creating immersive, large-scale, location-based experiences. It extends Unity's AR Foundation subsystems, allowing developers to seamlessly mix and match MultiSet's SDK features with Unity's existing AR framework. Any existing AR Foundation project can be upgraded with the MultiSet SDK. Developers can use the Unity documentation and tutorials on AR Foundation for basic AR concepts and then extend them to make use of MultiSet's powerful features.
## MultiSet Features
* **Large-Scale VPS:** Our Visual Positioning System (VPS) scales to spaces over 100,000 sq ft, perfect for large, location-based experiences.
* **Map Stitching:** Merge multiple maps using MapSet to cover even larger and more complex areas seamlessly.
* **Standardized Localization:** Delivers standard localization output in WGS 84 and GeoPose formats for universal compatibility.
* **Robust Precision:** Achieve centimeter-precise VPS performance even in challenging conditions with dynamic lighting and environmental changes.
* **Third-Party Scan Integration:** Easily use existing scans from Matterport (MatterPak, E57) and other scanners that support E57 files (e.g., Leica RTC 360, BLK 360, NavVis).
* **360 Video Capture:** Turn a single hand-held walk with an Insta360 X4 / X5 into a centimeter-accurate VPS map. Upload the raw `.insv` footage and MultiSet stitches and reconstructs it server-side. See [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans).
* **Map Versioning:** Re-scan a space at any time and merge the new capture into the same coordinate frame, so your content layer (anchors, navigation paths, instructions) keeps working without re-authoring. Powered by the MultiSet VPS rather than ICP, so it stays robust across months, hardware changes, and visual changes. See [Map Versioning](/multiset/basics/map-versioning).
* **Object Tracking:** Anchor and track 3D objects in the real world using any GLB file with our Object Tracking feature.
* **Cross-Platform SDKs:** Build your applications with comprehensive SDKs for Unity, WebXR, and native iOS and Android.
* **Shared AR Experiences:** Enable multiplayer and shared AR experiences with a unified coordinate system, allowing users to interact in the same digital space.
## Platform Architecture
MultiSet is an end-to-end VPS platform: scans go in on one side, 6-DoF pose comes out on the other, and everything in between runs in your account-isolated tenant on the MultiSet Cloud. The diagram below shows the four stages — **map ingestion**, **cloud processing**, **authentication**, and **VPS query from the SDK** — and how data is kept private to your account at every step.
End-to-end MultiSet pipeline with per-account data isolation
### How the pieces fit together
1. **Map ingestion.** You bring spatial data into the platform from the [MultiSet Mapping App](/multiset/basics/maps) (LiDAR iPhone / iPad), from [third-party scanners](/multiset/basics/third-party-scans) (Matterport, Leica, NavVis, Faro, Xgrids), from a [360° video capture](/multiset/basics/third-party-scans/insta360-scans) (Insta360 X4 / X5), or from a [Gaussian Splat](/multiset/basics/third-party-scans/gaussian-splat). For [Object Tracking](/multiset/basics/object-tracking), you upload a textured GLB. Every upload is authenticated and bound to your account.
2. **Cloud processing.** The cloud processes your scan and prepares a VPS-ready map. When processing finishes, the map's status moves to **Active** and is ready to be queried. Multiple maps can be combined into a [MapSet](/multiset/basics/mapset-multiple-maps) for large venues, any map can be [georeferenced](/multiset/basics/georeferencing-maps) to WGS 84 for GeoPose output, and a space can be re-scanned and merged into the same coordinate frame with [Map Versioning](/multiset/basics/map-versioning) so your content layer survives the update.
3. **Authentication.** All access is gated by [client credentials](/multiset/basics/credentials): you exchange a `clientId` + `clientSecret` for a short-lived JWT Bearer token via `/v1/m2m/token`. Every API request is checked for signature, expiry, scope (Query / Write / Delete), and CORS origin — see the full flow in [Authentication](/multiset/basics/rest-api-docs/authentication).
4. **VPS query.** A device captures one or more frames and sends them to the [Map Query](/multiset/basics/rest-api-docs/map-query) endpoint (directly via REST or through one of the SDKs). [Localization parameters](/multiset/basics/localization) like `hintPosition`, `hintFloorHeight`, `geoHint`, and `hintMapCodes` narrow the search before image retrieval runs. The response is a centimeter-precise 6-DoF pose, optionally returned in WGS 84 / GeoPose form.
5. **Cross-platform SDKs.** The [Unity SDK](https://github.com/MultiSet-AI/docs.multiset.ai/tree/Development/unity-sdk/README.md), [Quest SDK](https://github.com/MultiSet-AI/docs.multiset.ai/tree/Development/multiset-quest-sdk/README.md), [iOS Swift](/multiset/native-support/ios-swift-native) and [Android](/multiset/native-support/android-native) native SDKs, and direct REST integration (including WebXR) all consume the same VPS service — so a map captured on iOS works seamlessly across every supported runtime.
### Data isolation & privacy
MultiSet is a multi-tenant platform with strict per-account isolation:
* **Per-account scope.** Maps, MapSets, object-tracking models, credentials, and CORS settings live inside a single account. There is no concept of cross-account access — your tokens can only see and query your own data.
* **No public maps.** Account data is never indexed, shared, or exposed to other developers. Maps are not discoverable outside your account, and there is no shared "world map."
* **Authorization on every request.** Every API call is verified by JWT, gated by the credential's scope, and filtered by your account's CORS allowlist for browser origins. A read-only credential cannot write or delete, even with a valid token.
* **Credential hygiene.** Client secrets are shown once at creation, can be rotated or deactivated at any time, and are intended to be stored in environment variables or a secrets manager — never embedded in client-side code.
{% embed url="" %}
# Getting Started
The end-to-end path through MultiSet, from your first scan to a running app with global pose and object tracking.
This page walks the full journey: scan a space, build against it with the SDK of your choice, then layer on MapSets, Map Versions, Object Tracking, and georeferencing as your project needs them. Each step links to the detailed guide.
## Step 1: Create your account
1. Register or log in at the [MultiSet Developer Portal](https://developer.multiset.ai/).
2. Open **Credentials** and copy your **Client ID** and **Client Secret**. Every SDK and every REST call authenticates with this pair. See [Credentials](/multiset/basics/credentials).
{% hint style="info" %}
Calling the API from a browser? Whitelist your domain first, see [Configuring Allowed Domains (CORS)](/multiset/basics/credentials/configuring-allowed-domains-cors).
{% endhint %}
## Step 2: Scan your space
A **Map** is a single scanned space. You can create one with a phone or bring in data from professional survey hardware, whichever suits the size and accuracy your site needs.
### Option A: Scan with an iPhone or iPad
The fastest way to a working map. Install the MultiSet app on a **LiDAR-equipped iPhone or iPad**, walk the space, and upload. No extra hardware required.
* [**MultiSet on the App Store**](https://apps.apple.com/us/app/multiset/id6737130008) : scanning, localization, content space, and geo-referencing
* [**MultiSet on Google Play**](https://play.google.com/store/apps/details?id=ai.multiset.maps) : localization, content space setup, and geo-referencing
{% hint style="warning" %}
**Scanning is iOS only.** Mapping requires the LiDAR sensor on an iPhone or iPad, so the Android app cannot create maps. Use it to test localization, set up a content space, and geo-reference maps you have already captured.
{% endhint %}
* [Maps](/multiset/basics/maps) : create and manage a map from the app
* [Mapping Instruction](/multiset/basics/maps/mapping-instruction) : how to walk the space while scanning
* [Mapping Planning](/multiset/basics/maps/mapping-planning) : plan your coverage before you start
* [Mapping Equipment](/multiset/basics/maps/mapping-equipment) : what to bring for larger sites
### Option B: Upload a third-party scan
Already have survey-grade data, or need to cover a large site at high accuracy? Upload an existing scan instead of rescanning.
| Capture method | Hardware | Guide |
| --------------------- | ---------------------------- | ------------------------------------------------------------------- |
| **360 video** | Insta360 X4, X5 | [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans) |
| **Terrestrial LiDAR** | Matterport Pro2, Pro3 | [Matterport E57](/multiset/basics/third-party-scans/matterport-e57) |
| **Terrestrial LiDAR** | Leica RTC360, BLK360, BLK2GO | [Leica Scans](/multiset/basics/third-party-scans/leica-scans) |
| **Mobile mapping** | NavVis VLX, M6 | [NavVis Scans](/multiset/basics/third-party-scans/navvis-scans) |
| **Mobile mapping** | Faro Focus, Orbis, Flash | [Faro Scans](/multiset/basics/third-party-scans/faro-scans) |
| **SLAM scanner** | Xgrids L2 Pro, K1 | [Xgrids Scans](/multiset/basics/third-party-scans/xgrids-scans) |
| **Gaussian Splat** | Xgrids via Lixel CyberColor | [Gaussian Splat](/multiset/basics/third-party-scans/gaussian-splat) |
See [Third Party Scans](/multiset/basics/third-party-scans) for the full list of supported vendors, file types, and input requirements. Scanners not on the list usually work too, as long as the `.e57` is structured the same way.
## Step 3: Build your app
Once a map is **Active**, pick the integration path that matches your target platform. Every SDK handles authentication, localization, and content anchoring for you, so you rarely need to touch the REST API directly.
| Platform | Use it for | Docs | GitHub |
| --------------------------- | ----------------------------------------------------------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| **Unity** | Cross-platform AR on iOS and Android, the most complete SDK | [Getting Started](/multiset/unity-sdk/getting-started) | [multiset-unity-sdk](https://github.com/MultiSet-AI/multiset-unity-sdk) |
| **iOS Native (Swift)** | Native ARKit apps with no Unity dependency | [iOS Swift Native](/multiset/native-support/ios-swift-native) | [multiset-ios-sdk](https://github.com/MultiSet-AI/multiset-ios-sdk) |
| **Android Native (Kotlin)** | Native ARCore apps with no Unity dependency | [Android Native](/multiset/native-support/android-native) | [multiset-android-sdk](https://github.com/MultiSet-AI/multiset-android-sdk) |
| **WebXR** | Browser-based AR with Three.js or Needle Engine, no install | [WebXR SDK](/multiset/webxr-sdk/webxr-sdk) | [@multisetai/vps](https://www.npmjs.com/package/@multisetai/vps) (npm) |
| **Meta Quest** | Headset MR experiences on Quest 3 and Quest 3S | [Installation Guide](/multiset/multiset-quest-sdk/installation-guide) | [multiset-quest-sdk](https://github.com/MultiSet-AI/multiset-quest-sdk) |
| **Meta Wearables** | Meta Ray-Ban Smart Glasses via the Meta Wearables Device Access Toolkit | [Meta Ray-Ban Smart Glasses](#meta-ray-ban-smart-glasses) | [wearable-vps-samples](https://github.com/MultiSet-AI/wearable-vps-samples) |
| **REST API** | Your own client, robotics, backend services, custom engines | [REST API Docs](/multiset/basics/rest-api-docs) | [MultiSet-AI](https://github.com/MultiSet-AI) |
All SDKs and samples are published under [github.com/MultiSet-AI](https://github.com/MultiSet-AI).
### Meta Ray-Ban Smart Glasses
Wearables support ships as standalone sample apps rather than a packaged SDK. Both samples use the **Meta Wearables Device Access Toolkit (DAT SDK)** to pair with Meta Ray-Ban Smart Glasses, stream the live camera feed, and run VPS localization from the paired phone:
* [**iOS sample**](https://github.com/MultiSet-AI/wearable-vps-samples/tree/main/iOS) : localization, audio turn-by-turn navigation to Points of Interest, on-lens HUD navigation on Ray-Ban **Display** glasses, and multiplayer pose sharing with a host running the MultiSet iOS SDK.
* [**Android sample**](https://github.com/MultiSet-AI/wearable-vps-samples/tree/main/Android) : localization, audio turn-by-turn navigation, and a live 2D map view.
Both need the **Meta AI app with Developer Mode enabled**, a pair of Meta Ray-Ban Smart Glasses, and your MultiSet Client ID and Secret. For a Unity host sharing a session with a wearable client, see [Multiplayer Sample](/multiset/unity-sdk/sample-scenes/multiplayer-sample).
### Going direct with the REST API
If you are working on a platform we do not ship an SDK for, or you want full control on the client side, call the API yourself. [Authenticate](/multiset/basics/rest-api-docs/authentication) once for a JWT, then send camera frames to the [Map Query](/multiset/basics/rest-api-docs/map-query) endpoint and get a 6-DoF pose back. See [Localization](/multiset/basics/localization) for how single frame and multi frame queries differ and which hint parameters narrow the search.
## Step 4: Grow beyond a single map
Two features handle almost every map operation you will run into as a project grows.
### MapSet: cover a larger area
A **MapSet** joins multiple maps into one coordinate frame, so a venue too large for a single scan still behaves as one seamless space. Each map keeps its own detail while the MapSet holds the precise relative transforms between them. Query the `mapSetCode` and the VPS resolves the pose across every map in the set.
Add a map to a MapSet and it aligns against the whole set automatically, existing maps are untouched.
* [MapSet : Multiple Maps](/multiset/basics/mapset-multiple-maps)
* [Merging Maps with Overlap](/multiset/basics/mapset-multiple-maps/merging-maps-with-overlap) : joining two scans that share physical space
* [Merging Maps without Overlap](/multiset/basics/mapset-multiple-maps/merging-maps-without-overlap) : joining scans that do not touch
### Map Version: update a space that changed
A **Map Version** groups two or more scans of the *same* space captured at different times. One scan is the base, and every newer scan is aligned to it by the VPS. Whether you rescan a corner of a room or the entire floor, the update is added as a version instead of replacing the map.
The reason this matters: **your content layer never moves.** Anchors, navigation paths, and AR overlays stay authored against the base map's frame, even when the new scan comes from different hardware with a different coordinate system. MultiSet computes the transform once and translates every query transparently.
* [Map Versioning](/multiset/basics/map-versioning)
{% hint style="success" %}
Use a **MapSet** to make a space bigger. Use a **Map Version** to make a space newer.
{% endhint %}
## Step 5: Add Object Tracking
Maps are inside-out: they track where the user is inside a space. **Object Tracking** is outside-in: it tracks a specific physical object while the user moves around it. Reach for it when your use case needs to anchor content to a machine, product, exhibit, or prop rather than to the room.
Upload a textured GLB of the target object, and MultiSet builds a tracking map from its geometry and texture. Object Tracking runs **with or without** a VPS map, in the **same session or a separate one**, so you can localize in a building and track a machine inside it, or track the object on its own with no map at all.
* [Object Tracking](/multiset/basics/object-tracking) : concepts and model requirements
* [How to Setup Object Tracking](/multiset/basics/object-tracking/how-to-setup-a-object-tracking) : end-to-end setup
* [Object Tracking Query](/multiset/basics/rest-api-docs/object-query) : the REST endpoint
## Step 6: Georeference for global pose
By default a localization result is in the map's **local** coordinate frame. If your app needs **global** pose, latitude and longitude rather than local XYZ, georeference the map. Once it is georeferenced you can pass a `geoHint` from device GPS to speed up localization, and convert results back to WGS84 coordinates.
There are three ways to get there:
* **Automatic:** Walk the site with the MultiSet App, localize at several well-spread points, and let the server solve the transform from the paired VPS poses and GPS fixes. Best for outdoor maps with clear sky view. See [Auto Geo-reference a Map](/multiset/basics/georeferencing-maps/auto-georeference-map).
* **Manual:** Align the scan against satellite imagery by hand and set the origin and heading yourself. Use this indoors or wherever GPS is unreliable. See [How to Manually Align Scans](/multiset/basics/georeferencing-maps/how-to-align-scans).
* **Inherited from the scan:** If the `.e57` you uploaded is already georeferenced, the map arrives georeferenced with no extra step.
* [Georeferencing Maps](/multiset/basics/georeferencing-maps) : setup and GeoHint
* [GeoHint in Localization](/multiset/basics/localization/geohint-in-localization) : using GPS as a localization prior
* [GeoPose Support](/multiset/basics/localization/geopose-support) : getting results back in global coordinates
## Where to go next
* [Localization](/multiset/basics/localization) : query types and the hint parameters that make localization faster and more accurate
* [Simulation](/multiset/basics/localization/simulation) : test a map in the portal without returning to the site
* [Simulation Data Capture](/multiset/multiset-app/simulation-data-capture) : record test data once, replay it anywhere
* [FAQ](/multiset/quick-access/faq) : common questions about the platform
* [Support](/multiset/basics/support) : get help from the MultiSet team
# FAQ
FAQ's around MultiSet Developer platform
## General
**Q: What does MultiSet AI provide?**\
**A:** MultiSet AI is an enterprise spatial computing platform that gives phones, headsets, glasses, and robots precise 6-DoF localization in the real world. It powers navigation, inspection, training, and digital-twin overlays across indoor and outdoor sites.
**Q: How is MultiSet different from other VPS or AR toolkits?**\
**A:** MultiSet is scan-agnostic and cross-platform: bring data from any supported scanner and deploy to iOS, Android, headsets, browsers, or robots. There is no vendor lock-in and no proprietary hardware requirement.
**Q: Is MultiSet built on Google Cloud Anchors or Apple World Map?**\
**A:** No. MultiSet built its VPS from the ground up, which is what lets it scale to very large spaces and stay agnostic to device form factor and platform.
**Q: Which devices are supported?**\
**A:** Any modern phone, tablet, AR headset, or robot with a standard RGB camera. LiDAR adds accuracy when scanning but is not required for localization.
**Q: What SDKs are available?**\
**A:** Unity, native iOS, native Android, WebXR, and Meta Quest, plus sample apps for Meta Ray-Ban Smart Glasses. Anything else, including robotics, integrates directly against the [REST API](/multiset/basics/rest-api-docs).
**Q: How do I get started?**\
**A:** Follow [Getting Started](/multiset/quick-access/getting-started): create an account, scan or upload a space, then pick an SDK. Most teams localize their first device the same day.
**Q: What are the costs of using MultiSet?**\
**A:** A free tier supports prototyping. Production pricing scales by maps, cumulative area, and API calls, with custom SLAs available for private deployments.
## 3D Mapping
**Q: How can I map a space?**\
**A:** Scan with the [MultiSet app](https://apps.apple.com/us/app/multiset/id6737130008) on a LiDAR-equipped iPhone or iPad, or upload an existing third-party scan. See [Maps](/multiset/basics/maps).
**Q: Can I map with an Android device?**\
**A:** No. Scanning requires the LiDAR sensor on an iPhone or iPad. The [Android app](https://play.google.com/store/apps/details?id=ai.multiset.maps) is for testing localization, setting up a content space, and geo-referencing existing maps.
**Q: What scan formats does MultiSet accept?**\
**A:** Structured `.e57` with panoramas (Matterport, Leica, NavVis, Faro, Xgrids), Matterport MatterPak, Gaussian Splat (`.ply` plus `poses.json`), and 360 video (`.insv` from Insta360 X4/X5). See [Third Party Scans](/multiset/basics/third-party-scans).
**Q: My scanner is not on the supported list. Will it work?**\
**A:** Usually yes, as long as the `.e57` is structured the same way, with the point cloud and its panoramic images embedded. Pick the closest provider at upload and the backend detects the structure.
**Q: How does MapSet enable large-scale coverage?**\
**A:** A MapSet joins many smaller maps into one coordinate frame so a large venue behaves as a single seamless space. Each map keeps its own detail while the set holds the relative transforms. See [MapSet](/multiset/basics/mapset-multiple-maps).
**Q: How much overlap should adjacent maps have?**\
**A:** For automatic overlap-based merging, aim for 5 to 10 meters or roughly 15 to 20% shared area. Overlap is not mandatory: maps without it can be merged manually in the app or the portal.
**Q: Can I update one area without re-mapping the whole venue?**\
**A:** Yes, with [Map Versioning](/multiset/basics/map-versioning). Rescan just the changed zone, add it as a version of that map, and activate it. The rest of the map keeps serving queries unchanged.
**Q: Does my AR content need to be re-authored after a rescan?**\
**A:** No. Content stays authored against the base map's coordinate frame, and MultiSet computes the transform to each new version automatically, even when the new scan uses a different coordinate system.
**Q: What is the difference between a MapSet and a Map Version?**\
**A:** A MapSet groups maps covering *different* physical areas, making a space bigger. A Map Version groups scans of the *same* area captured at different times, making a space newer.
**Q: How do I geo-reference a map?**\
**A:** Three ways: automatically from the app by localizing at well-spread points, manually by aligning the scan over satellite imagery, or automatically on upload if your `.e57` is already geo-referenced. See [Georeferencing Maps](/multiset/basics/georeferencing-maps).
**Q: Can I export maps to other spatial tools?**\
**A:** Yes. Maps can be downloaded as raw or textured GLB files for use in BIM, game engines, or analytics platforms.
## Visual Positioning System (VPS)
**Q: What is VPS and why do enterprises need it?**\
**A:** A VPS gives a device its precise 6-DoF position and orientation so AR content aligns exactly to physical assets, indoors and at scale where GPS, Wi-Fi, and beacons fall short. Robots use it to navigate, and smart glasses use it to anchor hands-free instructions.
**Q: How accurate is the VPS?**\
**A:** MultiSet delivers centimeter-level localization, with a median positional error of about 6 cm in well-mapped spaces. Accuracy depends on scan quality, coverage density, and how much the space has changed since capture.
**Q: What is the difference between a single frame and a multi frame query?**\
**A:** A single frame query sends one image and returns a pose in about 2 seconds. A multi frame query sends 4 to 6 images plus tracking pose for a more robust result in up to about 5 seconds. See [Localization](/multiset/basics/localization).
**Q: Can I speed up localization with GPS or a known position?**\
**A:** Yes. Hints narrow the search before matching begins: `hintPosition` and `hintRadius` for a known point, `geoHint` for GPS, `hintFloorHeight` for a floor band, and `hintMapCodes` to limit which maps in a MapSet are searched.
**Q: How does MultiSet handle lighting and environmental changes?**\
**A:** The neural networks behind the VPS are trained across diverse lighting and dynamic scenes, so localization stays robust through typical lighting shifts, minor rearrangement, and people moving through frame.
**Q: Can I run localization offline, without a network call?**\
**A:** Yes, with [On-Device Localization](/multiset/unity-sdk/on-device-localization), an Enterprise feature for iOS and Android. Map bundles are cached locally, so after a one-time online verification the device localizes with no internet.
**Q: Can I test a map without going back to the site?**\
**A:** Yes. Capture [simulation data](/multiset/multiset-app/simulation-data-capture) once at the location, then replay it against any map in the Developer Portal or in the Unity Editor to compare query types and inspect the returned pose.
**Q: Does MultiSet work indoors, outdoors, and across multiple floors?**\
**A:** Yes. Maps can span factory floors, loading docks, and outdoor areas, and multi-floor venues are handled with MapSets plus floor-level hints.
**Q: Does MultiSet support indoor navigation and wayfinding?**\
**A:** Yes. The Unity SDK combines VPS localization with Unity NavMesh for turn-by-turn pathing, and MapSets keep a single coordinate frame across a multi-floor facility.
**Q: Are there VPS limitations I should know about?**\
**A:** Vision-based localization can struggle after drastic structural change, in feature-poor or heavily reflective areas, and in the dark. Plan periodic map refreshes through Map Versioning for spaces that change often.
**Q: Can I deploy without an app install?**\
**A:** Yes. The [WebXR SDK](/multiset/webxr-sdk/webxr-sdk) runs AR in the browser with no install, using Three.js or Needle Engine.
## Object Tracking
**Q: What does "markerless object tracking" mean?**\
**A:** No stickers, QR codes, or fiducials are needed. The system uses the object's own geometry and texture to recognize and track it in 3D.
**Q: How does Object Tracking work?**\
**A:** You upload a textured 3D model, and the cloud builds an optimized tracking map from its geometry and texture. The SDK then detects the object and maintains its pose locally for low-latency AR.
**Q: Does Object Tracking need a map?**\
**A:** No. It runs with or without a VPS map, in the same session or a separate one, so you can localize in a building and track a machine inside it, or track the object on its own.
**Q: Which file formats are supported?**\
**A:** `.glb` only. The Developer Portal accepts the file directly, while the [REST API](/multiset/basics/rest-api-docs/modelset-upload) expects it packaged in a `.zip`. CAD and raw scans must be converted to a textured polygonal mesh first.
**Q: What are the model requirements?**\
**A:** Real-world metric scale (1 unit equals 1 meter), `+Y` up axis, a rigid non-deforming object, and a diffuse texture map. Untextured or single-color models are rejected.
**Q: What are the size limits?**\
**A:** The object should be between 1 and 50 feet on its longest dimension, and the model file should stay under 50 MB. See [How to Setup Object Tracking](/multiset/basics/object-tracking/how-to-setup-a-object-tracking).
**Q: Which objects track best?**\
**A:** Asymmetrical objects with rich visual detail such as logos, panels, and varied geometry. Highly symmetrical, featureless, or mirror-finish items are the hardest.
**Q: Can it track moving or deformable objects?**\
**A:** No. Object Tracking is designed for static, rigid objects. The camera can move freely around the object, but the object itself must stay put.
**Q: What is the difference between 360 View and Side View?**\
**A:** 360 View recognizes the object from any angle and is the default. Side View optimizes for objects viewed mainly from the sides and processes slightly faster.
**Q: How long does object processing take?**\
**A:** Typically under 10 minutes, depending on the complexity of the model.
## Integration & Deployment
**Q: How do I authenticate?**\
**A:** Create a Client ID and Secret in the Developer Portal, exchange them for a JWT bearer token valid for 30 minutes, and send it with every request. See [Authentication](/multiset/basics/rest-api-docs/authentication).
**Q: Can I use MultiSet from a browser?**\
**A:** Yes, but never embed your Client Secret client-side. Whitelist your domain for [CORS](/multiset/basics/credentials/configuring-allowed-domains-cors) and mint short-lived tokens from your own backend.
**Q: Can my scan data stay on our own infrastructure?**\
**A:** Yes. [On-Premises Localization](/multiset/basics/localization/on-premises-localization) runs the MultiSet VPS on your own servers, so scan data never leaves your network. It is an Enterprise option with deployment support from our team.
**Q: Can I deploy fully offline?**\
**A:** Yes. On-device localization keeps map bundles and the license token cached locally, so after first-run verification the app needs no network at all, which suits air-gapped sites.
**Q: How secure is my mapping data?**\
**A:** Data is encrypted in transit and at rest, and the platform enforces strict per-account isolation with scoped API credentials. Private and on-premises deployments are available for stricter governance requirements.
**Q: Does MultiSet support robotics and headsets?**\
**A:** Yes. Headsets are covered by the Meta Quest SDK and the Meta Ray-Ban wearable samples. Robotics integrates through the REST API, which returns the same pose the SDKs consume.
# Changelog
Release notes for each version of the MultiSet platform. Select a version below to see what changed.
* [Version 2.2.0](/multiset/quick-access/changelog/version-2.2.0) — 30-July-2026
* [Version 2.1.0](/multiset/quick-access/changelog/version-2.1.0) — 11-July-2026
* [Version 2.0.0](/multiset/quick-access/changelog/version-2.0.0) — 4-June-2026
* [Version 1.14.0](/multiset/quick-access/changelog/version-1.14.0) — 13-May-2026
* [Version 1.12.0](/multiset/quick-access/changelog/version-1.12.0) — 23-April-2026
* [Version 1.11.1](/multiset/quick-access/changelog/version-1.11.1) — 30-March-2026
* [Version 1.11.0](/multiset/quick-access/changelog/version-1.11.0) — 02-Febuary-2026
* [Version 1.10.0](/multiset/quick-access/changelog/version-1.10.0) — 08-December-2025
* [Version 1.9.3](/multiset/quick-access/changelog/version-1.9.3) — 16-November-2025
* [Version 1.9.2](/multiset/quick-access/changelog/version-1.9.2) — 09-October-2025
* [Version 1.9.1](/multiset/quick-access/changelog/version-1.9.1) — 29-September-2025
* [Version 1.9.0](/multiset/quick-access/changelog/version-1.9.0) — 12-September-2025
* [Version 1.8.1](/multiset/quick-access/changelog/version-1.8.1) — 22-August-2025
* [Version 1.8.0](/multiset/quick-access/changelog/version-1.8.0) — 25-July-2025
* [Version 1.7.0](/multiset/quick-access/changelog/version-1.7.0) — 14-July-2025
* [Version 1.6.5](/multiset/quick-access/changelog/version-1.6.5) — 4-June-2025
* [Version 1.6.0](/multiset/quick-access/changelog/version-1.6.0) — 21-April-2025
* [Version 1.5.2](/multiset/quick-access/changelog/version-1.5.2) — 14-March-2025
* [Version 1.5.0](/multiset/quick-access/changelog/version-1.5.0) — 26-February-2025
* [Version 1.4.0](/multiset/quick-access/changelog/version-1.4.0) — 30-January-2025
* [Version 1.3.0](/multiset/quick-access/changelog/version-1.3.0) — 06-January-2025
* [Version 1.2.0](/multiset/quick-access/changelog/version-1.2.0) — 26-December-2024
* [Version 1.1.0](/multiset/quick-access/changelog/version-1.1.0) — 10-December-2024
# Version 2.2.0
### v2.2.0 30-July-2026
**Added**
1. **360° Virtual Tour (Pano) APIs.** New read APIs serve navigable 360° panoramic tours built from your maps. List pano-enabled maps, fetch a viewer-ready manifest of every node, or request a local navigation "window" around a node or a world position. Including continuous tours that span map versions and MapSets. [360° Virtual Tour (Pano)](/multiset/basics/rest-api-docs/pano-360-virtual-tour)
2. **Camera Intrinsics API.** Estimate a camera's intrinsic parameters (focal length and principal point) from one to five images, with an optional lens-distortion model. Useful as a prior when calibration data isn't available. [Camera Intrinsics](/multiset/basics/rest-api-docs/camera-intrinsics)
3. **Map Scale API.** Apply a metric scale factor to a standalone 360 (Insta360) map to correct its real-world scale. Runs asynchronously and updates the map in place. [Map Scale](/multiset/basics/rest-api-docs/map-scale)
4. **Object upload as a zip package.** Object (3D model) uploads are now packaged and uploaded as a `.zip`, aligning the object upload flow with map uploads. [Object Upload](/multiset/basics/rest-api-docs/modelset-upload)
5. **Resumable large-map uploads.** Large map uploads can now be paused and resumed instead of restarting from scratch. Create the map with `?resumable=true`, and if an upload is interrupted you can list the parts already uploaded, request fresh signed URLs for the remaining parts, and complete the upload, or abort it cleanly. Ideal for large scans on slow or unstable connections. [Map Upload](/multiset/basics/rest-api-docs/map-upload)
**Fixed**
1. **Faster, more accurate multi-frame query.** The multi-frame query is now more accurate and up to 50% faster. [Map Query](/multiset/basics/rest-api-docs/map-query)
2. More reliable large-file map uploads: interrupted multipart uploads no longer have to be restarted from the beginning. [Map Upload](/multiset/basics/rest-api-docs/map-upload)
# Version 2.1.0
### v2.1.0 11-July-2026
**Added**
1. **Team members and multi-account RBAC.** Accounts now support multiple members with **Owner**, **Admin**, and **Viewer** roles. Invite teammates to your account by email, switch between accounts you belong to, and roles are enforced across all routes. Owners manage billing and membership, Admins manage content and API credentials, and Viewers have read-only access. Manage your team from the new **Members** tab in the Developer Portal. [Team Members](/multiset/basics/team-members)
2. **New `vps-2` query mode for the Single Frame Query API.** The single-image Map Query endpoints now accept a `queryMode` parameter that selects the localization engine per query. `vps-2` runs the new Deep Search engine with up to **15% higher recall** and improved accuracy, at the cost of higher latency (\~3 to 4 seconds) — ideal for offline or background workflows where accuracy is preferred over speed. Existing integrations are unaffected: when `queryMode` is omitted, queries run on `vps-1`. [Query Mode](/multiset/basics/localization/query-mode)
3. **VPS-2 in the simulation viewer.** Deep Search can be tested on your own simulation datasets from the Developer Portal simulation viewer using the new **VPS-2** option, alongside the existing Single Frame and Multi Frame runs. [Simulation Data](/multiset/basics/rest-api-docs/simulation-data)
4. **Georeference 3D maps via API.** New API to georeference 3D maps using a pair of 3D and WGS 84 points. [Georeference Map](/multiset/basics/rest-api-docs/georeference)
5. **Add maps to a MapSet without picking an overlapping map.** When you add a new map to an existing MapSet, it is now aligned against the entire MapSet automatically — you no longer choose which existing map it overlaps. [MapSet : Multiple Maps](/multiset/basics/mapset-multiple-maps)
6. **GravityAligned merge for MapSet.** MapSet merging now supports a **GravityAligned** merge, recommended for most merges where the maps share the same plane. [MapSet : Multiple Maps](/multiset/basics/mapset-multiple-maps)
7. **Partial map updates in Map Versioning.** Map versioning now supports multiple active maps, so you can push updates to a section of a map instead of replacing the full map. Localization queries run against all active maps in the version and return a pose in the base map's coordinate frame. Choose which maps participate with the new **Query Active** toggle in the Developer Portal. [Map Versioning](/multiset/basics/map-versioning)
8. **Moving-object masking in 360-to-VPS.** The 360-to-VPS pipeline now masks moving objects such as vehicles and people in the splat, improving metric scaling accuracy. [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans)
**Fixed**
1. Geo-referenced E57 files can now be uploaded directly, and MultiSet geo-references them automatically on upload. [Georeferencing Maps](/multiset/basics/georeferencing-maps)
2. E57 to Mesh now produces a more accurate mesh with less noise.
3. Human masking in mesh texturing: people captured during scanning are now masked out when the mesh is textured.
# Version 2.0.0
### v2.0.0 4-June-2026
**Added**
1. **MultiSet VPS Gen2** is now live on all maps and objects created after 3 June 2026. Gen2 brings a higher recall rate in repetitive areas like train stations, basements, and corridors. Existing maps can be upgraded to Gen2 from the Developer Portal.
2. Improvements to the 360-to-VPS pipeline.
3. Gen2 also improves the recall rate for Object Tracking.
4. **Indoor and outdoor modes for 360-to-VPS.** 360 video processing now optimises the scene according to the environment. [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans)
5. **Geometric and high-fidelity splat outputs.** 360-to-VPS now produces both a geometric and a high-fidelity version of the splat.
**Fixed**
1. Unity SDK: added the missing **hintFloorHeight** property. [HintFloorHeight](/multiset/basics/localization/hint-floor-height)
2. Fixed the MapSet map version update issue. [Map Versioning](/multiset/basics/map-versioning)
# Version 1.14.0
### v1.14.0 13-May-2026
**Added**
1. **Map Versioning** for smooth map operations. Merge re-scans of the same space into a single coordinate frame so your content layer (anchors, navigation paths, instructions) keeps working when a map is updated. [Map Versioning](/multiset/basics/map-versioning), REST API: [Map Version](/multiset/basics/rest-api-docs/map-version)
2. **MapSet overlap merging** now uses the MultiSet VPS under the hood to compute the relative pose between two scans. It's more robust than previous geometric methods and works on maps with low overlap. [Merging Maps with Overlap](/multiset/basics/mapset-multiple-maps/merging-maps-with-overlap)
3. **Early access (Beta) to 360 scan support.** Capture with Insta360 X4 / X5, upload raw `.insv`, and MultiSet stitches and reconstructs server-side. Includes a printable ChArUco marker for metric scale. [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans)
4. **Improved mesh creation for E57 maps**, denser, cleaner meshes from Leica / Matterport / NavVis / Faro structured E57 exports. [Third Party Scans](/multiset/basics/third-party-scans)
5. **Faster single-frame query API calls**, reduced server-side latency for the single-frame localization endpoint. [Map Query](/multiset/basics/rest-api-docs/map-query)
6. Ability to update optimised textured and raw meshes in a map (new map details page in UI added)
7. **On-device localization on Android**, run map and MapSet localization locally on Android without a network round-trip.
8. **On-device object tracking** across iOS and Android native SDKs.
9. **On-device API parity for localization filters**, `hintPosition`, `hintFloorHeight`, `hintMapCodes`, and `geoHint` now work the same on-device as they do on-cloud. [Localization](/multiset/basics/localization)
{% hint style="info" %}
**Beta:** 360 scan support is an early-access feature. Capture quality, processing time, and supported camera configurations will continue to improve based on user feedback during the beta.
{% endhint %}
# Version 1.12.0
### v1.12.0 23-April-2026
**Added**
1. Conversion of 3D Gaussian Splats to VPS maps, starting with Xgrids Gaussian Splat export. Support for more providers is coming soon. [Gaussian Splat](/multiset/basics/third-party-scans/gaussian-splat)
2. Localization improvements to reduce false positive cases for the Single Frame Localization API.
3. Confidence localization for the single-frame API for Maps and Objects is now calculated using multiple metrics and may result in lower confidence bands in areas with minimum visual features.
4. Added **hintFloorHeight** parameter for height-specific filtering during localization. [HintFloorHeight](/multiset/basics/localization/hint-floor-height)
5. New Multiplayer Sample scene in the Unity SDK. [Multiplayer Sample](/multiset/unity-sdk/sample-scenes/multiplayer-sample)
6. Localization speed improvements on the Meta Ray-Ban SDK:
**Fixed**
1. Unity SDK: Object Tracking mesh visualization state bug is fixed.
**Known Issues**
1. Mapping App / SDK producing maps with no mesh results in map processing failures.
2. Low localization success rate with Matter**c**raft and Spaces due to high default confidence, set the default confidence to 0.3
# Version 1.11.1
### v1.11.1 30-March-2026
**Added**
1. New VPS Query **hintRadius** and **use2DFiltering** parameter to control the search radius and skip the altitude/height check while providing GeoHint or HintPosition. [Map Query](/multiset/basics/rest-api-docs/map-query)
2. Localization success heatmaps in the MapViewer and Analytics section of the developer dashboard
3. Meta Ray-Ban SDK sample for VPS Tracking and Navigation
4. Ability to create a MapSet manually in the developer portal [Merging Maps Manually](/multiset/basics/mapset-multiple-maps/merging-maps-manually)
**Fixed**
1. Added background tracking for objects to reduce drift
2. Content Space deeplinking issue
3. New MapSet viewer UI for better map management and alignment
4. Added Undo and Redo in the MapSet viewer
# Version 1.11.0
### v1.11.0 02-Febuary-2026
**Added**
1. Android and iOS Native SDK with all VPS parameters: [iOS Swift Native](/multiset/native-support/ios-swift-native) [Android Native](/multiset/native-support/android-native)
2. Unity SDK: The mapping scene is now modular, with UI and Core logic separate
3. Ability to upload glb files of up to 50 MB for [Object Tracking](/multiset/basics/object-tracking).
4. Support for glb files with PBR material and Draco compression for Object Tracking
5. CORS Control directly in the developer portal [Configuring Allowed Domains (CORS)](/multiset/basics/credentials/configuring-allowed-domains-cors)
6. NPM package for WebXR SDK:
7. Localization Query performance analytics in the developer portal
**Fixed**
1. ModeSet Tracking is now called **Object Tracking**
2. Localization accuracy improvements for On-Device VPS
3. Improved Map area calculations
4. Improved map viewer performance for large scans
# Version 1.10.0
### v1.10.0 08-December-2025
**Added**
1. Introducing OnDevice Localization for iOS:
2. ModelSet Tracking now supports tracking multiple objects in the same session:
3. MapSet alignment feature in Unity SDK:
4. Simulation data capture sample scene in Unity SDK:
5. Added Faro scanner support for VPS using the E57 export:
6. Developer Portal now shows query analytics for maps and model sets.
7. Ability to Archive Individual Maps and Maps that are part of a MapSet
**Fixed**
1. Localization performance improvement for Maps, Mapset and ModelSet
2. Confidence values in the VPS multi-frame query API response
3. Confidence check bug with the Single Frame Localization script in Unity SDK
4. Memory management while checking for blurred images in Unity SDK
# Version 1.9.3
### v1.9.3 16-November-2025
**Added**
1. Added Xgrids scanner support for VPS using the E57 export:
2. Map analytics now shows the total mapped area in sq ft.
**Fixed**
1. Unity SDK: Mapping Scene upload issue is fixed.
2. Mesh generation improvement for E57 scans imports.
3. Updated glTF and draco packages for Android 16 KB support.
4. Developer Portal: Mesh visualization optimization for loading large Maps/MapSets.
5. MapSet manual alignment transform update was resolved.
6. MultiSet App UI is optimized for iPad.
# Version 1.9.2
### v1.9.2 09-October-2025
**Added**
1. On-Cloud processing for the Matterport Matterpak file through the MultiSet developer portal:
2. REST API Endpoint to process Matterport MatterPak files either using API credentials or OAuth token
3. Meta Quest SDK now supports Localization simulatoin in Unity Editor and object tracking with a new sample scene.
4. Multi-frame localization in the Meta Quest SDK for robust localization performance.
**Fixed**
1. Authentication issue with Meta Quest SDK prefab.
# Version 1.9.1
### v1.9.1 29-September-2025
**Added**
1. Localization Simulation in Unity SDK for testing localization inside Unity Editor Play mode.
2. Support for E57 file with epirectangular panoramas to support output scans from Navvis
3. Updated the MultiSet Mobile app to capture Localization Simulation data
4. REST API endpoint to upload an e57 scan file:
**Fixed**
1. Unity SDK Android build issue is resolved.
# Version 1.9.0
### v1.9.0 12-September-2025
**Added**
1. Mapping scene in Unity SDK for integrating scanning capabilities into phones with LiDAR support.
2. Continuous localization until success for the first attempt in a session has been added
3. MultiSet Mobile App UI overhaul with the ability to upload .glb objects for object tracking.
4. Navigation scene added in Meta Quest SDK:
**Fixed**
1. Improved localization response time in Meta Quest SDK.
2. ModelSet object tracking improvements for smaller objects
3. WebXR SDK can handle localization in both portrait and landscape mode
4. 3D viewer launch orientation while rendering large scans on the developer portal is improved.
# Version 1.8.1
### v1.8.1 22-August-2025
**Added**
1. Meta Quest SDK can now be installed using Unity NPM**:**
2. Confidence check in ModelSet tracking manager (to filter tracking response based on confidence value)
**Fixed**
1. Meta Quest SDK and Unity AR Foundation SDK now support for Unity Universal 3D pipeline (SRP)
2. Frame drop during localization in Unity SDK fixed.
3. Simulation Mode with the new input pipeline is working now.
4. MapFoundry Desktop app crashing while processing large scans has been fixed.
# Version 1.8.0
### v1.8.0 25-July-2025
**Added**
1. **ModelSet** for object anchoring using textured meshes.
2. New Sample Scene in Unity SDK for ModelSet Tracking
3. WebXR supports ModelSet to track objects.
**Fixed**
1. Localisation accuracy and stability improvement in WebXR SDK
2. Simplified localisation response pose transformation for RHS systems (Native iOS and Android SDK)
# Version 1.7.0
### v1.7.0 14-July-2025
**Added**
1. Content Space in the MultiSet app to quickly prototype and share AR experience using MultiSet VPS.
2. Added GeoHint in localization API's for improved localization time and performance.
3. Localizatoin API's now send responses in Geo Coordinates (WGS 84).
**Fixed**
1. Unity SDK: Fixed issue with the background localization
# Version 1.6.5
### v1.6.5 4-June-2025
**Added**
1. E57 scan file import now generates better mesh with accurate normals and faces
2. Unity SDK: Add a new sample scene for AR enabled location based training
3. Native SDK for Android:
4. Native SDK for iOS:
5. Native SDK supports App Clips and instant apps for install-free location-based AR experience
Fixed
1. Unity SDK: Fixed issue with navigation and background localisation
# Version 1.6.0
### v1.6.0 21-April-2025
**Added**
1. Released the MapFoundry Desktop application that can process any 3D scene (Textured Meshes) to support visual localization, starting with support for Matterpak files.
2. Added the capability for background localization in the SDK, which reduces AR drift when users start walking during a session.
3. Included HintMapCodes and HintMapPosition in the localization API to improve localization time and accuracy in large maps. ([details](https://docs.multiset.ai/unity-sdk/on-cloud-localization/pose-prior-hintposition))
4. MultiSet app now supports background map uploads.
5. Unity 6 Support.
Fixed
1. SDK Map download issue with large maps fixed
# Version 1.5.2
### v1.5.2 14-March-2025
**Added**
1. Email notification once a Map is active
Fixed
1. Localization improvements for large maps scanned from the mobile app.
2. FPS drop during mapping is fixed.
3. MapSet merging from app is now precise.
4. Localization API request bug happening in devices running non-English language.
# Version 1.5.0
### v1.5.0 26-February-2025
**Added**
1. Improvement of localization accuracy under challenging conditions like changes in environment and viewports.
2. Multiset app capture improvements.
3. Ability to upload E57 as Zip files for faster uploading and processing
4. A new localization scene in Unity SDK that uses multiple query frames for robust localization accuracy.
Fixed
1. Mesh color texturing for E57 files has been fixed.
# Version 1.4.0
### v1.4.0 30-January-2025
**Added**
1. Support for third-party scans through the developer portal (.e57 files with overlapping panoramas).
2. Unity SDK: Sample scene for Navigation app
**Know issue**
1. Textured mesh with e57 scans has issues with disappearing mesh faces.
# Version 1.3.0
### v1.3.0 06-January-2025
**Added**
1. Unity SDK: Simulation mode for the Unity Editor.
2. Unity SDK: Ability to directly download meshes inside the Unity SDK instead of a manual process.
3. Unity SDK: Added a relocalization feature when AR tracking is limited or lost.
4. Unity SDK : Runtime Occlusion support added using Meshes
5. Create MapSet with an Overlap option in the Developer Portal
# Version 1.2.0
### v1.2.0 26-December-2024
**Added**
1. Improved localization experience and more flexibility to control localization trigger events.
2. New localization animation that can be customized.
3. Ability to adjust Map transformations after MapSet creation in the Developer Portal
**Fixed**
1. MultiSet App bug triggering a low duration issue while saving a Map.
2. Map upload issue fixed in case of low network.
# Version 1.1.0
### v1.1.0 10-December-2024
**Added**
1. Improved localization accuracy in challenging viewport cases.
2. Introducing Map Code and MapSet Code for SDK Map connections.
3. Option to download raw mesh of your map as .glb.
# Maps
Maps section lists out all the maps you have captured so far with their status
### Map details
Each map section displays the Map Name, Map ID, Map Size, and the Date-Time of creation. Use the View Map button to view and download Map GLB files compatible with our SDK.
### Map Status
**Uploading** : Map data is being uploaded from the device to the cloud.
**Pending** : Map raw data is being processed in our cloud
**Active** : Map is active to download and make VPS calls
**Failed** : Map processing is failed, try mapping again
{% hint style="success" %}
MultiSet Mapping App currently supports iPhones and iPads equipped with Lidar sensor
{% endhint %}
# Mapping Instruction
Follow these techniques to capture high-quality maps with the MultiSet Mapping App. Accurate scans produce faster localization, better accuracy, and lower drift, the foundation of every reliable AR experience.
{% hint style="success" %}
The MultiSet Mapping App currently supports iPhones and iPads equipped with a LiDAR sensor.
{% endhint %}
### 1. Ideal distance between you and the surface to map
Maintain a capture distance of **1–5 meters** between the device and the surface being mapped. If the maximum distance is not sufficient, move closer or reposition to a different vantage point. Too close loses context; too far loses feature detail.
Correct range
Too far — weak features
### 2. Avoid capturing the same area from the same viewpoint
Repeating the same area from identical viewpoints in a single session does **not** improve accuracy. Instead, capture the same surface from multiple different angles and distances — this gives the system the multi-perspective data it needs to reconstruct 3D geometry reliably.
Multiple viewpoints
Same viewpoint repeated
### 3. Avoid rapid movements — maintain a steady mapping speed
Rapid or jerky movements break frame-to-frame feature tracking, causing gaps and position tracking loss in the map. Move at a slow, **continuous and consistent pace**. If the on-device mesh becomes patchy or unstable, slow down and reorient toward previously scanned features.
Steady pace
Rapid / jerky movement
### 4. Map in landscape mode — capture floor and ceiling if required
Landscape mode captures a wider field of view and more environmental features per frame. However, it may miss the floor and ceiling in a standard sweep — if those surfaces need to be localized, **tilt the device up and down while moving** to include them in separate passes.
Landscape recommended
Tilt up for ceiling
Tilt down for floor
### 5. Map corridors from both directions
Long corridors are one of the most challenging localization environments. Scanning from only one end results in repetitive, symmetric visual features that are hard to distinguish. **Capture the same corridor walking in both directions** — this doubles viewpoint diversity and dramatically improves localization accuracy and stability along the full length.
Both directions
Single direction only
### 6. Ensure sufficient overlap between scan sessions
When a site requires multiple scan sessions, each new scan must **share visible features** with the previous session. Overlap at boundaries, doorways, and transitions connects multiple scans into a single coherent 3D map. Isolated scans that share no visible features cannot be merged.
Overlapping scans merge cleanly
No overlap — cannot merge
### 7. Minimize problematic surfaces
Certain surface types reduce localization quality. When possible, **keep these out of the center of frame** or avoid capturing them as the primary subject. Focus on areas with distinct, stable visual features.
Surfaces to avoid
**Instead, focus on:**
* Textured walls with paint, artwork, or signage
* Furniture, shelving, and equipment
* Columns, pillars, and architectural features
* Floors with distinct patterns or markings
* Fixed machinery and structures
{% hint style="info" %}
MultiSet's AI pipeline automatically filters many dynamic elements. However, minimizing them during capture always produces a more stable and accurate map.
{% endhint %}
***
## Quick reference
A checklist to review before and during every mapping session.
| # | Technique | Key rule |
| - | --------------------- | ------------------------------------------------ |
| 1 | Capture distance | Stay 1–5 m from the surface |
| 2 | Viewpoint diversity | Vary angles; never repeat the same viewpoint |
| 3 | Movement speed | Slow, continuous, no sudden stops or turns |
| 4 | Orientation | Landscape; tilt up/down for ceiling and floor |
| 5 | Corridors | Scan in both directions — always |
| 6 | Multi-session overlap | Share visible features at every session boundary |
| 7 | Surfaces to avoid | Mirrors, glass, blank walls, moving objects |
{% embed url="" %}
# Mapping Planning
**Optimizing Your Scanning Approach**
Plan an efficient path when scanning smaller spaces, keeping a distance of 0.5-4m from surfaces and using a fronto-parallel approach (scan objects directly in front of you).
**Follow these key principles:**
* Create a circular path along boundaries that covers all important features without overlapping
* Position start and end points near each other to avoid duplicating scans
* Scan tables and flat structures from one side only
* Skip inaccessible areas behind or beneath objects users won't approach
* Be aware that complex objects like plants are difficult to capture accurately
* Avoid scanning transparent or reflective surfaces, or cover them if necessary
{% hint style="warning" %}
Scanning empty rooms provides insufficient features for MultiSet VPS for localization
{% endhint %}
{% embed url="" %}
# Mapping Equipment
### Selecting Environments and Equipment for 3D Scanning
### Environment Selection Criteria
When choosing environments for scanning, prioritize static spaces where objects remain fixed and are unlikely to be moved. This ensures consistent augmentation results over time. For dynamic environments like exhibition booths or offices that contain movable elements or people, it's essential to verify that sufficient stable features remain consistently visible during augmentation scenarios. These stable elements might include:
* Wall decorations and fixtures
* Ceiling-based installations and lighting
* Distinctive floor coverings and patterns
* Heavy, stable furniture pieces
* Architectural elements and permanent structures
{% hint style="warning" %}
MultiSet App only support scanning indoors, for outdoors, we recommend using a professional scanner.
{% endhint %}
### Scanner Selection Guidelines by Environment Size
Various environments can be suitable candidates for scanning and localization for VPS, including exhibition spaces, cafés, apartments, production facilities, retail environments, museums, and transportation hubs. Each environment presents unique challenges based on its layout, complexity, and lighting conditions. To select the most appropriate scanning equipment:
#### Small Spaces (10m² to 500m² / \~100-5000 sq ft)
* **Recommended Equipment**: MultiSet app
* **Best for**: Small rooms, compact retail spaces, small café areas, compact exhibition booths
* **Advantages**: Portable, user-friendly, cost-effective for limited areas
#### Medium Spaces (500m² to 10,000m² / \~5,000-100,000 sq ft)
* **Recommended Equipment**: Matterport Pro2 & Pro3 models and Leica scanners
* **Best for**: Larger retail environments, museum galleries, office floors, larger exhibition spaces
* **Advantages**: Better depth accuracy, improved handling of medium-sized complex environments
#### Large Spaces (up to and beyond 30,000m² / \~300,000 sq ft)
* **Recommended Equipment**: NavVis scanners and Leica RTC360
* **Best for**: Airports, large museums, factory floors, shopping malls, extensive exhibition halls
* **Advantages**: High precision over large areas, advanced processing capabilities
* **Special Consideration**: In practice, these extensive environments are typically divided into separate scans and processed individually to improve authoring convenience and performance optimization
Selecting the appropriate scanning equipment based on these guidelines ensures optimal capture quality and tracking performance for your specific environment requirements
# Map Versioning
Merge scans of the same space captured at different times, without breaking the content layer.
## Overview
MultiSet Map Versioning lets developers merge scans captured at different intervals into a single coordinate system. Unlike traditional scan merging, which relies on algorithms like ICP and weak visual features, MultiSet Map Versioning is powered by the **MultiSet VPS** under the hood. This means it can merge scans taken months apart, even when there are significant visual and geometric changes between them.
With Map Versions, a developer's content layer coordinates don't need to be updated when a map is updated, even if a different local coordinate system is used, MultiSet handles the transformation automatically.
Map Version panel — toggle Query Active per map to control which maps are used for localization
## When to use Map Versioning
Use a Map Version when you are **updating a scan of a space you have already mapped**, whether you re-scanned the whole space or only part of it. The new scan covers the same physical area as the original, and versioning keeps your content layer anchored to the original coordinate frame.
{% hint style="info" %}
**Extending coverage instead? Use a MapSet.** If the new scan covers **additional area** rather than re-capturing existing area, for example a new wing, an adjacent building, or another floor, add it to a [MapSet](/multiset/basics/mapset-multiple-maps) instead of creating a Map Version. Map Versioning is for the same area over time; a MapSet is for stitching different areas together.
{% endhint %}
| Your situation | Use |
| -------------------------------------------------- | ---------------------------------------- |
| Re-scanned the whole space | Map Version |
| Re-scanned one section of the space | Map Version, with both maps Query Active |
| Scanned a new area to extend coverage | MapSet |
| Re-scanned one map that is already inside a MapSet | Map Version on that individual map |
## How it works
A Map Version groups two or more scans of the same physical space:
* One scan is the **base map**, its coordinate frame is the version's coordinate frame and the one your content layer is authored against.
* Every other scan added to the version is aligned to that base by the MultiSet VPS, which computes a rigid transform (`relativePose`) between the two scans.
* At any time, one or more maps in the version are **active**. When you query the base map, the VPS localizes against every active map in the version and transforms the returned pose back into the base map's coordinate frame. You choose which maps are active with the **Query Active** toggle in the Developer Portal.
### Why MultiSet VPS instead of ICP?
Traditional scan-merging techniques, such as ICP (Iterative Closest Point) and matchers built on low-level geometric or visual features, work well only when the two scans look almost identical. They fail in the cases that matter most in the real world:
* Furniture rearranged, walls repainted, fixtures swapped out
* Different hardware, so different coordinate systems: LHS, RHS Y Up, RHS Z Up, etc
* Different sensor or different point density (e.g. iPhone LiDAR vs. Matterport vs. Leica)
* Different lighting conditions or season
* Months between captures, with cumulative change to the space
MultiSet Map Versioning uses the same VPS pipeline that powers MultiSet localization. Because the VPS reasons about the scene at the level of recognizable structure rather than raw geometry, it stays robust under drastic change between captures. The output is a clean rigid transform between the new scan and the base map, even when ICP would have given up.
### Content layer stays put
The biggest practical benefit: **your content layer never has to move.**
Anchors, navigation paths, instruction overlays, and any other authored content live in the base map's coordinate frame. When you scan the space again, possibly with a different device that uses a different local coordinate system, you add the new scan as a version and activate it. MultiSet's VPS computes the transform once; from then on every query against the base map is transparently translated to the active map.
{% hint style="success" %}
The content you authored in your app (navigation paths, AR instructions, anchors) does **not** need to be re-authored when you re-scan the space, even if the new scan uses a completely different local coordinate system.
{% endhint %}
## Create a Map Version in the Developer Portal
The quickest way to version a map is from the Developer Portal. No API calls required.
**1. Open the map's Map Details.** In the **Maps** list, click the map you want to use as the base. On its **Map Details** screen, scroll to the **Map Version** section and click **+ Create Map Version**.
Map Details → Map Version section → Create Map Version.
**2. Pick the target map.** In the **Create Map Version** dialog, the map you opened is set as the **Source Map (Base)**: its coordinate frame becomes the version's frame. Search for and select the newer scan of the same space as the **Target Map**, then click **Create Version**.
Choose the newer scan as the target map, then Create Version.
**3. Wait for alignment.** The target map is added to the version with a **Pending** status while the MultiSet VPS computes the transform between the two scans. The base map is **Ready** and **Query Active** from the start.
The new scan shows Pending while the VPS aligns it to the base map.
**4. Activate the new scan.** Once the target map turns **Ready**, toggle **Query Active** to include it in localization queries. Use **View pose** to inspect the computed relative pose, and **+ Add Map Version** to add further scans over time. At least one map must remain Query Active.
Once Ready, toggle Query Active to include the scan in queries; View pose shows its relative pose.
## Query more than one map at a time
A version is not limited to a single active scan. **Any number of the maps in a version can be query active at once**, and a localization query against the base map runs against all of them, returning the pose in the base map's coordinate frame either way.
This is what makes **partial updates** practical. If you re-scan only one section of a space, you do not have to replace the whole map: keep the original active for the areas that have not changed, activate the new scan for the section that has, and queries resolve against whichever one the user is actually standing in.
### From the Developer Portal
Each map in the **Map Version** panel has its own **Query Active** toggle. Turn it on for every map you want localization to run against.
Both maps in this version are query active, so localization runs against both.
{% hint style="info" %}
A map has to reach **Ready** before it can be activated, and **at least one map must stay active** at all times, so the last active map cannot be switched off.
{% endhint %}
### Over the API
Use the activate endpoint and pass `mapCodes` with the full set of maps you want active:
```json
{
"mapCodes": ["MAP_BTTE1MOXYVT8", "MAP_QQYKIBHXZE01"]
}
```
The call **replaces** the active set, so any map in the version that is not listed is deactivated. To activate a single map, send `mapCode` instead. See [Activate Maps in Version](/multiset/basics/rest-api-docs/map-version#activate-maps-in-version) for the full request and response.
## Detecting and applying a version at runtime
When a map has been versioned, the **map details response** carries two new fields, this is how a client app discovers that a version exists:
```json
{
"versionCode": "MVER_3VG3VINCWUIR",
"activeMapCode": "MAP_KRXDJ14SY7ER"
}
```
Once your client sees these fields on a map response, the flow is:
1. Call **Get Map Version** with the `versionCode` to fetch the version's full state, including the **relative pose (offset)** of the active map against the base.
2. Download the mesh / map data for the **active map** (`activeMapCode`) instead of the base map. This is the geometry that VPS queries are actually running against.
3. **Apply the relative pose as an offset** when loading the active map's mesh into your scene. With this offset applied, the new mesh lines up exactly on top of your existing content layer (which was authored in the base map's coordinate frame).
The result: the new, fresher mesh is rendered in the right place, your anchors and navigation paths stay where you originally put them, and VPS query responses already arrive in the base map's frame, so no further client-side conversion is needed.
{% embed url="" %}
## Versioning an individual Map
The simplest case: a single base map, scanned again later.
1. Create a Map Version with the original map as `sourceMapCode` (it becomes the **base**) and the newer scan as `targetMapCode`.
2. The MultiSet VPS computes the transform between the two scans. The new map's status moves through `pending` → `computing` → `ready` as the job progresses.
3. Activate the newer scan with the **Activate Map in Version** API. From this point on, queries against the original map's code are routed to the newer scan, and the response is transformed back, so your content layer is unaffected.
4. When you scan again next quarter, add it as another version. The transform is computed against the **currently active map**, not against the original base, which keeps quality high as the space drifts further from the original scan over time.
See the [Map Version REST API](/multiset/basics/rest-api-docs/map-version) page for the full request/response shapes.
## Versioning a Map that is part of a MapSet
Map Versions and MapSets compose cleanly:
* A MapSet groups maps that **cover different physical areas** of a venue (e.g. several floors, or several rooms).
* A Map Version groups scans that **cover the same physical area** at different points in time, or from different hardware.
To re-scan one map inside a MapSet, create a Map Version on that individual map. The MapSet keeps pointing at the same `mapCode`, the version layer transparently swaps in the newer scan. The relative pose between maps in the MapSet does not need to be re-computed, only the transform between the old and new scan of the one updated map.
{% hint style="info" %}
You don't version the MapSet itself. You version the individual maps that compose it. This keeps each capture session isolated: re-scanning Floor 2 doesn't affect Floor 1's alignment.
{% endhint %}
## Mix hardware between versions
You don't have to use the same scanning hardware for every version of a map. Add versions captured with:
* **MultiSet Mapping App** (iPhone / iPad with LiDAR)
* **Matterport** (E57 / MatterPak)
* **Insta360** and other 360° cameras
* **Leica**, **NavVis**, **Faro**, **Xgrids** scanners
The VPS computes the alignment regardless of which device produced either scan. This lets you start with a quick phone capture, then upgrade to a higher-fidelity professional scan later, without losing the content layer you already built.
See [Third Party Scans](/multiset/basics/third-party-scans) for supported formats and upload flows.
## Lifecycle
| Status | Meaning |
| ----------- | --------------------------------------------------------------------------------------- |
| `pending` | The scan has been added to the version and is queued for alignment. |
| `computing` | VPS is actively computing the transform between this scan and the version's source map. |
| `ready` | Transform is computed; this map can be set as active (toggle **Query Active**). |
| `failed` | VPS could not align this scan to the source. `failureReason` will be populated. |
A version always has exactly one **base** map (its coordinate frame is the version's frame) and one or more **active** maps (the scans VPS queries run against). When a version is first created only the base map is active; use the **Query Active** toggle in the Developer Portal to activate or deactivate additional maps for querying. At least one map must stay active.
# MapSet : Multiple Maps
### Overview
A MapSet is a container structure that enables the unified management and interaction with multiple individual maps, primarily used for handling large-scale venues and areas where Visual Positioning System (VPS) capabilities are required. It acts as a coherent unit, allowing multiple map fragments to function as a single, seamless mapping solution.
### Key Concepts
#### Map Consolidation
* MapSets solve the challenge of managing extensive areas by combining multiple smaller, more manageable maps into a unified structure
* Each constituent map maintains its individual characteristics while contributing to the larger whole
* The system preserves the integrity and detail of individual maps while enabling seamless navigation across their boundaries
#### Spatial Relationships
* Maps within a MapSet are positioned with high-precision relative coordinates and rotations
* The accurate spatial relationships between maps ensure:
* Seamless transitions between adjacent areas
* Consistent positioning across the entire venue
* Accurate distance calculations spanning multiple maps
* When a MapSet is created, the relative coordinate is assigned with respect to the origin of the **first map** of the MapSet, so every map's coordinate is defined with respect to the first map
{% hint style="success" %}
When you add a new Map to a MapSet, it is aligned against the entire MapSet automatically — you no longer pick an overlapping map. Its transform is calculated with respect to the existing maps, and existing maps are not impacted.
{% endhint %}
#### User Experience
Despite the underlying complexity of multiple maps, end-users interact with a MapSet as if it were a single map:
* Continuous navigation across map boundaries
* Unified coordinate system across the entire venue
* Consistent positioning and orientation references
* Seamless VPS functionality throughout the covered area
### Best Practices
1. Divide large venues into logical, manageable sections
2. Ensure sufficient overlap between adjacent maps
3. Maintain consistent mapping quality across all constituent maps
####
# Merging Maps without Overlap
A MapSet is created when merging multiple maps together, this is especially required to capture large venues to enable VPS in large areas.
## Steps of creating MapSet
### 1. Capture individual maps
The first step is to capture individual sections independently in multiple sessions.\
For large scale, you can have multiple people mapping different areas of a venue at the same time, which can be merged later.This is helpful when you want to capture large venues in a short period of time.
\
Overlap between individual maps is recommended, but it's not mandatory, as we can merge maps without overlaps too.\
The ideal capture duration for an individual section could be up to 5 minutes.
### 2. Merge the first pair of maps together.
Select a Map from the list by pressing on (...) and then choose the "Merge Map" option
### 3. Choose a map that you want to merge into this Map.
Choose a map from the list of the maps that you have created so far to be merged.
{% hint style="info" %}
Maps can only be merged after processing is completed
{% endhint %}
### 4. Perform localization in both maps in the correct order
Localize in maps that you want to merge by following the on-screen instructions, start by localizing in the first map, then walk to the next map, and localize.\
Post successful localization of both maps, you will see a success pop-up and you can enter the name of your MapSet
### 5. View your Mapsets in-app or developer portal.
After successfully merging the first pairs of maps in-app, you can test the localization in the combined map as MapSet\
See the list of MapSet under the new tab and then select a MapSet to either test localization or extend MapSet further
In the Developer Portal, you should be able to view the combined maps meshes together as shown below
{% hint style="danger" %}
If there is any error in the MapSet merging process you will notice that individual meshes are not merged together well, in this case you have to remove that individual map and add it back in MapSet
{% endhint %}
# Extend a MapSet
Docs on how to merge more maps into an existing MapSet
### 1. Go into the MultiSet App and choose a MapSet to extend
Select the "Extend MapSet" option from the below screen and follow a similar process of selecting a new map that needs to be merged into the existing MapSet
### 2. Localize into any map of the existing Mapset, then localize the new map
To add a new map, just localize to any existing map of Mapset, then walk to the new map that needs to be added and localize.
{% hint style="warning" %}
Ideally, localize the closest map in the existing Mapset to the new map that's being added to reduce the walking distance and to reduce merging errors
{% endhint %}
# Merging Maps with Overlap
Overlap-based merging is powered by the **MultiSet VPS** under the hood, so it is far more robust than traditional geometric methods (ICP, low-level feature matching) and works even when two scans share only a small overlap area.
### Create a MapSet with Overlap
Click on Create MapSet -> **Merge Maps with Overlap**
### Select the Origin Map and then a Map that overlaps with it.
Select an Origin Map this could be the first map that you want to add into your MapSet, this map act as the origin of your MapSet coordinate system.
In the next steps, you MUST choose a Map that has a decent overlap area with the Origin Map (we recommend having around 5-10 meters of overlap between maps to be merged or 15-20% overlap area between that needs to be merged).
After this, give a MapSet name and wait for a couple of minutes for the Map merging process to be completed
### Add a new Map to an existing MapSet
Open the MapSet details page and then click on **Add Map** to add a new Map in a MapSet
First, select the new Map that you want to add and then choose an existing Map from MapSet that has overlap area with the new Map.
{% embed url="" %}
# Adjust Map Transformation
After merging several maps into a single MapSet, you might notice that some individual maps are not perfectly aligned. You can manually adjust the transformation properties (such as position and rotation) of each map individually using one of the following methods:
* **Web-based Viewer** – Use the MapSet viewer in the Developer Portal to fine-tune map placement directly in your browser.
* **Unity Editor** – Use the [MapSet Alignment Sample Scene](/multiset/unity-sdk/sample-scenes/mapset-alignment) in the Unity SDK. This is **recommended for large scans** as it provides better performance and more precise control.
By selecting a particular map within either tool, you can fine-tune its placement—shifting it slightly or rotating it—to better match the surrounding maps. These adjustments help in correcting any misalignments or rotation errors that occurred during the merge process, ensuring that the maps fit together seamlessly within the MapSet.
It's important to note that this adjustment step is optional. You only need to adjust individual map transformations if you observe errors or discrepancies after merging.
### Using the Web-based Viewer
Go to **Developer Portal** -> **MapSet** -> **View Mapsets** -> **View Maps**
Analyze the map transformations, and if you notice errors produced during merging, you can select individual maps and adjust their position and rotation to align them accurately with their real-world transformations.
Click on the **Update** button to save your transformation.
# Merging Maps Manually
Use this option when your maps do not have sufficient overlap for auto alignment and you are unable to visit the actual location to perform mobile-based merging.
Manual alignment lets you select two maps and merge them using visual geometry hints, think of it like assembling Lego pieces using individual 3D scans. You visually position and orient each map relative to the other based on recognizable spatial features in the 3D point clouds.
# Object Tracking
Object Tracking is a powerful feature that enables your AR applications to recognize, track, and anchor digital content to specific physical objects in the real world. By uploading a 3D model of a target object, you can achieve persistent, high-precision visual localization and tracking.
This technology is ideal for a wide range of applications, including:
* Interactive product visualizations
* AR maintenance and training guides for machinery
* Museum exhibits with augmented information
* Location-based AR games anchored to real-world props
### How It Works
Object Tracking combines the power of cloud processing with on-device performance to deliver a robust tracking experience.
1. **Cloud-based Processing**: When you upload your 3D model, our cloud service analyzes its geometry and texture to create a highly optimized tracking map. This map contains the unique visual features required for fast and accurate detection.
2. **On-device Tracking:** The Multiset SDK fetches the initial pose to the user's device. Using the device's camera, the SDK performs real-time local processing to detect the object and tracks the camera movement with the help of the underlying tracking system in 3D space.
This hybrid approach ensures that the initial heavy-lifting is done on the server, while the live tracking remains fast and responsive on the user's device.
***
### Understand the outside-in object tracking.
**Object Tracking: An "Outside-In" Concept**
* **Core Principle:** The object itself is the center of the tracking environment. Users localize and track the object by observing it from multiple angles around it.
* **Key Distinction:** This "outside-in" method is the opposite of "inside-out" systems (like localization maps), which track a user's position within a larger space.
* **Suitable Objects:** This technique is designed for tracking specific, well-defined items within a scanned area. Good examples include engines, products, or statues inside a building.
### Getting Started: Preparing Your 3D Model
The quality of your tracking experience is directly tied to the quality of the 3D model you provide. Before uploading, ensure your model adheres to the following requirements.
#### Model Requirements
These are mandatory specifications for an object to be processed for tracking
**1. Scale: Meters**
The 3D model **must** be authored at a real-world scale, where **1 unit corresponds to 1 meter (m)**. If your object is 50cm tall in reality, its model should be 0.5 units tall in your 3D software. Incorrect scale will result in tracking failure or incorrect content placement.
**2. Up Vector: +Y Axis**
The model's "up" direction **must** be aligned with the **positive Y-axis**. This ensures that the object's orientation is interpreted correctly by our tracking system. Most 3D authoring tools allow you to configure or export with a specific coordinate system.
**3. Static Objects**
Currently, Object Tracking is designed for **static, rigid objects**. The physical object you intend to track should not move, deform, or articulate during the AR session. Support for dynamic objects is planned for a future release.
> **Note:** This means the physical object in the world should be stationary. The user (camera) can, of course, move around it freely.
**4. Textured Mesh**
Tracking relies on visual features. Your model **must** have a texture map (e.g., a diffuse/albedo map) applied. Untextured or single-color models lack the necessary feature points for our computer vision algorithms to lock onto.
***
#### Best Practices for Optimal Tracking
Follow these recommendations to significantly improve the stability and accuracy of your Object tracking.
**✔️ Use High-Detail Textures and UVs**
A clear, high-resolution texture with well-distributed details is crucial. Good UV wrapping ensures the texture is applied without distortion, providing a rich set of unique feature points for the tracker. Avoid blurry or heavily compressed textures.
**✔️ Choose Asymmetrical Objects with Rich Features**
The ideal objects for tracking are visually complex and asymmetrical. Think of things with unique shapes, text, logos, and varied surface patterns.
* **Good Examples**: A detailed statue, a coffee machine with buttons and logos, a branded cereal box (with text and images), a piece of industrial machinery.
* **Challenging Examples**: A symmetrical wooden crate, a single-color yoga ball, a generic coffee mug, a mirrored or highly reflective object.
**❌ Avoid Symmetrical and Feature-Poor Objects**
Symmetrical objects, like a perfect cube or sphere, create ambiguity for the tracker. The system may struggle to determine the object's exact orientation because multiple sides look identical. Similarly, objects with large, uniform-color surfaces provide very few points to track.
***
# How to Setup Object Tracking
### Step 1: Prepare Your 3D Model
The first and most critical step is to source and prepare a 3D model that accurately represents the physical object you want to track.
{% hint style="warning" %}
Ensure the model file size is below 50 MB and the ideal physical size of the model should be between 1 foot and 50 feet.
{% endhint %}
#### Sourcing Your Model
You can acquire a 3D model from various sources. The best method depends on your specific object and resources:
* **3D Scanning**: Using dedicated 3D scanners or mobile apps (like Polycam or Luma AI) to capture a real-world object and convert it into a 3D mesh. This is often the best method for capturing existing objects with complex, organic surfaces and rich textures. (Please use Lidar-based mode while scanning.
* **CAD Files**: If the object was designed in CAD software (e.g., SolidWorks, AutoCAD, Fusion 360), you can use these files. However, CAD models are often untextured and mathematically perfect. You will need to convert them into a polygonal mesh and apply a realistic texture before they can be used as an Object.
* **Photogrammetry**: This technique involves taking many photos of an object from different angles and using software (like RealityCapture or Metashape) to reconstruct a textured 3D model from them. It is excellent for achieving high-fidelity textures. (make sure the model has real world scale)
Regardless of the source, you must export your final model in **.glb** format.
#### Model Preparation Guidelines
Before exporting, you **must** ensure your model meets the following technical requirements. Failing to do so will result in poor performance or processing failure.
> **✅ Checklist for a Valid Objects**
>
> 1. **Scale must be in Meters (m).**\
> The model's internal units must represent real-world scale, where 1 unit = 1 meter.
> 2. **Up vector should be the +Y axis.**\
> The model must be oriented so that its "up" direction aligns with the positive Y-axis.
> 3. **A Textured Mesh is required.**\
> The model must have a texture map (diffuse/albedo) applied. The visual information from the texture is essential for tracking.
***
### Step 2: Choose the Tracking Type
After preparing your model, you can proceed to the Multiset Developer Portal to upload it. During this process, you will be asked to select a **Tracking Type**. This setting optimizes the tracking map based on how you expect users to interact with the object.
There are currently two tracking types available:
#### 1. 360 View (Default)
This is the most comprehensive tracking mode. It creates a tracking map that allows the object to be detected from any angle—top, bottom, front, back, and all sides.
* **Use when**: You expect users to be able to walk completely around the object, or view it from above or below (e.g., a statue in an open room, a product on a tabletop).
* **This is the default and recommended setting for most use cases.**
#### 2. Side View
This mode optimizes tracking for scenarios where the object will primarily be viewed from the sides (front, back, left, and right). It does not include data for tracking from extreme top-down or bottom-up angles.
* **Use when**: The object is placed in such a way that it cannot be viewed from the top or bottom (e.g., a large machine fixed to the floor, an object mounted high on a wall). Choosing this can slightly speed up processing for applicable models.
***
### Step 3: Upload and Process
Once you have your prepared .glb file and have decided on the tracking type, you are ready to complete the process.
1. Navigate to the **Object Tracking** section of the Multiset dashboard.
2. Click **"Create Object"**.
3. Upload your 3D model file.
4. Select your desired **Tracking Type** ("360 View" or "Side View").
5. Begin the upload.
This will trigger the cloud processing job. Our servers will analyze your model's mesh and texture to build the optimized tracking map.
Processing time is typically **less than 10 minutes** but can vary depending on the complexity of your model.
Once processing is complete, your Object will be marked as "Active" in the dashboard. You can then copy its unique **Object Code** and begin using it in your application via our SDKs and APIs to bring your AR experience to life.
# 360° Virtual Tour
Walk through a mapped space as a navigable 360° panoramic tour.
## Overview
A **360° Virtual Tour** turns a mapped space into a walk-through experience. Instead of looking at a mesh or a point cloud, you stand at a capture point, look around in full 360°, and step to the next point, the same way you would explore a street-level map.
Tours are generated automatically for **360 (Insta360)** maps that have been processed for panoramic viewing. No extra capture step is required: if you have already mapped the space with a 360 camera, the tour comes from the same data.
{% hint style="info" %}
A map's [Map details](/multiset/basics/rest-api-docs/map-details) response includes `hasPano`, which tells you whether that map has a tour available.
{% endhint %}
## How it works
A tour is a **graph of nodes**. Each node is one 360° capture point and holds:
* the panorama image for that spot,
* its position and orientation in the map's coordinate frame,
* links to its neighbouring nodes, with the distance to each in metres.
Because the nodes carry real positions, the tour is spatially aware. A viewer can jump to the node nearest a given location, or bias its choice toward whichever direction the user is facing, rather than only stepping along a fixed path.
## View a tour in the Developer Portal
Open the map from the **Maps** list, then click **Pano View** in the Map Details toolbar:
Map Details toolbar. Open Pano View.
Inside the viewer, each ring on the floor is a neighbouring node you can click to move to. The minimap shows the full tour path and where you currently stand, and **Go to** lists nearby nodes with their distances.
Drag to look around, scroll to zoom, and click a ring or any point on the floor to move.
## Compare a space across map versions
When a map has been re-scanned as a [Map Version](/multiset/basics/map-versioning), the tour can show **the same viewpoint in each version**, so you can compare a space before and after a change while standing in one spot.
Compare mode: one viewpoint shown in two versions of the same space. Each match reports how far it sits from the anchor, in distance and angle.
## One continuous tour across a MapSet
The maps in a [MapSet](/multiset/basics/mapset-multiple-maps) can be presented as a **single continuous tour**. Where two maps meet, the tour bridges between them, so a visitor can walk from one map into the next without noticing a boundary.
A tour spanning a MapSet. The minimap shows the joined maps, and navigation flows from one map into the next.
## Build your own viewer
Everything the Developer Portal viewer does is available over REST, so you can embed a tour in your own application: list the maps that have tours, load a complete tour in one call, or fetch just the neighbourhood around the user as they move.
See [360° Virtual Tour (Pano) API](/multiset/basics/rest-api-docs/pano-360-virtual-tour) for the endpoints, the node shape, and request examples.
# Credentials
Create credentials to authenticate API's and SDKs once you create new credentials you can download or copy.
Go to **Developer Portal** -> **Credentials** -> **Create New** -> **Enter Name**
You will get the **Client ID** and **Client Secret**.
You can set the scope of each credential at the time of creation
**Query**: Read access to retrieve data and perform Localization queries
**Write**: Create and update access to modify existing Map, MapSet or Object Tracking data
**Delete**: Remove access to permanently delete Account data
{% hint style="warning" %}
Please download or copy credentials immediately after creating them as you can't access them later
{% endhint %}
## How Credentials Work
MultiSet uses an **OAuth 2.0–style client credentials flow** for machine-to-machine (M2M) authentication. Your `clientId` and `clientSecret` are long-lived identifiers for your application; they are exchanged at runtime for a short-lived **Bearer access token (JWT)** that authorizes individual API calls.
Client Credentials → Bearer JWT → Authenticated API request
### Step-by-step
1. **Create credentials in the Developer Portal.** A `clientId` and `clientSecret` are generated and bound to your account, with the scopes you select (Query / Write / Delete). The secret is shown once — store it somewhere safe (env vars, a secrets manager, server-side config). MultiSet does not store a recoverable copy.
2. **Exchange credentials for an access token.** Your application sends a `POST` request to `https://api.multiset.ai/v1/m2m/token` with an `Authorization: Basic base64(clientId:clientSecret)` header. The endpoint validates that the client is active, the secret matches, and the owning account is verified and active.
3. **Receive a Bearer token.** On success, the response contains a signed JWT and an `expiresOn` timestamp. Tokens are valid for **30 minutes** from issue.
4. **Call MultiSet APIs.** Attach the token to every request as `Authorization: Bearer `. This applies to REST endpoints (Map, MapSet, Localization, Object Tracking) and to SDK calls that route through the MultiSet cloud.
5. **Per-request validation.** For each API call the backend verifies the token signature and expiry, checks that the credential's scope permits the requested route and method, and enforces the CORS allowlist configured for your account.
6. **Refresh before expiry.** When the token expires (or you receive a `401`), repeat step 2 to obtain a fresh one. There is no separate refresh-token flow — re-authenticate with `clientId` + `clientSecret`.
### Scopes and authorization
Scopes are evaluated on every request. A token issued from a credential created with only the **Query** scope cannot perform write or delete operations, even if the route exists — the API will respond with `403 Forbidden`. Choose the minimum scope your integration needs, and create separate credentials for read-only services (e.g. a public-facing localization client) versus tools that manage map data.
### Security best practices
* **Never embed `clientSecret` in a shipped mobile app, public website, or git repository.** Treat it like a password.
* For server-side and back-office tools, store the secret in a secrets manager or environment variable and exchange it for a token at runtime.
* For browser-based experiences (e.g. WebXR demos), prefer a thin backend proxy that holds the secret and returns short-lived tokens to the client. Direct in-browser exchange is convenient for prototyping but exposes the secret to anyone who views the page source.
* **Rotate credentials** if you suspect exposure. Deactivate or delete the old credential in the Developer Portal — existing tokens issued from it remain valid until they expire (max 30 minutes), but no new tokens can be issued.
* **Restrict by origin.** Configure your CORS allowlist so that browser requests can only be made from the domains you control. See [Configuring Allowed Domains (CORS)](/multiset/basics/credentials/configuring-allowed-domains-cors).
* **Use distinct credentials per environment** (dev / staging / prod) so you can rotate or revoke one without affecting the others.
### Where to use the token
| Surface | How credentials are used |
| ------------------------- | --------------------------------------------------------------------------------------------------------------- |
| REST API | Exchange `clientId`/`clientSecret` for a token, then send `Authorization: Bearer ` on every request. |
| Unity SDK / Quest SDK | Configure `clientId` and `clientSecret` in the SDK; the SDK performs the token exchange and refresh internally. |
| iOS / Android native SDKs | Same as Unity — pass credentials to the SDK config and it handles token lifecycle. |
For the full token-endpoint reference and copy-paste code samples, see [Authentication](/multiset/basics/rest-api-docs/authentication).
# Configuring Allowed Domains (CORS)
If you are building a browser-based application (such as a React, Vue, or Angular app) that interacts directly with the MultiSet Public API, you must configure **Cross-Origin Resource Sharing (CORS)**.
For security reasons, browsers restrict cross-origin HTTP requests initiated from scripts. By adding your application's domain to the MultiSet whitelist, you instruct our servers to accept API requests coming from your specific web application.
### How to Add a Domain
1. Navigate to the **Credentials -> Settings** tab on the left sidebar.
2. Locate the **Domains** section.
3. Click the purple **Add +** button on the right side of the card.
4. Enter the full origin of your application. This includes the protocol and the port (if applicable).
#### Recommended Setup
We recommend adding entries for both your local development environment and your production environment.
**1. Local Development**
To test the API while developing on your local machine, add your localhost address.
* **Protocol:** http
* **Domain:** localhost:3000 (or whichever port your local server uses)
**2. Production**
Once your application is deployed, add your live domain.
* **Protocol:** https
* **Domain:** [www.your-application.com](http://www.your-application.com)
> **Note:** Do not include trailing slashes (e.g., use localhost:3000, not localhost:3000/).
***
# Team Members
Invite teammates to your MultiSet account, assign roles, and control what each person can access.
Team Members lets you share a single MultiSet account with the people you work with. Instead of sharing one login, you invite each teammate by email and give them a role that decides what they can see and do. Everyone works inside the same account, so your maps, MapSets, objects, and credentials stay in one place.
You manage your team from the **Members** tab in the Developer Portal.
Go to **Developer Portal** -> **Account** -> **Members**.
Manage who has access to your account and what they can do
## Roles
Every member has one of three roles. Roles are the same across the whole account, so a person's role decides what they can do everywhere in the portal.
| Role | What they can do |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| **Owner** | Full access. Manage maps, MapSets, objects, and credentials, plus billing and members. Each account has exactly one owner. |
| **Admin** | Create, edit, and delete content (maps, MapSets, objects) and manage API credentials. Admins cannot access billing or manage members. |
| **Viewer** | Read-only. Viewers can see everything in the account but cannot create, edit, or delete anything. |
{% hint style="info" %}
In short: **Admin** can create, edit, and delete content, but not billing or members. **Viewer** has read-only access.
{% endhint %}
The **Owner** is the person who created the account. Ownership is fixed: the owner cannot be removed and cannot leave the account. When you invite someone, you can assign them the **Admin** or **Viewer** role.
## Invite a member
Only the account owner can invite members.
1. Open **Account** -> **Members**.
2. Select **Add member**.
3. Enter the teammate's email address.
4. Choose a role, **Admin** or **Viewer**.
5. Select **Send invite**.
Invite a teammate by email and pick their role
The person receives an invitation email. The invite stays valid for **7 days**, after which you can send a new one.
You can invite someone who does not have a MultiSet account yet. When they sign up with the same email address, the invitation is linked to their new account automatically, and they can accept it.
{% hint style="warning" %}
The number of members you can add depends on your plan. Pending invitations count toward your seat limit. If you reach the limit, remove a member or upgrade your plan to add more. If your current plan does not include team members, contact .
{% endhint %}
## Accept an invitation
When you have been invited to an account, you see an invitation banner on your **Home** and **Account** pages once you sign in with the invited email.
The invitation banner shown to an invited user
Select **Accept invitation** to join. The account then appears in your account switcher, and you can start working in it with the role you were given.
## Switch between accounts
If you belong to more than one account (for example, your own account and one you were invited to), use the account switcher at the top of the portal to move between them. The switcher lists each account with your role in it, so you always know whether you are an Owner, Admin, or Viewer in the account you are viewing.
Switch between accounts you belong to, or leave one you no longer need
## Manage existing members
From the **Members** tab, the owner can:
* **Change a role.** Use the role dropdown next to a member to switch them between Admin and Viewer.
* **Remove a member.** Select the delete icon to revoke a member's access to the account.
The owner's own row cannot be changed or removed.
## Leave an account
Any member can leave an account they were invited to from the account switcher. The owner cannot leave their own account, because every account must always have an owner.
# Analytics and Usage
View the Map analytics like the number of maps created, total storage consumed by all maps and number of VPS API calls
Home Page: Basic usage analytics
Add alt text and captions to your images
**Account -> Plan**
# Downloads
The download page is a one-stop destination to find all the latest Apps, packages, and executables.\
\
**Developer Portal -> Download Section**
## Download MultiSet App
iOS:
Android:
## Download Unity SDK
Unity UPM:
Meta Quest SDK:
# Localization
### What is Localization?
Localization is the process of determining a device's precise position and orientation (6-DoF pose) within a pre-scanned map. When a query image is sent to the VPS API, the system searches the map's database of reference frames for visually similar candidates, then computes an exact pose using feature matching and geometry.
By default the search covers the entire map. In large maps or multi-floor buildings this can be slow and prone to confusion between visually similar areas. **Localization parameters** let you provide prior knowledge (floor level, known position, GPS coordinates, map scope) to narrow the search before it begins, improving both speed and accuracy.
### Query Types
MultiSet VPS offers two ways to localize a device against a map: a **single frame query** and a **multi frame query**. Both send camera imagery to the same localization engine and return a 6-DoF pose; they differ in how many frames are sent, how long the request takes, and how robust the result is.
#### Single Frame Query
A single frame query sends **one camera frame** (a base64-encoded image plus its metadata) to the VPS. It is the fastest option, with a typical response time of around 2 seconds, and is ideal when you need quick, frequent pose updates or are localizing in well-textured, stable environments.
* **Endpoints:** `POST /vps/map/query` (JSON body) and `POST /vps/map/query-form` (multipart form-data)
* **Input:** 1 query image (max 1280 px on the longer side) + metadata
* **Response time:** \~2 seconds
* **Best for:** fast, repeated localization in stable, feature-rich scenes
Single frame queries can also run in a higher-accuracy mode (`vps-2`) when latency is less important. See [Query Mode](/multiset/basics/localization/query-mode).
#### Multi Frame Query
A multi frame query sends **4 to 6 camera frames** together with the local SLAM tracking pose (position and rotation from ARKit or ARCore) captured at the moment of each image. Combining multiple viewpoints makes the result more robust and accurate, especially in challenging conditions such as repetitive textures, partial occlusion, or minor scene changes. The trade-off is higher latency, with a typical response time of up to around 5 seconds.
* **Endpoint:** `POST /vps/map/multi-image-query` (multipart form-data)
* **Input:** 4 to 6 query images + per-frame SLAM tracking pose
* **Response time:** up to \~5 seconds
* **Best for:** robust localization in difficult or visually repetitive environments
#### Choosing a Query Type
| | Single Frame Query | Multi Frame Query |
| --------------------- | --------------------------------------- | ---------------------------------- |
| Images per request | 1 | 4 to 6 |
| Endpoint | `/vps/map/query`, `/vps/map/query-form` | `/vps/map/multi-image-query` |
| Typical response time | \~2 seconds | up to \~5 seconds |
| Robustness | Good in stable scenes | Higher, handles challenging scenes |
| Best for | Fast, frequent updates | Accuracy in difficult conditions |
The localization parameters described below apply to both query types. See [Map Query](/multiset/basics/rest-api-docs/map-query) for full request and response details.
### Localization Parameters
#### hintPosition
A 3D coordinate `[x, y, z]` in map-local space (left-handed / Unity convention) that tells the server approximately where the device is. The search is limited to frames within a radius of that point (default 25 m, configurable via `hintRadius`).
| Format | Example |
| ------------------------------- | ----------------------------------- |
| Array of 3 numbers (JSON body) | `"hintPosition": [2.5, 0.1, 8.0]` |
| JSON-encoded string (form-data) | `"hintPosition": "[2.5, 0.1, 8.0]"` |
Coordinates use the same LHS coordinate system as the localization response (when `isRightHanded: false`, the default). You can pass a previous localization result's `position` directly as the hint for the next query.
Use when you have approximate position from BLE, UWB, WiFi fingerprinting, or a previous localization result. For GPS-georeferenced maps, use `geoHint` instead.
#### hintMapCodes *(MapSet only)*
A list of map codes that limits the search to specific maps within a MapSet. Useful in multi-floor buildings where floor-level context is already known (e.g. from a floor selector, QR code scan, or BLE beacon).
| Format | Example |
| ------------------------------- | ---------------------------------------------------------- |
| Array of strings (JSON body) | `"hintMapCodes": ["MAP_RJFKKWQ1787J", "MAP_XPQR3456ABCD"]` |
| JSON-encoded string (form-data) | `"hintMapCodes": "[\"MAP_RJFKKWQ1787J\"]"` |
Only valid when querying against a `mapSetCode`. Ignored for single-map queries.
#### hintFloorHeight
A vertical band `[y_min, y_max]` in map-local Y coordinates. Only frames whose camera position falls within this Y range are considered. Useful for floor-level filtering in multi-floor spaces without needing per-floor map codes.
| Format | Example |
| ------------------------------- | --------------------------------- |
| Array of 2 numbers (JSON body) | `"hintFloorHeight": [3.0, 6.0]` |
| JSON-encoded string (form-data) | `"hintFloorHeight": "[3.0, 6.0]"` |
Order does not matter — `[-3, -1]` and `[-1, -3]` both select y ∈ \[-3, -1]. See [HintFloorHeight](/multiset/basics/localization/hint-floor-height) for full details.
#### geoHint
The device's GPS coordinates `[latitude, longitude, altitude]` passed as a comma-separated string. For georeferenced maps, the server converts these to map-local coordinates and applies the same spatial radius filtering as `hintPosition`. Requires the map to be georeferenced in the MultiSet Developer Portal.
| Format | Example |
| ---------------------- | --------------------------------------- |
| Comma-separated string | `"geoHint": "37.4219983,-122.084,10.5"` |
Works for both single Maps and MapSets. Combine with `convertToGeoCoordinates: "true"` to receive the pose result back in GPS coordinates. See [GeoHint in Localization](/multiset/basics/localization/geohint-in-localization) for full details.
#### hintRadius
Controls the search radius around `hintPosition` or `geoHint`. Default is 25 m. Smaller values are faster but require a more accurate prior; larger values are more forgiving.
| Format | Range | Default |
| -------------------------- | ------- | ------- |
| Number (JSON body) | 1–100 m | 25 m |
| Numeric string (form-data) | `"25"` | 25 m |
Only applies when `hintPosition` or `geoHint` is provided. Has no effect otherwise. See [HintRadius](/multiset/basics/localization/hint-radius) for full details.
#### use2DFiltering
When `true`, the spatial radius filter ignores the Y-axis (altitude) and uses only horizontal distance (X and Z). Useful for tall buildings or outdoor environments where GPS altitude is unreliable.
| Format | Default |
| --------------------------------------- | --------- |
| Boolean (JSON body) | `false` |
| String `"true"` / `"false"` (form-data) | `"false"` |
Only applies when `geoHint` is provided. Works for both Maps and MapSets. Has no effect with `hintPosition` alone.
### How Parameters Combine
Parameters are applied in sequence — each one narrows the candidate set further:
```
All map frames
→ hintMapCodes (keep only frames from these maps)
→ hintPosition / geoHint + hintRadius (keep frames within radius)
→ use2DFiltering (if set, ignore Y in radius check)
→ hintFloorHeight (keep frames within vertical band)
→ visual matching (match the query image against the remaining candidates)
```
Using more hints together reduces the search space more aggressively and is particularly effective in large, multi-floor MapSets.
***
### Quick Reference
| Parameter | Type | Applies to | Purpose |
| ----------------- | ---------------- | ------------ | ---------------------------------------- |
| `hintPosition` | `[x, y, z]` | Map / MapSet | Spatial radius filter from a known point |
| `hintMapCodes` | `[string, ...]` | MapSet only | Limit search to specific maps |
| `hintFloorHeight` | `[y_min, y_max]` | Map / MapSet | Vertical band filter |
| `geoHint` | `"lat,lon,alt"` | Map / MapSet | GPS-based spatial filter |
| `hintRadius` | number (1–100) | Map / MapSet | Radius for hintPosition / geoHint |
| `use2DFiltering` | boolean | Map / MapSet | Ignore Y-axis in geoHint radius check |
# Query Mode
Choose between the fast default engine (VPS-1) and the slower, more accurate deep-search engine (VPS-2) for Single Frame Query.
### What is Query Mode?
Query mode selects which localization engine handles a **Single Frame Query**. It lets you trade latency for accuracy on a per-request basis, without changing anything about your map or the rest of your request.
| Mode | Speed | Accuracy | Use for |
| ------------------- | ------------------------- | ------------------------------------------------- | ------------------------------------------------- |
| `vps-1` *(default)* | Fastest (\~2 seconds) | Standard | Real-time, interactive localization |
| `vps-2` | Slower (\~3 to 4 seconds) | Up to **15% higher recall** and improved accuracy | Offline or background workflows, difficult scenes |
Query mode is set with the `queryMode` parameter. If you omit it, the request uses `vps-1`.
{% hint style="info" %}
Query mode applies to the **Single Frame Query** only (`/vps/map/query` and `/vps/map/query-form`). It has no effect on the [Multi Frame Query](/multiset/basics/rest-api-docs/map-query#vps-multi-image-query-api). For how single-frame and multi-frame queries differ, see [Localization](/multiset/basics/localization#query-types).
{% endhint %}
### VPS-1 (Default)
`vps-1` is the standard single-image engine and the default when `queryMode` is not provided. It returns a pose in roughly 2 seconds, which makes it the right choice for anything interactive.
**Use VPS-1 when:**
* You are localizing in real time (live AR, in-app navigation, continuous re-localization).
* Response time matters more than squeezing out the last few percent of accuracy.
* The scene is well textured and not visually repetitive.
### VPS-2 (Deep Search)
`vps-2` is a deep-search engine that searches more of the map before estimating a pose. It delivers up to **15% higher recall** and improved accuracy, at the cost of higher latency (around 3 to 4 seconds per query).
**Use VPS-2 when:**
* You are running an **offline or background** workflow where a few extra seconds do not matter.
* The environment is challenging or visually repetitive (long corridors, basements, warehouses, transit stations) and `vps-1` sometimes fails to find a pose.
* You want to verify or improve a result: retry a failed or low-confidence `vps-1` query with `vps-2`.
* Accuracy is strictly more important than speed for the task.
{% hint style="info" %}
A common pattern is to query with `vps-1` first for a fast result and fall back to `vps-2` only when the response has `poseFound: false` or low confidence.
{% endhint %}
### API Usage
Pass `queryMode` alongside your usual query parameters. It is optional and defaults to `vps-1`. Allowed values are `vps-1` and `vps-2`.
#### JSON body (`/vps/map/query`)
```json
{
"mapCode": "MAP_RJFKKWQ1787J",
"queryMode": "vps-2",
"queryImage": "data:image/png;base64,...",
"cameraIntrinsics": { "fx": 664.38, "fy": 664.38, "px": 478.97, "py": 364.99 }
}
```
#### Form-data (`/vps/map/query-form`)
On the form-data endpoint, `queryMode` is a plain string form field:
```
queryMode: vps-2
```
See [Map Query](/multiset/basics/rest-api-docs/map-query) for the full request and response reference.
# HintFloorHeight
Restrict localization search to a specific floor using a vertical Y-axis band
### Overview
`hintFloorHeight` is a localization hint that limits the search to a specific vertical range. This is particularly useful in multi-floor buildings where frames from different floors may look visually similar.
By filtering to the expected floor before running feature matching, you reduce both the search space and the likelihood of cross-floor mismatches.
### How It Works
The VPS system uses map-local Y coordinates to represent height. When you provide `hintFloorHeight: [y_min, y_max]`, the search runs within that Y range only.
**Coordinate system notes:**
* Y is the vertical axis in both single-Map and MapSet (global) coordinate spaces
* Positive Y is typically up, but the exact values depend on how your map was captured
* For MapSet queries, positions are expressed in the shared global coordinate system
### Parameter Format
| Endpoint type | Format | Example |
| --------------------------------------------------------------- | -------------------- | --------------------------------- |
| JSON body (`/vps/map/query`) | Array of two numbers | `"hintFloorHeight": [1.5, 4.5]` |
| Form-data (`/vps/map/query-form`, `/vps/map/multi-image-query`) | JSON-encoded string | `"hintFloorHeight": "[1.5, 4.5]"` |
Order does not matter — `[-3, -1]` and `[-1, -3]` both select frames with y ∈ \[-3, -1].
***
### Combination with hintPosition
`hintFloorHeight` and `hintPosition` can be used together. The pipeline applies them in sequence:
1. **hintPosition** — filters to frames within the 25 m horizontal radius
2. **hintFloorHeight** — further restricts to frames within the vertical band
This two-stage filter is ideal when you know both the approximate XZ location and the floor level.
***
### Examples
```json
// Floor 1: floor at y=0, ceiling at y=3.0
{
"hintFloorHeight": [0.0, 3.0]
}
// Floor 2: floor at y=3.0, ceiling at y=6.0
{
"hintFloorHeight": [3.0, 6.0]
}
// Basement (negative Y)
{
"hintFloorHeight": [-3.0, -0.5]
}
// Combined with hintPosition (JSON body)
{
"hintPosition": [12.5, 2.0, -7.3],
"hintFloorHeight": [0.0, 3.0]
}
// Form-data endpoint — pass as JSON string
"hintFloorHeight": "[1.5, 4.5]"
```
**MapSet multi-map example:** In a MapSet where Floor 1 maps span y ∈ \[0, 3] in global space, pass `"hintFloorHeight": [0, 3]` to restrict search to only Floor 1 maps across all constituent maps.
***
### Best Practices
* Measure floor and ceiling Y values from a known localization result or from the map's coordinate metadata. You can also read these Y values directly by picking a floor point and a ceiling point on the mesh, see [Find Hint Coordinates](/multiset/workflow/find-hint-coordinates).
* Add a small buffer (e.g. ±0.3 m) to account for device height variation and imprecise floor detection
* For tall single-floor spaces (e.g. warehouses), use a generous range that covers the full height of the space
* Combine with `hintPosition` when you have both horizontal and vertical context — this gives the tightest search scope and best latency
# HintPosition
HintPosition helps to reduce the localization search scope in large maps
### Overview
HintPosition is a feature that allows you to restrict the localization search scope within large maps, significantly improving both the speed and accuracy of the localization process. By providing a hint position in your localization query, the system will only search within a 25-meter radius of the specified coordinates, which is particularly valuable in environments with visually similar areas.
### Key Concepts
* **Hint Position**: A set of coordinates (x, y, z) in the map's left-handed (LHS / Unity) coordinate system
* **Search Radius**: 25 meters around the provided hint position (configurable via `hintRadius`)
* **Coordinate System**: Uses the same LHS coordinate system as the localization response (`isRightHanded: false`). A previous localization result's `position` can be passed directly as the hint.
* **Complementary Positioning**: Works alongside visual positioning for improved accuracy
### Implementation
To pass a hint position in your localization query:
```
// Add the hint position to your localization query
form.AddField("hintPosition", hintPosition);
// where hintPosition is an (x, y, z) coordinate in the map's Cartesian coordinate system
```
### Integration with Other Positioning Technologies
HintPosition enables powerful hybrid positioning solutions by combining multiset visual positioning with other location technologies:
#### GPS Integration
* Convert GPS WGS84 coordinates to your map's Cartesian coordinate system
* Implement geo-fencing to determine when users are within mappable areas
* Use the converted coordinates as hint positions for more precise indoor positioning
#### Bluetooth/WiFi Positioning
* Leverage Bluetooth beacons or WiFi fingerprinting for approximate positioning
* Use these approximate coordinates as hint positions
* Let visual positioning provide the final, high-precision location
### Benefits
* **Performance Optimization**: Significantly reduces computational load by limiting the search area
* **Increased Accuracy**: Minimizes confusion between visually similar areas in different locations
* **Hybrid Positioning**: Creates a seamless transition between outdoor (GPS) and indoor positioning
* **Resilience**: Provides backup positioning when visual features are temporarily obscured or changed
### Best Practices
* Update hint positions frequently when users are in motion
* Implement appropriate coordinate system transformations for your specific mapping setup
* Consider the accuracy limitations of your secondary positioning systems when defining search areas
* Validate hint positions before passing them to ensure they fall within valid map boundaries
### Example Use Cases
* **Multi-floor Navigation**: Using floor detection to narrow down the MapSet, then BLE positioning for the hint position
* **Campus Navigation**: GPS for outdoor areas transitioning to visual positioning for indoor spaces
* **Large Warehouse Operations**: Using zone-based RF positioning combined with visual positioning for precise inventory placement
### Technical Considerations
* Ensure your coordinate systems are properly aligned across positioning technologies
* Consider the latency implications when fusing multiple positioning sources
* Implement appropriate error handling for cases where the hint position may be significantly incorrect
# HintRadius
Control the search radius around hintPosition or geoHint to tune the localization search scope
### Overview
`hintRadius` sets the search radius, in meters, that the VPS uses around a spatial hint. When your localization query includes a `hintPosition` or a `geoHint`, the retrieval phase only searches the portion of the map within `hintRadius` meters of that point. The default is **25 m**, and the value can range from **1 to 100**.
A smaller radius gives faster queries and fewer mismatches in visually similar areas, but requires a more accurate prior. A larger radius is more forgiving when the hint itself is uncertain.
{% hint style="warning" %}
`hintRadius` only applies when `hintPosition` or `geoHint` is provided in the same query. On its own it has no reference point and no effect.
{% endhint %}
### How It Works
Localization runs in two phases: image retrieval first, then pose estimation. Spatial hints narrow the retrieval phase, and `hintRadius` controls how aggressively:
1. The hint (`hintPosition` in map-local coordinates, or `geoHint` converted from GPS) defines a center point.
2. Only the map portion within `hintRadius` meters of that point is kept as the search area.
3. Visual matching then runs against the selected map zone only.
Only the map zone within hintRadius of the hint point is searched; the rest of the map is skipped.
If `use2DFiltering` is enabled with a `geoHint`, the radius check ignores the Y-axis (altitude) and uses only horizontal distance (X and Z).
### Parameter Format
| Endpoint type | Format | Example |
| --------------------------------------------------------------- | -------------- | -------------------- |
| JSON body (`/vps/map/query`) | Number | `"hintRadius": 15` |
| Form-data (`/vps/map/query-form`, `/vps/map/multi-image-query`) | Numeric string | `"hintRadius": "15"` |
| Range | Default |
| ------- | ------- |
| 1–100 m | 25 m |
Values outside the range are clamped to 1–100 by the SDKs. Works for both single Maps and MapSets.
***
### Choosing a Radius
Match the radius to the accuracy of whatever produced the hint:
| Hint source | Suggested radius |
| ---------------------------------------------- | ---------------- |
| Previous localization result (device tracking) | 5–15 m |
| QR code or known start point | 5–10 m |
| BLE beacon / WiFi positioning | 10–25 m |
| Outdoor GPS (`geoHint`) | 25–50 m |
| Low-accuracy GPS (urban canyon, near windows) | 50–100 m |
If localization starts failing with a tight radius, the hint is probably drifting outside the search area: widen the radius or refresh the hint.
***
### Examples
```json
// Tight radius around a previous localization result (JSON body)
{
"hintPosition": [12.5, 2.0, -7.3],
"hintRadius": 10
}
// GPS hint with a wider radius and 2D filtering (JSON body)
{
"geoHint": [37.7749, -122.4194, 10.0],
"hintRadius": 50,
"use2DFiltering": true
}
// Form-data endpoint, pass as strings
"hintPosition": "[12.5, 2.0, -7.3]"
"hintRadius": "10"
```
***
### SDK Support
`hintRadius` is exposed in all MultiSet SDKs alongside the hint parameters:
* **Unity SDK / Quest SDK**: `hintRadius` (int) on `MapLocalizationManager` and `SingleFrameLocalizationManager`
* **Android Native**: `LocalizationConfig.hintRadius`
* **iOS Native**: `MultisetConfig` hint settings
* **WebXR SDK**: `hintRadius` in the localization options
***
### Best Practices
* Always pair `hintRadius` with `hintPosition` or `geoHint`. Sent alone, it is ignored by the server, and some SDKs skip sending it entirely.
* Start with the default 25 m and tighten it only once your hint source has proven reliable.
* Update the hint frequently when the user is moving, a stale hint with a tight radius can exclude the user's true position from the search area.
* For multi-floor buildings, combine with [`hintFloorHeight`](/multiset/basics/localization/hint-floor-height) to constrain the vertical range, since a 25 m sphere can span several floors.
* To read map-local coordinates for `hintPosition` directly off the mesh, see [Find Hint Coordinates](/multiset/workflow/find-hint-coordinates).
# Hint MapCodes
Pass Hint MapCodes in your MapSet query to reduce the localization search scope
#### Overview
When working with MapSets containing multiple maps, visual aliasing can occur where similar-looking areas (such as floors with identical features) confuse the localization system. By passing Hint MapCodes in your MapSet queries, you can restrict the localization search to specific maps within the MapSet, improving accuracy and performance.
#### Key Concepts
* **MapSet**: A collection of related maps (e.g., multiple floors of a building)
* **Hint MapCodes**: Identifiers that restrict the localization search scope
* **Visual Aliasing**: When visually similar areas cause localization confusion
{% hint style="warning" %}
Note: Update the Hint MapCode before a localization request in Unity SDK so that the values are passed during localization
{% endhint %}
#### Implementation
You can pass Hint MapCodes to the localization system using the following method:
```
public void UpdateHintMapCodes()
{
List hintMapCodes = new()
{
"HintmapCode1",
"HintmapCode2"
};
mapLocalizationManager.hintMapCodes = hintMapCodes;
}
```
#### Real-World Applications
Hint MapCodes can be provided through various user interactions:
* Floor selection in a building navigation interface
* QR code scanning at entry points
* Location-aware services
* User input through UI elements
#### Benefits
* **Improved Accuracy**: Reduces incorrect map matches
* **Faster Localization**: Narrows the search space
* **Better User Experience**: More reliable navigation, especially in multi-floor environments
#### Best Practices
* Update Hint MapCodes when context changes (e.g., when a user changes floors)
* Use descriptive MapCodes that align with your application's navigation model
* Consider implementing fallback mechanisms when Hint MapCodes don't yield results
#### Troubleshooting
If localization fails despite using Hint MapCodes:
* Verify the MapCodes exist in the current MapSet
* Check for typos in MapCode strings
* Ensure the MapLocalizationManager is properly initialized
# GeoHint in Localization
Release **v1.7.0** adds two major features to the MultiSet VPS for improved localization in large georeferenced maps.
1. **GeoHint in Localization Requests**
* **Purpose:** Pass the device's GPS coordinates (latitude, longitude, altitude) in localization requests after georeferencing your map.
* **Benefits:**
* Significantly reduces localization time in large outdoor environments.
* Mitigates issues of visual aliasing.
* Improves overall localization performance and accuracy.
2. **GeoCoordinates in Localization Responses**
* **Purpose:** Get localization results directly in global GPS coordinates (latitude, longitude, altitude).
* **Benefits:**
* Enables seamless integration with other applications or services that rely on GPS coordinates.
* Directly connects localization results to real-world locations.
***
### How to Use These Features
#### In Unity (MultiSet SDK Manager)
The following GPS-related options are available in the MultiSet SDK Manager component in Unity:
* **passGeoPose:** Enables sending device GPS as a GeoHint in requests.
* **geoCoordinatesInResponse:** Enables receiving localization results in global GPS coordinates.
* **hintRadius:** Search radius in meters for spatial filtering (default: 25m, range: 1–100). Only applies when geoHint or hintPosition is provided.
* **use2DFiltering:** When enabled, skips altitude (Y-axis) in geoHint spatial filtering and uses only horizontal distance (X and Z axes). Only applies when geoHint is provided.
**Setup Steps:**
1. **Update Map GeoCoordinates:**\
Before enabling these features, update the geo-coordinates of your map in the MultiSet Developer Portal.
2. **Enable Options in Unity:**
* Open your scene and select the MultiSet SDK Manager (or Single Frame Localization Manager).
* Check the box for `Pass GeoPose` to send device GPS as a hint.
* Check the box for `GeoCoordinates In Response` to receive results in global GPS coordinates.
* Adjust `Hint Radius` to control the spatial filtering search radius (default: 25 meters).
* Enable `Use 2D Filtering` if you want to ignore altitude differences during spatial filtering.
## GeoHint for MapSet
To enable GeoHint in MapSet, start by georeferencing the primary map. Afterward, merge all the maps to enable georeferencing for the entire MapSet.
***
#### Using MultiSet Localization API Directly
If you are integrating via the API (outside Unity), you can enable these features by including the following fields in your localization request:
**Request Parameters**
| Field | Type | Description |
| ------------------------- | ------ | --------------------------------------------------------------------------------------------------------------- |
| `geoHint` | string | Comma-separated string, e.g. `"latitude,longitude,altitude"`. Pass the device's GPS as a hint for localization. |
| `convertToGeoCoordinates` | string | Pass `"true"` to receive localization response in global coordinates. |
**Example Request Body and Response:**
JSON
```
{
"poseFound": true,
"position": {
"x": -0.12362211813361121,
"y": 1.563005737034826,
"z": 6.99455530632896
},
"rotation": {
"x": 0.0054829774148128785,
"y": -0.575302195022844,
"z": -0.009251237126862318,
"w": 0.8178702439703922
},
"mapIds": [
"68c8fe2ff8f229955c39eb02"
],
"GeoPose": {
"frame_spec": {
"model": "WGS-84",
"frame": "Y-Up-ENU"
},
"pose": {
"position": {
"lat": 12.952603117267845,
"lon": 77.61615745357129,
"h": 912.923009573482
},
"quaternion": {
"x": 0.01041866301787597,
"y": -0.985121678974319,
"z": -0.002664561995662227,
"w": 0.1715215123100019
}
}
}
}
```
### Feature Details
#### 1. Passing GeoHint in Localization
* **How:**\
When `passGeoPose` is enabled, the SDK will automatically attach the device's current GPS coordinates to your localization request.
* **Why:**\
This narrows down the search area in large outdoor georeferenced maps, speeding up localization and reducing errors.
#### 2. Receiving GeoCoordinates in Response
* **How:**\
When `geoCoordinatesInResponse` is enabled (or when `convertToGeoCoordinates: "true"` is included in API requests), the SDK converts the localization result into global GPS coordinates.
* **Why:**\
This is useful for interoperability with external systems or for mapping localization results to real-world positions.
***
### Important Notes
* **Map Preparation:**\
Ensure your map is properly geo-referenced and updated in the MultiSet Developer Portal before using these options.
* **Platform Support:**\
GPS functionality requires device location permissions; the SDK will prompt the user accordingly.
* **Unity Integration:**\
No additional scripting is needed—just enable the options in the SDK Manager.
***
# GeoPose Support
At MultiSet, our goal is to provide the most accurate and flexible localization possible. To enhance the portability and interoperability of our localization data, the MultiSet API response now includes a [**GeoPose**](https://www.geopose.org/) object.
This document explains what GeoPose is, how it's structured in our API response, and how you can use it to build powerful, large-scale, and interconnected applications.
## What is GeoPose?
[**GeoPose**](https://www.geopose.org/) is an open standard for expressing the position and orientation (a full 6-DoF pose) of an object in a real-world, geographic coordinate system (WGS-84).
Think of it as a universal language for location. Instead of describing a position relative to a local, arbitrary map origin (e.g., "5 meters from the corner of a building"), GeoPose describes the exact same position and orientation in a globally consistent frame of reference that any GIS, mapping, or geospatial tool can understand.
This solves a major challenge in AR and robotics: **data interoperability**.
#### The MultiSet Localization Response
When you perform a localization query, our API returns a JSON object containing two distinct but related coordinate systems: the traditional **Local Map Coordinates** and the new **Global GeoPose Coordinates**.
{% hint style="success" %}
To enable GeoPose in localization response, please pass convertToGeoCoordinates: true in the localization query api's request body
{% endhint %}
### **Sample API Response:**
```json
{
"poseFound": true,
"position": {
"x": -0.1236,
"y": 1.5630,
"z": 6.9945
},
"rotation": {
"x": 0.0054,
"y": -0.5753,
"z": -0.0092,
"w": 0.8178
},
"mapIds": [
"68c8fe2ff8f229955c39eb02"
],
"GeoPose": {
"frame_spec": {
"model": "WGS-84",
"frame": "Y-Up-ENU"
},
"pose": {
"position": {
"lat": 12.952603,
"lon": 77.616157,
"h": 912.9230
},
"quaternion": {
"x": 0.0104,
"y": -0.9851,
"z": -0.0026,
"w": 0.1715
}
}
}
}
```
#### Understanding the Two Coordinate Systems
### **1. Local Map Coordinates (position, rotation)**
The top-level position and rotation fields provide the device's pose relative to the internal origin of the specific MultiSet map (mapIds).
* **position**: {x, y, z} in meters from the map's origin.
* **rotation**: {x, y, z, w} quaternion describing the device's orientation in the map's local frame.
**When to use it:** This is the most direct and performant data to use for rendering content **within the context of that specific MultiSet map**. If your application's logic and assets all exist relative to the map you just localized against, this is the pose to use.
### **2. Global Coordinates (GeoPose)**
The nested GeoPose object provides the same pose but in a globally consistent reference frame.
* **frame\_spec**: Defines the coordinate system. MultiSet uses the standard Y-Up-ENU frame.
* model: WGS-84, the standard GPS ellipsoid model.
* frame: Y-Up-ENU, a right-handed coordinate system where +Y is Up, +Z is True North, and +X is East.
* **pose.position**:
* lat: Latitude in decimal degrees.
* lon: Longitude in decimal degrees.
* h: Ellipsoidal height in meters.
* **pose.quaternion**: The device's orientation relative to the ENU frame at its specific location.
**When to use it:** You should use the GeoPose object whenever you need to interact with external systems or data that is not tied to the specific MultiSet map.
#### Why This Matters: Use Cases for GeoPose
Providing both poses gives you maximum flexibility. The GeoPose object unlocks powerful new capabilities for interoperability:
1. **Building Digital Twins:**\
Fuse MultiSet's real-time localization with building information models (BIM), GIS data, or sensor feeds. Since GeoPose is a global standard, you can align data from multiple sources with high precision.
2. **Large-Scale, Multi-Map Experiences:**\
Create seamless AR experiences that span multiple, non-overlapping MultiSet maps. You can use the GeoPose to calculate the transformations between different maps and manage user location in a global context.
3. **Data Portability and Logging:**\
Log user or device trajectories in a format that is independent of any single map. This data can be analyzed later in standard geospatial tools or used to visualize paths on a 2D map like Mapbox or Google Maps.
4. **Inter-System Communication:**\
Pass the location of a device localized by MultiSet to another system that uses GPS or a different localization technology. For example, guide an autonomous robot localized with MultiSet to a specific GPS coordinate.
#### Practical Example
Imagine your application needs to show a 3D model of a proposed building (from a CAD file with georeferenced coordinates) within your AR scene.
1. Your app receives the localization response from the MultiSet API.
2. You use the GeoPose object to understand the device's absolute position and orientation on Earth.
3. Your app knows the absolute GeoPose of the proposed building.
4. You can now calculate the building's position and orientation relative to your device and render it correctly in your scene, even if your local scene's origin is completely different from the MultiSet map's origin.
The GeoPose acts as the "ground truth" that connects your MultiSet-powered world to any other georeferenced data source.
# On-Premises Localization
MultiSet VPS Self-Hosting allows Enterprise customers to deploy MultiSet core binaries directly within their private network infrastructure. This on-premises solution provides complete control over spatial data processing while maintaining the full functionality of MultiSet's AR localization capabilities.
### Enterprise Availability
**This feature is exclusively available to Enterprise customers.** The self-hosting option requires specialized deployment and integration support from the MultiSet team.
### Key Benefits
#### Enhanced Data Privacy & Security
* **Sensitive map data remains secure** within your private network perimeter
* Spatial data never leaves your controlled environment
* Full compliance with enterprise security policies and data governance requirements
* Zero exposure of proprietary location data to external services
#### Optimized Performance Architecture
* **Low latency localization queries** through local processing
* **Edge devices don't require massive map data installations** - eliminating the need to download and update gigabytes of spatial information
* **Improved AR SLAM tracking performance** as edge device processing power is focused on tracking rather than local localization computations
* Reduced bandwidth requirements for edge devices
#### Operational Advantages
* **Works in areas with no internet connectivity** - completely offline operation capability
* Reduced dependency on external network conditions
* Consistent performance regardless of internet quality or availability
* Centralized map data management and updates within your infrastructure
## How it works
The MultiSet VPS self-hosting architecture separates spatial processing from edge device operations:
1. **Core Binaries Deployment**: MultiSet core binaries are installed on your private VPS infrastructure
2. **SDK Integration**: Edge devices connect to your on-premises MultiSet instance through the MultiSet SDK
3. **Localized Processing**: All spatial queries and map data processing happen within your network
4. **Optimized Edge Performance**: Edge devices focus exclusively on AR SLAM tracking and rendering
### Getting Started
#### 1. Contact MultiSet Enterprise Team
To begin the self-hosting process, reach out to our Enterprise team:
[**Contact MultiSet Enterprise Team →**](https://share.hsforms.com/159qsFzsYSTGydaNc9kXZKAsq11p)
#### 2. Technical Specifications Review
The MultiSet team will collaborate with your Enterprise engineering and operations teams to:
* Share detailed technical specifications required to run MultiSet in your backend infrastructure
* Review your current infrastructure capabilities and requirements
* Provide system architecture recommendations
* Discuss integration points with your existing systems
#### 3. Installation & Integration Support
Our Enterprise team provides comprehensive deployment assistance:
* **Direct installation support**: MultiSet engineers work alongside your team to install on-premises binaries
* **SDK integration guidance**: Ensure seamless connection between your MultiSet VPS instance and MultiSet SDK implementations
* **Testing and validation**: Comprehensive testing to verify proper functionality within your environment
* **Documentation and training**: Custom documentation and team training for your specific deployment
### Architecture Benefits Summary
| Traditional Edge Processing | MultiSet VPS Self-Hosting |
| ----------------------------------------------- | ----------------------------------------- |
| Large map data downloads to each device | Centralized map data on private VPS |
| Internet dependency for updates | Local control of data and updates |
| Shared processing between SLAM and localization | Dedicated edge processing for AR SLAM |
| Potential data privacy concerns | Complete data privacy within your network |
| Variable latency based on connection | Consistent low-latency local queries |
# Simulation
Test and compare VPS localization against a map or MapSet in the Developer Portal using previously captured simulation data.
### What is Simulation?
Simulation lets you test VPS localization against one of your maps or MapSets directly in the MultiSet Developer Portal, without deploying an app or standing in the physical space. You run previously captured camera data against the map and inspect exactly where the VPS localized the device, how confident it was, and how different query types perform.
This is the fastest way to validate a map's localization quality, debug problem areas, and decide whether a single frame or multi frame query works best for a given location.
### Capture Simulation Data
Before you can run a simulation, you need simulation data associated with your account. Simulation data is a short capture of camera frames and sensor data recorded at a physical location. You can capture it in two ways:
* **MultiSet App:** Open the app, start a new capture, and choose **Simulation Data**. Walk to the location you want to test, capture for a few seconds, name the dataset, and upload it. It becomes instantly available to your account. See [Simulation Data Capture](/multiset/multiset-app/simulation-data-capture).
* **Unity SDK:** Use the `SimulationDataManager` and the `SimulationDataCapture` scene to record and upload simulation data from the Editor or a build. See [Localization Simulation](/multiset/unity-sdk/localization-simulation) for the full workflow.
Captured datasets are tied to your account, so any data you record is available to run against your maps in the portal.
### Run a Simulation in the Developer Portal
#### 1. Select a Map or MapSet
Open a map or MapSet in the Developer Portal and choose the **Simulation** option from the actions on its card.
Choose the Simulation option on a map or MapSet card.
#### 2. Load the Mesh and Your Captured Data
The portal loads the mesh of the selected map or MapSet. On the left, the **Simulation Datasets** panel lists all the simulation data captured against your account, each with its name, code, size, and capture time.
All datasets captured against your account, ready to run.
#### 3. Run and Inspect the Result
Each dataset has a **Single Frame** and a **Multi Frame** button. Click one to run that query type against the loaded map. On a successful localization, a **camera frustum** (the query frame) is placed in the mesh at the exact pose the VPS returned, with an orientation axis gizmo, so you can visually confirm the result against the real geometry.
The dataset card reports:
* The localization status and query type used (e.g. **Pose Found · Single-frame**)
* The **confidence** score
* The returned position (`X`, `Y`, `Z`)
* A thumbnail of the query frame that was localized
Use **Re-run** to run the same dataset again.
On a successful pose, the camera frustum is positioned exactly where the VPS localized, with confidence and pose values.
### Compare Query Types
For each dataset you can run the localization as a **Single Frame** or a **Multi Frame** query, and re-run as many times as you like. By comparing the pose accuracy and confidence each query type returns for the same location, you can decide which API performs better in that particular situation before you commit to it in your app.
{% hint style="info" %}
Single frame queries are faster but rely on one viewpoint, while multi frame queries combine several frames for more robust results in challenging areas. See [Localization](/multiset/basics/localization#query-types) for how the two query types work.
{% endhint %}
# REST API Docs
MultiSet provides endpoints for creating and managing VPS Maps, MapSets, Object Tracking and Localization Queries using a single or multiple frames in a request
{% hint style="warning" %}
To whitelist your domain, please follow these steps for domain setup: [CORS](/multiset/basics/credentials/configuring-allowed-domains-cors)
{% endhint %}
{% embed url="" %}
Here is a group of API's available to interact with the MultiSet Platform:
[Authentication](/multiset/basics/rest-api-docs/authentication): How to authenticate with MultiSet API service using JWT tokens
[Map Query](/multiset/basics/rest-api-docs/map-query): Localization query in a map or mapset using single or multiple images in a request
[Map Operations:](/multiset/basics/rest-api-docs/map-operations) APIs to download a map file or delete a map
[MapSet](/multiset/basics/rest-api-docs/mapset): APIs for creating and managing MapSets (collections of maps with relative positioning)
[Map Upload:](/multiset/basics/rest-api-docs/map-upload) APIs showing how to upload a new scan file for creating a map using third-party scans
[Object Upload](/multiset/basics/rest-api-docs/modelset-upload): API showing how to upload a 3D model for object tracking
[Map Details](/multiset/basics/rest-api-docs/map-details): APIs to retrieve map details across an account or for a specific map
[Georeference Map](/multiset/basics/rest-api-docs/georeference): API to georeference a 3D map from local-to-WGS84 control point pairs
[Object Tracking Query](/multiset/basics/rest-api-docs/object-query): API for performing localization query for object tracking
# Authentication
### M2M Auth
{% hint style="warning" %}
To whitelist your domain and avoid CORS issues, follow the steps in [Configuring Allowed Domains (CORS)](/multiset/basics/credentials/configuring-allowed-domains-cors)
{% endhint %}
{% hint style="info" %}
Expiry time of the token is 30 minutes from the time a fresh token is generated.
{% endhint %}
{% openapi src="/files/Bx6nb0NA6pS1eMimBWoz" path="/m2m/token" method="post" %}
[m2mauth.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-63b96990189b917ecc399b1c6b02bb67ffedf391%2Fm2mauth.yaml?alt=media)
{% endopenapi %}
{% hint style="info" %}
Credentials are passed only in the `Authorization` header as `Basic Base64(clientId:clientSecret)`. No request body is required.
{% endhint %}
## Code Examples
### Browser (JavaScript)
For client-side applications like WebXR, you can generate tokens directly in the browser:
```javascript
async function generateToken(clientId, clientSecret) {
const authorization = "Basic " + btoa(`${clientId}:${clientSecret}`);
const response = await fetch("https://api.multiset.ai/v1/m2m/token", {
method: "POST",
headers: {
Authorization: authorization,
"Content-Type": "application/json",
},
});
const { token, expiresOn, error } = await response.json();
if (error) {
throw new Error(error);
}
return { token, expiresOn };
}
// Usage
const { token, expiresOn } = await generateToken('YOUR_CLIENT_ID', 'YOUR_CLIENT_SECRET');
```
{% hint style="warning" %}
Browser-based authentication exposes your credentials in client-side code. This is suitable for prototyping and demos, but for production applications consider using a backend proxy to keep credentials secure.
{% endhint %}
### Node.js (Backend)
For server-side applications:
```javascript
async function generateToken(clientId, clientSecret) {
const authorization = "Basic " + Buffer.from(`${clientId}:${clientSecret}`).toString('base64');
const response = await fetch("https://api.multiset.ai/v1/m2m/token", {
method: "POST",
headers: {
Authorization: authorization,
"Content-Type": "application/json",
},
});
const { token, expiresOn, error } = await response.json();
if (error) {
throw new Error(error);
}
return { token, expiresOn };
}
```
# Map Query
### VPS Query API
Query your map to get the device position with respect to Map local origin. The query API takes query image (encoded as base64 string) and other image metadata.
{% hint style="warning" %}
The maximum query image resolution is **1280** pixels in either width or height.
{% endhint %}
{% openapi src="/files/ajrrlVmhtUvgT9mQ0Qyq" path="/vps/map/query" method="post" %}
[VPSquery.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-66c0b0070ef0c01467577e359db5beef8c4f8a66%2FVPSquery.yaml?alt=media)
{% endopenapi %}
#### Sample Response (JSON Body)
```json
{
"poseFound": true,
"position": {
"x": -5.89516855615433,
"y": 1.225031596452081,
"z": 2.2112895596804227
},
"rotation": {
"x": -0.007873432249486393,
"y": 0.8212519784928444,
"z": 0.03204363652415735,
"w": 0.5696107462509017
},
"confidence": 0.46875,
"mapIds": ["67e12d4bff7ecf561f2f8a0c"],
"mapCodes": ["MAP_RJFKKWQ1787J"],
"responseTime": 2572
}
```
{% openapi src="/files/4THolmFT4ct9qovYeocP" path="/vps/map/query-form" method="post" %}
[VPSqueryform.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-f51edc3b19b02d9f33b288a339b7dbdade59b22e%2FVPSqueryform.yaml?alt=media)
{% endopenapi %}
#### Sample Response (Form Data)
```json
{
"poseFound": true,
"position": {
"x": -2.8765866867008025,
"y": 1.4018881843419664,
"z": 7.677072632098843
},
"rotation": {
"x": -0.0033476528966347096,
"y": 0.6750939967230872,
"z": -0.00310829178339555,
"w": 0.7377175796541121
},
"confidence": 0.9175769612711023,
"mapIds": ["67e12d4bff7ecf561f2f8a0c"],
"mapCodes": ["MAP_RJFKKWQ1787J"],
"responseTime": 2669
}
```
{% hint style="warning" %}
**FormData Content-Type:** When using fetch or similar HTTP clients with FormData, do not manually set the Content-Type header. The browser automatically sets it to multipart/form-data with the required boundary string. Manually setting it will strip the boundary and cause a parse error.
{% endhint %}
#### Code Example (Python)
```python
import json
import requests
BASE_URL = "https://api.multiset.ai/v1"
TOKEN = "YOUR_M2M_TOKEN"
with open("query.jpg", "rb") as f:
response = requests.post(
f"{BASE_URL}/vps/map/query-form",
headers={"Authorization": f"Bearer {TOKEN}"},
data={
"mapCode": "MAP_RJFKKWQ1787J",
"isRightHanded": "false",
"width": "720",
"height": "960",
"fx": "670.4620971679688",
"fy": "670.4620971679688",
"px": "478.838623046875",
"py": "365.346618652343",
# Optional localization hints, all as strings
"hintPosition": "-3.452,0.252,-0.531",
"hintRadius": "15",
"hintFloorHeight": "[1.5, 4.5]",
},
files={"queryImage": ("query.jpg", f, "image/jpeg")},
)
print(response.status_code, response.json())
```
### Spatial Hint: hintPosition
The `hintPosition` parameter restricts the localization search to a radius around a given point in the map, speeding up the query and improving accuracy in visually repetitive scenes.
| Parameter | Type | Description |
| -------------- | ----------- | ----------------------------------------------------------------------- |
| `hintPosition` | `[x, y, z]` | Hint point in the map's **left-handed (LHS / Unity) coordinate system** |
| `hintRadius` | `number` | Search radius in meters (default `25`, range `1`–`100`) |
{% hint style="warning" %}
**`hintPosition` is in LHS (Unity) coordinates**, the same coordinate system returned in the `position` field of a successful localization response (`isRightHanded: false`). A previous localization result's `position` can be passed directly as the hint. If your application works in right-handed space (e.g. ROS, Three.js), convert to LHS before sending.
{% endhint %}
On form-data endpoints, pass `hintPosition` as a JSON-encoded string, e.g. `"[2.5, 0.1, 8.0]"`. See [Pose Prior : HintPosition](/multiset/basics/localization/pose-prior-hintposition) for the full guide.
### Floor-Level Search: hintFloorHeight
The `hintFloorHeight` parameter limits the search to a specific vertical band, enabling floor-level localization in multi-floor buildings.
| Parameter | Type | Description |
| ----------------- | ---------------- | ---------------------------------------- |
| `hintFloorHeight` | `[y_min, y_max]` | Vertical band in map-local Y coordinates |
**Rules:**
* Both values are in map-local coordinates (or global MapSet coordinates for MapSet queries)
* Order does not matter: `[-3, -1]` and `[-1, -3]` both select images with y ∈ \[-3, -1]
* Can be combined with `hintPosition`: spatial filter runs first, height filter applies on the result
**Examples:**
```json
// Floor 2 of a building where floor is at y=3.0, ceiling at y=6.0
"hintFloorHeight": [3.0, 6.0]
// Basement (negative Y)
"hintFloorHeight": [-3.0, -0.5]
// Form-data endpoints, pass as JSON string
"hintFloorHeight": "[1.5, 4.5]"
```
### Query Mode: queryMode
The `queryMode` parameter selects which localization engine runs your query, letting you trade latency for accuracy. The request and response format stay exactly the same on both engines.
| Value | Engine | Description |
| ------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `vps-1` | Standard (default) | The fast standard single-image localization engine, roughly 2 seconds. Best for real-time, interactive localization. |
| `vps-2` | Deep Search | A deep-search engine with up to **15% higher recall** and improved accuracy, at higher latency (around 3 to 4 seconds). Best for offline or background workflows and challenging, visually repetitive scenes. |
**Rules:**
* Optional. When omitted, the query runs on `vps-1`, so existing integrations are unaffected.
* Supported on both single-image endpoints: `/vps/map/query` (JSON) and `/vps/map/query-form` (form-data). It is ignored by the multi-image query endpoint.
* Can be combined with all localization parameters (`hintPosition`, `hintRadius`, `hintFloorHeight`, `geoHint`, `hintMapCodes`).
* Works with Maps, MapSets, and versioned maps.
**Examples:**
```json
// JSON body: run this query on the Deep Search engine
"queryMode": "vps-2"
// Form-data endpoints: pass as a plain string field
"queryMode": "vps-2"
```
{% hint style="info" %}
Use `vps-2` when accuracy matters more than latency, for example when placing persistent content or localizing in repetitive corridors. For high-frequency relocalization during an AR session, `vps-1` remains the recommended default. You can also try both engines on your own datasets from the Developer Portal simulation viewer. See [Query Mode](/multiset/basics/localization/query-mode) for guidance on choosing between the two.
{% endhint %}
### VPS Multi Image Query API
Multi-image query API requires a minimum of 4 images in each request. Up to 6 images can be passed. Higher image counts increase localization robustness and accuracy but may compromise latency.
{% hint style="info" %}
**About image#\_data properties:** The `image1_data`, `image2_data`, etc. fields contain the local SLAM tracking data (position and rotation) for when each image was captured. This is the device pose in the local coordinate system from ARKit (iOS) or ARCore (Android) at the moment the image was taken.
{% endhint %}
{% openapi src="/files/oOKFI0YnlAkR5K578gJk" path="/vps/map/multi-image-query" method="post" %}
[vps-maps-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-f4bdc26ef511f1388a04c68c729c157201c797fe%2Fvps-maps-api.yaml?alt=media)
{% endopenapi %}
#### Sample Response (Multi Image)
```json
{
"poseFound": true,
"estimatedPose": {
"position": {
"x": 3.9023228363353613,
"y": 2.2199969114542415,
"z": 7.684019738005462
},
"rotation": {
"x": 0.0008889080606250241,
"y": 0.7454695784085924,
"z": -0.022682644709609446,
"w": 0.6661529967948453
}
},
"trackingPose": {
"position": {
"x": 0.0022450201213359833,
"y": 2.471872329711914,
"z": -10.018059730529785
},
"rotation": {
"x": 0.030655404552817345,
"y": 0.14812710881233215,
"z": -0.00305502163246274,
"w": -0.9884883761405945
}
},
"imageId": "image4",
"mapIds": ["67e12d4bff7ecf561f2f8a0c"],
"confidence": 0.34274043817304123,
"mapCodes": ["MAP_RJFKKWQ1787J"],
"responseTime": 4919
}
```
{% hint style="warning" %}
**FormData Content-Type:** When using fetch or similar HTTP clients with FormData, do not manually set the Content-Type header. The browser automatically sets it to multipart/form-data with the required boundary string. Manually setting it will strip the boundary and cause a parse error.
{% endhint %}
#### Code Example (Python)
Split the request in two parts: the image binaries go in `files`, and every other field, including each `imageN_data` tracking pose, goes in `data` as a string.
```python
import json
import requests
from contextlib import ExitStack
BASE_URL = "https://api.multiset.ai/v1"
TOKEN = "YOUR_M2M_TOKEN"
image_paths = ["image1.jpg", "image2.jpg", "image3.jpg", "image4.jpg"]
# Local SLAM tracking pose (ARKit / ARCore) for each image, in the same order
tracking_poses = [
{"x": -5.1772, "y": 0.2936, "z": -2.6439, "qx": -0.0185, "qy": 0.9949, "qz": -0.0691, "qw": 0.0703},
{"x": -4.8210, "y": 0.3011, "z": -2.1094, "qx": -0.0201, "qy": 0.9932, "qz": -0.0805, "qw": 0.0812},
{"x": -4.3945, "y": 0.2874, "z": -1.7723, "qx": -0.0173, "qy": 0.9951, "qz": -0.0664, "qw": 0.0699},
{"x": -3.9012, "y": 0.2952, "z": -1.3388, "qx": -0.0190, "qy": 0.9940, "qz": -0.0731, "qw": 0.0764},
]
# Text fields, all sent as strings
text_fields = {
"mapCode": "MAP_RJFKKWQ1787J", # or "mapSetCode", exactly one of the two
"isRightHanded": "false",
"width": "720",
"height": "960",
"fx": "670.4620971679688",
"fy": "670.4620971679688",
"px": "478.838623046875",
"py": "365.346618652343",
# Optional localization hints
"hintPosition": "-3.452,0.252,-0.531",
}
# Each imageN file part needs a matching imageN_data text part
for index, pose in enumerate(tracking_poses, start=1):
text_fields[f"image{index}_data"] = json.dumps(pose)
with ExitStack() as stack:
files = [
(
f"image{index}",
(f"image{index}.jpg", stack.enter_context(open(path, "rb")), "image/jpeg"),
)
for index, path in enumerate(image_paths, start=1)
]
response = requests.post(
f"{BASE_URL}/vps/map/multi-image-query",
headers={"Authorization": f"Bearer {TOKEN}"},
data=text_fields,
files=files,
)
print(response.status_code, response.json())
```
{% hint style="info" %}
**Rules for the multipart body:**
* File parts must be named `image1` to `image6`, and each one needs a matching `imageN_data` text part with the same number. A missing `imageN_data` returns `400` with `Missing metadata for imageN`.
* Each `imageN_data` value is a JSON-encoded string containing `x`, `y`, `z`, `qx`, `qy`, `qz` and `qw`. All seven keys are required.
* Send between 4 and 6 images. Fewer returns `At least 4 image file is required!`, more returns `Maximum 6 image files allowed!`.
* Pass exactly one of `mapCode` or `mapSetCode`.
* Do not set the `Content-Type` header yourself. `requests` adds `multipart/form-data` with the boundary when you pass `files`.
{% endhint %}
# Map Operations
{% openapi src="/files/UV64lJchhd7SUxNgji6q" path="/file" method="get" %}
[filelink.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2FCTMTzA039Zqe3I3p1THH%2Ffilelink.yaml?alt=media\&token=213cb0b1-3e53-4e31-a643-870c715cf75b)
{% endopenapi %}
### Replace a map's mesh
Use `/vps/map/{mapCode}/mesh-upload-url` to replace a map's **textured** or **raw** mesh. The flow is:
1. `POST /vps/map/{mapCode}/mesh-upload-url` with the `meshType` you want to replace (`textured` or `raw`).
2. `PUT` the mesh file to the returned `uploadUrl`, using the `contentType` from the response as your `Content-Type` header. The URL is valid for 1 hour.
{% hint style="info" %}
You never construct a storage path. The upload location is derived from the map in the path, so the request can only replace that map's own mesh. The map must be **active**, and your role must allow write access (owner or admin).
{% endhint %}
{% openapi src="/files/cYpvLjgZRyUzuDKNwnnr" path="/vps/map/{mapCode}/mesh-upload-url" method="post" %}
[mesh-upload-url.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-5fcd439c933a2c5405a5e94d73dad63634020031%2Fmesh-upload-url.yaml?alt=media)
{% endopenapi %}
{% hint style="success" %}
Replacing a mesh does not affect localization accuracy. Queries run on the map's feature data, not on the mesh, so you can update the visual mesh at any time without re-processing the map.
{% endhint %}
{% openapi src="/files/7h9I87pKfSRJ4vlJK1Kg" path="/vps/map/{mapCode}" method="delete" %}
[map-delete.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-99caa797ce8ad5d21789558c59d17d55623fc78b%2Fmap-delete.yaml?alt=media)
{% endopenapi %}
# MapSet
MapSet APIs allow you to create and manage collections of maps with relative positioning for large-scale VPS coverage.
{% hint style="warning" %}
To whitelist your domain, follow: [Configuring Allowed Domains (CORS)](/multiset/basics/credentials/configuring-allowed-domains-cors)
{% endhint %}
## Overview
A MapSet is a collection of two or more maps positioned relative to each other. This enables:
* Large area coverage by combining multiple scans
* Seamless localization across connected spaces
* Flexible map arrangement and positioning
{% hint style="info" %}
**Using Codes Instead of IDs**
All MapSet APIs use human-readable codes:
* **MapSet Code** (e.g., `MSET_ZIRWP1NV0WBH`) - Use this for all MapSet operations
* **Map Code** (e.g., `MAP_WUTCLWDXTK6U`) - Use this when adding maps to a MapSet
You can find these codes in the MultiSet Developer Portal or from API responses.
{% endhint %}
## Endpoints
### Create MapSet using Overlap
Creates a new MapSet from two existing maps that share overlapping scan area — pass only the **Map Codes** of a source and a target map, and the cloud computes the relative pose between them automatically. No `relativePose` is required in the request.
{% hint style="info" %}
**When to use this:** when your two scans overlap physically (e.g. you scanned a corridor twice from each end, or two adjacent rooms share a doorway). The cloud aligns the maps for you. If your maps don't overlap, use [Create MapSet](#create-mapset) and supply `relativePose` manually instead.
{% endhint %}
**Constraints:**
* Both maps must be in your account and in `active` status.
* Neither map can already belong to a MapSet.
* The `sourceMapCode` becomes the origin (order=0) of the new MapSet.
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/overlap" method="post" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Add Map to MapSet using Overlap
Adds a new map to an existing MapSet. The new map is aligned against the entire MapSet, so you do not pick an overlapping map to anchor it against: send only the new map's Map Code. The cloud computes its `relativePose` automatically from the overlap, no `relativePose` is required, and the existing maps in the set are not affected.
{% hint style="info" %}
**When to use this:** when you've added a new scan that overlaps with the area a MapSet already covers (e.g. extending coverage to an adjacent area). If the new scan does not overlap the existing MapSet, use [Add Map to MapSet](#add-map-to-mapset) and supply `relativePose` manually instead.
{% endhint %}
**Constraints:**
* `targetMapCode` must be `active` and not already part of any MapSet.
* The new map must be on the same generation as the MapSet. Mixing generations returns `400`.
{% hint style="warning" %}
The request body accepts only `targetMapCode` plus the optional `isGravityAligned`. Sending `sourceMapCode` returns `400` with `{"error": "\"sourceMapCode\" is not allowed"}`. An anchor map was required by an earlier version of this endpoint and is no longer accepted.
{% endhint %}
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/overlap/{mapSetCode}" method="put" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Create MapSet
Creates a new MapSet with two or more maps. Use **Map Codes** to reference the maps.
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set" method="post" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Get MapSet Details
Retrieves details of a MapSet including all associated maps and their relative poses. Use the **MapSet Code** (e.g., `MSET_ZIRWP1NV0WBH`).
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/{mapSetCode}" method="get" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Update MapSet Details
Updates the name of an existing MapSet. Use the **MapSet Code**.
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/details/{mapSetCode}" method="put" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Add Map to MapSet
Adds a new map to an existing MapSet with its relative pose. Use the **MapSet Code** in the URL and **Map Code** in the request body.
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/{mapSetCode}" method="put" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Update Map Pose in MapSet
Updates the relative pose of a map within a MapSet.
{% hint style="info" %}
Use the `dataId` (MapSetData ID) from the **Get MapSet Details** response. This is the only operation that requires an internal ID.
{% endhint %}
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/data/{dataId}" method="put" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Delete Map from MapSet
Removes a map from a MapSet.
{% hint style="warning" %}
**Constraints:**
* Cannot delete the primary map (order=0)
* Cannot delete if it would leave less than 2 maps in the MapSet
* Use the `dataId` from the **Get MapSet Details** response
{% endhint %}
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/data/{dataId}" method="delete" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
***
### Delete MapSet
Deletes a MapSet and all associated map data entries. The maps themselves are not deleted, only their association with the MapSet. Use the **MapSet Code**.
{% openapi src="/files/c8XCzdnLgXIkClBQrXdX" path="/map-set/{mapSetCode}" method="delete" %}
[mapset-crud-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-aef2ea46da08131bc7d2dbc9f092a98ffbf954b1%2Fmapset-crud-api.yaml?alt=media)
{% endopenapi %}
## Schemas
### RelativePose
The `relativePose` object defines a map's position and orientation within the MapSet coordinate system:
| Field | Type | Description |
| ------------- | ------ | ---------------------------------- |
| `position.x` | number | X position coordinate |
| `position.y` | number | Y position coordinate |
| `position.z` | number | Z position coordinate |
| `rotation.qx` | number | X component of quaternion rotation |
| `rotation.qy` | number | Y component of quaternion rotation |
| `rotation.qz` | number | Z component of quaternion rotation |
| `rotation.qw` | number | W component of quaternion rotation |
{% hint style="info" %}
The first map in a MapSet (order=0) typically uses identity pose: position (0,0,0) and rotation quaternion (0,0,0,1).
{% endhint %}
# Map Upload
Map files are uploaded to S3 in parts, so a large scan can be sent as parallel chunks rather than one request. The process supports files up to **25 GB**, with retries on individual parts.
There are two ways to run an upload. Both use the same map metadata and the same `PUT` per part, and both finish with the same call.
| Flow | How it works | Use it when |
| ------------- | ------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- |
| **Standard** | The create call returns a presigned URL for every part up front. | The upload is small, or the connection is reliable. |
| **Resumable** | Part URLs are signed on demand, and an interrupted upload can be continued instead of restarted. | Large scans, slow or unstable connections. **Recommended for big files.** |
{% hint style="success" %}
If you are unsure, use the **resumable** flow. It behaves the same as the standard flow on a clean run, and it saves re-uploading everything if the connection drops.
{% endhint %}
## Map metadata
Both flows start with `POST /v2/vps/map` and take the same body.
| Field | Type | Required | Notes |
| ----------------------- | ------- | -------- | --------------------------------------------------------------------------------------------------------- |
| `mapName` | string | yes | Display name for the map. |
| `fileSize` | number | yes | Total upload size **in bytes**. Must be greater than 0. |
| `partSize` | integer | no | Part size in bytes. Recommended for resumable uploads, see [Choosing a part size](#choosing-a-part-size). |
| `coordinates.latitude` | number | yes | -90 to 90. |
| `coordinates.longitude` | number | yes | -180 to 180. |
| `coordinates.altitude` | number | yes | Metres. |
| `heading` | number | no | 0 to 360 degrees. |
| `source` | object | no | Describes the scan, see below. Defaults to a `zip` upload. |
### The source object
Set `source` to match the scan you are uploading:
```json
{
"source": {
"provider": "unity",
"fileType": "zip",
"coordinateSystem": "RHS"
}
}
```
| Field | Accepted values |
| ------------------ | --------------------------------------------------------------------- |
| `provider` | `unity`, `matterport`, `leica`, `navvis`, `xgrid`, `faro`, `insta360` |
| `fileType` | `zip`, `e57`, `splat`, `360` (defaults to `zip`) |
| `coordinateSystem` | `LHS`, `RHS`, `RHS-Z-UP`, `RHS-Y-UP` |
For a Matterport or NavVis E57 scan:
```json
{
"source": {
"provider": "matterport",
"fileType": "e57",
"coordinateSystem": "RHS"
}
}
```
```json
{
"source": {
"provider": "navvis",
"fileType": "e57",
"coordinateSystem": "RHS"
}
}
```
{% hint style="info" %}
Upload each part with the `Content-Type` that matches your `fileType`: `application/zip` for a `zip` upload, `application/octet-stream` otherwise. Presigned URLs are valid for **1 hour**.
{% endhint %}
## Standard upload
### 1. Create the map
`POST /v2/vps/map` with the metadata above. The response includes a presigned URL for every part:
```json
{
"message": "Map created successfully",
"mapCode": "MAP_QQYKIBHXZE01",
"mapId": "67e12d4bff7ecf561f2f8a0c",
"key": "…",
"uploadUrls": {
"uploadId": "…",
"signedUrls": [
{ "partNumber": 1, "signedUrl": "https://…" },
{ "partNumber": 2, "signedUrl": "https://…" }
]
}
}
```
### 2. Upload the parts
Split the file and `PUT` each chunk to its matching `signedUrl`. Each successful `PUT` returns an **`ETag`** header. Keep every `ETag` with its part number, you need them to finish the upload.
### 3. Complete the upload
`POST /v2/vps/map/complete-upload/{mapId}` with the parts you uploaded:
```json
{
"parts": [
{ "PartNumber": 1, "ETag": "\"a1b2c3…\"" },
{ "PartNumber": 2, "ETag": "\"d4e5f6…\"" }
]
}
```
The parts are assembled into a single file and the map enters the processing queue. Once processing finishes, the map is available for VPS queries.
{% openapi src="/files/KkX808qgMzlMCRvDY60Y" path="/map" method="post" %}
[map-upload.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ba07526960f765cebb0450c17643bfe399463f3b%2Fmap-upload.yaml?alt=media)
{% endopenapi %}
{% openapi src="/files/KkX808qgMzlMCRvDY60Y" path="/map/complete-upload/{id}" method="post" %}
[map-upload.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ba07526960f765cebb0450c17643bfe399463f3b%2Fmap-upload.yaml?alt=media)
{% endopenapi %}
## Resumable upload
### 1. Create the map in resumable mode
`POST /v2/vps/map?resumable=true` with the same metadata. Include `partSize` so the server can confirm the upload is complete before finalizing it.
The response is deliberately small, because no part URLs are issued yet:
```json
{
"message": "Multipart upload initialised",
"mapCode": "MAP_QQYKIBHXZE01",
"mapId": "67e12d4bff7ecf561f2f8a0c"
}
```
### 2. Request URLs for the parts you are about to send
`POST /v2/vps/map/sign-part` with the `mapId` and the part numbers you want, up to **100 per call**:
```json
{
"mapId": "67e12d4bff7ecf561f2f8a0c",
"partNumbers": [1, 2, 3, 4]
}
```
```json
{
"signedUrls": [
{ "partNumber": 1, "signedUrl": "https://…" },
{ "partNumber": 2, "signedUrl": "https://…" }
]
}
```
### 3. Upload the parts
`PUT` each chunk to its `signedUrl`, exactly as in the standard flow. You do not need to keep the `ETag` values here, because the server reads the uploaded parts when you complete.
### 4. Resume after an interruption
Ask which parts already arrived:
`GET /v2/vps/map/list-parts/{mapId}`
```json
{
"active": true,
"uploadedPartNumbers": [1, 2, 3, 5]
}
```
Skip those, request fresh URLs for the rest with **sign-part**, and upload only what is missing. Repeat as often as you need, the upload stays open.
### 5. Complete the upload
`POST /v2/vps/map/complete-upload/{mapId}` with an **empty body**:
```json
{}
```
The server assembles the upload from the parts it received. If you supplied `partSize` when creating the map, it first checks that every expected part is present and returns `409` if any are missing.
### Cancel an upload
`POST /v2/vps/map/abort-upload/{mapId}` discards the upload and removes the map.
{% openapi src="/files/KkX808qgMzlMCRvDY60Y" path="/map/sign-part" method="post" %}
[map-upload.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ba07526960f765cebb0450c17643bfe399463f3b%2Fmap-upload.yaml?alt=media)
{% endopenapi %}
{% openapi src="/files/KkX808qgMzlMCRvDY60Y" path="/map/list-parts/{id}" method="get" %}
[map-upload.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ba07526960f765cebb0450c17643bfe399463f3b%2Fmap-upload.yaml?alt=media)
{% endopenapi %}
{% openapi src="/files/KkX808qgMzlMCRvDY60Y" path="/map/abort-upload/{id}" method="post" %}
[map-upload.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ba07526960f765cebb0450c17643bfe399463f3b%2Fmap-upload.yaml?alt=media)
{% endopenapi %}
## Choosing a part size
`partSize` controls how the file is split. A smaller part means more requests but less to re-send after a failure. A larger part means fewer requests but more lost work when one fails.
* **8 MB to 16 MB** suits most uploads on a normal connection.
* Go smaller, around **5 MB**, on unstable or mobile connections so a dropped part costs little.
* The number of parts works out to `ceil(fileSize / partSize)`.
## Errors worth handling
| Status | Meaning |
| ------ | ---------------------------------------------------------------------------------------------- |
| `400` | Missing or invalid metadata, maps limit reached, or the plan has expired. |
| `403` | The map belongs to another account. |
| `409` | No active upload for this map, or the upload is incomplete because expected parts are missing. |
{% hint style="info" %}
The upload location is managed for you. You work with the `mapId` and the presigned URLs, and never construct a storage path yourself.
{% endhint %}
# MatterPak Upload
The VPS Map Creation API accepts a POST request to /v1/vps/map with a JSON payload containing map metadata and Matterport authentication.
{% openapi src="/files/p14V2A9n8i4PGpDMULga" path="/vps/map" method="post" %}
[matterpak.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-f89b46090cf9fda9a6323417ba61197e44495eeb%2Fmatterpak.yaml?alt=media)
{% endopenapi %}
# Gaussian Splat Upload
Upload a Gaussian Splat (.ply + poses.json) as a Multiset VPS map via the REST API.
Gaussian Splats are uploaded through the **same two-step multipart flow** as e57 maps ([POST `/v2/vps/map`](/multiset/basics/rest-api-docs/map-upload) → upload parts → `/complete-upload/{id}`). The only thing that changes is the `source` object you send in **Step 1** — it describes that the file is a Gaussian Splat and carries splat-specific metadata.
## What to upload
A **single `.zip`** containing both:
* `point_cloud.ply` — the Gaussian Splat file.
* `poses.json` — the training camera poses.
Both files must be at the root of the archive. See [Gaussian Splat](/multiset/basics/third-party-scans/gaussian-splat) for export details and the `poses.json` schema.
## `source` object for Gaussian Splat
```json
"source": {
"provider": "xgrid",
"fileType": "splat",
"coordinateSystem": "RHS-Z-UP",
"metadata": {
"mode": "indoor",
"hasPoses": true
}
}
```
| Field | Value | Notes |
| ------------------- | ------------------------- | ----------------------------------------------------------------- |
| `provider` | `"xgrid"` | Scanner provider. See note below for upcoming providers. |
| `fileType` | `"splat"` | Tells the server this is a Gaussian Splat upload (not e57 / zip). |
| `coordinateSystem` | `"RHS-Z-UP"` | Xgrids exports use right-handed, Z-up. |
| `metadata.mode` | `"indoor"` \| `"outdoor"` | Pick the one that matches the capture — affects VPS accuracy. |
| `metadata.hasPoses` | `true` | Must be `true`. `poses.json` is required for VPS. |
{% hint style="info" %}
The example on this page is specifically for **Xgrids** Gaussian Splats (exported from Lixel CyberColor). Support for additional Gaussian Splat providers — such as **LitechFeld** — is coming soon. The `provider` value will change per pipeline; the rest of the `source` shape (`fileType`, `metadata.mode`, `metadata.hasPoses`) will stay the same.
{% endhint %}
## Example — create a Gaussian Splat map (Step 1)
```bash
curl --location 'https://api.multiset.ai/v2/vps/map' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ' \
--data '{
"mapName": "Abode Splat No Pose",
"coordinates": {
"latitude": 37.7770709310221,
"longitude": -122.43656613753897,
"altitude": 70.0325927734375
},
"source": {
"provider": "xgrid",
"fileType": "splat",
"coordinateSystem": "RHS-Z-UP",
"metadata": {
"mode": "indoor",
"hasPoses": true
}
}
}'
```
The response contains `mapId`, `uploadId`, and the array of presigned `signedUrls` — exactly like the e57 flow.
## Next steps
After Step 1:
1. Split your `.zip` into chunks and `PUT` each chunk to the corresponding presigned URL. Capture the `ETag` of every part.
2. Call `POST /v2/vps/map/complete-upload/{mapId}` with the `uploadId`, S3 key, and the array of `{ PartNumber, ETag }`.
Both endpoints are identical to the e57 upload — see the [Map Upload overview](/multiset/basics/rest-api-docs/map-upload) for the full OpenAPI specs.
{% hint style="warning" %}
**`hasPoses` must be `true`.** Gaussian Splats without training poses cannot be registered for VPS. Also confirm your splat is **metric-scaled** (for Xgrids this means reconstructing with **Portability On** in Lixel CyberColor).
{% endhint %}
# Object Upload
To create a new Object for tracking using the upload API, make a POST request to **/v1/vps/object** with your authentication token, providing the object name, tracking type (full for 360-degree tracking or side for side-view only), and source details including the file type (glb), provider (web), and coordinate system (RHS for right-handed or LHS for left-handed).
The API will respond with a pre-signed S3 upload URL, unique object ID, and object code.
{% hint style="info" %}
**Objects are now uploaded as a `.zip`.** Package your `.glb` mesh into a zip archive and `PUT` it to the returned `uploadUrl` with **`Content-Type: application/zip`**. Once uploaded, the system automatically unpacks the archive and processes the model for Object tracking. (The `source.fileType` in your request stays `glb`. Only the upload container is a zip.)
{% endhint %}
Once uploaded, the system will automatically process the model for Object tracking.
{% openapi src="/files/kIZwTQ8R86F9qZZ7ESEx" path="/object" method="post" %}
[object-upload.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-3d29a36ca890d4963edbaff9be866728b504851f%2Fobject-upload.yaml?alt=media)
{% endopenapi %}
# Map details
This API endpoint retrieves comprehensive details of a VPS map by providing its unique map code in the URL path along with a bearer token for authentication, returning complete information including the map's status, geographic location, spatial metrics (floor areas, elevations, capture points), storage usage, etc.
{% hint style="info" %}
**`hasPano`** tells you whether the map has a 360° virtual tour. When it is `true`, you can load the tour with the [360° Virtual Tour (Pano)](/multiset/basics/rest-api-docs/pano-360-virtual-tour) APIs. If the map belongs to a Map Version, the response also carries `versionCode` and `activeMapCode`. See [Map Versioning](/multiset/basics/map-versioning) for how to apply them at runtime.
{% endhint %}
{% openapi src="/files/crAGFko3hVqHLa65gdiY" path="/vps/map/{mapCode}" method="get" %}
[get-map.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-d7af24680468fe3c6183fed5eea0ef44ab766f79%2Fget-map.yaml?alt=media)
{% endopenapi %}
### GET All VPS maps for an account
{% hint style="warning" %}
`limit` and `page` are both required on this endpoint. A request without them, such as `GET /vps/map`, returns `500` with `{"error": "Something went wrong!"}`. Use `GET /vps/map?limit=10&page=1` to fetch the first page.
{% endhint %}
{% openapi src="/files/WjX5DJ8IiGWIu7l50anH" path="/vps/map" method="get" %}
[openapi-maplist.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6e904ae42be69edb7ec5074c9a00d1fb822c5637%2Fopenapi-maplist.yaml?alt=media)
{% endopenapi %}
{% openapi src="/files/REo9Pu8Yu4xpabhcwiue" path="/vps/map-set" method="get" %}
[mapset-api-openapi.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fh5JvlrsX78bpolBaI0ec%2Fmapset-api-openapi.yaml?alt=media\&token=25af37b3-58e0-4ef8-b053-5d362183c412)
{% endopenapi %}
# Georeference Map
### Georeference API
Georeference a 3D map by tying its map-local coordinate frame to real-world WGS84 coordinates. You provide a set of **control points**, each pairing a map-local position (`x, y, z` in meters) with a known geographic location (`latitude, longitude, altitude`). The server solves the map's origin and heading from these pairs, so no compass heading is required.
{% hint style="warning" %}
The `local` (3D) coordinates must be in the map's **left-handed (LHS / Unity) coordinate system**, the same frame as the `position` field returned by a successful localization query (`isRightHanded: false`). If your data is in a right-handed system (e.g. ROS, Three.js), convert it to LHS before sending.
{% endhint %}
Once a map is georeferenced, you can use GPS-based localization features such as [GeoHint](/multiset/basics/localization/geohint-in-localization) and receive results in geographic coordinates. See [Georeferencing Maps](/multiset/basics/georeferencing-maps) for the underlying concept.
{% hint style="info" %}
Provide **at least 3 control points**. They must not all lie on the same vertical line (they cannot be horizontally coincident), otherwise the heading cannot be resolved. More well-distributed points improve accuracy. Use at least 6 decimal places for latitude and longitude to reach centimetre-level accuracy.
{% endhint %}
{% openapi src="/files/M6PGwC77ySiVvF1XFlNv" path="/vps/map/{mapCode}/georeference" method="post" %}
[georeference.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-05b120459efe23c61b90c573da66d031d148af05%2Fgeoreference.yaml?alt=media)
{% endopenapi %}
#### Sample Request
```json
{
"controlPoints": [
{
"name": "corner-A",
"geo": { "latitude": 37.774912, "longitude": -122.419425, "altitude": 12.0 },
"local": { "x": 0, "y": 0, "z": 0 }
},
{
"name": "corner-B",
"geo": { "latitude": 37.775012, "longitude": -122.419525, "altitude": 11.8 },
"local": { "x": 10.5, "y": 0, "z": -8.2 }
},
{
"name": "point-3",
"geo": { "latitude": 37.774812, "longitude": -122.419225, "altitude": 12.3 },
"local": { "x": -5.0, "y": 1.2, "z": 12.0 }
}
],
"solveScale": false,
"rejectOutliers": true,
"outlierThresholdMeters": 2.5
}
```
#### Sample Response
```json
{
"message": "Map geo-referenced successfully",
"mapId": "67e12d4bff7ecf561f2f8a0c",
"origin": { "latitude": 37.77491234, "longitude": -122.41942567, "altitude": 12.045 },
"heading": 37.5003,
"scale": 1.0,
"rmseMeters": 1.182,
"horizontalRmseMeters": 0.587,
"maxErrorMeters": 2.9,
"numControlPoints": 4,
"numInliers": 3,
"inliers": ["corner-A", "corner-B", "point-3"],
"rejected": ["outlier-point"],
"tiltCheck": {
"tiltAngleDeg": 0.0006,
"tiltDirectionDeg": 66.9,
"thresholdDeg": 1.5,
"gravityAlignmentLikelyOk": true
},
"residuals": [
{
"name": "corner-A",
"errorMeters": 1.15,
"horizontalMeters": 0.53,
"verticalMeters": 0.94,
"isInlier": true
}
]
}
```
### Reading the Result
The response reports both the solved georeference and quality metrics so you can judge the fit:
* **`origin`** and **`heading`** are the solved values applied to the map. `origin` is the WGS84 location of map-local `(0, 0, 0)`, and `heading` is degrees clockwise from true north.
* **`horizontalRmseMeters`** is the most reliable accuracy indicator. GPS altitude is noisy, so horizontal error is the number to watch.
* **`scale`** is reported only when `solveScale` is `true` and is not applied at query time. A metric scan should return a value close to `1.0`; a value far from `1.0` suggests a scaling problem in the scan.
* **`inliers`** / **`rejected`** and **`residuals`** show which control points were used and how far each landed from the final fit, so you can spot and remove bad GPS readings.
* **`tiltCheck`** verifies that the solved up axis is close to true vertical, confirming the map is gravity aligned.
{% hint style="info" %}
Georeferencing is idempotent: calling the endpoint again overwrites the map's existing georeference with the new solution.
{% endhint %}
# Map Scale
You can now upload **360° videos without any physical scale markers**. The capture is processed into a map, and once the map is **active** you set its real-world scale yourself: measure something in the map whose length you know and enter that length **in metres**. MultiSet applies the corresponding **metric scale factor** so the map's geometry and stored metrics match the real world.
This is useful whenever a marker-less 360 capture needs its true scale set (or corrected) after processing.
{% hint style="info" %}
Map scaling is available only for **standalone Insta360 maps** that are currently **active**. A map that belongs to a MapSet or a version chain cannot be scaled directly.
{% endhint %}
### Where to find it
In the Developer Portal, open the map's **Map Details** page and click **View Splat** in the toolbar:
Map Details toolbar. Open View Splat.
Then click the **Scaling** button (top right of the Splat View) to open the measure-to-scale tool:
1. Click the two ends of something you know the real size of: a door, a window, a tile.
2. Drag the dots to line them up exactly.
3. Enter its real length **in metres** and confirm.
MultiSet computes the factor as **`scaleFactor = known length (in metres) ÷ length measured in the map`**. For example, a door that is 2 m in reality but measures 2.35 in the map's current units gives `2 ÷ 2.35 ≈ 0.85`, scaling the map to 85%. You can also apply scaling programmatically with the API below.
Inside the Splat View, click Scaling (highlighted, top right) to open the measure-to-scale tool. Measure something of a known real length, enter its length in metres, and confirm to scale the map.
### Map Scale API
Make a `POST` request to **`/v1/vps/map/{mapId}/scale`** with your authentication token and a single scale factor. `{mapId}` accepts the map id or map code.
* **`scaleFactor`**: number, **required**, must be greater than `0`. Values below `1` shrink the map, above `1` enlarge it (e.g. `0.85` scales the map to 85% of its current size).
The operation is **asynchronous**: the endpoint responds immediately with `202 Accepted` and sets the map to `pending`. The map returns to `active` once scaling completes, or `failed` if it could not be applied.
#### Sample Request
```bash
curl -X POST "https://api.multiset.ai/v1/vps/map/{mapId}/scale" \
-H "Authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{ "scaleFactor": 0.85 }'
```
#### Sample Response
```json
{
"mapId": "67e12d4bff7ecf561f2f8a0c",
"scaleFactor": 0.85,
"status": "pending"
}
```
### Behaviour
* The map is set to `status: "pending"` while scaling runs, then flips to `active` on success or `failed` if it could not be applied.
* Scaling is multiplicative. Each call scales relative to the map's **current** size, so applying `0.85` twice results in `0.85 × 0.85`.
* Requests are rejected if the map is not found, not owned by your account, not an Insta360 map, not currently active, or part of a MapSet or version chain.
{% openapi src="/files/T4Pf8FEkIHZXF1GEdFYm" path="/vps/map/{mapId}/scale" method="post" %}
[map-scale.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-0431d6de65dca67c72a9d5dffbee977a2b570dac%2Fmap-scale.yaml?alt=media)
{% endopenapi %}
# Camera Intrinsics
Estimate a camera's **intrinsic parameters** (focal length and principal point, with optional lens distortion) from one to five images of the same scene. Use it as a prior when you don't have factory calibration data for a camera.
{% hint style="info" %}
All images in a single request must come from the **same camera at the same resolution**. Sending 1–5 images of the same scene improves the estimate. Images of different resolutions (or from different cameras) are rejected.
{% endhint %}
### Camera Intrinsics API
Make a `POST` request to **`/v1/vps/camera-intrinsics`** with your authentication token. The request is `multipart/form-data`:
* **`images`**: 1 to 5 image files (field name `images`). Each file must be an image and **≤ 15 MB**.
* **`cameraModel`**: optional string, one of `pinhole` (default), `simple_radial`, `radial`, `simple_divisional`. Use a distortion model (`simple_radial` / `radial` / `simple_divisional`) for wide-angle or fisheye lenses.
The endpoint is synchronous and returns the estimated intrinsics in the response.
#### Sample Request
```bash
curl -X POST "https://api.multiset.ai/v1/vps/camera-intrinsics" \
-H "Authorization: Bearer " \
-F "images=@frame_01.jpg" \
-F "images=@frame_02.jpg" \
-F "cameraModel=pinhole"
```
#### Sample Response
```json
{
"requestId": "b1c2d3e4-....",
"cameraModel": "pinhole",
"imageCount": 2,
"intrinsics": { "fx": 1462.3, "fy": 1462.3, "px": 960.0, "py": 540.0 },
"resolution": { "width": 1920, "height": 1080 },
"focalUncertainty": 12.4,
"focalUncertaintyRatio": 0.0085,
"distortion": null
}
```
### Reading the Result
* **`intrinsics.fx` / `intrinsics.fy`**: focal length in pixels. **`px` / `py`**: the principal point (image centre).
* **`resolution`**: the pixel dimensions the intrinsics correspond to. If you resize the image, scale the intrinsics by the same factor.
* **`focalUncertainty`** / **`focalUncertaintyRatio`**: the estimate's uncertainty in pixels and as a fraction of `fx`. Lower is more confident.
* **`distortion`**: present only when a distortion model is requested and coefficients are returned; `null` for `pinhole`.
{% hint style="warning" %}
This is a **prior/estimate**, not a substitute for a full calibration target. Treat the result as a good starting point that a downstream process can refine.
{% endhint %}
{% openapi src="/files/c9LGlr2GJL4BlKVHq2ap" path="/vps/camera-intrinsics" method="post" %}
[camera-intrinsics.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-8467843fe3cd7dd90e313d3028ea732e325fb6bb%2Fcamera-intrinsics.yaml?alt=media)
{% endopenapi %}
# 360° Virtual Tour (Pano)
The Pano APIs serve navigable **360° panoramic virtual tours** built from your maps. A tour is a graph of **nodes**. Each node is a 360° capture point with a position, orientation, a panorama image, and links to its neighbours. Use these read APIs to build a "walk-through" viewer where a user steps from node to node.
All Pano endpoints are read-only and require your authentication token. They are mounted under **`/v1/pano`**.
{% hint style="info" %}
Pano data is generated for **360 (Insta360)** maps that have been processed for panoramic tours. Maps without pano data return `404`. See [360° Virtual Tour](/multiset/basics/360-virtual-tour) for the concepts behind these calls and how to preview a tour in the Developer Portal.
{% endhint %}
### The node object
Most responses return one or more **node** objects with this shape:
```json
{
"nodeId": "n_012",
"keyframe": 12,
"timeS": 4.5,
"position": [1.20, 1.60, -3.40],
"rotation": [0.0, 0.0, 0.0, 1.0],
"rgbKey": "…/Pano/n_012.jpg",
"maskKey": null,
"neighbors": [ { "nodeId": "n_013", "distance": 1.05 } ]
}
```
* **`position`**: node location `[x, y, z]` in the map's coordinate frame.
* **`rotation`**: orientation quaternion `[qx, qy, qz, qw]`.
* **`rgbKey`**: storage key for the node's 360° panorama image. **`maskKey`**: optional mask image key (`null` if none).
* **`neighbors`**: the nodes you can step to from here, with the distance in metres.
### Endpoints
#### List pano-enabled maps
**`GET /v1/pano`**: a paginated library of maps that have a virtual tour.
Query params: `page` (default `1`), `limit` (1–100, default `20`).
```json
{
"totalCount": 8,
"page": 1,
"limit": 20,
"panos": [
{
"mapCode": "MAP-ABC123",
"mapId": "67e1…",
"mapName": "Ground Floor",
"thumbnailKey": "…/Pano/thumb.jpg",
"metric": true,
"panoSize": 512,
"minSpacingM": 1.0,
"nodeCount": 240,
"updatedAt": "2026-07-20T10:15:00.000Z"
}
]
}
```
{% openapi src="/files/T8MNstj6ZwhVHDx9RLt6" path="/pano" method="get" %}
[pano-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6abf43287a4d76f02891cffd6e0f501cc53fde1c%2Fpano-api.yaml?alt=media)
{% endopenapi %}
#### Tour summary
**`GET /v1/pano/{id}`**: header/summary for one map's tour (`{id}` = map id or map code). Includes `nodeCount`, `hasMasks`, and navigation hints (`version`, `mapSet`) when the map belongs to a version chain or MapSet.
{% openapi src="/files/T8MNstj6ZwhVHDx9RLt6" path="/pano/{id}" method="get" %}
[pano-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6abf43287a4d76f02891cffd6e0f501cc53fde1c%2Fpano-api.yaml?alt=media)
{% endopenapi %}
#### Viewer-ready manifest
**`GET /v1/pano/{id}/manifest`**: the tour header plus **every node**, sorted by `nodeId`. Load this once to render a complete tour.
{% openapi src="/files/T8MNstj6ZwhVHDx9RLt6" path="/pano/{id}/manifest" method="get" %}
[pano-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6abf43287a4d76f02891cffd6e0f501cc53fde1c%2Fpano-api.yaml?alt=media)
{% endopenapi %}
#### Paginated nodes
**`GET /v1/pano/{id}/nodes`**: nodes in pages, for large tours. Query params: `page` (default `1`), `limit` (1–200, default `50`). Returns `{ totalCount, page, limit, nodes }`.
{% openapi src="/files/T8MNstj6ZwhVHDx9RLt6" path="/pano/{id}/nodes" method="get" %}
[pano-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6abf43287a4d76f02891cffd6e0f501cc53fde1c%2Fpano-api.yaml?alt=media)
{% endopenapi %}
#### Navigation window
**`GET /v1/pano/{id}/window`**: the local neighbourhood around a starting point, for stepping through a tour without loading it all. All query params are optional:
* **`nodeId`**: centre on a specific node.
* **`x`, `y`, `z`**: centre on the node nearest a world position (provide all three).
* **`qx`, `qy`, `qz`, `qw`**: an optional orientation quaternion (provide all four, together with a position) to prefer nodes the user is facing.
* **`depth`**: how many graph hops to include from the centre (1–4, default `2`).
You cannot pass both `nodeId` and a position. Precedence is `nodeId` → position → the tour's entry node.
```json
{
"mapCode": "MAP-ABC123",
"mapId": "67e1…",
"center": "n_012",
"depth": 2,
"nodeCount": 9,
"nodes": [ /* node objects within `depth` hops of the centre */ ]
}
```
{% openapi src="/files/T8MNstj6ZwhVHDx9RLt6" path="/pano/{id}/window" method="get" %}
[pano-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6abf43287a4d76f02891cffd6e0f501cc53fde1c%2Fpano-api.yaml?alt=media)
{% endopenapi %}
### Tours across versions and MapSets
#### Same view across map versions
**`GET /v1/pano/map-version/{versionCode}/window`**: returns the "same viewpoint" seen across the maps in a version chain, so a viewer can switch between versions (e.g. *before* / *after* an update) at the same spot. Optional query params: `mapCodes` (comma-separated subset of the version), `anchorMapCode` (which version drives navigation), `nodeId`, `depth` (1–4, default `2`), `maxDistanceM` (how close a node must be to count as the "same view"). Positions are returned in the base map's frame.
{% openapi src="/files/T8MNstj6ZwhVHDx9RLt6" path="/pano/map-version/{versionCode}/window" method="get" %}
[pano-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6abf43287a4d76f02891cffd6e0f501cc53fde1c%2Fpano-api.yaml?alt=media)
{% endopenapi %}
#### One continuous tour across a MapSet
**`GET /v1/pano/map-set/{id}/window`**: treats the pano-enabled maps in a MapSet as a **single continuous tour**, bridging between maps where they meet. Optional query params: `mapCodes`, `mapCode` (the current node's map), `nodeId`, `depth` (1–4, default `2`), `bridgeRadiusM` (how close nodes in different maps must be to link). Each returned node is identified by `{ mapCode, nodeId }`, and neighbours that cross into another map are marked `"cross": true`.
{% openapi src="/files/T8MNstj6ZwhVHDx9RLt6" path="/pano/map-set/{id}/window" method="get" %}
[pano-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-6abf43287a4d76f02891cffd6e0f501cc53fde1c%2Fpano-api.yaml?alt=media)
{% endopenapi %}
{% hint style="info" %}
For large tours, prefer the **window** endpoints and load neighbours as the user moves, rather than fetching the full manifest up front.
{% endhint %}
# Object Tracking Query
### Object Tracking Query API
Send a single query frame to detect and localize one of your tracked objects. The request returns a 6-DoF pose for the object that is visible in the frame.
The request takes an `objectCode` (a single string or an **array of up to 10 codes**), which is what enables multi-object tracking in a single session.
#### Tracking multiple objects in one query
`objectCode` can be an array — pass **1 to 10** object codes in a single query when your application is set up to track several different objects but only one is expected to be in the camera frame at any moment.
This is useful when:
* Your scene contains a small set of known trackable objects (e.g. five different products on a shelf, or three machines in a maintenance bay) and you want MultiSet to figure out which one the user is currently looking at.
* You don't want to run separate per-object queries every frame and pick the best result client-side — let the cloud do the matching across the candidate objects in one round trip.
**How it works:** if you pass, say, 5 object codes, the cloud searches the query frame against the tracking maps for **all 5** candidates and detects whichever object is visible in the frame. If multiple candidates are in view, the system returns the best match.
{% hint style="warning" %}
**Only one tracking pose is returned per query.** Even when you send multiple object codes, the response contains the pose for the **single object** the system detected in that frame. The detected object is identified in the `objectCodes` field of the response (an array containing the matched code).
To track two or more objects **simultaneously** in the same frame, send separate queries — multi-object pose return in a single response is not currently supported.
{% endhint %}
{% hint style="info" %}
All object codes in a single query must belong to your account and be in `active` status. Unknown, inactive, or cross-account codes are rejected. Maximum 10 candidates per query.
{% endhint %}
#### Request body fields
| Field | Type | Required | Notes |
| ----------------- | -------------------------- | -------- | -------------------------------------------------------------------- |
| `queryImage` | file (multipart) | yes | Single RGB frame from the device camera. |
| `objectCode` | string \| string\[] (1–10) | yes | Object code(s) to match against. |
| `fx`, `fy` | number | yes | Camera focal length in pixels. |
| `px`, `py` | number | yes | Camera principal point in pixels. |
| `width`, `height` | number | yes | Query image resolution in pixels. |
| `isRightHanded` | boolean | no | Default `false` (LHS / Unity). Set `true` for RHS coordinate output. |
{% openapi src="/files/zGORL0gZAQeTc3S056sv" path="/vps/object/query" method="post" %}
[object-query.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bf7be55c553e078129ae6899e135d33e8bfd2134%2Fobject-query.yaml?alt=media)
{% endopenapi %}
#### Sample Response — pose found
When the cloud detects one of the candidate objects in the query frame:
```json
{
"poseFound": true,
"position": {
"x": 1.4085917235274734,
"y": 0.4753337271392171,
"z": 0.6372062117019925
},
"rotation": {
"x": -0.10826634872129595,
"y": 0.48317258532326446,
"z": 0.07714477353215363,
"w": 0.8653735230773266
},
"confidence": 0.46301103179753406,
"objectCodes": [
"OBJ_CL4JCEKIHWAJ"
]
}
```
* `position` / `rotation` — the detected object's 6-DoF pose in the camera coordinate frame (LHS / Unity by default; RHS when `isRightHanded=true`).
* `objectCodes` — array containing the **single matched object code**, even if multiple candidates were sent in the request. Use this field to tell which object was detected.
* `confidence` — match confidence in `[0, 1]`.
#### Sample Response — pose not found
When none of the candidate objects could be detected in the frame:
```json
{
"poseFound": false,
"errorCode": "001",
"errorMessage": "Pose not found"
}
```
This is the expected response when the user hasn't pointed the camera at any of the tracked objects yet, or when the view is too occluded / motion-blurred. Continue querying with new frames until `poseFound` is `true`.
### List Objects API
Returns the object codes available in your account — useful for building the candidate list you'll send to `/vps/object/query`.
{% openapi src="/files/zGORL0gZAQeTc3S056sv" path="/vps/object" method="get" %}
[object-query.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bf7be55c553e078129ae6899e135d33e8bfd2134%2Fobject-query.yaml?alt=media)
{% endopenapi %}
# Map Version
Create, inspect, and manage Map Versions over the REST API.
These endpoints let you create a Map Version, add additional scans to it, control which scans are active for querying, and tear the version down. See [Map Versioning](/multiset/basics/map-versioning) for the concepts behind these calls.
{% hint style="warning" %}
To whitelist your domain, follow: [Configuring Allowed Domains (CORS)](/multiset/basics/credentials/configuring-allowed-domains-cors)
{% endhint %}
{% hint style="info" %}
**Codes used in these APIs**
* **Version Code** (e.g. `MVER_W6NTBJ0ALVUN`), identifies a Map Version. Returned when you create one.
* **Map Code** (e.g. `MAP_BTTE1MOXYVT8`), identifies an individual map. Used as the base, target, or active map.
{% endhint %}
## Endpoints
### Create Map Version
Creates a new Map Version. The `sourceMapCode` becomes the **base map**, its coordinate frame becomes the version's frame, and your content layer should be authored against it. The `targetMapCode` is added as the first additional version.
The response is `202 Accepted`, the VPS computes the rigid transform asynchronously. Poll **Get Map Version** until the target map's status flips from `computing` to `ready`.
**Constraints:**
* Both maps must be in your account.
* Neither map can already belong to a Map Version.
* `sourceMapCode` and `targetMapCode` must differ.
{% openapi src="/files/raC9U46pEdVX4vbsnwnn" path="/map-version" method="post" %}
[map-version-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bb68f496e1e1d93f12f36f4254c75cab31a1cf5a%2Fmap-version-api.yaml?alt=media)
{% endopenapi %}
### Get Map Version
Returns the full state of a Map Version: every scan in it, each scan's per-map `relativePose`, the current active map, and processing status.
{% openapi src="/files/raC9U46pEdVX4vbsnwnn" path="/map-version/{versionCode}" method="get" %}
[map-version-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bb68f496e1e1d93f12f36f4254c75cab31a1cf5a%2Fmap-version-api.yaml?alt=media)
{% endopenapi %}
### Add Map to Map Version
Adds a new scan to an existing Map Version. The transform is computed against the **currently active map** in the version (not the original base). This is what lets you keep adding fresh scans over time without alignment quality decaying back to the original capture.
**Constraints:**
* The target map must be in your account and must not already belong to a Map Version.
{% openapi src="/files/raC9U46pEdVX4vbsnwnn" path="/map-version/{versionCode}/maps" method="post" %}
[map-version-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bb68f496e1e1d93f12f36f4254c75cab31a1cf5a%2Fmap-version-api.yaml?alt=media)
{% endopenapi %}
### Activate Maps in Version
Sets which map(s) in the version are **active** (the "query-active" set). After this call, VPS queries against the base map's `mapCode` are localized against **every active map** in the version, and each returned pose is transformed back into the base map's coordinate frame, so your content layer keeps working.
You can activate a **single** map or **multiple** maps at once by providing **exactly one** of:
* `mapCode`: a single Map Code, or
* `mapCodes`: an array of Map Codes (the full active set).
{% hint style="info" %}
This call **replaces** the active set: any map in the version that isn't listed is deactivated. This is the same control as the **Query Active** toggle in the Developer Portal. See [Map Versioning](/multiset/basics/map-versioning#create-a-map-version-in-the-developer-portal).
{% endhint %}
**Constraints:**
* Every listed map must already be part of this version.
* Every listed map must be in `ready` status. You cannot activate a map whose alignment is still `computing` or `failed`.
* At least one map must remain active.
#### Sample Request (activate multiple maps)
```json
{
"mapCodes": ["MAP_BTTE1MOXYVT8", "MAP_QQYKIBHXZE01"]
}
```
To activate just one map, send `{ "mapCode": "MAP_QQYKIBHXZE01" }` instead.
{% openapi src="/files/raC9U46pEdVX4vbsnwnn" path="/map-version/{versionCode}/activate" method="put" %}
[map-version-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bb68f496e1e1d93f12f36f4254c75cab31a1cf5a%2Fmap-version-api.yaml?alt=media)
{% endopenapi %}
***
### Remove Map from Version
Removes a single scan from a Map Version. The map itself is not deleted, only its association with the version.
**Constraints:**
* You cannot remove the base map of the version.
* You cannot remove the currently active map. Activate a different map first, then remove this one.
{% openapi src="/files/raC9U46pEdVX4vbsnwnn" path="/map-version/{versionCode}/maps/{targetMapCode}" method="delete" %}
[map-version-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bb68f496e1e1d93f12f36f4254c75cab31a1cf5a%2Fmap-version-api.yaml?alt=media)
{% endopenapi %}
***
### Delete Map Version
Deletes the Map Version and clears the version association from every map in it. The maps themselves are kept.
{% openapi src="/files/raC9U46pEdVX4vbsnwnn" path="/map-version/{versionCode}" method="delete" %}
[map-version-api.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-bb68f496e1e1d93f12f36f4254c75cab31a1cf5a%2Fmap-version-api.yaml?alt=media)
{% endopenapi %}
## Status values
| Status | Meaning |
| ----------- | ----------------------------------------------------------------------------------------------- |
| `computing` | VPS is computing the transform between this scan and the version's source map. |
| `ready` | Transform is computed. The map can be set as active. |
| `failed` | The VPS could not align this scan. Inspect `failureReason` on the **Get Map Version** response. |
# Simulation Data
Manage simulation data used to test localization.
Simulation data lets you test the localization pipeline without being physically present at the mapped site. Each simulation data record is a zip file containing pre-recorded camera frames and metadata that can be replayed against the localization APIs.
A typical workflow is:
1. **Record** simulation data at the site.
2. **Upload** the resulting zip via `POST /v1/simulation-data`, the response gives you a unique `simulationCode`.
3. **Reference** the `simulationCode` when replaying the captured frames against your maps.
4. **Update, download, or delete** the simulation as your test set evolves.
{% hint style="info" %}
Max upload size is **10 MB** and the file must be a `.zip`. The uploaded part's `Content-Type` must be `application/zip` or `application/x-zip-compressed`, any other value (including the `application/octet-stream` default that some HTTP clients send) is rejected with `Only zip files are allowed`. The returned `simulationCode` is an 8-character alphanumeric identifier, use it in all subsequent calls.
{% endhint %}
{% hint style="warning" %}
**Setting the part's Content-Type from curl:** append `;type=application/zip` to the `-F` value, for example `-F 'file=@capture.zip;type=application/zip'`. Browsers and SDKs normally set this header from the file's extension or detected type, so no extra work is needed there.
{% endhint %}
### Zip File Format
The uploaded `.zip` represents a single capture session, a small set of AR camera frames plus the device pose and camera intrinsics at the moment each frame was captured. Replaying this archive against a map reproduces a real-world localization session offline.
#### What a capture contains
Each capture session contains **exactly one** dataset:
1. **A fixed number of camera frames** (4–6, default 5) stored as JPEGs.
2. **The 6-DoF camera pose** at the instant each frame was acquired, `(x, y, z)` position and `(qx, qy, qz, qw)` quaternion rotation, in a **left-handed (LHS) world coordinate system** (Unity convention). See [Coordinate system](#coordinate-system) below if your capture pipeline is right-handed.
3. **The camera intrinsics** for the session, focal length `(fx, fy)`, principal point `(px, py)`, and the image dimensions `(width, height)`. Captured once and assumed constant for the rest of the session.
#### Coordinate system
All poses in the manifest must be expressed in a **left-handed coordinate system (LHS)**, the same convention the MultiSet Unity SDK and the localization replay pipeline use. Uploading right-handed (RHS) poses without converting them first will produce poses that look mirrored along the X axis when replayed, and localization will not match the original capture.
If your capture pipeline produces RHS poses (for example, native iOS / Android, ARKit / ARCore raw transforms, or any OpenGL-style stack), convert each pose to LHS before writing it into the manifest. The full conversion is: **negate `position.x`, negate `rotation.qy` and `rotation.qz`, keep the rest unchanged**.
```javascript
/**
* Convert a 6-DoF pose from a right-handed coordinate system (RHS) to the
* left-handed coordinate system (LHS) expected by the simulation data
* manifest. Apply this to every per-frame pose before writing the JSON.
*/
function flipPoseHandedness(pose) {
return {
position: {
x: -pose.position.x,
y: pose.position.y,
z: pose.position.z,
},
rotation: {
qx: pose.rotation.qx,
qy: -pose.rotation.qy,
qz: -pose.rotation.qz,
qw: pose.rotation.qw,
},
};
}
```
{% hint style="info" %}
Captures produced by the MultiSet Unity SDK are already LHS, no conversion is needed.
{% endhint %}
#### Zip layout
Entries inside the zip are **flat**, there is no top-level folder:
```
SimulationData.zip
├── Image_0_.jpg
├── Image_1_.jpg
├── Image_2_.jpg
├── Image_3_.jpg
├── Image_4_.jpg
└── SimulationData_.json
```
* Image filenames follow `Image__.jpg`, where `` is the zero-based capture order. Images must appear in the **same order** as the entries in the JSON manifest's `imageDataList` so they can be paired positionally.
* The timestamp suffix `` is identical for every file in the session.
* JPEGs should be encoded at quality \~80.
#### `SimulationData_.json` manifest
The single JSON manifest at the root of the zip describes the intrinsics and the per-frame pose, in capture order:
| Field | Type | Notes |
| ----------------------------------------- | ----- | ------------------------------------------------------------------- |
| `width`, `height` | int | Working image resolution in pixels (must match the JPEGs). |
| `fx`, `fy` | float | Camera focal length in pixels, scaled to match `width`/`height`. |
| `px`, `py` | float | Camera principal point in pixels, scaled to match `width`/`height`. |
| `imageDataList` | array | One entry per image. Same order as `Image_0`, `Image_1`, … |
| `imageDataList[].x`, `.y`, `.z` | float | Camera position at frame acquisition. |
| `imageDataList[].qx`, `.qy`, `.qz`, `.qw` | float | Camera rotation as a quaternion. |
Example:
```json
{
"width": 720,
"height": 960,
"px": 361.3074645996094,
"py": 481.46038818359377,
"fx": 665.6408081054688,
"fy": 665.6408081054688,
"imageDataList": [
{
"x": 0.31831833720207217,
"y": 1.156510353088379,
"z": 1.3285411596298218,
"qx": -0.009507684037089348,
"qy": 0.30342480540275576,
"qz": -0.00325992563739419,
"qw": -0.9528024196624756
},
{
"x": 0.3255421817302704,
"y": 1.154714822769165,
"z": 1.3292453289031983,
"qx": -0.0043470230884850029,
"qy": 0.27293628454208376,
"qz": -0.006081813480705023,
"qw": -0.9620030522346497
},
{
"x": 0.3384963274002075,
"y": 1.156157374382019,
"z": 1.3352329730987549,
"qx": -0.0042652105912566189,
"qy": 0.18202008306980134,
"qz": -0.007582413498312235,
"qw": -0.9832563400268555
},
{
"x": 0.37630680203437807,
"y": 1.1523985862731934,
"z": 1.3415110111236573,
"qx": -0.0032018644269555809,
"qy": 0.0914652869105339,
"qz": -0.0009664428071118891,
"qw": -0.995802640914917
},
{
"x": 0.42654383182525637,
"y": 1.1490235328674317,
"z": 1.3500515222549439,
"qx": -0.0047492762096226219,
"qy": 0.0032952935434877874,
"qz": -0.0013450992992147804,
"qw": -0.9999824166297913
}
]
}
```
{% hint style="warning" %}
`imageDataList.length` must equal the number of `Image__*.jpg` files in the zip, and each index `n` must have a corresponding entry at position `n` in the list. Mismatched counts or out-of-order indices cause the simulation to be rejected at replay time.
{% endhint %}
### Upload Simulation Data
Upload a new simulation data zip. Returns a `simulationCode` you use to reference it later.
{% openapi src="/files/Z73rNpLR5ivcc3uKTeTT" path="/simulation-data" method="post" %}
[simulation-data.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ea098d99dde3d64ccce074808a7f3684565631b2%2Fsimulation-data.yaml?alt=media)
{% endopenapi %}
### List Simulation Data
Paginated list of simulation data for the authenticated account. Supports search by name and filtering by status.
{% openapi src="/files/Z73rNpLR5ivcc3uKTeTT" path="/simulation-data" method="get" %}
[simulation-data.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ea098d99dde3d64ccce074808a7f3684565631b2%2Fsimulation-data.yaml?alt=media)
{% endopenapi %}
### Get Simulation Data by Code
Retrieve details (name, description, status, file size, timestamps) for a single simulation data record.
{% openapi src="/files/Z73rNpLR5ivcc3uKTeTT" path="/simulation-data/{simulationCode}" method="get" %}
[simulation-data.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ea098d99dde3d64ccce074808a7f3684565631b2%2Fsimulation-data.yaml?alt=media)
{% endopenapi %}
### Update Simulation Data
Update the `name` and/or `description` of an existing simulation data record. At least one of the two fields must be supplied.
{% hint style="warning" %}
This endpoint updates **metadata only**. To replace the underlying zip file, delete the simulation and upload a new one.
{% endhint %}
{% openapi src="/files/Z73rNpLR5ivcc3uKTeTT" path="/simulation-data/{simulationCode}" method="put" %}
[simulation-data.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ea098d99dde3d64ccce074808a7f3684565631b2%2Fsimulation-data.yaml?alt=media)
{% endopenapi %}
### Delete Simulation Data
Removes the simulation data record and deletes the underlying zip file from storage. This action is irreversible.
{% openapi src="/files/Z73rNpLR5ivcc3uKTeTT" path="/simulation-data/{simulationCode}" method="delete" %}
[simulation-data.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ea098d99dde3d64ccce074808a7f3684565631b2%2Fsimulation-data.yaml?alt=media)
{% endopenapi %}
### Download Simulation Data
Generates a short-lived pre-signed URL for downloading the zip. The response includes the URL, the original filename, and the file size.
{% openapi src="/files/Z73rNpLR5ivcc3uKTeTT" path="/simulation-data/{simulationCode}/download" method="get" %}
[simulation-data.yaml](https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2Fgit-blob-ea098d99dde3d64ccce074808a7f3684565631b2%2Fsimulation-data.yaml?alt=media)
{% endopenapi %}
# Third Party Scans
The Multiset developer portal supports third-party capture platforms, including **Matterport**, **NavVis**, **Leica**, **Faro**, **Xgrids** and **Insta360**, to streamline the creation of high-accuracy **Visual Positioning Systems (VPS)**. Developers can use these industry-leading tools to generate the data required for precise localization and AR experiences.
## Supported Input Formats
Multiset accepts three kinds of input for building a map:
| Input type | What you upload | Notes |
| ------------------ | ------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| **Point cloud** | Structured **`.e57`** with embedded panoramas, compressed to `.zip` | The most common path. Panoramic images must be included in the export. |
| **Gaussian Splat** | **`.ply`** and **`poses.json`** in a single `.zip` | Must be metric-scaled. See [Gaussian Splat](/multiset/basics/third-party-scans/gaussian-splat). |
| **360 video** | Raw **`.insv`**, one file per `.zip` | Hand-held 360 capture, no LiDAR hardware required. See [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans). |
## Supported Vendors and File Types
| Vendor | Supported devices | File type | Upload package | Guide |
| --------------------------- | ---------------------------------- | ------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- |
| **Matterport** | Pro2, Pro3 | Structured `.e57` with panoramas | `.e57` inside a `.zip` | [Matterport E57](/multiset/basics/third-party-scans/matterport-e57) |
| **Matterport (MatterPak)** | Pro2, Pro3 | MatterPak bundle | Fetched from your Matterport account, no manual upload | [MatterPak Files](/multiset/basics/third-party-scans/matterport-e57/matterpak-files) |
| **Leica** | RTC360, BLK360 G2, BLK2GO, BLK2FLY | Structured `.e57` with panoramas | `.e57` inside a `.zip` | [Leica Scans](/multiset/basics/third-party-scans/leica-scans) |
| **NavVis** | VLX, M6 | `.e57` with equirectangular panoramas | `.e57` inside a `.zip` | [NavVis Scans](/multiset/basics/third-party-scans/navvis-scans) |
| **Faro** | Focus series, Orbis, Flash/Hybrid | `.e57` with Color Panorama Image enabled | `.e57` inside a `.zip` | [Faro Scans](/multiset/basics/third-party-scans/faro-scans) |
| **Xgrids** | L2 Pro, K1 | `.e57` converted from LAS using `poses.csv` | `.e57` inside a `.zip` | [Xgrids Scans](/multiset/basics/third-party-scans/xgrids-scans) |
| **Xgrids (Gaussian Splat)** | PortalCam, L2 Pro, K1, K2 | `.ply` and `poses.json` | Both files in a single `.zip` | [Xgrids Gaussian Splat Export](/multiset/basics/third-party-scans/gaussian-splat/xgrids-gaussian-splat) |
| **Insta360** | X4, X5 | Raw `.insv` 360 video (dual-fisheye) | Exactly one `.insv` per `.zip` | [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans) |
{% hint style="info" %}
**Using a scanner that is not listed?** Structured `.e57` files from other scanners will also work, as long as the data is laid out the same way as the exports above, meaning the point cloud and its panoramic images with pose information are embedded in the file. Simply pick the closest matching provider in the upload dropdown and the Multiset backend will detect the structure and process it. The provider selection does not have to match your hardware exactly, since the panorama format is detected automatically.
{% endhint %}
{% hint style="success" %}
**Panoramas can be cubic or equirectangular, based on the vendor. The system detects the pano format.**
{% endhint %}
## Input Requirements
### Point cloud (`.e57`)
* **Panoramas**: Panoramic images must be included in the export. These are essential for capturing comprehensive environmental data.
* **Scan Density**: The density of both the scans and the accompanying images should be **high**. Increased density ensures better visual mapping and localization accuracy.
* **Panorama Spacing**: Keep the distance between panoramas (scan positions or capture trigger interval) between **1 and 3 meters**, depending on the device: 1 to 2 m for Faro and Xgrids (1 m indoors, 2 m outdoors), 1.5 to 2.5 m for Matterport, 2 to 3 m for Leica tripod scanners, and up to 3 m for NavVis in open spaces. See each scanner page for details.
### Gaussian Splat
* **Metric scale is required.** Arbitrary-scale reconstructions cannot be used for VPS.
* **`poses.json` is required**, the splat cannot be localized against without the camera poses used to train it.
* Both files must sit at the root of the `.zip`, with no nested folders.
### 360 video
* Record in the camera's **360 video mode**: 5.7K at 60 fps for indoor and low light, 8K at 30 fps for outdoor and natural light. Flat or single-lens footage is not supported.
* Include a **printed ChArUco calibration marker** in the scene. This is what gives the reconstruction its real-world metric scale.
* Walk at a steady **1.0 to 1.5 m/s** with the camera at head height, and close the loop by returning to your starting point.
* Upload the **raw `.insv`**, no stitching required. Multiset stitches to equirectangular server-side.
* Keep each `.zip` **under 50 GB**, with exactly one `.insv` inside. One `.insv` produces one map.
{% hint style="info" %}
**360 video support (new):** Multiset accepts hand-held **360 video** as input for VPS, so you can build a map without LiDAR or SLAM hardware. The first supported pipeline is **Insta360** (**X4** and **X5**), uploaded as a raw `.insv` file. See [Insta360 Scans](/multiset/basics/third-party-scans/insta360-scans) for camera settings, the calibration marker, and walking patterns.
{% endhint %}
{% hint style="info" %}
**Gaussian Splat support:** Multiset accepts **Gaussian Splats** as input for VPS. The first supported pipeline is **Xgrids**, where all Xgrids scanners (**PortalCam**, L2 Pro, **K1/K2**) can export Gaussian Splats in `.ply` format via Lixel CyberColor. See [Gaussian Splat](/multiset/basics/third-party-scans/gaussian-splat) for details.
{% endhint %}
#### Benefits of High-Density Scans:
* Enhanced environmental detail for improved VPS accuracy.
* Better alignment and anchoring of AR experiences in real-world environments.
* Seamless integration with Multiset’s tools for large-scale AR deployments.
By leveraging the capabilities of Matterport, NavVis, Leica, Faro, Xgrids and Insta360, developers can create precise and persistent AR experiences.
# Matterport E57
Support for Matterport scans
The Multiset Developer Portal supports **Matterport™ scans** for generating high-accuracy **Visual Positioning System (VPS)**. Developers can leverage Matterport™ Pro3 and Pro2 cameras to produce detailed 3D models of physical spaces. These models, exported as **.e57 files**, serve as the foundation for creating immersive and precise AR experiences.
This guide provides an overview of the workflow, equipment, and best practices for using Matterport™ scanners to produce VPS.
***
#### **Supported Devices**
* **Matterport™ Pro3 Camera**
* **Matterport™ Pro2 Camera**
***
#### **Scanning Workflow**
Follow these steps to ensure successful scanning and processing of Matterport™ data for VPS:
**1. Prerequisites**
* **Matterport™ Account**:
* Create an account at [MyMatterport.com](https://my.matterport.com) and upgrade to a Professional subscription plan or higher.
* **Equipment**:
* Ensure you have access to a Matterport™ Pro2 or Pro3 camera and a compatible mobile device with the **Matterport™ Capture App** installed.
* **Output Format**:
* Scans must be exported as **.e57 files**. Panoramic images should be included to enhance the accuracy of VPS processing.
***
**2. Required Equipment**
1. **Camera Accessories**:
* A professional-grade tripod with 3/8”-16 UNC mount to ensure stability during scanning.
* Quick-release clamps for efficient setup and movement of the camera.
2. **Mobile Device**:
* An iPad, iPhone, or Android device to control the Matterport™ Capture App. Ensure the device is fully charged before use.
3. **Additional Essentials**:
* Sturdy carrying cases for secure transport.
***
**3. Preparing for the Scan**
* **Camera Setup**:
* Mount the camera securely on a tripod and connect it to the Matterport™ Capture App.
* Verify the camera’s connection and functionality within the app.
* **Environment Preparation**:
* Minimize direct sunlight and reflective surfaces to reduce alignment errors.
* Provide adequate ambient lighting for indoor scans.
***
**4. Capturing the Environment**
* Launch the Matterport™ Capture App and create a new project.
* Position the camera to capture comprehensive coverage of the space.
* Keep scan positions **1.5–2.5 meters (5–8 feet)** apart so consecutive panoramas overlap well. Close spacing is critical for visual positioning accuracy.
* Use the Capture button to perform 360-degree scans. Avoid standing in the camera’s field of view.
* Leverage tools in the Capture App to mark reflective surfaces or windows for better alignment.
***
**5. Uploading and Processing**
* After completing the scan, upload the data to your Matterport™ cloud account via the Capture App.
* Processing times vary depending on the size and complexity of the scan. Ensure a stable network connection during upload.
***
**6. Exporting .e57 Files**
* Once the scan is processed, export the model as a **.e57 file** from your Matterport™ cloud account.
* Ensure that high-resolution panoramic images are included in the export for enhanced VPS performance.
***
**7. Uploading to Multiset Portal**
* Log in to the Multiset Developer Portal and upload the **.e57 file**.
* To upload, click on **Create Map -> Upload -> Choose Matterport** and .e57 file
* The portal processes the file to generate a VPS-compatible map.
After processing, the map status will be active, and if anything goes wrong during the process, the status will be "failed"
{% hint style="warning" %}
Note : Processing Matterport E57 files can take upto an hour to be active
{% endhint %}
Below is the video tutorial of the process
{% embed url="\_" %}
#### **Best Practices**
* **High Scan Density**: Perform scans at closer intervals, roughly every **1.5–2.5 meters (5–8 feet)**, to capture detailed and accurate environmental data with overlapping panoramic images.
* **Include Panoramas**: Ensure overlapping panoramic images are embedded in the export for visual localization.
* **Post-Processing**: Use the Matterport™ Capture App to refine the scans and remove unnecessary artifacts before exporting.
***
# MatterPak Files
The MultiSet Visual Positioning System now supports on-cloud processing of Matterport MatterPak files. This allows for a more streamlined and efficient workflow for converting your Matterport scans into usable VPS data.
#### Benefits of On-Cloud Processing
* **No High-End Hardware Required:** Instead of relying on the processing power of your local machine, on-cloud processing removes the dependency on local RAM and processing capabilities. This means you can process scans of any size without needing a powerful desktop computer.
* **Parallel Processing:** On-cloud processing enables you to run multiple scans simultaneously. This parallel processing capability significantly reduces the time required to process large batches of scans.
### Connect with your Matterport account using the Developer token
Once you login to your Matterport, select the right organisation from Top-right where scan is added.
Then in your Matterport account, navigate to settings -> Developer tools -> Apply for Production "**Private Use.**" You will need to fill out a Matterport form to get production access to **Secret ID and Secret Token.**
Once you have granted Production Use, you need to create a new **API Token** to get a Secret **ID and Secret Token**
#### The On-Cloud Processing Workflow
Here is a step-by-step guide to processing your Matterport MatterPak files on the cloud with MultiSet:
**Step 1: Select Matterport as the Provider**
Navigate to your MultiSet Dashboard and go to **Maps -> Create Map -> Upload existing scan**. From the provider options, select **Matterport**. In the "File Format" dropdown, choose **Matterpak (fetched from Matterport account)**.
**Step 2: Authenticate Your Matterport Account**
To connect your MultiSet and Matterport accounts, you will need to provide your Matterport API credentials. Enter your **Token ID** and **Token Secret**. You can generate these credentials in your Matterport account settings under "Developer Tools."
**Step 3: Choose the Scan to Process**
Once your accounts are connected, you will see a list of your available Matterport spaces. Select the scan that you want to process. Please ensure that you have already purchased the MatterPak bundle for the selected scan in your Matterport account.
**Step 4: Add Metadata and Begin Processing**
Provide a **Map Name** for your project and set the location metadata. After you have filled in the necessary information, click on **Next** to begin the on-cloud processing.
# Leica Scans
MultiSet AI’s Visual Positioning System (VPS) accepts **structured `.e57` point-cloud files exported from Leica’s Cyclone REGISTER 360 PLUS**. By pairing Leica’s high-fidelity LiDAR capture with MultiSet’s VPS you can turn large venues into centimeter-accurate AR canvases in hours instead of days. The guide below walks you through recommended devices, capture workflow, export settings and the upload process to the MultiSet Developer Portal.
**Supported Devices**
* **Leica RTC360**
* **Leica BLK360 G2**
* **Leica BLK2GO / BLK2FLY**
> MultiSet parses the panoramic images and intrinsic calibration blocks embedded in a **structured E57**. Unstructured “raw” exports will be rejected during processing.
### Scanning Workflow
#### 1 Prerequisites
| Item | Notes |
| -------------------- | ------------------------------------------------------------------------------------------- |
| **Scanner & power** | Fully charge two batteries; firmware ≥ 2023.1 on RTC360/BLK360. |
| **Mobile app** | Leica **Cyclone FIELD 360** on iOS/Android for live QA and alignment. |
| **Desktop software** | Leica **Cyclone REGISTER 360 PLUS** 2023.1 or later (adds one-click structured E57 export). |
#### 2 Required Equipment
* **Sturdy tripod** (3⁄8-16 UNC), quick-release clamp.
* **Target spheres / checkerboards** if survey-grade alignment is needed.
* **Tablet/phone** with FIELD 360 (Wi-Fi 6 preferred for RTC360’s onboard hotspot).[Google Play](https://play.google.com/store/apps/details?hl=en_US\&id=com.leicageosystems.cyclone.field360\&utm_source=chatgpt.com)
#### 3 Prepare the Environment
1. **Declutter walkways** and remove moving objects if possible.
2. **Set low-resolution LiDAR** when AR is the only deliverable; high-res mode multiplies file size with no VPS benefit.
3. **Turn off HDR** on BLK360; use LDR panoramas to avoid over-saturated textures.
#### 4 Capture the Space
* **RTC360 / BLK360** – place the scanner roughly every 2–3 m; 30 % overlap between setups.
* **BLK2GO** – maintain slow walking speed (< 1 m/s) and follow loop-closure paths.
* Monitor FIELD 360: green spheres = complete scan, yellow = alignment pending.
#### 5 Register & Clean
1. Import data into **Cyclone REGISTER 360 PLUS**.
2. Run **Auto-Cloud > Cloud** for coarse alignment; refine with target-based or visual alignment as needed
3. Crop excess ceiling sky boxes or outdoor noise for faster VPS processing.
#### 6 Export Structured E57
* Choose **“Export > E57 (Structured)”** and tick **Include Panoramas**.
* Leave “Compatibility Mode” **unchecked** unless another software requires it; it downsamples points.
> Expect \~1 GB per 1 k sq ft at low-res settings. The MultiSet portal currently accepts files up to 20 **GB for free plan**.
***
### Uploading to MultiSet
* Create Map → Select Leica → Upload.
* Drag-and-drop the zip of E57 file.
* Please allow up to one hour to receive an email notification once your scan is complete
### Best Practices
| Do | Avoid |
| ------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------ |
| Keep the scanner out of mirrors & windows. | Walking within 2 m of shiny cars or glass facades (causes ghost points). |
| Use **low-res LiDAR + LDR pano** for VPS-only jobs. | High-res HDR if final AR textures aren’t required (adds 3–4× size). |
| Validate overlap in FIELD 360 before leaving the site. | Expecting Register 360 to fix large gaps automatically. |
| Export **one file per level** for multi-story buildings, then combine with MultiSet Merge tools. | A single 40 GB monolithic export. |
# NavVis Scans
The NavVis Ivion platform generates equirectangular panoramas instead of cubic images like Matterport or Leica, and MultiSet handles this easily. By combining NavVis's high-quality mobile mapping systems with MultiSet's Visual Positioning System (VPS), you can efficiently create large-scale, AR-ready environments. This guide provides a walkthrough of the recommended devices, capture workflow, export settings, and the process for uploading to the MultiSet Developer Portal.
### Supported Devices
* NavVis VLX
* NavVis M6
### Scanning Workflow
The following is the recommended workflow for capturing data with NavVis devices for use with MultiSet.
#### 1. Prerequisites
| Item | Notes |
| -------------------- | -------------------------------------------------------- |
| **Scanner & Power** | Ensure your NavVis device is fully charged. |
| **Desktop Software** | NavVis IVION for processing and exporting the scan data. |
#### 2. Prepare the Environment
To ensure a high-quality scan, prepare the scanning area as follows:
* Make sure the area is well-lit. For outdoor areas, scan on a cloudy day to avoid direct sunlight.
* Remove any people or moving objects from the environment.
* Use door stoppers to keep all doors open.
* Hide any confidential materials.
* Plan your mapping route in advance.
#### 3. Capture the Space
* For general mapping guidelines, consult the NavVis documentation.
* In open spaces, set the automatic panorama capture trigger to a predefined length of up to 3 meters.
* In more complex environments, use the manual trigger for panorama capture.
#### 4. Process and Export in NavVis IVION
* Before processing your raw data, enable the **Include panoramas in .e57 point cloud** option.
* It is recommended to use a point-cloud density of 1 cm (0.4 inches).
* When exporting, choose the **.e57 with panoramas** option.
* For the Coordinate System, select **Dataset** to ensure the origin of the site is within the point-cloud
#### 5. Uploading to MultiSet
* In the MultiSet Developer Portal, navigate to **Create Map → Select NavVis → Upload**.
* Compress the exported .e57 file into a **.zip** and drag and drop the zip file.
* Processing time will vary depending on the file size. You will receive an email notification when your scan is complete.
### Best Practices
| Do | Avoid |
| ---------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| **Use low-resolution LiDAR** for projects where AR is the only deliverable. | **High-resolution HDR**, as it can significantly increase file size with no benefit to the VPS. |
| **Keep the scanner out of mirrors** and windows.\[ | **Walking within 2 meters of shiny surfaces**, such as glass facades, which can cause ghost points. |
| **Export one file per level** for multi-story buildings and then use the MultiSet Merge tools. | **A single, large export** for a multi-story building. |
# Xgrids Scans
This document provides developers with instructions on how to process scan data from Xgrids SLAM-based scanners for use with the Multiset developer portal.
You will need Xgrids Lixel Studio and its supporting documentation, which can be found here:
#### Supported Devices
The following Xgrids scanners are supported:
* **L2 Pro:** Ideal for large-scale environments.
* **K1:** Recommended for medium to small-scale environments.
#### Data Processing Workflow
After completing a scan, the raw data needs to be processed into a specific format before it can be uploaded to the Multiset developer portal. The following steps outline the process of converting the initial LAS file to a zipped e57 file, including the necessary settings for panoramic images.
{% hint style="warning" %}
Please clean up your point cloud to remove all the noise and outliers to get an accurate mesh output, or you can directly use the mesh generated from Lixel Studio.
{% endhint %}
**Step 1: Project Processing**
1. Launch the LixelStudio software and create a new project.
2. Navigate to the **Processing** tab and select **Project Processing**.
3. In the "Project Processing" window, select your project file.
4. Under the "Coloring" section, ensure that the **Output panoramic image** checkbox is selected. This will generate the necessary panoramic images to be included in the final e57 file.
**Step 2: Data Format Conversion**
1. Once the initial project processing is complete, a poses.csv file will be generated.
2. In the main LixelStudio interface, go to the **Tool** menu.
3. Under "Data Management", select the **las to e57** tool. This will open the "Data format conversion" window.
4. In the "Data format conversion" window, select the poses.csv file as the "Select Pose File".
5. Set the **Panorama interval (m).** For large outdoor scans, a setting of **1 or 2 meters** is recommended (2 meters for outdoors and 1 meter for indoors)
6. Specify the "Output path" where the converted e57 file will be saved.
7. Click **OK** to begin the conversion process.
**Step 3: File Compression**
After the e57 file has been successfully exported, it must be compressed into a ZIP file before being uploaded to the Multiset developer portal.
1. Locate the exported e57 file in the output path you specified.
2. Right-click on the e57 file and select the "Compress" or "Send to > Compressed (zipped) folder" option, depending on your operating system.
3. The resulting ZIP file is now ready to be uploaded to the Multiset developer portal.
# Faro Scans
MultiSet supports point cloud data captured using Faro laser scanners. To ensure the best localization performance, the data must be captured, processed, and exported according to specific guidelines.
### Supported Hardware
The following Faro devices are fully supported:
* **Faro Focus Series**
* **Faro Orbis**
* **Faro Flash/Hybrid** (formerly Blink/Swift workflows)
### Scanning Guidelines
High-quality data capture is essential for accurate localization. Please adhere to the following best practices during the scanning process:
* **Scan Spacing:** Maintain a distance of **1–2 meters** between scan locations. Close spacing ensures high overlap, which is critical for visual positioning.
* **Lighting:** Ensure the environment is well-lit. Avoid scanning in pitch-black areas or environments with rapidly changing lighting conditions.
* **Static Environment:** Minimize movement during the scan. Avoid areas with heavy foot traffic or moving machinery. If people are present, ask them to stand still or mask them out during post-processing.
#### Altimeter Settings (Critical)
Before starting your scan, you must **disable the Altimeter** on the scanner or controller app.
MultiSet requires the point cloud to use **Local Y coordinates** (relative to the first scan) rather than absolute geodetic height. Using the altimeter can introduce vertical shifts that misalign the map with the device's camera feed.
### Point Cloud density
Exporting dense point clouds can crash the map processing and significantly increase processing times. We recommend a **medium** point cloud density or a voxel size of **0.5 - 1.0 CM.**
### Post-Processing in Faro SCENE
Once data capture is complete, import your raw scans into **Faro SCENE** software.
1. **Registration:** Register your scans to create a cohesive project.
2. **Cleanup:** Manually clean the point cloud to remove noise, reflections (from mirrors or glass), and "ghosts" caused by moving people or objects. Clean data results in faster and more accurate localization.
### Exporting for MultiSet
To generate a compatible map file, you must export the data as an **.e57** file with specific settings enabled.
1. In Faro SCENE, right-click your processed cluster or project and select **Export > Export Scan Points**.
2. Set the **Format** to E57 Files (\*.e57).
3. **Crucial:** Ensure the **Color Panorama Image** checkbox is **CHECKED**. MultiSet relies on these panoramic images for visual feature extraction.
4. Ensure **Full Scan** is selected (do not use "Selection" unless you specifically intend to crop the map).
5. Click **Export**.
#### Upload
Once the .e57 file is generated, zip it and then log in to the MultiSet Developer Portal and upload the file to create your new Map.
# Insta360 Scans
Capture 360° video with an Insta360 camera and turn it into a MultiSet VPS map.
MultiSet AI accepts **raw `.insv` (dual-fisheye) footage** from Insta360 cameras as input for the VPS reconstruction pipeline, the stitch to equirectangular happens server-side. With a single hand-held walk through the space plus a printed calibration marker for metric scale, you can turn a 360° capture into centimeter-accurate VPS maps, without any external SLAM hardware. This guide walks you through camera settings, the calibration marker, the walking pattern, and the upload flow on the MultiSet Developer Portal.
### Supported Devices
* **Insta360 X4**
* **Insta360 X5**
> Record in the camera's standard 360 video mode: **5.7K at 60 fps for indoor & low light**, **8K at 30 fps for outdoor & natural light**. Upload the **raw `.insv` file directly**, MultiSet stitches to equirectangular on the server. Footage shot in flat or single-lens mode is not supported.
### Scanning Workflow
#### 1. Prerequisites
| Item | Notes |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------- |
| **Camera & power** | Fully charge the camera, bring a spare battery for sessions longer than 20 minutes. |
| **Selfie / invisible stick** | Hold the camera above head height so the operator's body is masked out of the stitched panorama. |
| **Calibration marker** | A4 ChArUco board printed at 100% scale (see [Calibration marker](#2-calibration-marker-for-metric-scale)). |
| **Mobile app** | Insta360 app, only required for verifying camera settings before capture. |
#### 2. Calibration marker (for metric scale)
A single ChArUco marker placed in the scene is what lets the reconstruction lock onto a **real-world metric scale**. Without it, the reconstructed map will be geometrically correct but at an arbitrary scale.
**Download the marker** (ChArUco, A4, 4×6, 48 mm / 36 mm, DICT\_4X4):
{% file src="/files/zZ6bw21hBbN5GVUjsUBo" %}
**Print:**
* Print on **flat A4 paper at exactly 100% scale**. Do **not** "fit to page", scaling the marker will scale the entire reconstructed map.
* Flatten any curl, place under a clear sheet of glass or acrylic if needed. A wrinkled marker will fail to register cleanly.
**Place:**
* Lay the marker **flat on the floor** (or a flat table) near the start of your capture path.
* Keep the marker stationary for the whole capture, do **not** hold it in your hand.
* Place it where you can pass it **twice**, at the start and again at the end of the walk, by routing your loop over it.
**Capture against the marker:**
* When you reach the marker, **rotate very slowly** above it so the camera sees the full board from several angles.
* The marker should fill roughly **1/4 of the camera frame** for at least a couple of seconds per pass.
* Slow rotation matters more than anything else here, fast motion blurs the marker and the metric-scale step will fail to lock.
####
{% embed url="" %}
#### 3. Camera settings
**Shooting mode**
Set this first, before adjusting any other setting.
1. Push **Shooting Mode**.
2. Choose **Video**.
3. Choose **360°**.
**Exposure and white balance**
Lock these before you press record. Auto-exposure is the single biggest cause of grey, smeared, or low-detail reconstructions.
| Setting | Value | Why |
| ---------------------------------------------- | ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Shutter (indoor)** | **Fixed, between 1/120 and 1/640** | Locks exposure so every frame has the same brightness. Pick a value in this range based on how bright the space is (slower end for dim spaces, faster end for well-lit ones). Auto-shutter is the #1 cause of grey or blurry results indoors. |
| **Shutter (outdoor)** | **Auto** | Outdoor lighting changes too quickly (sun, shade, sky) for a fixed shutter to track. Let the camera adjust. |
| **White balance** | **Fixed** (any preset, just not Auto) | Locks colour cast, no per-frame colour shift, sharper textures. |
| **ISO** | Auto | Fine to leave on Auto, as long as shutter is set correctly for the environment. |
| **Resolution / FPS (indoor & low light)** | **5.7K at 60 fps** | The higher frame rate keeps frames sharp in dim, hand-held capture where motion blur is the main risk. |
| **Resolution / FPS (outdoor & natural light)** | **8K at 30 fps** | Plenty of light means motion blur is less of a concern, so the extra resolution gives the pipeline more usable feature detail. Always shoot in 360 video mode. The raw `.insv` is dual-fisheye, MultiSet stitches it to equirectangular server-side. |
If you can only fix one setting indoors, fix the **shutter**.
**Where to find these settings**
1. Swipe down from the top of the camera screen.
2. Tap the **gear / settings** icon to open camera settings.
3. Open **Image Settings**.
Then set:
| Setting | Value |
| ------------------- | --------- |
| **Codec** | **H.265** |
| **Bitrate** | **High** |
| **Video Sharpness** | **High** |
#### 4. Walking pattern
* **Speed: 1.0–1.5 m/s**, slow walking pace, "museum tempo".
* **Smooth, continuous motion**, no stops, no jerks, no rotation in place.
* Turn at corners with a **wide arc**, not a pivot.
* Stay **0.5–1 m away from walls and furniture**.
* Keep the camera at a steady height, don't bob or swing the stick.
* Hold the camera at **head height or just above**, so the operator's body is masked in the stitch.
A 360° camera sees in every direction at once, so unlike a phone scan you don't need to think about where you're pointing it, only **where you walk**. The path you choose is what guarantees full coverage and clean loop closure. Match the pattern to your space type below.
**Choose your pattern**
| Space type | Pattern | Notes |
| ------------------------------------------------------------ | -------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| Small room (< 50 m²) | **Single loop** | One perimeter pass, offset \~0.7 m from the walls. The 360° camera sees the whole interior as you walk. |
| Large open area (indoor hall, plaza, courtyard, parking lot) | **Perimeter + lattice** | Loop the perimeter first, then fill the interior with parallel sweeps \~3–5 m apart. |
| Two or more connected rooms | **Figure-8 / double loop** | Loop each room separately, but pass through each doorway **twice** (once in each direction). |
| Long, narrow space (corridor, aisle, hallway) | **Corridor sweep** | Walk down one side, turn at the end with a wide arc, walk back along the other side. |
| Object or landmark (statue, kiosk, exhibit) | **Orbit** | Walk a full circle around the object at \~1.5–2 m distance, then a second wider circle if there's surrounding context. |
#### 5. Loop closure
* **Return to your starting point** at the end of the walk.
* Overlap the **first \~5 metres of the path** at the end so the start and end see the same content from a similar viewpoint.
* Loop closure is what lets the reconstruction recognise that the start and end are the same place. Without it the scene can drift and end up warped or "open".
* Place the calibration marker **near the start/end of the loop** so the camera passes over it twice, that gives the metric-scale step the best chance of locking on.
#### 6. Export
* Export the `.insv` file from the camera or the Insta360 app, no stitching is required, MultiSet's pipeline handles stitching internally.
* **Each upload `.zip` must contain exactly one `.insv` file.** One `.insv` produces one map.
* Keep the `.zip` size **under 50 GB**.
{% hint style="info" %}
**Capturing a larger area?** Record it as **multiple separate captures**, each in its own `.insv` file. Upload each `.insv` as its own `.zip` so MultiSet processes them as independent maps, then combine them into a single coordinate frame with [MapSet : Multiple Maps](/multiset/basics/mapset-multiple-maps).
{% endhint %}
***
### Uploading to MultiSet
1. Open the [MultiSet Developer Portal](https://developer.multiset.ai/) and choose **Upload Existing Map**.
2. **Scan Type:** select 360 video.
3. **Select Provider:** choose **Insta360**.
4. **File Format:** select **Zip (.insv file compressed in .zip format)**.
5. Pick the map location, then drag-and-drop the `.zip` (containing a single `.insv`) on the Upload Map step.
{% hint style="warning" %}
**Keep in mind**
1. Record in 360 video mode: **5.7K at 60 fps for indoor & low light**, **8K at 30 fps for outdoor & natural light**. Upload the raw `.insv`, MultiSet stitches server-side.
2. Walk at a steady pace with the camera at head height.
3. One `.insv` file per `.zip`, one `.zip` produces one map. For larger areas, upload each `.insv` separately and combine them with a MapSet.
4. Ensure file size is **not more than 50 GB**.
{% endhint %}
Allow up to one hour for processing. You'll receive an email notification once the map is ready.
### Anti-checklist (do NOT do)
* Don't shoot indoors with auto-shutter or auto-exposure on (outdoor capture is the exception, leave shutter on Auto there).
* Don't shoot with auto white balance.
* Don't pause the recording mid-walk.
* Don't rotate in one spot (except slowly above the calibration marker).
* Don't hold the calibration marker in your hand during the recording.
* Don't modify the scene during the capture (moving chairs, toggling lights, opening or closing doors).
* Don't make LED display screens the main subject, they emit light that changes with viewing angle, and the reconstruction can't recover crisp detail on them.
* Don't shoot in mixed harsh / direct-window light without dimming or covering the windows. Strong sunlight reflections and glare are the hardest content to reconstruct.
# Gaussian Splat
Use Gaussian Splat reconstructions as input for Multiset VPS.
Multiset now supports **Gaussian Splats** as input for building a VPS map. Instead of uploading a point cloud (e57), you can upload a trained Gaussian Splat reconstruction together with the camera poses used to train it, and Multiset will use that as the source of truth for localization.
#### What you need to upload
A **single `.zip`** containing two files:
1. **`.ply`** — the Gaussian Splat file exported from your reconstruction tool.
2. **`poses.json`** — the camera poses (position + rotation) associated with the training frames, in the format shown below.
Both files must be at the root of the zip (no nested folders).
#### poses.json format
```json
{
"poses": [
{
"ts": "1776075484.516121864",
"T": [0.000021, 0.000021, 0.000008],
"R": [0.707448, -0.706765, 0.001143, 0.000027],
"RGB": null
}
]
}
```
Fields:
* `ts` — timestamp (string).
* `T` — translation `[x, y, z]`.
* `R` — rotation quaternion `[x, y, z, w]`.
* `RGB` — optional, can be `null`.
#### Requirements
* **Metric scale is required.** The Gaussian Splat must be in real-world metric scale for VPS to work.
* **Indoor or outdoor** — you'll specify this at upload time; selecting the correct mode improves VPS accuracy.
* **poses.json is required** — the splat cannot be localized against without the training poses.
{% hint style="warning" %}
Gaussian Splats generated without metric scale (arbitrary-scale reconstructions) cannot be used for VPS.
{% endhint %}
## Supported pipelines
Currently supported Gaussian Splat export sources:
* [Xgrids (Lixel CyberColor)](/multiset/basics/third-party-scans/gaussian-splat/xgrids-gaussian-splat) — all Xgrids scanners (PortalCam, L2 Pro, K1/K2) via the Lixel CyberColor reconstruction app.
More providers will be added as they ship metric-scaled Gaussian Splat exports.
## Upload flow
On the Multiset developer portal, choose **Upload Existing Map**, select your scanner provider (e.g. **XGRIDS**), then pick **Gaussian Splat (ply and poses.json compressed in same zip)** as the file format. You will be asked to confirm:
* Whether the Gaussian Splat is metric-scaled (must be **Yes** for VPS).
* Whether the capture is **indoor** or **outdoor**.
* Whether the zip includes `poses.json` (required).
Gaussian Splat configuration — confirm scale, indoor/outdoor, and poses.json.
# Xgrids Gaussian Splat Export
Export a Gaussian Splat from Xgrids Lixel CyberColor for use with Multiset VPS.
All Xgrids scanners (PortalCam, L2 Pro, K1/K2) can produce a Gaussian Splat reconstruction via **Lixel CyberColor**. This page walks through the export steps and how to package the output for upload to Multiset.
{% hint style="info" %}
For the e57 point-cloud workflow, see [Xgrids Scans](/multiset/basics/third-party-scans/xgrids-scans) instead. The steps below are specific to the **Gaussian Splat** upload path.
{% endhint %}
#### You will need
* An Xgrids scan (PortalCam, L2 Pro, K1/K2).
* **Lixel CyberColor** (v1.12.1 or newer) — available from Xgrids support downloads.
## Step 1: Generate the Gaussian Splat model
1. Open **Lixel CyberColor** and click **Create** to start a new model.
2. Choose **Single Model** as the reconstruction type.
3. Give the project a name (e.g. `splats_for_vps`) and set the **Scan data** path to your captured `portalcam_capture_data` folder.
4. In the **Parameters** panel on the right:
* **Reconstruction Settings**: choose **Slow** for the best quality suitable for VPS.
* **Maximum Gaussian Splats**: leave at the recommended value (e.g. `28M`) or raise it for larger scenes.
* **Portability**: **On**. This is important, it produces a metric-scaled, portable output that Multiset VPS can use.
* Leave **Exposure Optimization** and **Low Memory Reconstruction** at defaults unless you have a specific reason to change them.
5. Click **Start** and let the reconstruction run.
Lixel CyberColor — Generate Model with Portability On and Slow reconstruction.
{% hint style="warning" %}
**Portability must be On.** Without it the Gaussian Splat will not be in metric scale and cannot be used for VPS.
{% endhint %}
## Step 2: Locate the exported `.ply`
Once the reconstruction finishes, Lixel CyberColor writes the point cloud (`.ply`) outputs under the project's output directory:
```