> For the complete documentation index, see [llms.txt](https://docs.multiset.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.multiset.ai/native-sdk/android-native.md). # Android Native Add VPS localization, object tracking and Non-LiDAR map scanning to an Android app with the platform agnostic MultiSet SDK, including quick start and requirements. ## Overview The MultiSet Android SDK provides Visual Positioning System (VPS) localization, object tracking and map scanning for Android applications. It enables precise indoor and outdoor localization using camera-based visual recognition against pre-mapped 3D environments, detects pre-registered physical objects in AR with an animated 3D outline, and records new maps on devices without LiDAR. The SDK is **platform-agnostic**. It carries no ARCore, Sceneform or Jetpack XR dependency of its own. Your app converts each camera frame into a plain `CameraFrame` and hands it to the SDK through the [`FrameSource`](/native-sdk/android-native/api-reference/framesource.md) interface, so the same library runs on ARCore or on any other Android XR runtime. The repository ships two reference apps: an ARCore and Sceneform sample for phones, and an XREAL Aura sample for Android XR glasses. **Current version:** 2.1.0 **GitHub Repository:** ## What's New in 2.1.0 | Change | Notes | | ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Non-LiDAR map scanning** | Record a new map with the device camera, save it as a draft and upload it with the SDK's own credentials. See [Map Scanning](/native-sdk/android-native/api-reference/map-scanning.md). | | **XREAL Aura sample** | Map and MapSet localization and object tracking on Android XR glasses, with the same SDK and credentials. | | **Relocalization after a tracking break** | The session re-localizes by itself once tracking recovers, and tells you when to hide the map. See [Tracking breaks](/native-sdk/android-native/api-reference/framesource.md#tracking-breaks-and-the-return-from-the-background). | | **Object tracking across breaks** | Re-tracks once tracking recovers and hides an object's content until that object is tracked again. Object tracking has its own settings on `MultiSetSDKConfig`. | | **Run stages** | Both sessions report each run (`Capturing`, `Uploading`, `AwaitingResponse`, `Idle`) so your UI can show progress. See [Run stages](/native-sdk/android-native/api-reference/framesource.md#run-stages). | | **VPS-2 for multi-frame** | `queryMode` is sent with multi-frame queries too, for a map or a MapSet. | | **Hints and GPS reach the query** | `hintPosition`, `hintFloorHeight`, `hintRadius`, `use2DFiltering`, the GPS hint and `includeGeoCoordinatesInResponse` are now sent. The result carries the new `geoPose`. | | **First localization until success** | Capped at 5 silent retries and independent of background localization. | | **Query image size** | `imageLongSide` sets the long side of the query image, at most 1280. Frames keep their aspect ratio. | | **New defaults** | Single-frame localization with VPS-1, and `poseConsistencyCheck = false`. | See the [Version 2.4.1 changelog](/quick-access/changelog/v2/version-2.4.1.md) for the fixes in this release. {% hint style="warning" %} **Upgrading from 2.0.0** needs no source changes, but recompile against the new AAR: some method signatures gained parameters. Check the new defaults above. **Upgrading from 1.16.0** is source-breaking. `poseConsistencyThreshold` now takes 1.5 to 15 metres (a value outside it throws), `GateTuning` dropped `rejectionBaseM`, `driftRateMPerSec` and `rejectionMaxM`, and `FalsePositiveInfo` gained fields. The [repository README](https://github.com/MultiSet-AI/multiset-android-sdk) has the full upgrade notes. {% endhint %} ## Table of Contents ### Sample Activities * [MainActivity](/native-sdk/android-native/sample-activities/mainactivity.md) - Authentication, SDK initialization, and navigation * [MultiSetLocalizationActivity](/native-sdk/android-native/sample-activities/multisetlocalizationactivity.md) - Unified AR localization (single-frame & multi-frame) * [ObjectTrackingActivity](/native-sdk/android-native/sample-activities/objecttrackingactivity.md) - AR object detection and tracking with animated mesh overlay ### API Reference * [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md) - The platform-agnostic boundary and the session APIs * [LocalizationConfig](/native-sdk/android-native/api-reference/localizationconfig.md) - Configuration parameters for localization behavior * [ObjectTrackingConfig](/native-sdk/android-native/api-reference/objecttrackingconfig.md) - Configuration parameters for object tracking behavior * [Map Scanning (Non-LiDAR)](/native-sdk/android-native/api-reference/map-scanning.md) - Recording, saving and uploading a new map ## Quick Start ### 1. Add Dependencies Add the MultiSet SDK AAR to your app's `libs/` directory and configure dependencies: ```gradle dependencies { implementation files('libs/multiset-sdk.aar') // Required by the SDK implementation 'com.squareup.okhttp3:okhttp:4.12.0' implementation 'com.squareup.okhttp3:logging-interceptor:4.12.0' implementation 'com.google.code.gson:gson:2.10.1' implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2' implementation 'com.google.mlkit:vision-common:17.3.0' implementation 'androidx.core:core-ktx:1.16.0' // Only needed if you render AR with ARCore and Sceneform, as the sample app does implementation 'com.google.ar:core:1.46.0' implementation 'com.github.RGregat:sceneform-android:1.24.6' } ``` {% hint style="warning" %} An AAR referenced as a file dependency carries no dependency metadata, so nothing is pulled in transitively. Every library listed above under "Required by the SDK" must be declared in your own build file, or the app will compile and then fail at runtime with `NoClassDefFoundError`. {% endhint %} Sceneform is published on JitPack, so add that repository to your project settings if you render AR: ```gradle maven { url = uri("https://jitpack.io") } ``` Sceneform's native libraries also require legacy JNI packaging for 16 KB page size compatibility: ```gradle packaging { jniLibs { useLegacyPackaging = true } } ``` ### 2. Credentials Setup Create a `multiset.properties` file in your project root with your credentials, and keep it out of version control: ```properties # API Host (Optional). Leave empty to use the production host. MULTISET_BASE_URL= MULTISET_CLIENT_ID=your_client_id MULTISET_CLIENT_SECRET=your_client_secret MULTISET_MAP_CODE=your_map_code MULTISET_MAP_SET_CODE= MULTISET_OBJECT_CODES=OBJ_001,OBJ_002 ``` | Property | Description | Required | | ------------------------ | ------------------------------------------ | ---------------------------- | | `MULTISET_BASE_URL` | API host override. Empty means production. | No | | `MULTISET_CLIENT_ID` | Your client identifier | Yes | | `MULTISET_CLIENT_SECRET` | Your secret key | Yes | | `MULTISET_MAP_CODE` | Single map identifier | One of these is required | | `MULTISET_MAP_SET_CODE` | MapSet identifier | One of these is required | | `MULTISET_OBJECT_CODES` | Comma-separated object code list | Required for object tracking | {% hint style="warning" %} `MULTISET_BASE_URL` accepts an `http://` host only in debug builds, because release builds disable cleartext traffic. Production hosts must use `https://`. {% endhint %} ### 3. Initialize the SDK ```kotlin val config = MultiSetSDKConfig.Builder(clientId, clientSecret) .mapCode(mapCode) // or .mapSetCode(mapSetCode) .localizationMode(LocalizationMode.SINGLE_FRAME) // the default, with QueryMode.VPS1 .build() MultiSetSDK.initialize(applicationContext, config, callback) ``` `MultiSetSDKConfig` validates on construction. Blank credentials, a missing map or object identifier, `numberOfFrames` outside 4 to 6, `imageQuality` outside 1 to 100, `imageLongSide` outside 1 to 1280, `confidenceThreshold` outside 0 to 1, `poseConsistencyThreshold` outside 1.5 to 15 metres, `poseConsistencyYawThreshold` outside 10 to 60 degrees, more than 10 object codes, `hintRadius` outside 1 to 100, or a malformed `baseUrl` all throw immediately. Calling `initialize` again with the same account, for example from an activity recreated by a rotation, sends no request while the token is still valid. The callback hears `onAuthenticationSuccess` again. A few settings exist only on `MultiSetSDKConfig`: | Setting | Default | Description | | ------------------------------------------ | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `imageLongSide` | `960` | Longer side of the query image in pixels, at most 1280. Frames keep their aspect ratio. | | `baseUrl` | production | API host override, as `MULTISET_BASE_URL` above. | | `poseConsistencyTuning`, `mapAliasingRisk` | `null`, empty | Advanced pose consistency settings. See [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md#localizationsession). | ### 4. Implement Callbacks ```kotlin class MainActivity : AppCompatActivity(), MultiSetSDKCallback { override fun onSDKReady() { /* SDK initialized */ } override fun onAuthenticationSuccess() { /* Ready for localization and tracking */ } override fun onAuthenticationFailure(error: String) { /* Handle error */ } override fun onLocalizationSuccess(result: LocalizationResult) { // Access result.mapCode, result.mapCodes, result.position, result.rotation, result.confidence } override fun onLocalizationFailure(error: String) { /* Handle failure */ } override fun onLocalizationFalsePositive(info: FalsePositiveInfo) { // Result discarded as inconsistent with the device trajectory. The scene is untouched. } override fun onRecoveryPhaseChanged(phase: RecoveryPhase, needsRealignment: Boolean) { // Hide map content while needsRealignment is true } override fun onTrackingStateChanged(state: TrackingState) { /* Handle state change */ } override fun onObjectTrackingSuccess(result: ObjectTrackingResult) { // Access result.objectCode, result.objectCodes, result.position, result.rotation, result.confidence } override fun onObjectTrackingFailure(error: String) { /* Handle tracking failure */ } } ``` Only `onSDKReady`, `onAuthenticationSuccess`, `onAuthenticationFailure`, `onLocalizationSuccess`, `onLocalizationFailure` and `onTrackingStateChanged` must be implemented; every other callback has a default no-op implementation. Sessions built by `MultiSetSDK` report to this callback as well as through their own lambdas. See the [full callback list](/native-sdk/android-native/api-reference.md#multisetsdkcallback-interface). ### 5. Run a Localization Session Results are delivered through the session you start, rather than being polled from the SDK. Supply frames through a `FrameSource`, handle the session callbacks, and report the AR tracking state: ```kotlin val session = MultiSetSDK.localizationSession(myFrameSource).apply { onLocalizationSuccess = { result -> // Apply result.position and result.rotation to your scene } onLocalizationFailure = { error -> // error.kind is SERVER, AUTH, NETWORK or TRANSIENT } onRecoveryPhaseChanged = { phase, needsRealignment -> // Hide the map while the scene is out of place after a tracking break } } session.reportTrackingState(TrackingState.TRACKING) // the first state, then every change session.start() ``` Call `session.stop()` when you tear the scene down. It cancels both the scheduled capture and any request already in flight, so a late response cannot re-apply a pose after you have stopped. See [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md) for the full session API, tracking breaks and a worked `FrameSource` implementation. ## Requirements * Android API Level 28+ (Android 9.0) * Target SDK 36 * Java 17 * Kotlin 2.2.0+ * Gradle 8.11.1 with AGP 8.10.0 * Camera permission and internet connectivity * ARCore compatible device (only if you render AR with the sample app's approach) * For the XREAL Aura sample: an Android XR device with a world-facing camera, Android 14 or newer ## License Copyright (c) 2026 MultiSet AI. All rights reserved. Licensed under the MultiSet License. For license details, visit [www.multiset.ai](https://www.multiset.ai). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.multiset.ai/native-sdk/android-native.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.