> For the complete documentation index, see [llms.txt](https://docs.multiset.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.multiset.ai/native-sdk/android-native.md). # Android Native Add VPS localization and object tracking to an Android app with the platform agnostic MultiSet SDK, including quick start and requirements. ## Overview The MultiSet Android SDK provides Visual Positioning System (VPS) localization and Object Tracking capabilities for Android applications. It enables precise indoor and outdoor localization using camera-based visual recognition against pre-mapped 3D environments, and detects pre-registered physical objects in AR with animated 3D mesh visualization. As of version 1.16.0 the SDK is **platform-agnostic**. It carries no ARCore or Sceneform dependency of its own. Your app converts each camera frame into a plain `CameraFrame` and hands it to the SDK through the [`FrameSource`](/native-sdk/android-native/api-reference/framesource.md) interface, so the same library runs on ARCore or on any other Android XR runtime. The bundled sample app is an ARCore and Sceneform reference implementation. **GitHub Repository:** ## What's New in 1.16.0 | Change | Notes | | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **AR-free SDK** | ARCore and Sceneform are no longer dependencies of the SDK. They are used only by the sample app. | | **Frame sources and sessions** | Localization and object tracking now run as long-lived session objects fed by a `FrameSource` you implement. See [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md). | | **False-positive rejection** | Localization results that contradict the device's own AR trajectory are discarded and reported through `onLocalizationFalsePositive` instead of moving your content to the wrong place. | | **Configurable API host** | `MULTISET_BASE_URL` points the SDK at a staging or on-premise deployment. Leave it empty for production. | | **Query mode** | `QueryMode.VPS1` and `QueryMode.VPS2` select the search strategy for single-frame localization. | | **Automatic token refresh** | Short-lived M2M tokens are renewed transparently, with a single retry on HTTP 401. | {% hint style="info" %} **Renamed types.** `MultiSetConfig` is now `MultiSetSDKConfig` and `MultiSetCallback` is now `MultiSetSDKCallback`. The removed `MultiSetSDK.getLastLocalizationResult()` is replaced by the session callbacks described below. {% endhint %} ## Table of Contents ### Sample Activities * [MainActivity](/native-sdk/android-native/sample-activities/mainactivity.md) - Authentication, SDK initialization, and navigation * [MultiSetLocalizationActivity](/native-sdk/android-native/sample-activities/multisetlocalizationactivity.md) - Unified AR localization (single-frame & multi-frame) * [ObjectTrackingActivity](/native-sdk/android-native/sample-activities/objecttrackingactivity.md) - AR object detection and tracking with animated mesh overlay ### API Reference * [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md) - The platform-agnostic boundary and the session APIs * [LocalizationConfig](/native-sdk/android-native/api-reference/localizationconfig.md) - Configuration parameters for localization behavior * [ObjectTrackingConfig](/native-sdk/android-native/api-reference/objecttrackingconfig.md) - Configuration parameters for object tracking behavior ## Quick Start ### 1. Add Dependencies Add the MultiSet SDK AAR to your app's `libs/` directory and configure dependencies: ```gradle dependencies { implementation files('libs/multiset-sdk.aar') // Required by the SDK implementation 'com.squareup.okhttp3:okhttp:4.12.0' implementation 'com.squareup.okhttp3:logging-interceptor:4.12.0' implementation 'com.google.code.gson:gson:2.10.1' implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2' implementation 'com.google.mlkit:vision-common:17.3.0' implementation 'androidx.core:core-ktx:1.16.0' // Only needed if you render AR with ARCore and Sceneform, as the sample app does implementation 'com.google.ar:core:1.46.0' implementation 'com.github.RGregat:sceneform-android:1.24.6' } ``` {% hint style="warning" %} An AAR referenced as a file dependency carries no dependency metadata, so nothing is pulled in transitively. Every library listed above under "Required by the SDK" must be declared in your own build file, or the app will compile and then fail at runtime with `NoClassDefFoundError`. {% endhint %} Sceneform is published on JitPack, so add that repository to your project settings if you render AR: ```gradle maven { url = uri("https://jitpack.io") } ``` Sceneform's native libraries also require legacy JNI packaging for 16 KB page size compatibility: ```gradle packaging { jniLibs { useLegacyPackaging = true } } ``` ### 2. Credentials Setup Create a `multiset.properties` file in your project root with your credentials, and keep it out of version control: ```properties # API Host (Optional). Leave empty to use the production host. MULTISET_BASE_URL= MULTISET_CLIENT_ID=your_client_id MULTISET_CLIENT_SECRET=your_client_secret MULTISET_MAP_CODE=your_map_code MULTISET_MAP_SET_CODE= MULTISET_OBJECT_CODES=OBJ_001,OBJ_002 ``` | Property | Description | Required | | ------------------------ | ------------------------------------------ | ---------------------------- | | `MULTISET_BASE_URL` | API host override. Empty means production. | No | | `MULTISET_CLIENT_ID` | Your client identifier | Yes | | `MULTISET_CLIENT_SECRET` | Your secret key | Yes | | `MULTISET_MAP_CODE` | Single map identifier | One of these is required | | `MULTISET_MAP_SET_CODE` | MapSet identifier | One of these is required | | `MULTISET_OBJECT_CODES` | Comma-separated object code list | Required for object tracking | {% hint style="warning" %} `MULTISET_BASE_URL` accepts an `http://` host only in debug builds, because release builds disable cleartext traffic. Production hosts must use `https://`. {% endhint %} ### 3. Initialize the SDK ```kotlin val config = MultiSetSDKConfig.Builder(clientId, clientSecret) .mapCode(mapCode) .localizationMode(LocalizationMode.MULTI_FRAME) .enableMeshVisualization(true) .backgroundLocalization(true) .poseConsistencyCheck(true) .build() MultiSetSDK.initialize(context, config, callback) ``` `MultiSetSDKConfig` validates on construction. Blank credentials, a missing map or object identifier, `numberOfFrames` outside 4 to 6, `imageQuality` outside 1 to 100, `confidenceThreshold` outside 0 to 1, `poseConsistencyThreshold` outside 3 to 30 metres, more than 10 object codes, `hintRadius` outside 1 to 100, or a malformed `baseUrl` all throw immediately. ### 4. Implement Callbacks ```kotlin class MainActivity : AppCompatActivity(), MultiSetSDKCallback { override fun onSDKReady() { /* SDK initialized */ } override fun onAuthenticationSuccess() { /* Ready for localization and tracking */ } override fun onAuthenticationFailure(error: String) { /* Handle error */ } override fun onLocalizationSuccess(result: LocalizationResult) { // Access result.mapCode, result.mapCodes, result.position, result.rotation, result.confidence } override fun onLocalizationFailure(error: String) { /* Handle failure */ } override fun onLocalizationFalsePositive(info: FalsePositiveInfo) { // Result discarded as inconsistent with the device trajectory. The scene is untouched. } override fun onTrackingStateChanged(state: TrackingState) { /* Handle state change */ } override fun onObjectTrackingSuccess(result: ObjectTrackingResult) { // Access result.objectCode, result.objectCodes, result.position, result.rotation, result.confidence } override fun onObjectTrackingFailure(error: String) { /* Handle tracking failure */ } } ``` `onLocalizationFalsePositive`, `onObjectTrackingSuccess` and `onObjectTrackingFailure` have default no-op implementations, so existing code keeps compiling when you upgrade. ### 5. Run a Localization Session Results are delivered through the session you start, rather than being polled from the SDK. Supply frames through a `FrameSource` and handle the session callbacks: ```kotlin val session = MultiSetSDK.localizationSession(myFrameSource, LocalizationMode.MULTI_FRAME).apply { onLocalizationSuccess = { result -> // Apply result.position and result.rotation to your scene } onLocalizationFailure = { error -> // error.kind is SERVER, AUTH, NETWORK or TRANSIENT } onLocalizationFalsePositive = { info -> // Discarded as inconsistent with the device trajectory } } session.start() ``` Call `session.stop()` when you tear the scene down. It cancels both the scheduled capture and any request already in flight, so a late response cannot re-apply a pose after you have stopped. See [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md) for the full session API and a worked `FrameSource` implementation. ## Requirements * Android API Level 28+ (Android 9.0) * Target SDK 36 * Java 17 * Kotlin 2.2.0+ * Gradle 8.11.1 with AGP 8.10.0 * Camera permission and internet connectivity * ARCore compatible device (only if you render AR with the sample app's approach) ## License Copyright (c) 2026 MultiSet AI. All rights reserved. Licensed under the MultiSet License. For license details, visit [www.multiset.ai](https://www.multiset.ai). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.multiset.ai/native-sdk/android-native.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.