Android Native

Overview

The MultiSet Android SDK provides Visual Positioning System (VPS) localization and Object Tracking capabilities for Android applications. It enables precise indoor and outdoor localization using camera-based visual recognition against pre-mapped 3D environments, and detects pre-registered physical objects in AR with animated 3D mesh visualization.

As of version 1.16.0 the SDK is platform-agnostic. It carries no ARCore or Sceneform dependency of its own. Your app converts each camera frame into a plain CameraFrame and hands it to the SDK through the FrameSource interface, so the same library runs on ARCore or on any other Android XR runtime. The bundled sample app is an ARCore and Sceneform reference implementation.

GitHub Repository: https://github.com/MultiSet-AI/multiset-android-sdk.git

What's New in 1.16.0

Change	Notes
AR-free SDK	ARCore and Sceneform are no longer dependencies of the SDK. They are used only by the sample app.
Frame sources and sessions	Localization and object tracking now run as long-lived session objects fed by a FrameSource you implement. See FrameSource and Sessions.
False-positive rejection	Localization results that contradict the device's own AR trajectory are discarded and reported through onLocalizationFalsePositive instead of moving your content to the wrong place.
Configurable API host	MULTISET_BASE_URL points the SDK at a staging or on-premise deployment. Leave it empty for production.
Query mode	QueryMode.VPS1 and QueryMode.VPS2 select the search strategy for single-frame localization.
Automatic token refresh	Short-lived M2M tokens are renewed transparently, with a single retry on HTTP 401.

Renamed types. MultiSetConfig is now MultiSetSDKConfig and MultiSetCallback is now MultiSetSDKCallback. The removed MultiSetSDK.getLastLocalizationResult() is replaced by the session callbacks described below.

Table of Contents

Sample Activities

• MainActivity - Authentication, SDK initialization, and navigation

• MultiSetLocalizationActivity - Unified AR localization (single-frame & multi-frame)

• ObjectTrackingActivity - AR object detection and tracking with animated mesh overlay

API Reference

• FrameSource and Sessions - The platform-agnostic boundary and the session APIs

• LocalizationConfig - Configuration parameters for localization behavior

• ObjectTrackingConfig - Configuration parameters for object tracking behavior

Quick Start

1. Add Dependencies

Add the MultiSet SDK AAR to your app's libs/ directory and configure dependencies:

dependencies {
    implementation files('libs/multiset-sdk.aar')

    // Required by the SDK
    implementation 'com.squareup.okhttp3:okhttp:4.12.0'
    implementation 'com.squareup.okhttp3:logging-interceptor:4.12.0'
    implementation 'com.google.code.gson:gson:2.10.1'
    implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2'
    implementation 'com.google.mlkit:vision-common:17.3.0'
    implementation 'androidx.core:core-ktx:1.16.0'

    // Only needed if you render AR with ARCore and Sceneform, as the sample app does
    implementation 'com.google.ar:core:1.46.0'
    implementation 'com.github.RGregat:sceneform-android:1.24.6'
}

An AAR referenced as a file dependency carries no dependency metadata, so nothing is pulled in transitively. Every library listed above under "Required by the SDK" must be declared in your own build file, or the app will compile and then fail at runtime with NoClassDefFoundError.

Sceneform is published on JitPack, so add that repository to your project settings if you render AR:

maven { url = uri("https://jitpack.io") }

Sceneform's native libraries also require legacy JNI packaging for 16 KB page size compatibility:

packaging {
    jniLibs { useLegacyPackaging = true }
}

2. Credentials Setup

Create a multiset.properties file in your project root with your credentials, and keep it out of version control:

# API Host (Optional). Leave empty to use the production host.
MULTISET_BASE_URL=

MULTISET_CLIENT_ID=your_client_id
MULTISET_CLIENT_SECRET=your_client_secret
MULTISET_MAP_CODE=your_map_code
MULTISET_MAP_SET_CODE=
MULTISET_OBJECT_CODES=OBJ_001,OBJ_002

Property	Description	Required
MULTISET_BASE_URL	API host override. Empty means production.	No
MULTISET_CLIENT_ID	Your client identifier	Yes
MULTISET_CLIENT_SECRET	Your secret key	Yes
MULTISET_MAP_CODE	Single map identifier	One of these is required
MULTISET_MAP_SET_CODE	MapSet identifier	One of these is required
MULTISET_OBJECT_CODES	Comma-separated object code list	Required for object tracking

MULTISET_BASE_URL accepts an http:// host only in debug builds, because release builds disable cleartext traffic. Production hosts must use https://.

3. Initialize the SDK

val config = MultiSetSDKConfig.Builder(clientId, clientSecret)
    .mapCode(mapCode)
    .localizationMode(LocalizationMode.MULTI_FRAME)
    .enableMeshVisualization(true)
    .backgroundLocalization(true)
    .poseConsistencyCheck(true)
    .build()

MultiSetSDK.initialize(context, config, callback)

MultiSetSDKConfig validates on construction. Blank credentials, a missing map or object identifier, numberOfFrames outside 4 to 6, imageQuality outside 1 to 100, confidenceThreshold outside 0 to 1, poseConsistencyThreshold outside 3 to 30 metres, more than 10 object codes, hintRadius outside 1 to 100, or a malformed baseUrl all throw immediately.

4. Implement Callbacks

class MainActivity : AppCompatActivity(), MultiSetSDKCallback {
    override fun onSDKReady() { /* SDK initialized */ }
    override fun onAuthenticationSuccess() { /* Ready for localization and tracking */ }
    override fun onAuthenticationFailure(error: String) { /* Handle error */ }
    override fun onLocalizationSuccess(result: LocalizationResult) {
        // Access result.mapCode, result.mapCodes, result.position, result.rotation, result.confidence
    }
    override fun onLocalizationFailure(error: String) { /* Handle failure */ }
    override fun onLocalizationFalsePositive(info: FalsePositiveInfo) {
        // Result discarded as inconsistent with the device trajectory. The scene is untouched.
    }
    override fun onTrackingStateChanged(state: TrackingState) { /* Handle state change */ }
    override fun onObjectTrackingSuccess(result: ObjectTrackingResult) {
        // Access result.objectCode, result.objectCodes, result.position, result.rotation, result.confidence
    }
    override fun onObjectTrackingFailure(error: String) { /* Handle tracking failure */ }
}

onLocalizationFalsePositive, onObjectTrackingSuccess and onObjectTrackingFailure have default no-op implementations, so existing code keeps compiling when you upgrade.

5. Run a Localization Session

Results are delivered through the session you start, rather than being polled from the SDK. Supply frames through a FrameSource and handle the session callbacks:

val session = MultiSetSDK.localizationSession(myFrameSource, LocalizationMode.MULTI_FRAME).apply {
    onLocalizationSuccess = { result ->
        // Apply result.position and result.rotation to your scene
    }
    onLocalizationFailure = { error ->
        // error.kind is SERVER, AUTH, NETWORK or TRANSIENT
    }
    onLocalizationFalsePositive = { info ->
        // Discarded as inconsistent with the device trajectory
    }
}

session.start()

Call session.stop() when you tear the scene down. It cancels both the scheduled capture and any request already in flight, so a late response cannot re-apply a pose after you have stopped.

See FrameSource and Sessions for the full session API and a worked FrameSource implementation.

Requirements

• Android API Level 28+ (Android 9.0)

• Target SDK 36

• Java 17

• Kotlin 2.2.0+

• Gradle 8.11.1 with AGP 8.10.0

• Camera permission and internet connectivity

• ARCore compatible device (only if you render AR with the sample app's approach)

License

Copyright (c) 2026 MultiSet AI. All rights reserved. Licensed under the MultiSet License. For license details, visit www.multiset.ai.