> For the complete documentation index, see [llms.txt](https://docs.multiset.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.multiset.ai/native-sdk/android-native/sample-activities/multisetlocalizationactivity.md). # MultiSetLocalizationActivity The unified AR localization activity on Android, covering single and multi frame modes, auto and background localization, relocalization after a tracking break, hints and mesh display. ## Overview The `MultiSetLocalizationActivity` is a unified AR localization activity that supports both single-frame and multi-frame localization modes. The mode is selected at launch time via an intent extra, and defaults to single frame. ## Description This is the unified AR localization activity that handles both single-frame and multi-frame localization modes in a single implementation. **Single-frame mode** is ideal for: * Quick localization with low latency * Environments with distinct visual features * Limited network bandwidth scenarios **Multi-frame mode** is ideal for: * Higher accuracy requirements * Environments with repetitive or sparse visual features * When the user can move the device slightly during capture The activity handles: * AR session management using ARCore and Sceneform * Reporting ARCore tracking to the SDK, and relocalization after a tracking break or a return from the background * Run progress: capture and server-wait overlays and the round status card * Pose calculation and gizmo positioning * 3D mesh visualization with radial reveal animation, a mesh toggle and a loading strip * Background localization * GPS hint integration * False-positive alerts with a Reset option *** ## Launching the Activity The localization mode is passed via intent extra: ```kotlin val intent = Intent(this, MultiSetLocalizationActivity::class.java) intent.putExtra( MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE, LocalizationMode.SINGLE_FRAME.name // the default; or LocalizationMode.MULTI_FRAME.name ) startActivity(intent) ``` ### Intent Extras | Extra | Type | Description | | ------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------- | | `EXTRA_LOCALIZATION_MODE` | `String` | The localization mode name: `"SINGLE_FRAME"` or `"MULTI_FRAME"`. Missing or unknown values fall back to single frame. | *** ## Key Features ### Run Progress A run the user is waiting on (a tap, the first fix and its silent retries) covers the scene with a capture overlay and a frame count ("Capturing image 2 of 4"), then a server-wait overlay. Every run, background ones included, also shows on the round status card at the top left. Both are driven by the session's [run stages](/native-sdk/android-native/api-reference/framesource.md#run-stages). ### Auto-Localization When `LocalizationConfig.autoLocalize` is enabled, localization starts automatically once ARCore reports `TRACKING`. ### Background Localization When `LocalizationConfig.backgroundLocalization` is enabled, the session re-localizes at the configured interval to refine positioning. Background runs show only on the status card, and the Localize button stays in place, disabled, while one runs. ### Relocalization When `LocalizationConfig.relocalization` is enabled, a tracking break after a fix, or a return from the background, hides the gizmo and map mesh. Once tracking has held for 1 s the SDK re-localizes, retrying every second without alerts, and the map reappears with the new fix. See [Tracking breaks](/native-sdk/android-native/api-reference/framesource.md#tracking-breaks-and-the-return-from-the-background). ### Query Mode `LocalizationConfig.queryMode` selects VPS-1 or VPS-2 in either mode. ### GPS Hint When `LocalizationConfig.enableGeoHint` is enabled, the activity asks for the location permission, feeds the platform location fix to `MultiSetSDK.geoHintProvider` (a fix at most 2 minutes old) and lights the GPS indicator while a fix is available. ### Localization Hints Before sending a query, the session applies optional localization hints from `LocalizationConfig` to narrow the search space, in both modes: * `hintMapCodes`: restricts a **mapSet** query to a subset of maps (ignored for single-map localization). * `hintPosition` (`"x,y,z"`) and `hintFloorHeight` (`"floor,ceiling"`): bias the search around a known area. * `hintRadius` and `use2DFiltering`: spatial filters applied only when a geo hint or `hintPosition` is present. See [LocalizationConfig → Localization Hints](/native-sdk/android-native/api-reference/localizationconfig.md#localization-hints) for the full reference. ### Mesh Visualization When `LocalizationConfig.enableMeshVisualization` is enabled, a 3D mesh overlay with radial reveal animation is rendered after successful localization. A loading strip shows while the mesh downloads, and a toggle at the bottom left hides or shows it. A fix on the map already shown only restarts the reveal. In a MapSet, each localized map adds its own mesh and the meshes already shown stay, so the set fills in as the user walks through it. ### Multi-Frame Capture In multi-frame mode, captures multiple frames (configurable via `LocalizationConfig.numberOfFrames`) with intervals between captures (configurable via `LocalizationConfig.frameCaptureIntervalMs`). Each frame includes: * Image data (JPEG compressed) * Camera position (X, Y, Z) * Camera rotation (quaternion) *** ## Session Wiring The activity does not implement a capture loop of its own. It builds an `ArFrameSource`, creates a `LocalizationSession` from the SDK, applies `LocalizationConfig` to it, and attaches callbacks. The SDK owns scheduling, retry, background re-localization, recovery and confidence gating. ```kotlin val frameSource = ArFrameSource(arFragment, this, ImageProcessor()) val session = MultiSetSDK.localizationSession(frameSource, localizationMode).apply { backgroundLocalization = LocalizationConfig.backgroundLocalization bgLocalizationDurationMs = (LocalizationConfig.backgroundLocalizationIntervalSeconds * 1000).toLong() relocalization = LocalizationConfig.relocalization firstLocalizationUntilSuccess = LocalizationConfig.firstLocalizationUntilSuccess numberOfFrames = LocalizationConfig.numberOfFrames frameCaptureIntervalMs = LocalizationConfig.frameCaptureIntervalMs confidenceCheck = LocalizationConfig.confidenceCheck confidenceThreshold = LocalizationConfig.confidenceThreshold imageQuality = LocalizationConfig.imageQuality queryMode = LocalizationConfig.queryMode poseConsistencyCheck = LocalizationConfig.poseConsistencyCheck poseConsistencyThreshold = LocalizationConfig.poseConsistencyThreshold poseConsistencyYawThreshold = LocalizationConfig.poseConsistencyYawThreshold hintMapCodes = LocalizationConfig.hintMapCodes hintPosition = LocalizationConfig.hintPosition hintFloorHeight = LocalizationConfig.hintFloorHeight hintRadius = LocalizationConfig.hintRadius use2DFiltering = LocalizationConfig.use2DFiltering enableGeoHint = LocalizationConfig.enableGeoHint includeGeoCoordinatesInResponse = LocalizationConfig.includeGeoCoordinatesInResponse onStageChanged = { stage, isBackground -> /* overlays and the status card */ } onRecoveryPhaseChanged = { phase, needsRealignment -> /* hide the map while needsRealignment */ } onLocalizationSuccess = { result -> /* place the gizmo and mesh */ } onLocalizationFailure = { error -> /* error.kind classifies the failure */ } onLocalizationFalsePositive = { info -> /* result discarded, scene untouched */ } } ``` ### Starting capture Capture is deferred until ARCore actually reports `TRACKING`. Starting before then fails with "Failed to capture frame". The activity sets a `pendingAutoStart` flag and calls `session.start()` from its scene update once the camera is tracking. ### The Localize button Calls `session.start()` directly, since the user only sees the button once the scene is live. The button is hidden under a run's overlay and disabled while `session.isBusy`. ### The Reset button Returns the scene to its initial state: ```kotlin session.resetSession() // clears the anchor and recovery, re-arms the silent first-fix retry meshRenderer.removeMesh() gizmoNode.hide() ``` {% hint style="info" %} `resetSession()` drops a request already in flight as well as the scheduled one. Without that, a response landing just after Reset would put the pre-reset pose straight back into the scene. {% endhint %} ### Tracking breaks From its scene update the activity reports ARCore's tracking state to the session on the first frame, and again whenever the state or its failure reason changes, through `reportArTracking(state, reason)` in `ar/ArTrackingSignals.kt`. That reports the state, then maps `CAMERA_UNAVAILABLE` to `notifyTrackingInterrupted()` and `STOPPED` or `BAD_STATE` to `notifyWorldOriginReset()`. ### The return from the background ARCore cannot reset its world frame, and a resumed session may no longer match the room. So in `onPause`, before `super.onPause()`, the activity marks the frames taken so far as stale, reports `PAUSED` and calls `notifyWorldOriginReset()`. The map stays hidden after the return until the SDK has re-localized. *** ## State Management | Property | Type | Description | | --------------------- | ------------------ | --------------------------------------------------------------------------------------- | | `isSessionConfigured` | `Boolean` | Whether the ARCore session has been configured | | `sessionStarted` | `Boolean` | Whether capture is currently running | | `pendingAutoStart` | `Boolean` | Auto-localization is waiting for ARCore to reach `TRACKING` | | `lastTrackingState` | `TrackingState?` | Previous AR tracking state; `null` until the first frame, so that frame is reported too | | `localizationMode` | `LocalizationMode` | Mode selected via the launch intent | Capture state such as whether a query is in flight, whether this is the first localization, and the frames captured for a multi-frame query is owned by `LocalizationSession` inside the SDK. The activity reads `isBusy`, `stage` and `needsRealignment` from it. *** ## Configuration The activity reads configuration from `LocalizationConfig`: ```kotlin LocalizationConfig.autoLocalize = true LocalizationConfig.backgroundLocalization = true LocalizationConfig.backgroundLocalizationIntervalSeconds = 30f // valid range 10 - 180 LocalizationConfig.relocalization = true LocalizationConfig.numberOfFrames = 4 // multi-frame only LocalizationConfig.frameCaptureIntervalMs = 500L // multi-frame only LocalizationConfig.confidenceCheck = true LocalizationConfig.confidenceThreshold = 0.3f // valid range 0.2 - 0.8 LocalizationConfig.firstLocalizationUntilSuccess = true LocalizationConfig.showAlerts = true LocalizationConfig.enableMeshVisualization = true LocalizationConfig.enableGeoHint = false LocalizationConfig.includeGeoCoordinatesInResponse = false // False-positive rejection (off by default) LocalizationConfig.poseConsistencyCheck = false LocalizationConfig.poseConsistencyThreshold = 4f // metres, valid range 1.5 - 15 LocalizationConfig.poseConsistencyYawThreshold = 25f // degrees, valid range 10 - 60 // Search strategy, single-frame and multi-frame LocalizationConfig.queryMode = QueryMode.VPS1 // Localization hints (optional), narrow the search before the query LocalizationConfig.hintMapCodes = emptyList() // mapSet: subset of maps LocalizationConfig.hintPosition = "" // "x,y,z" LocalizationConfig.hintFloorHeight = "" // "floor,ceiling" LocalizationConfig.hintRadius = 25 // meters (1 - 100) LocalizationConfig.use2DFiltering = false LocalizationConfig.imageQuality = 90 ``` *** ## Usage Examples ### Single-Frame Localization ```kotlin // Configure for single-frame LocalizationConfig.autoLocalize = true LocalizationConfig.enableMeshVisualization = true LocalizationConfig.validate() // Launch activity val intent = Intent(this, MultiSetLocalizationActivity::class.java) intent.putExtra( MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE, LocalizationMode.SINGLE_FRAME.name ) startActivity(intent) ``` ### Multi-Frame Localization ```kotlin // Configure for multi-frame LocalizationConfig.numberOfFrames = 5 LocalizationConfig.frameCaptureIntervalMs = 600L LocalizationConfig.autoLocalize = true LocalizationConfig.backgroundLocalization = true LocalizationConfig.enableMeshVisualization = true LocalizationConfig.validate() // Launch activity val intent = Intent(this, MultiSetLocalizationActivity::class.java) intent.putExtra( MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE, LocalizationMode.MULTI_FRAME.name ) startActivity(intent) ``` *** ## Related * [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md) * [MainActivity](/native-sdk/android-native/sample-activities/mainactivity.md) * [LocalizationConfig](/native-sdk/android-native/api-reference/localizationconfig.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.multiset.ai/native-sdk/android-native/sample-activities/multisetlocalizationactivity.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.