> For the complete documentation index, see [llms.txt](https://docs.multiset.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.multiset.ai/native-sdk/android-native/sample-activities/multisetlocalizationactivity.md).

# MultiSetLocalizationActivity

The unified AR localization activity on Android, covering single and multi frame modes, auto and background localization, hints and mesh display.

## Overview

The `MultiSetLocalizationActivity` is a unified AR localization activity that supports both single-frame and multi-frame localization modes. The mode is selected at launch time via an intent extra.

## Description

This is the unified AR localization activity that handles both single-frame and multi-frame localization modes in a single implementation.

**Single-frame mode** is ideal for:

* Quick localization with low latency
* Environments with distinct visual features
* Limited network bandwidth scenarios

**Multi-frame mode** is ideal for:

* Higher accuracy requirements
* Environments with repetitive or sparse visual features
* When the user can move the device slightly during capture

The activity handles:

* AR session management using ARCore and Sceneform
* Single-frame and multi-frame capture and processing
* Localization animation with visual feedback
* Localization API requests
* Pose calculation and gizmo positioning
* 3D mesh visualization with radial reveal animation
* Background localization
* GPS hint integration

***

## Launching the Activity

The localization mode is passed via intent extra:

```kotlin
val intent = Intent(this, MultiSetLocalizationActivity::class.java)
intent.putExtra(
    MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE,
    LocalizationMode.MULTI_FRAME.name  // or LocalizationMode.SINGLE_FRAME.name
)
startActivity(intent)
```

### Intent Extras

| Extra                     | Type     | Description                                                     |
| ------------------------- | -------- | --------------------------------------------------------------- |
| `EXTRA_LOCALIZATION_MODE` | `String` | The localization mode name: `"SINGLE_FRAME"` or `"MULTI_FRAME"` |

***

## Key Features

### Localization Animation

Both modes display an animated phone icon with visual feedback during capture, guiding users to move their device for better coverage.

### Auto-Localization

When `LocalizationConfig.autoLocalize` is enabled, localization starts automatically after the AR session is ready.

### Background Localization

When `LocalizationConfig.backgroundLocalization` is enabled, the activity periodically sends localization requests to refine positioning.

### Relocalization

When `LocalizationConfig.relocalization` is enabled, automatic relocalization is triggered when AR tracking state becomes PAUSED or STOPPED.

### GPS Hint

When `LocalizationConfig.enableGeoHint` is enabled, GPS coordinates are captured and sent as a hint to improve localization accuracy for large-scale maps.

### Localization Hints

Before sending a query, the activity reads optional localization hints from `LocalizationConfig` to narrow the search space:

* `hintMapCodes`: restricts a **mapSet** query to a subset of maps (ignored for single-map localization).
* `hintPosition` (`"x,y,z"`) and `hintFloorHeight` (`"floor,ceiling"`): bias the search around a known area.
* `hintRadius` and `use2DFiltering`: spatial filters applied only when a geo hint or `hintPosition` is present.

See [LocalizationConfig → Localization Hints](/native-sdk/android-native/api-reference/localizationconfig.md#localization-hints) for the full reference.

### Mesh Visualization

When `LocalizationConfig.enableMeshVisualization` is enabled, a 3D mesh overlay with radial reveal animation is rendered after successful localization.

### Multi-Frame Capture

In multi-frame mode, captures multiple frames (configurable via `LocalizationConfig.numberOfFrames`) with intervals between captures (configurable via `LocalizationConfig.frameCaptureIntervalMs`). Each frame includes:

* Image data (JPEG compressed)
* Camera position (X, Y, Z)
* Camera rotation (quaternion)

***

## Session Wiring

The activity does not implement a capture loop of its own. It builds an `ArFrameSource`, creates a `LocalizationSession` from the SDK, applies `LocalizationConfig` to it, and attaches callbacks. The SDK owns scheduling, retry, background re-localization and confidence gating.

```kotlin
val frameSource = ArFrameSource(arFragment, this, ImageProcessor())

val session = MultiSetSDK.localizationSession(frameSource, localizationMode).apply {
    backgroundLocalization = LocalizationConfig.backgroundLocalization
    numberOfFrames = LocalizationConfig.numberOfFrames
    confidenceCheck = LocalizationConfig.confidenceCheck
    confidenceThreshold = LocalizationConfig.confidenceThreshold
    queryMode = LocalizationConfig.queryMode
    poseConsistencyCheck = LocalizationConfig.poseConsistencyCheck
    poseConsistencyThreshold = LocalizationConfig.poseConsistencyThreshold

    onLocalizationRequested = { /* show scanning overlay */ }
    onLocalizationSuccess = { result -> /* place the gizmo and mesh */ }
    onLocalizationFailure = { error -> /* error.kind classifies the failure */ }
    onLocalizationFalsePositive = { info -> /* result discarded, scene untouched */ }
}
```

### Starting capture

Capture is deferred until ARCore actually reports `TRACKING`. Starting before then fails with "Failed to capture frame". The activity sets a `pendingAutoStart` flag and calls `session.start()` from its scene update once the camera is tracking.

### The Localize button

Calls `session.start()` directly, since the user only sees the button once the scene is live.

### The Reset button

Returns the scene to its initial state:

```kotlin
session.stop()                 // cancels scheduled AND in-flight work
session.resetPoseReference()   // the next accepted fix becomes the new reference
meshRenderer.removeMesh()
gizmoNode.hide()
```

{% hint style="info" %}
`stop()` cancels a request already in flight as well as the scheduled one. Without that, a response landing just after Reset would put the pre-reset pose straight back into the scene.
{% endhint %}

### Tracking loss

When ARCore drops out of `TRACKING`, the activity calls `session.notifyTrackingInterrupted()` so the pose consistency gate stops trusting a reference the tracker no longer backs. If `relocalization` is enabled it then restarts the session.

***

## State Management

| Property              | Type               | Description                                                 |
| --------------------- | ------------------ | ----------------------------------------------------------- |
| `isSessionConfigured` | `Boolean`          | Whether the ARCore session has been configured              |
| `sessionStarted`      | `Boolean`          | Whether capture is currently running                        |
| `pendingAutoStart`    | `Boolean`          | Auto-localization is waiting for ARCore to reach `TRACKING` |
| `lastTrackingState`   | `TrackingState`    | Previous AR tracking state                                  |
| `localizationMode`    | `LocalizationMode` | Mode selected via the launch intent                         |

Capture state such as whether a query is in flight, whether this is the first localization, and the frames captured for a multi-frame query is owned by `LocalizationSession` inside the SDK and is not exposed to the activity.

***

## Configuration

The activity reads configuration from `LocalizationConfig`:

```kotlin
LocalizationConfig.autoLocalize = true
LocalizationConfig.backgroundLocalization = true
LocalizationConfig.backgroundLocalizationIntervalSeconds = 30f
LocalizationConfig.relocalization = true
LocalizationConfig.numberOfFrames = 4                    // multi-frame only
LocalizationConfig.frameCaptureIntervalMs = 500L         // multi-frame only
LocalizationConfig.confidenceCheck = true
LocalizationConfig.confidenceThreshold = 0.3f            // valid range 0.2 - 0.8
LocalizationConfig.firstLocalizationUntilSuccess = true
LocalizationConfig.showAlerts = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.enableGeoHint = false
LocalizationConfig.includeGeoCoordinatesInResponse = false

// False-positive rejection
LocalizationConfig.poseConsistencyCheck = true
LocalizationConfig.poseConsistencyThreshold = 10f        // metres, valid range 3 - 30

// Search strategy (single-frame only; multi-frame always runs VPS-1)
LocalizationConfig.queryMode = QueryMode.VPS1

// Localization hints (optional), narrow the search before the query
LocalizationConfig.hintMapCodes = emptyList()            // mapSet: subset of maps
LocalizationConfig.hintPosition = ""                     // "x,y,z"
LocalizationConfig.hintFloorHeight = ""                  // "floor,ceiling"
LocalizationConfig.hintRadius = 25                       // meters (1 - 100)
LocalizationConfig.use2DFiltering = false

LocalizationConfig.imageQuality = 90
```

***

## Usage Examples

### Single-Frame Localization

```kotlin
// Configure for single-frame
LocalizationConfig.autoLocalize = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.validate()

// Launch activity
val intent = Intent(this, MultiSetLocalizationActivity::class.java)
intent.putExtra(
    MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE,
    LocalizationMode.SINGLE_FRAME.name
)
startActivity(intent)
```

### Multi-Frame Localization

```kotlin
// Configure for multi-frame
LocalizationConfig.numberOfFrames = 5
LocalizationConfig.frameCaptureIntervalMs = 600L
LocalizationConfig.autoLocalize = true
LocalizationConfig.backgroundLocalization = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.validate()

// Launch activity
val intent = Intent(this, MultiSetLocalizationActivity::class.java)
intent.putExtra(
    MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE,
    LocalizationMode.MULTI_FRAME.name
)
startActivity(intent)
```

***

## Related

* [FrameSource and Sessions](/native-sdk/android-native/api-reference/framesource.md)
* [MainActivity](/native-sdk/android-native/sample-activities/mainactivity.md)
* [LocalizationConfig](/native-sdk/android-native/api-reference/localizationconfig.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.multiset.ai/native-sdk/android-native/sample-activities/multisetlocalizationactivity.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.