MultiSetLocalizationActivity

Overview

The MultiSetLocalizationActivity is a unified AR localization activity that supports both single-frame and multi-frame localization modes. The mode is selected at launch time via an intent extra.

Description

This is the unified AR localization activity that handles both single-frame and multi-frame localization modes in a single implementation.

Single-frame mode is ideal for:

• Quick localization with low latency

• Environments with distinct visual features

• Limited network bandwidth scenarios

Multi-frame mode is ideal for:

• Higher accuracy requirements

• Environments with repetitive or sparse visual features

• When the user can move the device slightly during capture

The activity handles:

• AR session management using ARCore and Sceneform

• Single-frame and multi-frame capture and processing

• Localization animation with visual feedback

• Localization API requests

• Pose calculation and gizmo positioning

• 3D mesh visualization with radial reveal animation

• Background localization

• GPS hint integration

---

Launching the Activity

The localization mode is passed via intent extra:

val intent = Intent(this, MultiSetLocalizationActivity::class.java)
intent.putExtra(
    MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE,
    LocalizationMode.MULTI_FRAME.name  // or LocalizationMode.SINGLE_FRAME.name
)
startActivity(intent)

Intent Extras

Extra	Type	Description
EXTRA_LOCALIZATION_MODE	String	The localization mode name: "SINGLE_FRAME" or "MULTI_FRAME"

---

Key Features

Localization Animation

Both modes display an animated phone icon with visual feedback during capture, guiding users to move their device for better coverage.

Auto-Localization

When LocalizationConfig.autoLocalize is enabled, localization starts automatically after the AR session is ready.

Background Localization

When LocalizationConfig.backgroundLocalization is enabled, the activity periodically sends localization requests to refine positioning.

Relocalization

When LocalizationConfig.relocalization is enabled, automatic relocalization is triggered when AR tracking state becomes PAUSED or STOPPED.

GPS Hint

When LocalizationConfig.enableGeoHint is enabled, GPS coordinates are captured and sent as a hint to improve localization accuracy for large-scale maps.

Localization Hints

Before sending a query, the activity reads optional localization hints from LocalizationConfig to narrow the search space:

• hintMapCodes: restricts a mapSet query to a subset of maps (ignored for single-map localization).

• hintPosition ("x,y,z") and hintFloorHeight ("floor,ceiling"): bias the search around a known area.

• hintRadius and use2DFiltering: spatial filters applied only when a geo hint or hintPosition is present.

See LocalizationConfig → Localization Hints for the full reference.

Mesh Visualization

When LocalizationConfig.enableMeshVisualization is enabled, a 3D mesh overlay with radial reveal animation is rendered after successful localization.

Multi-Frame Capture

In multi-frame mode, captures multiple frames (configurable via LocalizationConfig.numberOfFrames) with intervals between captures (configurable via LocalizationConfig.frameCaptureIntervalMs). Each frame includes:

• Image data (JPEG compressed)

• Camera position (X, Y, Z)

• Camera rotation (quaternion)

---

Session Wiring

The activity does not implement a capture loop of its own. It builds an ArFrameSource, creates a LocalizationSession from the SDK, applies LocalizationConfig to it, and attaches callbacks. The SDK owns scheduling, retry, background re-localization and confidence gating.

val frameSource = ArFrameSource(arFragment, this, ImageProcessor())

val session = MultiSetSDK.localizationSession(frameSource, localizationMode).apply {
    backgroundLocalization = LocalizationConfig.backgroundLocalization
    numberOfFrames = LocalizationConfig.numberOfFrames
    confidenceCheck = LocalizationConfig.confidenceCheck
    confidenceThreshold = LocalizationConfig.confidenceThreshold
    queryMode = LocalizationConfig.queryMode
    poseConsistencyCheck = LocalizationConfig.poseConsistencyCheck
    poseConsistencyThreshold = LocalizationConfig.poseConsistencyThreshold

    onLocalizationRequested = { /* show scanning overlay */ }
    onLocalizationSuccess = { result -> /* place the gizmo and mesh */ }
    onLocalizationFailure = { error -> /* error.kind classifies the failure */ }
    onLocalizationFalsePositive = { info -> /* result discarded, scene untouched */ }
}

Starting capture

Capture is deferred until ARCore actually reports TRACKING. Starting before then fails with "Failed to capture frame". The activity sets a pendingAutoStart flag and calls session.start() from its scene update once the camera is tracking.

The Localize button

Calls session.start() directly, since the user only sees the button once the scene is live.

The Reset button

Returns the scene to its initial state:

session.stop()                 // cancels scheduled AND in-flight work
session.resetPoseReference()   // the next accepted fix becomes the new reference
meshRenderer.removeMesh()
gizmoNode.hide()

stop() cancels a request already in flight as well as the scheduled one. Without that, a response landing just after Reset would put the pre-reset pose straight back into the scene.

Tracking loss

When ARCore drops out of TRACKING, the activity calls session.notifyTrackingInterrupted() so the pose consistency gate stops trusting a reference the tracker no longer backs. If relocalization is enabled it then restarts the session.

---

State Management

Property	Type	Description
isSessionConfigured	Boolean	Whether the ARCore session has been configured
sessionStarted	Boolean	Whether capture is currently running
pendingAutoStart	Boolean	Auto-localization is waiting for ARCore to reach TRACKING
lastTrackingState	TrackingState	Previous AR tracking state
localizationMode	LocalizationMode	Mode selected via the launch intent

Capture state such as whether a query is in flight, whether this is the first localization, and the frames captured for a multi-frame query is owned by LocalizationSession inside the SDK and is not exposed to the activity.

---

Configuration

The activity reads configuration from LocalizationConfig:

LocalizationConfig.autoLocalize = true
LocalizationConfig.backgroundLocalization = true
LocalizationConfig.backgroundLocalizationIntervalSeconds = 30f
LocalizationConfig.relocalization = true
LocalizationConfig.numberOfFrames = 4                    // multi-frame only
LocalizationConfig.frameCaptureIntervalMs = 500L         // multi-frame only
LocalizationConfig.confidenceCheck = true
LocalizationConfig.confidenceThreshold = 0.3f            // valid range 0.2 - 0.8
LocalizationConfig.firstLocalizationUntilSuccess = true
LocalizationConfig.showAlerts = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.enableGeoHint = false
LocalizationConfig.includeGeoCoordinatesInResponse = false

// False-positive rejection
LocalizationConfig.poseConsistencyCheck = true
LocalizationConfig.poseConsistencyThreshold = 10f        // metres, valid range 3 - 30

// Search strategy (single-frame only; multi-frame always runs VPS-1)
LocalizationConfig.queryMode = QueryMode.VPS1

// Localization hints (optional), narrow the search before the query
LocalizationConfig.hintMapCodes = emptyList()            // mapSet: subset of maps
LocalizationConfig.hintPosition = ""                     // "x,y,z"
LocalizationConfig.hintFloorHeight = ""                  // "floor,ceiling"
LocalizationConfig.hintRadius = 25                       // meters (1 - 100)
LocalizationConfig.use2DFiltering = false

LocalizationConfig.imageQuality = 90

---

Usage Examples

Single-Frame Localization

// Configure for single-frame
LocalizationConfig.autoLocalize = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.validate()

// Launch activity
val intent = Intent(this, MultiSetLocalizationActivity::class.java)
intent.putExtra(
    MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE,
    LocalizationMode.SINGLE_FRAME.name
)
startActivity(intent)

Multi-Frame Localization

// Configure for multi-frame
LocalizationConfig.numberOfFrames = 5
LocalizationConfig.frameCaptureIntervalMs = 600L
LocalizationConfig.autoLocalize = true
LocalizationConfig.backgroundLocalization = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.validate()

// Launch activity
val intent = Intent(this, MultiSetLocalizationActivity::class.java)
intent.putExtra(
    MultiSetLocalizationActivity.EXTRA_LOCALIZATION_MODE,
    LocalizationMode.MULTI_FRAME.name
)
startActivity(intent)

---

Related

• FrameSource and Sessions

• MainActivity

• LocalizationConfig