SingleFrameARActivity

Overview

The SingleFrameARActivity provides AR-based localization using a single camera frame. This activity captures one frame from the AR camera and sends it to the MultiSet backend for pose estimation.

Description

Single-frame localization is ideal for scenarios where:

• Quick localization is needed

• The environment has distinct visual features

• Network bandwidth is limited

• Lower accuracy is acceptable

The activity handles:

• AR session management using ARCore and Sceneform

• Single frame capture and processing

• Localization API requests

• Pose calculation and gizmo positioning

• Mesh visualization

• Background localization

• GPS hint integration

---

How It Works

1. Initialization Flow

onCreate()
  └─> setupAR()
        └─> configureSession()
              └─> addGizmoToScene()
                    └─> initializeMeshComponents()
                          └─> initializeLocalization()

2. Localization Flow

localizeFrame()
  └─> startLocalization(frame)
        └─> captureAndProcessFrame()
              └─> sendLocalizationRequest()
                    └─> handleLocalizationResponse()
                          └─> updateGizmoPosition()
                                └─> loadMeshAfterLocalization()

---

Key Features

Auto-Localization

When LocalizationConfig.autoLocalize is enabled, localization starts automatically after the AR session is ready.

Background Localization

When LocalizationConfig.backgroundLocalization is enabled, the activity periodically sends localization requests to refine positioning.

Relocalization

When LocalizationConfig.relocalization is enabled, automatic relocalization is triggered when AR tracking state becomes PAUSED or STOPPED.

GPS Hint

When LocalizationConfig.enableGeoHint is enabled, GPS coordinates are captured and sent as a hint to improve localization accuracy for large-scale maps.

Mesh Visualization

When LocalizationConfig.enableMeshVisualization is enabled, a 3D mesh overlay is rendered after successful localization.

---

Public Methods

localizeFrame()

Manually triggers the localization process.

Declaration:

fun localizeFrame()

Description: Captures a single frame from the AR camera and sends it to the MultiSet API for localization. This is the primary method to call for manual localization triggers.

Usage:

binding.captureButton.setOnClickListener {
    localizeFrame()
}

---

resetWorldOrigin()

Resets the AR world origin and clears the current localization.

Declaration:

fun resetWorldOrigin()

Description:

• Pauses and reconfigures the AR session

• Cancels any ongoing background localization

• Removes the mesh visualization

• Resets the gizmo position

• Optionally restarts auto-localization

---

Localization Process

Frame Capture

Captures and processes a single frame:

1. Acquires the camera image from ARCore

2. Converts YUV to RGB bitmap

3. Applies orientation correction for portrait mode

4. Compresses to JPEG with configurable quality

5. Extracts camera intrinsics (fx, fy, px, py)

API Request

Sends the captured frame to the MultiSet API:

• Includes camera intrinsics

• Includes map code or map set code

• Optionally includes GPS hint

• Optionally requests geo-coordinates in response

Response Handling

Processes the localization response:

1. Checks if pose was found

2. Validates confidence threshold (if enabled)

3. Calculates final pose

4. Updates gizmo position

5. Loads mesh visualization

6. Notifies callback

---

State Management

Property	Type	Description
isLocalizing	Boolean	Whether localization is in progress
isFirstLocalization	Boolean	Whether this is the first localization attempt
isBackgroundLocalizationRequest	Boolean	Whether current request is from background
lastTrackingState	TrackingState	Previous AR tracking state
isSessionConfigured	Boolean	Whether AR session has been configured

---

Configuration

The activity reads configuration from LocalizationConfig:

LocalizationConfig.autoLocalize = true
LocalizationConfig.backgroundLocalization = true
LocalizationConfig.backgroundLocalizationIntervalSeconds = 30f
LocalizationConfig.relocalization = true
LocalizationConfig.confidenceCheck = false
LocalizationConfig.confidenceThreshold = 0.3f
LocalizationConfig.firstLocalizationUntilSuccess = true
LocalizationConfig.showAlerts = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.enableGeoHint = false
LocalizationConfig.includeGeoCoordinatesInResponse = false
LocalizationConfig.imageQuality = 90

---

Usage Example

Launching the Activity

// From MainActivity after authentication
val intent = Intent(this, SingleFrameARActivity::class.java)
startActivity(intent)

Configuring Before Launch

// Customize localization behavior
LocalizationConfig.autoLocalize = true
LocalizationConfig.backgroundLocalization = true
LocalizationConfig.enableMeshVisualization = true
LocalizationConfig.showAlerts = true
LocalizationConfig.validate()

// Launch activity
val intent = Intent(this, SingleFrameARActivity::class.java)
startActivity(intent)

---

Lifecycle

onCreate()

• Validates SDK authentication

• Inflates layout

• Loads and validates configuration

• Sets up AR and UI

onDestroy()

• Stops GPS status updates

• Cancels background localization job

• Stops GPS handler

• Removes mesh visualization

---

Related

• MainActivity

• MultiFrameARActivity

• LocalizationConfig