ARLocalizationView

Overview

The ARLocalizationView provides AR-based localization using the MultiSetSDK framework. This view supports both single-frame and multi-frame localization modes, capturing frames from the AR camera and sending them to the MultiSet backend for pose estimation.

Description

The AR localization view handles both localization modes:

Single-Frame Localization is ideal for scenarios where:

• Quick localization is needed

• The environment has distinct visual features

• Network bandwidth is limited

• Lower accuracy is acceptable

Multi-Frame Localization is ideal for scenarios where:

• Higher accuracy is required

• The environment has repetitive or sparse visual features

• The user can move the device slightly during capture

• More robust localization is needed

The view handles:

• AR session management using ARKit and RealityKit

• Frame capture and processing (single or multi-frame)

• Localization API requests

• Pose calculation and gizmo positioning

• Mesh visualization

• Background localization

• GPS hint integration

---

How It Works

1. Initialization Flow

ARLocalizationView appears
  └─> setupCallbacks()
        └─> Configure SDK callbacks for results
              └─> startGpsUpdates() (if GPS enabled)
                    └─> startAutoLocalizationIfNeeded()

2. Localization Flow

User taps Capture / Auto-localization triggers
  └─> MultiSet.shared.localize()
        └─> SDK captures frame(s)
              └─> [Multi-frame: Loop capture with intervals]
                    └─> Send to API
                          └─> onLocalizationSuccess / onLocalizationFailure
                                └─> Update gizmo position
                                      └─> Load mesh visualization

---

Key Features

Auto-Localization

When MultiSetConfig.autoLocalize is enabled, localization starts automatically after the AR session is ready (with a 1.5 second delay to ensure tracking stability).

Background Localization

When MultiSetConfig.backgroundLocalization is enabled, the SDK periodically sends localization requests to refine positioning at the interval specified by bgLocalizationDurationSeconds.

Relocalization

When MultiSetConfig.relocalization is enabled, automatic relocalization is triggered when AR tracking state becomes paused or stopped.

GPS Hint

When MultiSetConfig.passGeoPose is enabled, GPS coordinates are captured and sent as a hint to improve localization accuracy for large-scale maps.

Mesh Visualization

When MultiSetConfig.meshVisualization is enabled, a 3D mesh overlay is rendered after successful localization.

Multi-Frame Capture

For multi-frame mode, captures multiple frames (configured via numberOfFrames) with intervals between captures (configured via frameCaptureIntervalMs). Each frame includes:

• Image data (JPEG compressed)

• Camera position (X, Y, Z)

• Camera rotation (quaternion)

Visual Feedback

During multi-frame capture, an animated phone icon guides users to move their device, ensuring diverse viewpoints for better localization.

---

SDK Methods Used

Localization Control

Method	Description
MultiSet.shared.localize()	Triggers the localization process
MultiSet.shared.stopLocalization()	Stops ongoing localization
MultiSet.shared.clearMesh()	Removes mesh visualization
MultiSet.shared.setLocalizationMode(_:)	Changes localization mode at runtime
MultiSet.shared.setQueryMode(_:)	Changes the single-frame matching pipeline (VPS-1 or VPS-2)
MultiSet.shared.resetPoseConsistencyReference()	Forgets the reference pose the false-positive check compares against

GPS Control

Method	Description
MultiSet.shared.startGpsUpdates()	Starts GPS location updates
MultiSet.shared.stopGpsUpdates()	Stops GPS location updates

State Properties

Property	Type	Description
MultiSet.shared.isLocalizing	Bool	Whether localization is in progress
MultiSet.shared.hasLocalized	Bool	Whether localization has succeeded at least once
MultiSet.shared.isShowingOverlay	Bool	Whether API loading overlay should be shown
MultiSet.shared.isCapturingFrames	Bool	Whether frames are being captured
MultiSet.shared.config	MultiSetConfig?	Current SDK configuration

---

State Management

View State Properties

Property	Type	Description
isTrackingNormal	Bool	Whether AR tracking state is normal
hasLocalized	Bool	Whether localization has succeeded
isLocalizing	Bool	Polled from SDK - localization in progress
isShowingOverlay	Bool	Polled from SDK - showing API overlay
isCapturingFrames	Bool	Polled from SDK - capturing frames
showCloseConfirmation	Bool	Controls close confirmation alert
showToast	Bool	Controls toast message visibility

State Polling

The view polls SDK state every 0.1 seconds to update UI:

let stateTimer = Timer.publish(every: 0.1, on: .main, in: .common).autoconnect()

.onReceive(stateTimer) { _ in
    isLocalizing = MultiSet.shared.isLocalizing
    isShowingOverlay = MultiSet.shared.isShowingOverlay
    isCapturingFrames = MultiSet.shared.isCapturingFrames
}

---

Callbacks

onLocalizationSuccess

Handles successful localization result:

sdkDelegate.onLocalizationSuccess = { result in
    hasLocalized = true
    if MultiSet.shared.config?.showAlerts == true {
        showToastMessage("Localization successful", success: true)
    }
}

onLocalizationFailure

Handles localization failure:

sdkDelegate.onLocalizationFailure = { error in
    if MultiSet.shared.config?.showAlerts == true {
        showToastMessage(error, success: false)
    }
}

onLocalizationFalsePositive

Handles a response the pose consistency check discarded. This fires instead of onLocalizationSuccess, and only when poseConsistencyCheck is enabled. The AR scene is left untouched, so the gizmo and mesh keep the last accepted pose:

sdkDelegate.onLocalizationFalsePositive = { info in
    showFalsePositiveAlert = true
    print("Discarded: \(info.summary)")
}

The sample view shows an alert with the jump distance and the threshold that was in force. After several consecutive rejections it also offers a reset, which calls MultiSet.shared.resetPoseConsistencyReference() so the next result is trusted and becomes the new reference.

There is no automatic fast retry after a false positive, because localizing again from the same spot tends to reproduce the same wrong match. Prompt the user to move to a more distinctive viewpoint first. See MultiSetCallback.

onTrackingChanged

Handles AR tracking state changes:

sdkDelegate.onTrackingChanged = { state in
    isTrackingNormal = (state == .tracking)
}

onMeshLoaded

Handles mesh loading completion:

sdkDelegate.onMeshLoaded = { mapId in
    print("Mesh loaded for map: \(mapId)")
}

---

Actions

startLocalization()

Manually triggers the localization process:

private func startLocalization() {
    MultiSet.shared.localize()
}

resetWorldOrigin()

Resets the AR world origin and clears current localization:

private func resetWorldOrigin() {
    MultiSet.shared.stopLocalization()
    MultiSet.shared.clearMesh()
    hasLocalized = false
    showToastMessage("World origin reset", success: true)

    // Restart auto-localization if enabled
    startAutoLocalizationIfNeeded()
}

cleanup()

Cleans up resources when view disappears:

private func cleanup() {
    MultiSet.shared.stopLocalization()
    MultiSet.shared.stopGpsUpdates()
}

---

Configuration

The view reads configuration from MultiSet.shared.config:

// Example configuration settings
config.localizationMode = .multiFrame       // or .singleFrame
config.autoLocalize = true
config.backgroundLocalization = true
config.bgLocalizationDurationSeconds = 30.0
config.relocalization = true
config.numberOfFrames = 4                   // For multi-frame
config.frameCaptureIntervalMs = 500         // For multi-frame
config.confidenceCheck = true
config.confidenceThreshold = 0.3
config.firstLocalizationUntilSuccess = true
config.showAlerts = true
config.meshVisualization = true
config.passGeoPose = false
config.geoCoordinatesInResponse = false
config.imageQuality = 90

---

Usage Example

Basic Usage

import SwiftUI
import MultiSetSDK

struct MyARView: View {
    @ObservedObject var sdkDelegate: MultiSetSDKDelegate
    let localizationMode: LocalizationMode

    var body: some View {
        ZStack {
            // AR View from SDK
            MultiSetARView()
                .ignoresSafeArea()

            // Capture button
            VStack {
                Spacer()
                Button("Localize") {
                    MultiSet.shared.localize()
                }
                .padding()
            }
        }
        .onAppear {
            setupCallbacks()
        }
    }

    private func setupCallbacks() {
        sdkDelegate.onLocalizationSuccess = { result in
            print("Localized at: \(result.position)")
        }
    }
}

Navigating from LandingView

NavigationStack {
    // ... LandingView content ...
}
.navigationDestination(isPresented: $navigateToAR) {
    ARLocalizationView(
        sdkDelegate: sdkDelegate,
        localizationMode: selectedMode
    )
    .navigationBarBackButtonHidden(true)
}

---

Comparison: Single-Frame vs Multi-Frame

Feature	Single-Frame	Multi-Frame
Frames captured	1	4-6 (configurable)
Capture time	Instant	~2-3 seconds
Accuracy	Good	Better
Network usage	Lower	Higher
Movement during capture	Not required	Recommended
Best for	Quick localization	Precise positioning
Visual feedback	Simple overlay	Animated phone guide

---

Lifecycle

onAppear

• Sets up SDK callbacks for results

• Starts GPS updates if enabled

• Triggers auto-localization if enabled

onDisappear

• Stops localization

• Stops GPS updates

• Cleans up resources

Timer

• Polls SDK state every 0.1 seconds for UI updates

---

Overlay Components

LocalizationOverlay

Shows during frame capture with animated phone image:

• Semi-transparent black background

• Background AR illustration

• Animated phone icon (moving left to right)

• "Hold steady and scan your surroundings" instruction

• Status text showing capture progress

APILoadingOverlay

Shows while waiting for server response:

• Semi-transparent black background

• Circular progress spinner

• "Localizing..." text

---

Related

• LandingView

• MultiSetConfig

• MultiSetCallback

• LocalizationResult