# ObjectTrackingActivity

## Overview

The `ObjectTrackingActivity` is a dedicated AR activity for detecting and tracking pre-registered physical objects using the MultiSet SDK. Once an object is successfully tracked, its 3D mesh is fetched from the MultiSet platform and rendered in the AR scene with an animated glowing outline shader.

## Description

This activity is responsible for:

1. Establishing an AR session with ARCore and Sceneform
2. Fetching and rendering 3D object meshes from the MultiSet platform
3. Sending camera frames to the MultiSet Object Tracking API
4. Applying an animated outline-only shader to tracked object meshes
5. Managing background tracking and auto-retry behavior
6. Forwarding tracking results to the host application via `MultiSetCallback`

***

## Launching the Activity

Object codes can be passed either directly via an intent extra, or pre-configured on `ObjectTrackingConfig`.

```kotlin
// Option A: pass codes via Intent extra
val objectCodes = arrayOf("OBJ_001", "OBJ_002")
val intent = Intent(this, ObjectTrackingActivity::class.java)
intent.putExtra(ObjectTrackingActivity.EXTRA_OBJECT_CODES, objectCodes)
startActivity(intent)

// Option B: configure via ObjectTrackingConfig (codes are read automatically)
ObjectTrackingConfig.objectCodes = arrayOf("OBJ_001", "OBJ_002")
ObjectTrackingConfig.validate()
startActivity(Intent(this, ObjectTrackingActivity::class.java))
```

### Intent Extras

| Extra                | Type            | Description                             |
| -------------------- | --------------- | --------------------------------------- |
| `EXTRA_OBJECT_CODES` | `Array<String>` | Array of object codes to track (max 10) |

***

## Prerequisites

The SDK must be initialized and authenticated before launching this activity. If the SDK is not authenticated, the activity will finish immediately with a toast message.

```kotlin
// Ensure SDK is initialized before launching
MultiSetSDK.initialize(context, config, callback)

// Launch only after onAuthenticationSuccess()
override fun onAuthenticationSuccess() {
    val intent = Intent(this, ObjectTrackingActivity::class.java)
    intent.putExtra(ObjectTrackingActivity.EXTRA_OBJECT_CODES, arrayOf("OBJ_001"))
    startActivity(intent)
}
```

***

## Key Features

### Auto-Tracking

When `ObjectTrackingConfig.autoTracking` is `true`, tracking starts automatically once the AR session is ready and the device camera is in a stable tracking state. A 1-second stabilization delay ensures a good initial frame.

### Background Tracking

When `ObjectTrackingConfig.backgroundTracking` is `true`, the activity continues sending tracking requests at the configured interval (`bgTrackingDurationSeconds`) after the first successful track. This keeps the object pose updated over time.

### Manual Tracking Trigger

A capture button is visible in the AR view. Tapping it manually triggers a new tracking request regardless of auto or background settings.

### Animated Outline Mesh

Once an object is tracked, its 3D GLB mesh is fetched from the MultiSet platform (with local caching) and placed in the AR scene. A custom Filament shader renders only the mesh outline with:

* Rim-lit edge glow
* Animated spark particles (dual-layer noise)
* Traveling wave along the Y axis
* Pulse modulation
* Configurable glow intensity and color

### Confidence Filtering

When `ObjectTrackingConfig.confidenceCheck` is `true`, tracking results below `confidenceThreshold` are discarded and the activity retries automatically.

### Re-tracking on AR Loss

When `ObjectTrackingConfig.restartTracking` is `true`, the activity detects AR tracking state changes (PAUSED/STOPPED) and automatically re-initiates tracking when the device recovers.

***

## Configuration

The activity reads all settings from `ObjectTrackingConfig` at session startup:

```kotlin
ObjectTrackingConfig.objectCodes = arrayOf("OBJ_001", "OBJ_002")
ObjectTrackingConfig.autoTracking = true
ObjectTrackingConfig.backgroundTracking = true
ObjectTrackingConfig.bgTrackingDurationSeconds = 15f
ObjectTrackingConfig.restartTracking = true
ObjectTrackingConfig.firstTrackingUntilSuccess = true
ObjectTrackingConfig.captureDelayMs = 1000L
ObjectTrackingConfig.confidenceCheck = true
ObjectTrackingConfig.confidenceThreshold = 0.3f
ObjectTrackingConfig.showAlerts = true
ObjectTrackingConfig.imageQuality = 80
ObjectTrackingConfig.validate()
```

See [ObjectTrackingConfig](https://docs.multiset.ai/native-support/android-native/api-reference/objecttrackingconfig) for full property details.

***

## Public Methods

### startObjectTracking()

Manually triggers a tracking request.

**Declaration:**

```kotlin
fun startObjectTracking()
```

**Description:** Captures a frame from the AR camera and sends it to the MultiSet Object Tracking API. Called automatically on session start (if `autoTracking` is enabled) or when the user taps the capture button.

***

## Callbacks (MultiSetCallback Interface)

Results are forwarded to the callback registered via `MultiSetSDK.initialize()`.

### onObjectTrackingSuccess(result: ObjectTrackingResult)

Called when an object is successfully tracked.

| Parameter | Type                   | Description                                |
| --------- | ---------------------- | ------------------------------------------ |
| `result`  | `ObjectTrackingResult` | Contains object code, pose, and confidence |

```kotlin
override fun onObjectTrackingSuccess(result: ObjectTrackingResult) {
    Log.d(TAG, "Tracked: ${result.objectCode}")
    Log.d(TAG, "Position: ${result.position.joinToString()}")
    Log.d(TAG, "Rotation: ${result.rotation.joinToString()}")
    Log.d(TAG, "Confidence: ${result.confidence}")
}
```

### onObjectTrackingFailure(error: String)

Called when tracking fails (after all retries are exhausted, if `firstTrackingUntilSuccess` is `false`).

| Parameter | Type     | Description                          |
| --------- | -------- | ------------------------------------ |
| `error`   | `String` | Error message describing the failure |

```kotlin
override fun onObjectTrackingFailure(error: String) {
    Log.e(TAG, "Tracking failed: $error")
}
```

### onTrackingStateChanged(state: TrackingState)

Called when the underlying AR tracking state changes.

| Parameter | Type            | Description                                       |
| --------- | --------------- | ------------------------------------------------- |
| `state`   | `TrackingState` | Current state: `TRACKING`, `PAUSED`, or `STOPPED` |

***

## ObjectTrackingResult

| Property      | Type           | Description                          |
| ------------- | -------------- | ------------------------------------ |
| `objectCode`  | `String`       | Code of the tracked object           |
| `objectCodes` | `List<String>` | All object codes returned by the API |
| `position`    | `FloatArray`   | XYZ position in ARCore world space   |
| `rotation`    | `FloatArray`   | XYZW quaternion orientation          |
| `confidence`  | `Float`        | Tracking confidence score (0.0–1.0)  |

***

## AR Session Setup

The activity configures the ARCore session with settings optimized for object tracking:

| Setting          | Value                 |
| ---------------- | --------------------- |
| Update Mode      | `LATEST_CAMERA_IMAGE` |
| Focus Mode       | `AUTO`                |
| Light Estimation | `DISABLED`            |
| Depth Mode       | `AUTOMATIC`           |
| Plane Finding    | `DISABLED`            |

Plane rendering is disabled to keep the view clean during object tracking.

***

## Lifecycle

| Event       | Behavior                                                       |
| ----------- | -------------------------------------------------------------- |
| `onCreate`  | Validates SDK auth, sets up AR, fetches mesh, starts animation |
| `onDestroy` | Stops animation, releases tracking manager and mesh handler    |

The activity does not persist the SDK instance — this is managed by the host `MainActivity`.

***

## Related

* [MainActivity](https://docs.multiset.ai/native-support/android-native/sample-activities/mainactivity)
* [ObjectTrackingConfig](https://docs.multiset.ai/native-support/android-native/api-reference/objecttrackingconfig)
* [MultiSetLocalizationActivity](https://docs.multiset.ai/native-support/android-native/sample-activities/multisetlocalizationactivity)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.multiset.ai/native-support/android-native/sample-activities/objecttrackingactivity.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.