# WebXR Integration

Build AR web experiences with precise indoor localization using the MultiSet VPS WebXR SDK.

{% hint style="success" %}
To try WebAR sample localization, open the developer portal in Chrome browser for Android, navigate to the Maps section and click on the AR button
{% endhint %}

<figure><img src="https://3163433004-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FokTDI7QVY04Zvb1pQ8Ry%2Fuploads%2FmxDpFnaCyOiRbjOFjUG6%2FWebARsample.jpeg?alt=media&#x26;token=9f04d3d9-2bf1-4510-b9f6-fa681d6110b1" alt="" width="270"><figcaption></figcaption></figure>

## GitHub Repository

{% embed url="<https://github.com/MultiSet-AI/multiset-vps-webxr>" %}

## Prerequisites

* Node.js 16+
* AR-capable Android device with ARCore support
* HTTPS connection (required for WebXR APIs)
* Chrome browser with WebXR support

{% hint style="warning" %}
To whitelist your domain and avoid CORS issues, follow the steps in [Configuring Allowed Domains (CORS)](https://docs.multiset.ai/basics/credentials/configuring-allowed-domains-cors)
{% endhint %}

## Installation

```bash
npm install @multisetai/vps three
```

## Configuration

Get your credentials from the [MultiSet Dashboard](https://developer.multiset.ai):

1. **Client ID & Secret** - Found under Credentials section
2. **Map Code** - Found in the Maps section (e.g., `MAP_XXXXXXXXXX`)

## Quick Start

### Step 1: Initialize the Client

```typescript
import { MultisetClient, WebxrController } from '@multisetai/vps';

const client = new MultisetClient({
  clientId: 'YOUR_CLIENT_ID',
  clientSecret: 'YOUR_CLIENT_SECRET',
  code: 'MAP_XXXXXXXXXX',
  mapType: 'map', // or 'map-set' for MapSets
});
```

### Step 2: Authorize and Set Up WebXR

```typescript
// Authorize with MultiSet API
await client.authorize();

// Initialize WebXR controller
const controller = new WebxrController({
  client,
  canvas: document.getElementById('canvas')
});
await controller.initialize();

// Access Three.js scene to add your AR content
const scene = controller.getScene();
```

### Step 3: Capture Frames for Localization

```typescript
// During active AR session, capture frame to get pose
const result = await controller.captureFrame();

if (result) {
  console.log('Position:', result.position); // { x, y, z }
  console.log('Rotation:', result.rotation); // { x, y, z, w }
  console.log('Confidence:', result.confidence);
}
```

## Key Components

### MultisetClient

Handles authentication and API communication:

* `authorize()` - Exchanges credentials for access token
* `localize()` - Submits camera frame for localization
* `getMapDetails()` - Retrieves map metadata

### WebxrController

Manages WebXR session and Three.js integration:

* `initialize()` - Sets up renderer, camera, and AR session
* `getScene()` - Returns Three.js scene for adding AR content
* `captureFrame()` - Captures frame and returns localization result
* `dispose()` - Cleans up resources

## Localization Response

The `captureFrame()` method returns:

| Field        | Type           | Description                    |
| ------------ | -------------- | ------------------------------ |
| `position`   | `{x, y, z}`    | 3D position in map coordinates |
| `rotation`   | `{x, y, z, w}` | Quaternion rotation            |
| `confidence` | `number`       | Localization confidence score  |

## Example Projects

The SDK includes ready-to-use examples:

* **React Example** - [examples/react](https://github.com/MultiSet-AI/multiset-vps-webxr/tree/main/examples/react)
* **Vanilla JS Example** - [examples/vanilla](https://github.com/MultiSet-AI/multiset-vps-webxr/tree/main/examples/vanilla)

To run an example:

```bash
cd examples/react  # or examples/vanilla
npm install
npm run dev
```

{% hint style="info" %}
Examples must be served over HTTPS and accessed from a whitelisted domain.
{% endhint %}

## Demo

{% embed url="<https://youtube.com/shorts/hRveey-df6k?feature=share>" fullWidth="false" %}
WebXR VPS demo
{% endembed %}