> For the complete documentation index, see [llms.txt](https://docs.multiset.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.multiset.ai/webxr-sdk/threejs-integration.md).

# Three.js Integration

`ThreeAdapter` wires `XRSessionManager` to a Three.js renderer. It manages the XR render loop, synchronizes the camera, downloads the map mesh, and gives you a `worldFromMap` matrix in `onLocalizationSuccess` so you can place content at any map-local coordinate.

### Installation

```bash
npm install @multisetai/vps three
```

Three.js 0.169.0 or higher is required.

### Complete Working Example

The following sets up a full AR session with auto-localization. Place this in your entry file and serve over HTTPS.

```typescript
import * as THREE from 'three';
import { MultisetClient, XRSessionManager } from '@multisetai/vps/core';
import { ThreeAdapter } from '@multisetai/vps/three';

// --- Scene setup ---
const renderer = new THREE.WebGLRenderer({ antialias: true, alpha: true });
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.xr.enabled = true;
document.body.appendChild(renderer.domElement);

const scene = new THREE.Scene();
scene.background = null; // transparent so the camera feed shows through

const camera = new THREE.PerspectiveCamera(
  70,
  window.innerWidth / window.innerHeight,
  0.01,
  100
);
scene.add(camera);

scene.add(new THREE.AmbientLight(0xffffff, 1));

window.addEventListener('resize', () => {
  camera.aspect = window.innerWidth / window.innerHeight;
  camera.updateProjectionMatrix();
  renderer.setSize(window.innerWidth, window.innerHeight);
});

// --- Multiset client ---
const client = new MultisetClient({
  clientId: import.meta.env.VITE_MULTISET_CLIENT_ID,
  clientSecret: import.meta.env.VITE_MULTISET_CLIENT_SECRET,
  mapType: 'map',
  code: import.meta.env.VITE_MAP_CODE,
});

await client.authorize();

// --- XR session ---
const session = new XRSessionManager(
  renderer.getContext() as WebGL2RenderingContext,
  {
    client,
    autoLocalize: true,
    onLocalizationFailure: (reason) => console.warn('Localization failed:', reason),
    onError: (error) => console.error('Session error:', error),
  }
);

// --- Three.js adapter ---
const adapter = new ThreeAdapter({
  session,
  renderer,
  scene,
  camera,
  showMesh: true,    // download and display the map mesh after localization
  showGizmo: true,   // show transform gizmo at map origin

  onLocalizationSuccess: (result, worldFromMap) => {
    // worldFromMap is a THREE.Matrix4 that converts map-local to world coordinates.
    // Use it to place objects at specific map-local positions.
    const box = new THREE.Mesh(
      new THREE.BoxGeometry(0.3, 0.3, 0.3),
      new THREE.MeshStandardMaterial({ color: 0x0099ff })
    );

    // Place the box at map origin
    box.applyMatrix4(worldFromMap);
    scene.add(box);

    console.log('Confidence:', result.localizeData.confidence);
  },
});

// --- Start ---
// initialize() starts the preview loop and mounts the AR button
adapter.initialize();
```

### Placing Content at Map Coordinates

The `worldFromMap` matrix in `onLocalizationSuccess` transforms any map-local position to Three.js world space. Use it to position objects at known coordinates within your map.

```typescript
onLocalizationSuccess: (result, worldFromMap) => {
  // Example: place a marker at map-local position (2, 0, -3)
  const marker = new THREE.Mesh(
    new THREE.SphereGeometry(0.1),
    new THREE.MeshBasicMaterial({ color: 0xff0000 })
  );

  const localPosition = new THREE.Vector3(2, 0, -3);
  localPosition.applyMatrix4(worldFromMap);
  marker.position.copy(localPosition);
  scene.add(marker);
}
```

### MapSet Mode

Switch `mapType` to `'map-set'` and use your map set code. Everything else stays the same.

```typescript
const client = new MultisetClient({
  clientId: '...',
  clientSecret: '...',
  mapType: 'map-set',
  code: 'MAPSET_XXXXXXXXXX',
});
```

### Manual Localization (No Auto-Localize)

Disable `autoLocalize` and trigger localization from your own UI:

```typescript
const session = new XRSessionManager(
  renderer.getContext() as WebGL2RenderingContext,
  {
    client,
    autoLocalize: false,
  }
);

const adapter = new ThreeAdapter({ session, renderer, scene, camera });
adapter.initialize();

document.getElementById('localize-btn').addEventListener('click', async () => {
  const result = await adapter.localizeFrame();
  if (result) {
    console.log('Localized with confidence:', result.localizeData.confidence);
  } else {
    console.log('No pose found');
  }
});
```

### Custom AR Button

Disable the built-in button and build your own:

```typescript
const adapter = new ThreeAdapter({
  session,
  renderer,
  scene,
  camera,
  useDefaultButton: false,
});

adapter.initialize();

myStartButton.addEventListener('click', () => adapter.startSession());
myStopButton.addEventListener('click', () => adapter.stopSession());
```

### Frame-Level Access

Use `onXRFrame` to run code every frame during an active AR session, after camera matrices are synced but before rendering:

```typescript
const adapter = new ThreeAdapter({
  session,
  renderer,
  scene,
  camera,
  onXRFrame: ({ frame, pose, deltaSeconds }) => {
    // Update animations, physics, etc.
    myObject.rotation.y += deltaSeconds;
  },
});
```

### Cleanup

```typescript
adapter.dispose(); // ends session, removes listeners, releases GL resources
```


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.multiset.ai/webxr-sdk/threejs-integration.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.