Needle Engine Integration

Integrate Multiset VPS into Needle Engine projects using Inspector components or the NeedleAdapter API

The SDK includes three Needle Engine components that cover the full VPS workflow. You can use them via the Unity Inspector (no code required for basic setups) or programmatically via NeedleAdapter for custom logic.

Installation

npm install @multisetai/vps

Needle Engine ships its own Three.js fork, so do not install three separately.

Inspector-Driven Setup (Recommended for Unity Developers)

This path lets you configure VPS entirely from the Unity Inspector and export to web with Needle Engine.

Step 1: Copy the Templates

Copy the three template files into your Needle Engine web project's scripts folder:

cp node_modules/@multisetai/vps/templates/MultisetVPS.ts web/src/scripts/
cp node_modules/@multisetai/vps/templates/MapAnchor.ts web/src/scripts/
cp node_modules/@multisetai/vps/templates/MapType.ts web/src/scripts/
cp node_modules/@multisetai/vps/templates/MapType.cs Assets/Scripts/

MapType.cs is a C# file for Unity — it exposes the MapType enum in the Inspector dropdown. MapType.ts is the TypeScript counterpart used by Needle Engine at runtime.

Step 2: Add MultisetVPS to a GameObject

In Unity, add the MultisetVPS component to any GameObject. Fill in the Inspector fields:

Field	Description
Client Id	From the Multiset Developer Portal
Client Secret	From the Multiset Developer Portal
Map Code	Map code, MapSet code, or comma-separated object codes
Map Type	SingleMap, MapSet, or ObjectTracking
Auto Localize	Start localization automatically when the AR session begins
Show Mesh	Download and display the map mesh after localization
Show Gizmo	Display a transform gizmo at the map origin
Relocalization	Re-localize when AR tracking is lost
Background Localization	Continuously re-localize to reduce drift
Confidence Check	Reject results below the confidence threshold
Confidence Threshold	Minimum accepted confidence (0.2 to 0.8)

Step 3: Add MapAnchor to Content GameObjects

Add the MapAnchor component to any GameObject you want anchored to the VPS map.

Field	Description
Offset	Position offset from the map origin in metres
Match Orientation	Align object rotation to the map orientation
Rotation Offset	Additional rotation applied after matching orientation
Hide Until Localized	Keep the object hidden until the first successful localization
Is Right Handed	Enable if entering Three.js coordinates directly. Leave off for Unity coordinates (auto-converts)

MapAnchor hides the entire GameObject when hideUntilLocalized is on. If the object has sibling components that need to run before localization, move them to a separate always-visible GameObject.

Step 4: Export and Run

Export the scene to web via Needle Engine. The AR button appears automatically on the exported page. No additional JavaScript is required.

---

Programmatic Setup (Custom Logic)

Use NeedleAdapter directly when you need to add listeners, react to localization events in other Behaviours, or build custom UI.

import { Behaviour, serializable } from '@needle-tools/engine';
import { MultisetClient } from '@multisetai/vps/core';
import { NeedleAdapter } from '@multisetai/vps/needle';

export class MyVPSController extends Behaviour {

  @serializable()
  clientId: string = '';

  @serializable()
  clientSecret: string = '';

  @serializable()
  mapCode: string = '';

  private adapter: NeedleAdapter | null = null;

  async start() {
    const client = new MultisetClient({
      clientId: this.clientId,
      clientSecret: this.clientSecret,
      mapType: 'map',
      code: this.mapCode,
    });

    await client.authorize();

    this.adapter = new NeedleAdapter({
      client,
      sessionOptions: {
        autoLocalize: true,
        confidenceCheck: true,
        confidenceThreshold: 0.6,
      },
      showMesh: true,
      onLocalizationSuccess: (result, worldFromMap) => {
        console.log('Localized! Confidence:', result.localizeData.confidence);
        // worldFromMap is a THREE.Matrix4
      },
    });
  }

  onDestroy() {
    this.adapter?.dispose();
  }
}

Listening to Localization Events from Another Behaviour

NeedleAdapter supports multiple listeners, so other components can subscribe without coupling to MultisetVPS:

import { Behaviour } from '@needle-tools/engine';
import { NeedleAdapter } from '@multisetai/vps/needle';
import type { ILocalizeAndMapDetails } from '@multisetai/vps/core';
import * as THREE from 'three';

export class MyContentPlacer extends Behaviour {

  start() {
    // Find the adapter from MultisetVPS on the scene
    const vps = this.gameObject.scene.getComponentInChildren(MultisetVPS);
    if (!vps?.adapter) return;

    vps.adapter.addLocalizationListener(this.onLocalized);
    vps.adapter.addSessionEndListener(this.onSessionEnd);
  }

  private onLocalized = (result: ILocalizeAndMapDetails, worldFromMap: THREE.Matrix4) => {
    // Place content at a map-local position
    const localPos = new THREE.Vector3(1.5, 0, -2);
    localPos.applyMatrix4(worldFromMap);
    this.gameObject.position.copy(localPos);
  };

  private onSessionEnd = () => {
    // Reset content visibility
    this.gameObject.visible = false;
  };

  onDestroy() {
    const vps = this.gameObject.scene.getComponentInChildren(MultisetVPS);
    vps?.adapter?.removeLocalizationListener(this.onLocalized);
    vps?.adapter?.removeSessionEndListener(this.onSessionEnd);
  }
}

Runtime-Spawned MapAnchors

If you instantiate a GameObject containing a MapAnchor after the scene loads, register it with the adapter so it receives the last localization result immediately:

import { instantiate } from '@needle-tools/engine';
import { MapAnchor } from './MapAnchor';

const prefab = ...; // your prefab reference
const instance = instantiate(prefab);
const anchor = instance.getComponent(MapAnchor);

if (anchor && vps.adapter) {
  vps.adapter.registerAnchor(anchor); // replays last localization if available
}

Manual Session Control

// Check support before showing the AR button
const supported = await NeedleAdapter.isSupported();

// Start and stop from your own UI
await adapter.startSession(); // must be inside a user gesture handler
adapter.stopSession();

// Trigger localization manually (when autoLocalize is false)
const result = await adapter.localizeFrame();