> For the complete documentation index, see [llms.txt](https://docs.multiset.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.multiset.ai/webxr-sdk/needle-engine.md). # Needle Engine Integration The SDK includes three Needle Engine components that cover the full VPS workflow. You can use them via the Unity Inspector (no code required for basic setups) or programmatically via `NeedleAdapter` for custom logic. ### Installation ```bash npm install @multisetai/vps ``` Needle Engine ships its own Three.js fork, so do not install `three` separately. ### Inspector-Driven Setup (Recommended for Unity Developers) This path lets you configure VPS entirely from the Unity Inspector and export to web with Needle Engine. #### Step 1: Copy the Templates Copy the three template files into your Needle Engine web project's scripts folder: ```bash cp node_modules/@multisetai/vps/templates/MultisetVPS.ts web/src/scripts/ cp node_modules/@multisetai/vps/templates/MapAnchor.ts web/src/scripts/ cp node_modules/@multisetai/vps/templates/MapType.ts web/src/scripts/ cp node_modules/@multisetai/vps/templates/MapType.cs Assets/Scripts/ ``` `MapType.cs` is a C# file for Unity — it exposes the `MapType` enum in the Inspector dropdown. `MapType.ts` is the TypeScript counterpart used by Needle Engine at runtime. #### Step 2: Add MultisetVPS to a GameObject In Unity, add the **MultisetVPS** component to any GameObject. Fill in the Inspector fields: | Field | Description | | --------------------------- | ----------------------------------------------------------- | | **Client Id** | From the Multiset Developer Portal | | **Client Secret** | From the Multiset Developer Portal | | **Map Code** | Map code, MapSet code, or comma-separated object codes | | **Map Type** | SingleMap, MapSet, or ObjectTracking | | **Auto Localize** | Start localization automatically when the AR session begins | | **Show Mesh** | Download and display the map mesh after localization | | **Show Gizmo** | Display a transform gizmo at the map origin | | **Relocalization** | Re-localize when AR tracking is lost | | **Background Localization** | Continuously re-localize to reduce drift | | **Confidence Check** | Reject results below the confidence threshold | | **Confidence Threshold** | Minimum accepted confidence (0.2 to 0.8) | #### Step 3: Add MapAnchor to Content GameObjects Add the **MapAnchor** component to any GameObject you want anchored to the VPS map. | Field | Description | | ------------------------ | ------------------------------------------------------------------------------------------------- | | **Offset** | Position offset from the map origin in metres | | **Match Orientation** | Align object rotation to the map orientation | | **Rotation Offset** | Additional rotation applied after matching orientation | | **Hide Until Localized** | Keep the object hidden until the first successful localization | | **Is Right Handed** | Enable if entering Three.js coordinates directly. Leave off for Unity coordinates (auto-converts) | {% hint style="info" %} `MapAnchor` hides the entire GameObject when `hideUntilLocalized` is on. If the object has sibling components that need to run before localization, move them to a separate always-visible GameObject. {% endhint %} #### Step 4: Export and Run Export the scene to web via Needle Engine. The AR button appears automatically on the exported page. No additional JavaScript is required. *** ### Programmatic Setup (Custom Logic) Use `NeedleAdapter` directly when you need to add listeners, react to localization events in other Behaviours, or build custom UI. ```typescript import { Behaviour, serializable } from '@needle-tools/engine'; import { MultisetClient } from '@multisetai/vps/core'; import { NeedleAdapter } from '@multisetai/vps/needle'; export class MyVPSController extends Behaviour { @serializable() clientId: string = ''; @serializable() clientSecret: string = ''; @serializable() mapCode: string = ''; private adapter: NeedleAdapter | null = null; async start() { const client = new MultisetClient({ clientId: this.clientId, clientSecret: this.clientSecret, mapType: 'map', code: this.mapCode, }); await client.authorize(); this.adapter = new NeedleAdapter({ client, sessionOptions: { autoLocalize: true, confidenceCheck: true, confidenceThreshold: 0.6, }, showMesh: true, onLocalizationSuccess: (result, worldFromMap) => { console.log('Localized! Confidence:', result.localizeData.confidence); // worldFromMap is a THREE.Matrix4 }, }); } onDestroy() { this.adapter?.dispose(); } } ``` ### Listening to Localization Events from Another Behaviour `NeedleAdapter` supports multiple listeners, so other components can subscribe without coupling to `MultisetVPS`: ```typescript import { Behaviour } from '@needle-tools/engine'; import { NeedleAdapter } from '@multisetai/vps/needle'; import type { ILocalizeAndMapDetails } from '@multisetai/vps/core'; import * as THREE from 'three'; export class MyContentPlacer extends Behaviour { start() { // Find the adapter from MultisetVPS on the scene const vps = this.gameObject.scene.getComponentInChildren(MultisetVPS); if (!vps?.adapter) return; vps.adapter.addLocalizationListener(this.onLocalized); vps.adapter.addSessionEndListener(this.onSessionEnd); } private onLocalized = (result: ILocalizeAndMapDetails, worldFromMap: THREE.Matrix4) => { // Place content at a map-local position const localPos = new THREE.Vector3(1.5, 0, -2); localPos.applyMatrix4(worldFromMap); this.gameObject.position.copy(localPos); }; private onSessionEnd = () => { // Reset content visibility this.gameObject.visible = false; }; onDestroy() { const vps = this.gameObject.scene.getComponentInChildren(MultisetVPS); vps?.adapter?.removeLocalizationListener(this.onLocalized); vps?.adapter?.removeSessionEndListener(this.onSessionEnd); } } ``` ### Runtime-Spawned MapAnchors If you instantiate a GameObject containing a `MapAnchor` after the scene loads, register it with the adapter so it receives the last localization result immediately: ```typescript import { instantiate } from '@needle-tools/engine'; import { MapAnchor } from './MapAnchor'; const prefab = ...; // your prefab reference const instance = instantiate(prefab); const anchor = instance.getComponent(MapAnchor); if (anchor && vps.adapter) { vps.adapter.registerAnchor(anchor); // replays last localization if available } ``` ### Manual Session Control ```typescript // Check support before showing the AR button const supported = await NeedleAdapter.isSupported(); // Start and stop from your own UI await adapter.startSession(); // must be inside a user gesture handler adapter.stopSession(); // Trigger localization manually (when autoLocalize is false) const result = await adapter.localizeFrame(); ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.multiset.ai/webxr-sdk/needle-engine.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.