ModelSet: Object Anchoring

ModelSet: Object Anchoring

ModelSet is a powerful feature that enables your AR applications to recognize, track, and anchor digital content to specific physical objects in the real world. By uploading a 3D model of a target object, you can achieve persistent, high-precision visual localization and tracking.

This technology is ideal for a wide range of applications, including:

• Interactive product visualizations

• AR maintenance and training guides for machinery

• Museum exhibits with augmented information

• Location-based AR games anchored to real-world props

How It Works

ModelSet combines the power of cloud processing with on-device performance to deliver a robust tracking experience.

1. Cloud-based Processing: When you upload your 3D model, our cloud service analyzes its geometry and texture to create a highly optimized tracking map. This map contains the unique visual features required for fast and accurate detection.

2. On-device Tracking: The Multiset SDK fetches the initial pose to the user's device. Using the device's camera, the SDK performs real-time local processing to detect the object and tracks the camera movement with the help of the underlying tracking system in 3D space.

This hybrid approach ensures that the initial heavy-lifting is done on the server, while the live tracking remains fast and responsive on the user's device.

Understand the outside-in object tracking.

ModelSet Object Tracking: An "Outside-In" Concept

• Core Principle: The object itself is the center of the tracking environment. Users localize and track the object by observing it from multiple angles around it.

• Key Distinction: This "outside-in" method is the opposite of "inside-out" systems (like localization maps), which track a user's position within a larger space.

• Suitable Objects: This technique is designed for tracking specific, well-defined items within a scanned area. Good examples include engines, products, or statues inside a building.

Getting Started: Preparing Your 3D Model

The quality of your tracking experience is directly tied to the quality of the 3D model you provide. Before uploading, ensure your model adheres to the following requirements.

Model Requirements

These are mandatory specifications for a model to be processed as a ModelSet.

1. Scale: Meters

The 3D model must be authored at a real-world scale, where 1 unit corresponds to 1 meter (m). If your object is 50cm tall in reality, its model should be 0.5 units tall in your 3D software. Incorrect scale will result in tracking failure or incorrect content placement.

2. Up Vector: +Y Axis

The model's "up" direction must be aligned with the positive Y-axis. This ensures that the object's orientation is interpreted correctly by our tracking system. Most 3D authoring tools allow you to configure or export with a specific coordinate system.

3. Static Objects

Currently, ModelSet is designed for static, rigid objects. The physical object you intend to track should not move, deform, or articulate during the AR session. Support for dynamic objects is planned for a future release.

Note: This means the physical object in the world should be stationary. The user (camera) can, of course, move around it freely.

4. Textured Mesh

Tracking relies on visual features. Your model must have a texture map (e.g., a diffuse/albedo map) applied. Untextured or single-color models lack the necessary feature points for our computer vision algorithms to lock onto.

Best Practices for Optimal Tracking

Follow these recommendations to significantly improve the stability and accuracy of your ModelSet tracking.

✔️ Use High-Detail Textures and UVs

A clear, high-resolution texture with well-distributed details is crucial. Good UV wrapping ensures the texture is applied without distortion, providing a rich set of unique feature points for the tracker. Avoid blurry or heavily compressed textures.

✔️ Choose Asymmetrical Objects with Rich Features

The ideal objects for tracking are visually complex and asymmetrical. Think of things with unique shapes, text, logos, and varied surface patterns.

• Good Examples: A detailed statue, a coffee machine with buttons and logos, a branded cereal box (with text and images), a piece of industrial machinery.

• Challenging Examples: A symmetrical wooden crate, a single-color yoga ball, a generic coffee mug, a mirrored or highly reflective object.

❌ Avoid Symmetrical and Feature-Poor Objects

Symmetrical objects, like a perfect cube or sphere, create ambiguity for the tracker. The system may struggle to determine the object's exact orientation because multiple sides look identical. Similarly, objects with large, uniform-color surfaces provide very few points to track.