OptiTrack Unreal Engine 4 Plugin

Back to Page References


Plugin Overview

The OptiTrack Unreal Engine 4 Plugin allows real-time streaming of live rigid body tracking data from Motive into Unreal Engine 4 (abbreviated as UE4). The plugin also features HMD tracking integration for virtual reality applications. Once the plugins are added to the Unreal Engine directory, OptiTrack – NatNet Streaming Client, OptiTrack – Oculus Rift. and OptiTrack - Gear VR plugins will be available within the UE4 project. The NatNet Streaming Client plugin will allow you to stream rigid body tracking data from Motive onto Unreal Engine, and the Oculus Rift plugin will allow you to implement HMD tracking onto your scene. Once the plugins are enabled, OptiTrack Client Origin object can be called. The Client Origin object connects the project to the server application via NatNet protocol and imports the tracking data. When the Client Origin object is imported into the scene, it reconciles the global origin in Motive, and all streamed tracking data will be referenced against to it. This origin can be positioned anywhere in the scene and scaled to different sizes. When the project is connected to the server application, imported tracking data can be used to instantiate the HMD and animate the objects in virtual reality applications.


Virtual Reality Experience

OptiTrack motion capture systems provide rigid body and full full-body human motion tracking with outstanding accuracy, and all of which can be accomplished with extremely low latency. Utilizing the tracking capability, an OptiTrack system can be used to provide fully immersive virtual reality experience by integrating head mounted display (HMD) device into the system. From this combined setup, even tiny movements can be precisely captured and reflected in the virtual space, and such accurate representation will deliver an immersive virtual experience. The OptiTrack Unreal Engine 4 Plugin allows you to stream real-time rigid body and HMD tracking data into UE4 using Motive. This page details instructions on the streaming workflow of the plugin. For general instructions on setting up the system, please refer to the Getting Started guide.

As of OptiTrack Unreal Engine 4 Plugin 1.3.0, Oculus DK2, CV1, and Gear VR HMD models are supported


Version Requirements

This plugin is supported for the following versions of Unreal Engine, and it also requires the corresponding version of Oculus runtime as indicated in the table.
Plugin Version UE Version Oculus SDK
1.0 UE 4.10 0.8
1.1 UE 4.11 1.3
1.2 UE 4.12 Latest
1.3 UE 4.13 Latest
1.4 UE 4.14 Latest
1.5 UE 4.15 Latest


Motive Data Streaming Setup (Server)[edit]


To stream the rigid body data, follow the general data streaming workflow provided from the Data Streaming page. Make sure appropriate network interface is defined and the Stream Rigid Bodies is enabled under the streaming settings.


Stereaming in Motive



  • Open the Data Streaming Pane in Motive
  • Select the network interface address for streaming data.
  • Select desired data types to stream under streaming options.
  • When streaming skeletons, set the appropriate bone naming convention for client application.
  • Check Broadcast Frame Data at the top.
  • Configure streaming settings and designate the corresponding IP address from client applications
  • Stream live or playback captures


Unreal Engine Setup (Client)[edit]


This section includes instructions on installing and configuring the Unreal Engine plugin to receive tracking data from Motive.

Setting Up the Plugin[edit]

  1. Download the plugin ZIP file. (Wait until this is publically available).
  2. Extract the contents from the ZIP file.
  3. Open the OptiTrack folder, transfer the OptiTrackNatNet and OptiTrackOculusRift folders into the Unreal Engine plugin directory. By default, this directory is installed in the C:\Program Files\Epic Games\4.11\Engine\Plugins folder. There will be other plugins installed in the folder already. Copy and paste the OptiTrack plugin folders into this directory.
  4. Open/Create a UE4 project, and under the plugin devices, enable both of the OptiTrack plugin devices.
  5. Under the Edit menu on the main command bar, open the plugin panel.
  6. Browse to Virtual Reality section and enable the OptiTrack plugins (OptiTrack - NatNet Streaming Client and OptiTrack - Oculus Rift) that are added to the folder.


UE4 plugins panel listing the enabled OptiTrack plugins.


Note: When using the Oculus Plugin, disable the built-in Oculus Rift (or the Gear VR) and Stream VR HMD plugins under virtual reality plugins category, as they will by default have a higher priority than the OptiTrack Oculus Plugin, and if enabled, will be used in preference to it.



Unreal Engine Client Setup[edit]

OptiTrack Client Origin[edit]

Once the plugin is properly added, the Client Origin Object can be found under Modes panel in UE4.

The OptiTrack Client Origin enables the UE4 client application to communicate with the NatNet server where Motive’s rigid body and HMD data is obtained. Using this object, the communication address and the rigid body ID for the HMD is specified. To add this object, simply drag and drop the OptiTrack Client Origin from the Modes panel into the scene.


Global Origin: Location of this client object will reconcile the global origin of Motive in Unreal Engine. In other words, the tracking data will be represented according to where this Client Origin object is placed in the scene.

Connecting Unreal Engine to Motive[edit]

  1. Once the plugin is properly added and enabled, OptiTrack Client Origin class will be available from the Modes panel in UE.
  2. Drag and drop the OptiTrack Client Origin into the scene.
  3. Select the instantiated OptiTrackClientOrigin object from the World Outliner panel.
  4. Under the Details panel, expand the OptiTrack Client Origin properties.
  5. Enter matching server and client network address information that was defined from the Data Streaming Pane in Motive.
  6. In Motive, make sure the Broadcast Frame Data entry is checked.
  7. Now the UE4 project is connected with Motive, and the tracking data will be available in UE4 when the scene is played.
  • Data streaming settings in Motive
  • OptiTrack Client Origin properties defined with corresponding server and client address.


Animating Rigid Body[edit]

Actor objects in Unreal Engine can be animated using rigid body tracking data from Motive. This can be used to animate props in the scene. To do this, you will need to create a rigid body asset in Motive, and it must be linked with the object in Unreal Engine using the User Data ID.

  1. Select an object in the scene.
  2. Under the details panel, use the +Add Component button to browse and associate the OptiTrack: OptiTrack Rigid Body component with the object.
  3. Under the OptiTrack Client Origin component properties, match the Tracking ID with User Data ID of the rigid body asset that was defined in Motive.
  • Once the OptiTrack NatNet streaming client plugin is properly installed, the OptiTrack Rigid Body component will be available within the components list in UE4.
  • Within the OptiTrack Rigid Body Component, input the User Data ID for the corresponding rigid body asset in Motive.

Animating Skeleton[edit]

Integrating skeleton tracking in Unreal Engine requires a 3D retargeting process. This can be accomplished through a third party software that retargets the 3D data to produce realistic motion in a scene. For more information, contact us.


Integrating HMD[edit]

After the Client Origin object is called into the project, the HMD can be integrated into Unreal Engine. First of all, make sure the OptiTrack – Oculus Rift plugin is enabled in your project. In order to instantiate HMD object tracking in UE, you will need to input the User Data ID for the HMD, and use a Game Mode with the OptiTrackHMDPawn class defined as the default pawn class.

When integrating the HMD, make sure to follow the #HMD Tracking Setup steps and prepare your HMD object in Motive.

Linking the HMD Tracking Data[edit]

  1. Enable the OptiTrack – Oculus Rift plugin from the plugins panel.
  2. Open the World Settings panel under the Window tab.
  3. If not done already, create a new GameMode by adding a new blueprint under the Game Mode section.
  4. Change the Default Pawn Class to OptiTrackHmdPawn. You may need to restart the editor before this change takes effect.
  5. In UE, select the OptitrackClientOrigin and access the Details panel.
  6. Expand the OptiTrack Client Origin section.
  7. In the HMD Rigid Body ID entry, input the respective User Data value that was defined from the rigid body properties in Motive.
  8. Now the tracking data is linked to the Unreal Engine project, and the HMD will display the scene when it is played.
  • Defining the default pawn class from World Setttings: GameMode in UE4.
  • Inputting the User Data ID information for tracking the HMD.


Live Streaming[edit]

Once you have connected the client object and configured the script components on the assets from the scene. Play the scene, and if the properties are properly configured, the assets in the UE scene will be animated according to the position and orientation of the rigid bodies and the HMD in Motive. When playing the scene, the HMD and the assigned actor objects will be placed in respect to their location in Motive’s coordinate system disregards to their position in the UE4 editor.


HMD Tracking Setup[edit]


Setup and optimize the motion capture volume as explained in the Getting Started guide or the Hardware Setup documentations. If you plan to install any obstacles (e.g. walls) within the capture volume, make sure they are non-reflective, and place and orient the cameras so that every corner is thoroughly captured by multiple cameras. For typical rigid body tracking, attach the rigid body markers as instructed from the Rigid Body Tracking page. However, when tracking an HMD, extra attention to placing the markers is necessary.


HMD Marker Setup[edit]

Attachment[edit]

When attaching retroreflective markers, make sure markers are securely attached and readily captured by the cameras. For attaching the markers, we recommend using our 20 mm wide and 30 mm tall M4 threaded plastic marker bases with Acrylic adhesives, available at the webstore, to attach the markers onto the HMD.

Placement[edit]

A markered HMD will be defined as a rigid body in Motive. When placing markers, make sure the placement asymmetry is respected in the arrangement within the HMD. Also, the marker arrangements between multiple HMDs must be incongruent. For more details, read about marker placement from the Rigid Body Tracking page. Also, for tracking the HMD, two landmark markers must be placed in the following locations:

Eye-level Side Markers (2)[edit]

Place two markers on left and right side of the HMD, these markers will serve two additional purposes. First, they will indicate yaw of the HMD, and they will be used to align the rigid body orientation with the orientation of the actual HMD component. Thus, a line interconnecting the two markers must be parallel to the frontal plane, or the display, of the HMD. Second, these markers will be used to locate the elevation of the eyes when creating the rigid body in Motive. In summary, the two landmark markers must be carefully placed considering the following:

  • The markers should align along eye-level of the user when the HMD is mounted.
  • Most importantly, place these markers in the exactly same location of the left and right side so that they form a precisely symmetrical arrangement.
  • Same dimension attachment bases must be used for both of the markers.


  • Sample marker placement: Front view. The two side markers are placed parallel to the HMD display
  • Sample marker placement: Top view.
  • Sample marker placement: Side view. The two side markers are aligned along the eye-level of the user.


HMD Rigid Body in Motive[edit]

The two landmark markers closely aligned to the x-axis of the global axes. When the rigid body is created, the orientation of the rigid body itself closely aligns with the global coordinate system.

Creating HMD Rigid Body[edit]

For HMD tracking, a modified rigid body needs to be defined in Motive. One important note here is that the rigid body orientation must closely align with the display for the best experience. To accomplish this, it is best to align both the HMD and the rigid body orientation along the global axes. This will ensure an appropriate orientation of the HMD rigid body to be exported.

  1. Place the HMD upright on the floor
  2. Click Viewport2.png and switch to orthographic top view.
  3. While monitoring the View pane, align the two landmark markers along the global x-axis in Motive.
  4. Select the HMD markers and create a rigid body.


Creating a rigid body this way will roughly align its orientation with the orientation of the physical HMD component as well as the global coordinate axes in Motive. This will make it easier when precisely adjusting the pivot point position and orientation later. To display the rigid body orientation axes, set the Orientation to true in the rigid body display settings.


Rigid Body Settings[edit]

Rigid body properties for the HMD asset.

After creating a rigid body for the HMD, select the asset in the project pane, and access the corresponding rigid body properties. Adjust the rigid body properties as the following:


Deflection[edit]

Deflection setting is the tolerable distance, in millimeters, that a rigid body marker may deviate from its expected position before it is unlabeled and unassociated with the rigid body. The deflection is set to 4 mm by default. For HMD tracking applications, we recommended lowering this value to 3 or 2. This will reduce the amount of computation required for labeling, and overall latency may be reduced.

Tracking Algorithm[edit]

Use the level 3 tracking algorithm (or ray-based tracking). The tracking algorithm setting determines which protocol to use for solving the rigid body in Motive. This algorithm is recommended because it is less susceptible to marker occlusions and solves rigid bodies more robustly without introducing trajectory gaps. This is more suitable for tracking the HMD since even minor occlusions can acutely affect the experience.

User Data[edit]

The User Data value for each rigid body is used as an identification number to reference the rigid body in external applications. Log this value for each rigid body (including the HMD(s)) in the scene. This number will be used to associate the assets in UE4 using the plugin devices.

Min Marker Count[edit]

The minimum marker count required for the rigid body solve is set to 3 by default. To prevent the swapping of the rigid body definition, you may want to increase this value to 4. When tracking multiple HMDs, there could be limitations to a variety of unique marker arrangements that could be achieved. If this value is set to 3, a set of three markers on an HMD may be congruent to another set in a different HMD, and the rigid body definitions may be switched in Motive.


Rigid Body Pivot Point[edit]

As mentioned previously, the position and orientation of the HMD rigid body pivot point need to be carefully adjusted. This section provides recommended methods and steps to accomplish this. When adjusting the pivot point, the following criteria must be achieved:

  • The pivot point is placed on the bridge of the nose; specifically, right at the midpoint between two eyes.
  • Orientation axes of the rigid body should precisely align with the physical HMD component.


Pivot Point Position[edit]

For best virtual experiences, the pivot point of the HMD rigid body, in Motive, needs to be positioned on the midpoint between two eyes, of the user when the HMD is put on. To locate this, use the side and top-center landmark markers as references. For more information on adjusting rigid body pivot points, please read through the Rigid Body Tracking page.

1. Set the pivot point over the landmark marker. Use the Set Pivot Point to Selected Marker feature to assign the pivot point to the marker. This will set the elevation of the pivot point along the eye-level.


  • Top view: Rigid body pivot point is assigned to the left landmark marker.
  • Side view: Now both the landmark marker and the pivot point is positioned along the user's eye-level elevation.


2. Place the pivot point at the midpoint between the two markers. Enable Two Marker Distance visual aid Viewport16.png from the perspective pane, and select the two landmark markers in Motive. This will provide a distance between two markers. Then, using this information, translate the pivot point laterally by half of the distance so that it is placed right on the midpoint between two markers.

Translating the pivot point by half of the measured distance to place it at the midpoint.

3. Translate the pivot point along the z-axis using the translation tool. For the most accurate position, you may need to physically measure the sagittal, z-axis, distance from the landmark marker to the root of nose, and apply the measured offset.

  • Pivot point translated along the z-axis
  • Perspective view of the adjusted pivot point position.


Orientation[edit]

Now that you have translated the pivot point, you need to make detailed adjustments to the orientation using the orientation transformation tool. For best results, align the two front markers along the x-axis grid and roughly center the rigid body along the z-axis grid. Then, check to make sure that each of the rigid body orientation axes is parallel to the grids lines in Motive. If there is any deviation, apply rotation to adjust the offset. If needed, transparency of the axes and the grids can be adjusted from the Application settings.

  • In Unreal Engine: the X-axis of the HMD rigid body must be directed forward.
  • In Unity: the Z-axis of the HMD rigid body must be directed forward.
  • Slight offset in the rigid body orientation.
  • After Adjusting, the rigid body orientation axes are precisely aligned with the global axes of Motive

Tip: Once you have the rigid body asset for the HMD configured, you can export the asset into a TRA file for future uses. Importing the TRA file (e.g. CV1.tra) will load the rigid body (HMD) asset and make it available for use; however, the marker placement must remain unchanged in order to re-load previously created rigid bodies.