Labeling of markers allows Motive to distinguish different 3D reconstructions within a capture. 3D trajectories of the labeled reconstructions can be exported individually or used (solved) altogether to compute positions and orientations of tracked subjects including rigid bodies and skeletons. Thus, all of the target 3D markers needs to be labeled in Motive. This page provides instructions on labeling 3D markers in Motive.
Motive provides several views for identifying and resolving labeled or unlabeled trajectories:
- The Timeline Tracks View gives a list of labels and their continuity during a captured Take.
- The scrubber in the Timeline pane gets highlighted in red for frames where the selected markers are unlabeled.
- The Labeling pane lists out the marker labels for each of the involved labels and indicates their corresponding percentage completeness.
There are two main approaches for labeling markers in Motive:
- Auto-label: Automatically label sets of rigid body markers and skeleton markers using corresponding asset definitions.
- Manual Label: Manually labeling individual markers using the Labeling pane.
When tracking rigid bodies and skeletons, Motive can automatically label associated markers both in real-time and post-processing. The auto-labeler uses active asset definitions to search for related markers and label them with pre-defined labels throughout each frame of 3D data. Using the auto-labeler, it wouldn't be necessary to individually assign marker labels for trackable assets.
There are times, however, when it is necessary to manually label a section or all of a trajectory, either because the markers of a rigid body or a skeleton were misidentified (or unidentified) during capture or because individual markers need to be manually labeled. In these cases, the Labeling Pane in Motive is used to perform manual labeling of individual trajectories. Note that manual labeling is supported only for post-processing of a capture when a Take file (TAK) has been loaded with 3D data as its playback type. In case of 2D data only capture, the Take must be Reconstructed first from the Project Pane in order to assign, or edit, the marker labels in its 3D data. This manual labeling process is typically referred to as post processing of recorded mocap data along with the process of editing reconstructed marker data. For more details on editing 3D data, see more in the Data Editing page.
The auto-labeler labels 3D markers that are associated with active rigid body or skeleton assets (enabled from the Project pane) in captured Takes. To auto-label Take(s) in post-processing:
- Select Takes from the Project pane
- Right-click to bring up the context menu
- Click either auto-label or reconstruct and auto-label to process selected Takes. The combined reconstruct and auto-label will create a new set of 3D data and auto-label the markers from it.
- This will label all the markers that matches the corresponding asset definition.
Each trackable asset definition stores respective arrangements of the involved markers, which was recorded when the asset was first created. During the auto-labeling process, Motive labels a set of reconstructed 3D points that resemble marker arrangements of the active assets, and then the corresponding rigid bodies and skeletons get modeled in the 3D view. Markers that are not associated with any of the assets are marked as Unlabeled unless they are manually labeled. After creating new assets in recorded capture, the Take must be auto-labeled again in order to assign respective marker labels.
For Motive 1.10 and above, a separate auto-label pipeline has been added. To auto-label a take, the selected Take must contain 3D data. Unlike the combined reconstruct and auto-label pipeline, the standalone auto-label pipeline labels existing 3D data without reconstructing a new set of 3D data. If 3D data already contains labeled markers, the auto-labeler will respect the existing labels and will not re-label them. Only the unlabeled markers will be labeled via this pipeline, and any post-processing edits that were made will be preserved.
The settings for the auto-labeling engine are defined in the Auto-labeler section of the Reconstruction pane. The auto-labeler parameters can be modified during post-processing pipelines, and they can be optimized for stable labeling of markers throughout the Take.
The MarkerSets are the most fundamental method of grouping related markers in Motive.
The MarkerSet is a type asset in Motive. MarkerSets are lists of marker labels that are used to group and manually label individual markers in Motive using the Labeling pane. Create Marker Set icon in the Project pane creates a new MarkerSet. Desired labels can be typed in or added by copying and pasting a list of labels (a carriage-return delimited) from the windows clipboard (Press Ctrl+V in the Marker List window). You can also quickly create a new MarkerSet from existing unlabeled markers by selecting them in the Labeling Pane and selecting 'Create' from the context menu, as shown from the image on the right. Note that Marker Sets are used for manual labeling after the data has been captured only. For automatic labeling during live mode, a Rigid Body asset or a Skeleton asset is necessary.
Since creating rigid bodies, or skeletons, group the markers in the set and automatically label them, MarkerSets are not commonly used in the workflow. However, they are still useful for marker-specific tracking applications or when the labeling process is done in pipelines other than auto-labeling. Also, marker sets are useful when organizing and reassigning the labels. ↑
The Labeling pane is used to assign, remove, and edit marker labels in the 3D data. The Tracks View under the Timeline Pane can be used in conjunction with the Labeling pane to monitor which markers and gaps are associated. The Labeling pane is also used to examine the number of occluded gaps in each label, and it can be used along with the Editing Tools for complete post-processing.
Using the Labeling pane, you can assign marker labels for each asset (MarkerSet, Rigid Body, and Skeleton) via the QuickLabel Mode . The Labeling Pane also shows a list of labels involved in the Take and their corresponding percent completeness values. The percent completeness values indicate frame percentages of a Take for which the trajectory has been labeled. If the trajectory has no gaps (100% complete), no number will be shown. You can use this pane together with the Timeline to quickly locate gaps in a trajectory.
For a given frame, all labels are color-coded. For each frame of 3D data, assigned marker labels are shown in white, labels without reconstructions are shown in red, and unlabeled reconstructions are shown in orange; similar to how they are presented in the 3D View.
See the Labeling Pane page for detailed explanation on each option.
The QuickLabel mode allows you to tag labels with one-click in the view pane, and it is a handy way to reassign or modify marker labels throughout the capture. When the QuickLabel mode is toggled, the mouse cursor changes to a finger icon with the selected label name attached next to it. Also, when the display label option is enabled in the perspective view, all of assigned marker labels will be displayed next to each marker in the 3D viewport, as shown in the image below. Select the marker set you wish to label, and tag the appropriate labels to each marker throughout the capture.
When assigning labels using the Quick Label Mode, the labeling scope is configured from the labeling range settings. You can restrict the labeling operation to apply from the current frame backward, current frame forward, or both depending on the trajectory. You may also restrict labeling operations to apply the selected label to all frames in the Take, to a selected frame range, or to a trajectory 'fragment' enclosed by gaps or spikes. The fragment/spike setting is used by default and this best identifies mislabeled frame ranges and assigns marker labels. See the Labeling pane page for details on each feature.
Labeling using the QuickLabel Mode
- Under the drop-down menu in the Labeling pane, select an asset you wish to label.
- All of the involved markers will be displayed under the columns.
- From the label list, select unlabeled or mislabeled markers.
- Inspect the behavior of the selected trajectory and decide whether you want to apply the selected label for frames before the current frame or frames after the current frame by toggling each icon. If you wish to assign selected label to a trajectory of entire capture, enable both of them.
- Switch to QuickLabeling Mode (Hotkey: D).
- In the Perspective View pane. Assign the selected label to a marker reconstruction. If the Increment Selection is enabled, label selection in the Labeling pane will automatically advance each time you assign them.
- After assigning all labels, switch back to normal Select Mode .
General Labeling Steps
The following steps describe typical labeling workflows. Note that these steps may vary depending on the capture, and alternative approaches could also be used. Utilize the reconstruction and auto-labeling pipelines in combination with the Labeling pane to best reconstruct and label the 3D data of your capture.
- Note: From Motive 1.10 and above, the standalone auto-labeling pipeline has been added, and it can be utilized to more efficiently and precisely label the markers.
Combined Reconstruction and Auto-label
Step 1. In the Project pane, Reconstruct and auto-label the take with all of the desired assets enabled.
Step 2. In the Timeline, examine the trajectories and navigate to the frame where labeling errors are frequent.
Step 3. Open the Labeling pane.
Step 4. Select an asset that you wish to label.
Step 5. From the label columns, Click on a marker label that you wish to re-assign.
Step 6. Inspect behavior of a selected trajectory and its labeling errors and set the appropriate labeling settings (allowable gap size, maximum spike and applied frame ranges).
Step 7. Switch to the QuickLabel mode (Hotkey: D).
Step 8. On the Perspective View, assign the labels onto the corresponding marker reconstructions by clicking on them.
Step 9. When all markers have been labeled, switch back to the Select Mode.
Standalone Reconstruction and Auto-label
Step 1. Start with 2D data of a captured Take with model assets (skeletons and rigid bodies).
Step 2. Reconstruct and Auto-Label, or just Reconstruct, the Take with all of the desired assets enabled under the Project pane. If you use reconstruct only, you can skip step 3 and 5 for the first iteration.
Step 3. Examine the reconstructed 3D data, and inspect the frame range where markers are mislabeled.
Step 4. Using the Labeling pane, manually fix/assign marker labels, paying attention to your label settings (direction, max gap, max spike, selected duration).
Step 5. Unlabel all trajectories you want to re-auto-label.
Step 6. Auto-Label the Take again. Only the unlabeled markers will get re-labeled, and all existing labels will be kept the same.
Step 7. Re-examine the marker labels. If some of the labels are still not assigned correctly from any of the frames, repeat the steps 3-6 until complete.
Labeling Error Fix
The general process for resolving labeling error is:
- Identify the trajectory with the labeling error.
- Determine if the error is a swap, an occlusion, or unlabeled.
- Resolve the error with the correct tool.
- Swap: Use the Swap Fix tool ( Edit Tools ) or just re-assign each label ( Labeling Panel ).
- When manually labeling markers to fix swaps, set appropriate settings for the labeling direction, max spike, and selected range settings.
- Occlusion: Use the Gap Fill tool ( Edit Tools ).
- Unlabeled: Manually label an unlabeled trajectory with the correct label ( Labeling Panel ).
For more data editing options, read through the Data Editing page.
Sample Scenarios and Tutorial Videos
Sample Scenario 1 :: All Good
When recorded 3D data have been labeled properly and entirely throughout the Take, you will not need to edit marker labels. If you don't have 3D data recorded, you can reconstruct and auto-label the Take to obtain 3D data and label all of the skeleton and rigid body markers. If all of the markers are well reconstructed and there are no significant occlusions, auto-labeled 3D data may be acceptable right away. In this case, you can proceed without post-processing of marker labels.
- Recorded 3D data has no gaps in the labels, or the Reconstruct and Auto-label works perfectly the first time without additional post-processing.
- Examine the Take(s). Check the Labeling pane, or the tracks view, to make sure no occlusion exists within the capture, and all markers are consistently labeled.
Sample Scenario 2 :: Labeling errors in the middle of a Take
When skeleton markers are mislabeled only within specific frame ranges of a Take, you will have to manually re-label the markers. This may occur when a subject performs dynamic movements or come into contact with another object during the recorded Take. After correcting the mislabeled markers, you can also use the auto-labeler to assign remaining missing labels.
- Start with recorded 3D data or Reconstruct and auto-label the Take to obtain newly labeled 3D data.
- Inspect the Take to pick out the frame ranges with bad tracking.
- If necessary, unlabel the bad frame range or the entire Take.
- Scrub the timeline to a frame just before the bad tracking frame range.
- Using the Labeling pane, manually label the skeleton. Depending on the severity of the mislabels, you can either label the entire skeleton or just the key segments starting from the hip.
- Scrub the timeline to a frame after the bad tracking frame range.
- Manually label the same skeleton.
- Auto-label the Take (Motive 1.10 and above).
- Check the frames again and correct any remaining mislabels using the Labeling pane.
Sample Scenario 3 :: Skeletons never acquire perfectly throughout a Take
For Take(s) where skeletons are never perfectly tracked and the markers are consistently mislabeled, you will need to manually assign the correct labels for the skeleton asset(s). Situations like this could happen when the skeleton(s) are never in an easily trackable pose throughout the Take (e.g. captures where the actors are rolling on the ground). It is usually recommended that all skeleton ‘’Takes’’’ start and end with T-pose in order to easily distinguish the skeleton markers. This also helps the skeleton solver to correctly auto-label the associated markers; however, in some cases, only specific section of a Take needs be trimmed out, or including the calibration poses might not be possible. Manually assigning labels can help the auto-labeler to correctly label markers and have skeletons acquire properly in a Take. You will get best results if you manually label the entire skeleton, but doing so can be time-consuming. You can also label only the mislabeled segment or the key segment (hip bone) and run the auto-labeler to see if it correctly assigns the labels with the small help.
- Start with recorded 3D data or Reconstruct the Take.
- At a certain point of the Take (usually at a frame where you can best identify the pose of the skeleton), use the Labeling pane to manually assign the marker labels for skeletons that are not labeling correctly. Depending on the severity of the mislabels, you can either label the entire skeleton or only the key segments starting from the hip.
- After manually assigning the labels, auto-label the Take. Make sure the corresponding assets are enabled in the Project pane.
- Check to see if all markers are correctly assigned throughout the take. If not, re-label or unlabel, any mislabeled markers and run auto-label again if needed.
Sample Scenario 4 :: Mislabeling/unlabeling of a marker due to occlusions
Marker occlusions can be critical to the auto-labeling process. After having a gap for multiple frames, occluded markers can be unlabeled entirely, or nearby reconstructions can be mistakenly recognized as the occluded marker and result in labeling swaps or mislabels. Skeleton and rigid body asset definitions may accommodate labeling for such occlusions, but in some cases, labeling errors may persist throughout the Take. The following steps can be used to re-assign the labels in this case.
If tracked markers are relatively stationary during the occluded frames, you may want to increase the Maximum Marker Label Gap value under the Auto-Labeler settings in the Reconstruction pane to allow the occluded marker to maintain its label after auto-labeling the Take. However, note that adjusting this setting will not be useful if the marker is moving dynamically beyond the Prediction Radius (mm) settings during occlusion.
- Start with recorded 3D data or Reconstruct and auto-label the Take
- Examine through the Take, and go to a frame where markers are mislabeled right after an occlusion.
- In the Labeling pane, disable the Increment Label Selection option, and select a marker set and a label that is frequently occluded.
- In the Labeling pane, disable the Apply Labels to Previous Frames option, and leave only the Apply Labels to Upcoming Frames option enabled.
- Using the Quick Label Mode, correct the labeling errors.
- Move onto next occluded frames. When the marker reappears, correct the labels.
- After correcting the labels, Auto-label the Take again.
- Use the Fill Gaps tool in the Editing tools to interpolate the occluded trajectories.