MOTION CAPTURE LAB DOCUMENTATION:
Project: Data Cleaning and Retargeting
Date: 2019.09.21 4 p.m.
Location: NYU Black Box Theatre
Participant: Yundi Jude Zhu, Chaoyue Huang, Ryan Grippi
Goals:
Pick a partner or two and sign up for time in the Black Box. Record a team member in mocap. Review the data for any gaps, and fix it. Take screenshots of the data, before and after (or give us the filepath to review), and export fbx files for retargeting.
Create an Avatar using MakeHuman, Fuse, or your software of choice (Bonus points for Tiltbrush, Medium, Blocks, or Quill). Retarget your cleaned data to the character and import into UE4 and place them in the world you’ve been working on. Record some video using Quicktime, your phone, Open Broadcaster, etc…
Steps:
Get motion capture record and clean the data
Make 3D model with MakeHuman
Get model rigged with Mixamo
Sync motion capture data with 3D model and export as .fbx
Use the data to make animation in Unreal
Motion Capture and Data Cleaning
Lynn makes a T-Pose to be captured by the 16 cameras.
Key notes:
We invited Lynn, a friend from NYU’s computer science program, to be the dancing model. She is a natural dancer! So good!
We took three takes and the data is surprisingly good. We didn’t get many untitled markers or missing gaps. Most of the unlabeled markers are ghost.
When cleaning the data, we did find out a hand was twisting weirdly. We tried out different ways to fix it. It turns out that the left hand wrist out might have been switched with left hand out for a few frames. We switched the two markers. Eventually, the movement looks much better with “smooth” button.
We captured surprisingly good data for the second take, which most markers are 99% good and only one unlabeled marker turns out to be a ghost.
Lynn dances. The data is not bad. Most of the markers are 100% and only few unlabeled markers.
In the original take, the hand flipped weirdly.
My phone was playing music for Lynn, so we didn’t capture a reference video for the take. We asked Lynn to flip her hand and see how we can fix it.
It turns out that the left hand wrist out might have been switched with left hand out for a few frames. We switched the two markers and make it smooth. It turns out great.
Making 3D Model and Retargeting
Key notes:
I created multiple models with MakeHuman. I downloaded 1.1.1 version for my Mac but there was no option for clothing. Then I downloaded the test version for a PC.
Our team had trouble when inputing motion capture data into Motion Builder. The animation was not playing. Thanks to Chaoyue, we booked another studio hour to get the files exported right. It should be .fbx files only.
The retargeting process is really straight forward. Mixamo for plotting skeleton and Motion Builder for syncing the motion capture data with the model.
In all the process of retargeting, the files should be .fbx. Make sure to rename the different .fbx files to make it organized.
Making randomized human. MakeHuman is very disturbing…It’s super stereotyped, and I feel bad using sliders to control the appearance.
Plotting skeleton into the model I made with MakeHuman on Mixamo.
My 3D baby is alive!
Super proud moment when the baby starts to dance along with Lynn in Motion Builder.
Making Music Video in Unreal
I created four models in total to test out the virtual dance.
Here is the ugly baby dance:
Then I imported three different models to play three different animation. I also played around with the models’ material to make it surreal.
Here is the creepy group dance:
