Towards a real-time photo-realistic animation of a human body using 2D tools
WISDOM Faculty Retreat
May 2012
Yosef Yomdin
Yevgeniy Kagan
Grisha Dinkin
Dima Batenkov
Our goal
Allow a non-expert user to create an interesting movie from her album photographs
- Single-image object recognition
- Imitate natural body movements
- Near real-time performance
Geometric image processing
Low-level geometric image representation (vectorization)
Geometric models for:
- visual patterns specification,
- object recognition,
- image animation.
Human model
Manual fitting - skeleton
Manual fitting - contours
Fitting
(Semi-)automatic
2D animation
Problem: need accurate 3d body movements!
Solution: use motion capture data
Carnegie-Mellon mocap database
CMU database
CMU database
Process overview
- Obtain 3d model + mocap data
- Offline 3d-2d translation
- 2d-2d translation from "standard 2d" model to the actual model from image
- Hope the result is visually satisfying
3d-2d projection
Need to specify joint positions + layer scaling
- 3d/2d skeleton matching
- "Grow flesh" and measure projected segment lengths
- Limbs occlusion
- Fix unacceptable limb distortions
2d-2d translation
- Unknown model proportions
- Unknown initial pose
- Assume initial pose to be roughly enface
- Preserve angles between limbs during animation
- Some animations require "forced grounding"
Example
Example
Self-motions
(I.Sella, T.Flash, R.Malach)
| Motion/Form | Self-motion | Other's motion |
| Self-picture | ||
| Other's picture |
Future work
- "Glue up" an interesting long animation from basic building blocks
- Improve visual quality by introducing additional "layers"
- Use 3D model for pose detection and "semi-automatic animation"