Disney Research Technology Brings Colored Drawings to Life

Posted on October 9, 2015

A new technology from Disney Research brings colored drawings to life. An app has been developed that uses augment reality to animate a 3-D version of a character a child is drawing or coloring in.

Disney Research says the drawing is detected and tracked and the video stream is augmented with an animated 3-D version of the character that is textured according to the child's coloring. The image processing and rendering happens in real-time.

Disney Research says its interactive coloring book app for Android and iOS uses the Unity game engine. The app uses Unity to access the camera and fetch the pixels of the drawing. This data is passed to a C++ library and the deformable surface tracking algorithm is implement. The C++ library tells Unity whether a drawing template is detected and if so it returns the 3-D shape of the colored drawing in the camera coordinates. The app rectifies the image of the colored drawing by projecting the vertices of the triangular mesh representing the drawing 3-D shape into the image plane. These projected points are used as a texture coordinate to draw a rectified grid with the camera image as texture. Disney Research says the app then renders the camera image using a screen-aligned quad and overlays the corresponding animated 3-D character.

A PDF version of a research paper describing the technology can be found here.

Here is a video from Disney Researching explaining the innovative AR coloring book technology:

The researchers do not say when the app they have developed will be made available to consumers. If the app can work as smoothly as it appears in the video it would likely be something kids would enjoy. RocketNews points out a similar technology called Quiver Augmented Reality.

TECH