This New Drawing Tool Can Turn Sketches Into 3D Images

Waldo was behind the corner all along!

Mental Canvas is a new technology that reimagines static images in a whole new dimension: 3D. While pictures on screens tend to be flat 2D objects layered on 2D objects, the project, which was funded by the National Science Foundation on research headed by Yale computer scientist Julie Dorsey, creates sketches that can be navigated in three dimensions. Ever wanted to look around the corner in a comic book? If it was rendered by Mental Canvas on a tablet, one could.

Here, take a look:

image courtesy of Julie Dorsey, Mental Canvas

In the works since 2007, the software takes drawings and adapts them to 3D. Unlike 3D modeling, where the entire object is created in 3D at the time, Mental Canvas builds up from the art that’s already there and creates either a very minimal rendering or a very advanced doodle, depending on perspective. Think of it like an architect drawing on the back of a napkin to explain a building sketch to a client.

Here’s how Julie Dorsey describes it:

One example is an adaptation of the children’s picture book, “The Other Side” by Istvan Banyai. In the print book, the conceit is that the perspective of the art changes with the flip of every page. In Mental Canvas’ software, that flip becomes something a reader gets to explore, moving a parallax image around and then diving forward to see where it goes. Watch it in action, here:

image courtesy of Julie Dorsey, Mental Canvas

The release from the National Science Foundation is a little vague on how the product itself works, but they expect a commercial version of the product to go on sale later this year, with specialized versions to come in the future.