PetaPixel

This Crazy Software Extracts 3D Objects from Photos with a Few Clicks

If you were wanting to have your mind blown today, the video above might do it. It’s a demonstration of a piece of 3D object extraction and manipulation software that made its debut at SIGGRAPH 2013, and it may just offer a glimpse into the future of photo manipulation.

The software was developed by Tao Chen, Zhe Zhu, Ariel Shamir, Shi-Min Hu and Daniel Cohen-Or, and even in its infancy it’s capable of some pretty amazing feats.

Throughout the demonstration, 3D objects are quickly and easily extracted from regular old 2D photos using a “3-sweep” method: two strokes to define the profile of the object, and one along the main axis. Depending on the complexity of the shape, sometimes parts have to be outlined individually, but the result it always the same: the software pulls an editable, movable 3D model straight out of a 2D photo.

3dextraction1

As the video goes on, the examples get more and more complex and impressive. Once outlined, each bit of the above telescope is scaled and altered at will, and the whole thing can be tilted, turned and shifted on every axis.

That’s not to say that the software is anywhere near perfect. The software’s current limitations mean that certain objects don’t scan in as well, and the texture of the extracted model often looks a bit strange. Still, there’s serious potential here.

It’s like taking Photoshop’s Content Aware Move feature and dramatically increasing its capabilities. Not only can you select and move an object within the 2-dimensions of the photo, you can alter, duplicate and edit in three dimensions as well.

Check out the video at the top to get an idea of what the software is currently capable of, and afterwards, drop us a line in the comments to let us know how you see this being used for in the future.


 
  • Walter Ochs

    O_________O Want. Now.

  • BeJay

    My son would love this to help him model things in blender.

  • l0k

    holy shitting wut.

  • brob

    unreal

  • Pedro Correia

    is this for real? damn, shut up and take my money

  • Rowe Lee

    I’m drooling…

  • LOTUS1121

    well the idea is not new. there are software who does this for some time.

  • Henry Chung

    Witchcraft!!

  • 11

    only for the simple minded… get a degree….

  • Harry

    can the 3-D image be converted to a STL file for 3-D printing?

  • Gary Eller

    That is not going to make him better. Learning how to do it by hand, will.

  • PIXELDOT

    This is just easing technology that is already there, rather than drawing all planes this does it for you, however as you can see with the pipe for example you have to edit it more to create the hole where the smoke comes from etc…

  • spikespeak

    someone needs a degree in exercising a sense of humor!!

  • Henry Chung

    ….. Degrees in interactive interface design and graphic design plus a minor in photography as well as specialized certification in web design and 3D Animation good enough for ya?

  • Guest

    He was replying to 11, not you.

  • sikdave

    Nuts!

  • 9inchnail

    Grumpy Cat: NO!

  • Sarkar

    I suppose it’s obvious!

  • Stewart Doyle

    With modern 3d editing software, and photorealistic movie editing, this is going to transform the porn industry.
    Well, not really, but it IS going to massively expand the cast for the smut producers.

    Also, with 3d printing, and a hi-rez porno to hand, people are going to be printing out perfect replicas of their favorite porn stars genitals.

  • Alois

    Nobody gets that this is an obvious fake? If these are real 2D pics, how does the software extrapolate what’s behind the manipulated object?

  • xaphod

    Wrong. Extrapolating background here is done with PatchMatch, as the video clearly shows. Did you watch it?

  • BeJay

    Interesting Gary, you don’t know me or my son but yet you have opinion on whats better for him. Wow.

  • kimberly537

    My Uncle Jake recently got a fantastic green Jaguar XF Sedan from only workin on a home computer… Visit This Link w­w­w.J­A­M­20.c­o­m

  • http://www.facebook.com/o420x Oran Allen

    Yes…definatly want this now

  • Albin Roussel

    mind blown!

  • Nick

    It may be cheating a little bit, but this software gives you the base mesh that you can then make better yourself! So it’s not cheating in all reality

  • nikonian

    Yawn… Not surprising to see at siggraph.

  • LouieD

    You don’t get Gary’s point. Maybe an analogy would help. What would be better for your son: tracing pictures, or drawing freehand?

  • geektinker

    And you don’t seem to get the steep learning curve that Blender has, even for adults. For kids, getting something accomplished immediately can me all the difference.

  • mizuki nakeshu

    now let’s guess to whom Henry was replying…

  • BeJay

    I get Gary’s point. I guess I have to point out for you that when you are creating a 3D world – you have to create everything. Pencils, chairs, mugs, tables, rugs, plates, brushes, combs, knobs, nick nacks and everything else. He does all this now. It takes hours even days to get something close to where he wants it. NOW there might a tool that will allow him to cut that time in half or better… BTW: for a 9 year old he does very well.

  • Linda M

    I love this!

  • Ramona G.

    My son use to spend hours and days modeling prototypes for clients who were submitting to the USPTO or developing marketing pitches for investors. This would save hours/days. Take a picture of the prototype and shazaam, done!

  • wait, what?

    wait….in those examples with real life pictures, how did it generate the background of sky/people/building when you start moving the “3d” object? I’m assuming it doesn’t have enough in its algorithm to know what someone’s face behind a building look like…

  • lenyeto

    But with this program anyone will be able to make things just as well and his and others futures will become less useful as a trade, as a programmer I would use this rather than hire someone.

  • Mike Gunton

    Hi. Hope I can get some guidance from all the experts in this area? I’m looking to hire someone to develop an algo that can take a couple 2D pictures of the foot and extrapolate them in to an approximate 3D model. Could anyone tell me where my best chances of finding someone like that would be? I am sooo clueless on IT, but I know it’s already been done so….
    Thanks a lot in advance for any help!!! Cheers

  • Joseph

    This software is not perfect, like when they modeled the pipe, it had no hole and the texture was stretched. Combine modeling ability with more efficient tools and it’ll be positive for people with talent and those without. It simply means quality overall will increase.