Taking a photograph of a wrinkled sticky note is pretty easy. Point a camera and shoot. But that image won't tell us what the sticky note looks like from the back. Or from the side. Or zoomed in. So we can't really make a cool animation like in the video above. For that, we'll need a 3D model, and like with anything 3D, the complexity grows exponentially compared to a 2D photo.
One technique is getting your hands dirty in a 3D app like Blender, spending days, maybe even weeks, getting all the 3D coordinates right. There are great videos on Youtube that will teach you that, but what if we could make that model in 5 minutes, with no experience?
Another technique is photogrammetry. The idea is that if you let a computer see a sticky note from a lot of different angles, it can figure out its 3D shape - all on its own, while we drink coffee.
It's is not a new idea (150 years) What's new is that the tech will ship to hundreds of millions of Macs this fall, making it trivial for app developers to let their users create 3D models from images.
I thought I'd give it a try. Here's what I used
The app for taking the photos is optional, but it has an auto-mode where it fires every second or so, and it generates a few extra formats like depth and gravity.
I took 58 photos thanks to my being impatient. Apple recommends up to 200, depending on the object. Getting the photos out of the app was a bit tricky, but I ended up using the Files app. Find the Capture folder, then AirDrop the session folder with all the photos in one go.
On the Mac, I built this command line tool
And then I really just dragged the built product from Xcode into my Terminal and ran it with two parameters:
1. The path to the folder with the images
2. A name for the USDZ model, e.g. stickynote.usdz
A few minutes later I had my model. I double clicked it and it opened in Xcode's Scenekit editor.
I used another beta, Rotato with Custom Model import to create my scene, and then just animated it as if I animated any other Rotato scene.
Here's the result. You should be able to view and download in 4K through Vimeo.
You can also grab the .rotato files here and play around with it. It didn't go over well in the macOS beta, but my Big Sur M1 had no problem rendering, so probably just a hiccup in the very first beta of the year.
Okay, so I mentioned I had been lazy and only used 58 photos. What would happen if I picked a harder object, but took, say, 297 photos? I went to the kitchen and grabbed the first messy object I could find. To get all sides, I picked up the flour bag with my fingers, hoping they wouldn't make a cameo on the final model.
And luckily, they didn't