Series

A Single Photo, Turned Into 3D — Faking Instagram's 3D Images with a Depth Map

2026-09-11 · SYSOP · 4 views

I think I bought it right around launch — I turned on my Oculus Quest 2, was using it, and happened to open Instagram, and it was pretty wild how photos and videos seemed to automatically render in 3D. Tilt your head, and whoever's in front shifts a lot while the background barely moves. It's probably done through depth estimation and object separation, and it looked quite natural, though a few photos showed its limits. Seeing it through a VR headset made it feel genuinely immersive and fun, so I spent a good while scrolling through my feed just to watch it happen.

A VR headset fundamentally relies on the parallax between two screens, so reproducing the same thing on an ordinary monitor is pretty difficult. It reminded me of the old illusion books where the picture seemed to pop out at you, so I decided to use anaglyph — the kind where one lens is red and the other cyan and things appear to jump off the page. It's a technique for tricking the brain on a single screen. I decided to test it with a photo I once took on Sowol-ro. "ILLUSO COFFEE AND MUSIC." The shop's name fits this project perfectly.

With this one photo, I want to imitate the 3D layering Instagram does in VR. Build a depth map using monocular depth estimation, then shift near things a lot and far things a little based on gaze (mouse or tilt) to create a sense of depth, running in real time in the browser. Output a single HTML file. It'd be nice to see the original and the illusion version side by side.

That's two sentences. But the real difficulty was hiding inside the one line, "build a depth map."

The Original

First, the photo I had in hand.

Original black-and-white street photo of a man standing in front of ILLUSO COFFEE AND MUSIC on Sowol-ro, Seoul

It's a composition facing the building head-on. A man in black stands on the sidewalk in front, and behind him are a brick building, a wall of glass, and a dark doorway thrown wide open. The person is closest, the sidewalk next, the building wall farther, and the inside of the open door is deepest. Translating this ordering into a grayscale map was the first job.

What Came Out

A single HTML file came out. Side by side with the original sits the illusion version, where the scene shifts with your gaze when you hover the mouse over it. Since a page can't show motion, I turned what happens when the screen auto-rocks left and right into a short looping clip.

When the gaze rocks left and right. The man and sidewalk in front shift a lot; the building and open door barely move. The difference in how far near and far things move is what creates the sense of depth.

The man up front slides left and right across the background. The building wall and sign, meanwhile, stay almost fixed in place. That difference in how much things move is the whole trick. The brain reads this difference as depth, drawing on a lifetime of experience that "things closer move faster." The segmentation isn't perfectly precise, so a few spots look a bit awkward, but overall it works pretty well.

If you have red-cyan glasses, there's another way to see it: an anaglyph, where the left-eye and right-eye parallax are split into red and cyan and overlaid in one image. Anyone with a bit of mileage has probably tried this at some point.

An anaglyph that appears to lift off the screen when viewed with red-cyan glasses. The closer the man, the wider the color offset.

The closer the man, the wider the gap between the red and cyan; the farther building has a narrower gap. The width of that offset is depth, made visible.

There's an interactive version here where you can actually move the mouse and switch modes — 👉 try it yourself in the browser. Use the mouse on a PC, or tilt the device on a phone.

How It Worked

The core of it is the depth map. There were three steps.

1. Estimate the depth. Feed a photo into a monocular depth estimation model called MiDaS, and it outputs a map that paints each pixel's brightness according to how close it is. It's a model that infers depth from an ordinary photo taken with a single camera.

The depth map the model produced. Brighter means closer. The man glows white, the sidewalk is bright, and the building and open doorway are dark — meaning far.

2. Shift the pixels according to gaze. The rest comes down to a single line of shader code. Each pixel on screen is repainted not from its original position but from the color at original position + gaze direction × strength × (that pixel's depth − reference plane). Pixels that are close sit far from the reference plane and shift a lot; distant pixels shift little. This calculation reruns every sixtieth of a second as the mouse moves. It's essentially the same computation Instagram does on a VR headset, done as a trick on a single flat screen.

Where It Got Stuck

The model mistook the man for background. I laughed when I saw the first depth map that came out. The perspective on the sidewalk and building looked plausible, but the actual subject — the man in black — had been painted as "far away" simply because he was dark, and got swallowed into the background. To a model trained to think brighter means closer, a black jacket just read as a distant shadow. If the man doesn't pop, the whole trick fails.

So I cut him out separately. Using a classic segmentation technique called GrabCut, I extracted the silhouette of the person and forced just that region to the "closest" value in the depth map. The reason the man looks especially stark white in the depth map above is this correction. It amounted to a human fixing one spot in an AI-generated draft — and this time, that one spot was everything.

The limits of a flat surface remain. Since it's a single screen, it's ultimately a trick of the eye. Move your gaze far enough and the edges of the silhouette start to look slightly stretched. The pixels that were hidden behind never existed to begin with, so when the object in front shifts aside, the background has to be invented. A headset eases this by giving each eye a different image, but a single monitor can't help but stop at "convincing enough." Still, when I let the screen slowly rock on its own, it produced something quite close to that subtle floating feeling I'd seen on Instagram.

In front of a shop named illuso — illusion — I pulled one flat photo just slightly off the screen.

It strikes me that with just a basic understanding of vision algorithms or visual systems, there's a lot of pretty enjoyable work you could do. Making anything truly polished is still hard, but with a bit of know-how and effort added on, it seems entirely possible to build something at a commercial level. Honestly, this process doesn't look like something that can be done with a single click.

It was nice to get to do something this interesting with a VR headset and Instagram after a long while. I'd like to try converting a few other photos too, or uploading them to Instagram to see them rendered in 3D.