Running Depth Anything V2 in the browser: a free depth map tool with no server

The short answer

Depth Anything V2 Small runs in a browser tab with ONNX Runtime Web. Load the 49.6 MB fp16 ONNX file from Hugging Face, resize the photo to a multiple of 14 pixels, normalise it with ImageNet's mean and standard deviation, run it, and stretch the output to 0..1. That's the whole of Gifsy's depth map generator: no server, no account, about 6 seconds per map once the model is cached.

The rest of this post is the parts that took longer than they should have: the model that wouldn't load, the cache header that re-downloads 50 MB on every visit, a guided filter for crisper edges, and a PNG encoder, because canvas.toBlob can't write the file I needed.

Why no server

I didn't want to store anyone's photos. Not “we delete it after an hour”. Just never have it. So the rule on Gifsy is simple: if something can run in the browser without making the tool painful to use, it runs in the browser.

A depth map passes that test easily. The model is small, the input is one photo, and the output is one grey image. The visitor's laptop does the work after a one-time 50 MB download, and there's nothing on our side to store, leak or pay for.

Not everything passes. The 3D photo maker fills in the background behind the subject with LaMa, an inpainting model of about 200 MB. In a browser that would have meant a huge download and a long wait before anyone saw a result, so it runs on our server, once, when someone publishes a scene. Publishing is also the one moment the finished scene gets uploaded, because an embed has to be hosted somewhere.

I also wanted the privacy claim to be checkable rather than a line in a policy. With everything local you can open DevTools, watch the Network tab, and see the model come down from Hugging Face and nothing go back up.

Getting a 50 MB model into a tab

onnx-community/depth-anything-v2-small publishes several ONNX variants. The tempting pick is the 27 MB int8 file. It fails at session creation: the int8 graph uses ConvInteger, which ONNX Runtime Web's WebAssembly backend doesn't implement. The fp16 file (49.6 MB, Apache-2.0) loads fine and takes float32 input.

import * as ort from "onnxruntime-web";

ort.env.wasm.wasmPaths =
  "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.21.0/dist/";

const MODEL =
  "https://huggingface.co/onnx-community/depth-anything-v2-small" +
  "/resolve/main/onnx/model_fp16.onnx";

// cachedFetch = fetch through the Cache Storage API (see below)
const buf = await (await cachedFetch(MODEL)).arrayBuffer();
const session = await ort.InferenceSession.create(buf, {
  executionProviders: ["wasm"],
  graphOptimizationLevel: "all",
});

Hugging Face sends CORS headers, so a plain fetch works from any origin. It also sends Cache-Control: no-store on the file (I checked on 23 September 2026), which means the browser's HTTP cache throws it away and every visit downloads 50 MB again. The fix is the Cache Storage API, the one service workers use, which ignores that header: open a named cache, return a hit if there is one, otherwise fetch and put a clone. If the stored copy is ever corrupt, the session creation throws, and we evict the entry and fetch once more.

Two small things made the first run feel less broken. The body is read with a stream reader so the progress bar moves with real bytes, and the session is created once per page and reused, because creating it takes seconds and there's no reason to do it twice.

Preprocessing

The model's preprocessor config asks for three things: keep the aspect ratio, make both sides a multiple of 14 (the ViT patch size), and normalise with ImageNet's mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225]. The native size is 518. On machines that report at least 8 GB of memory and 8 cores I use 770, which gives the edge filter later more to work with.

const scale = Math.min(size / w, size / h);       // size = 518 or 770
const iw = Math.max(14, Math.round((w * scale) / 14) * 14);
const ih = Math.max(14, Math.round((h * scale) / 14) * 14);

ctx.drawImage(img, 0, 0, iw, ih);                // the canvas does the resize
const px = ctx.getImageData(0, 0, iw, ih).data;  // RGBA, interleaved
const n = iw * ih;
const data = new Float32Array(3 * n);            // planar RGB (NCHW)
for (let i = 0; i < n; i++)
  for (let c = 0; c < 3; c++)
    data[c * n + i] = (px[i * 4 + c] / 255 - MEAN[c]) / STD[c];

const input = new ort.Tensor("float32", data, [1, 3, ih, iw]);

One shortcut I took knowingly: the reference preprocessor resizes with bicubic, and I let the canvas do it instead. I haven't compared the two side by side; it saved writing a resampler, and the output looked right on every photo I tried. The loop that matters is the one that turns interleaved RGBA into planar RGB, because ONNX wants [1, 3, H, W].

Running it and normalising

The output, predicted_depth, is a float32 grid the same size as the input, and larger values mean closer. The absolute numbers mean nothing on their own: they're relative, and they drift from photo to photo. So each map is stretched to 0..1 between its own 0.5th and 99.5th percentiles, which stops a single bright specular highlight or a patch of sky from squashing everything else into grey mush.

const out = await session.run({ [session.inputNames[0]]: input });
const raw = out[session.outputNames[0]].data as Float32Array; // ih × iw

const sorted = raw.slice().sort();
const lo = sorted[Math.floor(sorted.length * 0.005)];
const hi = sorted[Math.ceil(sorted.length * 0.995)];
const span = Math.max(hi - lo, 1e-6);
for (let i = 0; i < raw.length; i++)
  raw[i] = Math.min(1, Math.max(0, (raw[i] - lo) / span));

Sorting a copy is O(n log n) on about 430,000 values for a 770 × 560 grid. A selection algorithm would be faster; I haven't needed it yet.

Sharpening edges with a guided filter

Raw monocular depth is soft at object boundaries. The model sees the world in 14-pixel patches, so the edge of a dog against a field turns into a smear a few pixels wide. For a depth map that feeds a displacement modifier or a lens blur, that smear is exactly where the artifacts appear.

The fix is He, Sun and Tang's guided filter, with the photo's own luminance as the guide. Inside each window it fits depth as a linear function of brightness, so where the photo has a hard edge the depth gets one too, and where the photo is flat the depth is smoothed. Every step is a box mean, and a box mean over an integral image costs the same whatever the radius, so the whole filter is O(n).

// I = photo luminance, p = depth; both 0..1 at the model's resolution
const r = Math.max(3, Math.min(16, Math.round(Math.min(w, h) * 0.02)));
const eps = 4e-3;

const mI = box(I), mp = box(p), mII = box(I * I), mIp = box(I * p);
const a = (mIp - mI * mp) / (mII - mI * mI + eps);  // per pixel
const b = mp - a * mI;
const q = box(a) * I + box(b);                      // sharpened depth

The radius is 2% of the grid's short side, clamped to 3–16: 11 pixels on a 770 × 560 grid, 8 on a 518 × 378 one. eps decides what counts as an edge; at 4e-3 on 0..1 luminance, low-contrast texture gets smoothed while a strong outline, like a dog against a field, keeps its edge. The filter has one known failure: it can't tell a shape edge from a printed one, so strong text or a logo can show up as a faint ridge. That's why “Sharpen edges” is a switch and not a hard-coded step.

Upscaling and Invert

People want a depth map the size of their photo, not 770 pixels wide. The download is bilinearly upscaled to the photo's size, capped at 2048 on the long edge. I say “upscaled” on the page because it's true: no detail gets added above the working resolution, and I'd rather not print “4K” on something that isn't.

Invert is one subtraction. It exists because apps disagree on which way round a depth map goes: Blender's Displace modifier and ControlNet read white as near, Photoshop's Lens Blur reads black as near. Both toggles work on the cached result, so flipping one takes about a tenth of a second and never re-runs the model.

Writing a real 16-bit PNG

This was the part I didn't expect to write. canvas.toBlob("image/png") always gives you an RGBA file at 8 bits per channel. For a depth map that means three identical channels, an alpha channel you don't need, and only 256 grey levels. Drive a displacement with 256 levels and you get terraces: flat steps where a slope should be.

A grayscale PNG is a simple format, and the browser already ships the one hard part, zlib. CompressionStream("deflate") produces the zlib-wrapped stream PNG expects ("deflate-raw" is the unwrapped one). What's left is the 8-byte signature, an IHDR chunk, one IDAT chunk, IEND, and a CRC-32 over each chunk's type and data, built from a 256-entry table.

// one filter byte per row (0 = None), then big-endian samples
raw[row] = 0;
const q = Math.round(v * 65535);
raw[row + 1 + x * 2] = q >> 8;
raw[row + 2 + x * 2] = q & 0xff;

// IHDR: width, height, bit depth 16, colour type 0 (grayscale)
ihdr[8] = 16;
ihdr[9] = 0;

// "deflate" in CompressionStream is the zlib format PNG wants
const idat = await new Response(
  new Blob([raw]).stream().pipeThrough(new CompressionStream("deflate")),
).arrayBuffer();

It comes to about 60 lines without comments, with no dependency. Colour type 0 is grayscale, bit depth 8 or 16, samples big-endian. I checked the output with Python's Pillow: the 16-bit file opens as mode I;16 with 65,508 distinct values out of 65,536, and the 8-bit one as mode L with 256.

One thing I skipped: PNG row filters. Every row uses filter 0, None. The Up or Paeth filters would shrink the 16-bit file a lot, since neighbouring depth values are close. It's on the list; for now a 1.8 MB file for a 1200 × 872 map is fine.

Measured numbers

Measured on 23 September 2026 in headless Chromium on a 10-core, 16 GB Mac (so the 770-pixel working size), with our end-to-end script driving the real page:

The 6 seconds covers everything after the click: decoding the photo, the tensor loop, inference, the guided filter, upscaling and encoding the preview. I haven't tried the WebGPU backend for this model yet; the sticker cut-out on this site went from about 10 s to 4 s when it moved to WebGPU in a worker, so that's the obvious next experiment.

The harder version: a free limit on a model that runs in the browser

Update, October 2026: Gifsy's 3D photos are now free and unlimited, so there's no meter any more. Every account downloads the head and runs the whole model locally, and the server head is only a fallback for when that fails. The rest of this section is how the limit worked while there was one.

The depth map tool is free with no limit, so it had it easy. The 3D photo maker used the same model and gave free accounts 3 generations. That raised a question I didn't have a good answer to at first: if the model is cached in the visitor's browser, how does our server know they've used their 3?

My first version asked the server nicely. Before each 3D generation the browser called an endpoint that counted it. That works until someone opens DevTools, deletes the call and keeps generating. The count was advisory, and anyone who reads JavaScript could skip it.

The fix was to split the model in two. Depth Anything V2 is an encoder (a vision transformer, about 44 MB in fp16) followed by a small head (the DPT decoder, about 5 MB) that turns the encoder's features into a depth map. I cut the ONNX graph at the boundary between them: four feature planes of 384 channels each, plus the grid size.

• The browser always runs the encoder. Its output is activations, not a depth map, so on its own it can't make a scene.
• On the free plan, the browser posted those activations to our server, which ran the head and sent the depth map back. The counting happened in that request, with an atomic claim in the database so two tabs can't both spend the last generation. The meter sits on the request that does the work, which is the one request a client can't skip.
• Pro accounts downloaded the head once and ran the whole model locally. There was nothing to count, and nothing left their machine while the scene was made. Since October 2026 that is the path for every account.

It fit the rule about data too: a free generation sent numbers derived from the photo, never the photo itself.

Two things bit me. First, Vercel rejects request bodies over 4.5 MB before they reach the function. At the 770-pixel working size the four fp16 planes come to 9.29 MB, so every free generation on a fast machine failed in production with a bare 413, while working perfectly on localhost. The server path now shrinks the working size until the upload stays under 1,100 patches, about 3.4 MB.

Second, on current Chrome a float16 tensor's .data is a Float16Array, and onnxruntime-web's Tensor constructor wants a Uint16Array. I was rebuilding the encoder's outputs as new tensors before feeding them to the local head. The rebuild threw, the fallback caught it, and every Pro generation quietly went to the server instead, where it hit the same 413. The fix was to stop rebuilding: the split kept the tensor names, so the encoder's outputs go straight into the head as they are.

To be clear about what this is and isn't: Depth Anything V2 Small is public on Hugging Face, and this article shows how to run the whole thing. Someone determined could rebuild the 3D pipeline around the full model, and the split doesn't stop that. What it did was make the easy path through Gifsy's own app the counted one, which is all a 3-generation free tier needed.

Limits

• Relative, not metric. The map knows the cup is in front of the window. It doesn't know it's 40 cm away.
• Resolution. The model sees 518 or 770 pixels. Everything above that is upscaled, with the guided filter doing what it can for edges.
• Mirrors, glass and sky. A mirror reads as a room behind the wall, a window as whatever is past it, and a plain sky has nothing to measure.
• Fine hair and fur. Strands thinner than a 14-pixel patch blend into the background depth.
• First visit. 50 MB is a real download on a phone connection. After that it's cached.

If you want to try it, the depth map generator is free and doesn't need an account. And if what you actually want is the photo itself moving, with the subject sliding in front of the background, that's what the 3D photo maker does.

FAQ