3D Emoji Kitchen3D Emoji Kitchen

From a sticker to a solid

Emoji Kitchen has 147,442 combinations, and every one of them is a flat sticker. Google redrew its 4,000 base emoji in 3D this year; the combinations stayed 2D. I wanted to know how many of the combinations an image-to-3D model could turn into solid objects on its own, with one rule for every result: seen from the front, the model has to match its sticker.

So far 44 are in the gallery, each one beside the sticker it came from. This is how the pipeline works, what I tried along the way, and what still goes wrong.

The stack

The cluster runs Blackwell GPUs, and neither model's pinned version of PyTorch supported them. Both needed fresh environments on the CUDA 12.8 builds of PyTorch, plus a CUDA 12.8 and gcc 12 toolchain to compile Hunyuan's rasterizer extensions. Everything after generation runs on my Mac.

Inflating the traced shapes

I had already traced the stickers into flat regions of color for the vector work, so the first attempt reused them. inflate.py turned each region into a cushion: its height rises with its layer, and it puffs up with the square root of the distance from its edge. The huitlacoche sticker became 141 cushions and a 67,000-vertex mesh with a flat back.

It read as a relief, a sticker with depth. I wanted models that could exist as physical objects, so I dropped it after one try.

Three image-to-3D models

TripoSR ran on my Mac's CPU and made a watertight 61,000-vertex mesh in a few minutes. It came out lumpy and low in resolution, with contours that wandered from the sticker's.

TRELLIS.2, Microsoft's 4 billion parameter model, took more setup: sparse attention through xformers, a DINOv3 image encoder from a mirror while my access to Meta's copy was pending, and a stubbed background remover. The huitlacoche took 133 seconds, peaked at 6.7 GB of GPU memory and came out at 5.2 million vertices in a 26.5 MB file. It looked the most real of anything up to that point, with too many kernels, a few holes and colors that were too bright. Frog plus stew and heart eyes plus © both failed. I deleted its 22 GB of weights a month later.

Hunyuan3D 2.1 made the shape in 40 seconds at 8.2 GB, with 450,000 to 760,000 vertices, watertight every time. My note on its first shape was one word: impeccable. Its paint model started at 6 views of 512 px. I compared that with 12 views at 768 px on 15 combinations and kept 12 at 768. That became the recipe: shape, paint, pack, and no paint fixes afterward.

Fixing the paint afterward

Hunyuan paints the parts of the model the sticker never shows, and it guesses. Frog plus stew came back with a white back, its average brightness 185 out of 255 behind a dark front. I spent a day trying to correct the paint after generation, and every attempt made it worse.

Projecting the sticker onto the mesh painted straight through it, so the back showed the front. Filling unseen surfaces from their seen neighbors turned the © discs yellow; 94% of the filled texels came out yellow. Swapping the paint model's own front view for the real sticker made all 17 test combinations worse, with the sticker visibly projected across the model. Snapping colors to 14 palette colors taken from the sticker merged the rocket's white into its gray, and I rejected every snapped model.

Reseeding the paint changed nothing at first: the seed was hard-coded to 0 inside the paint model's code, so every texture came out byte for byte the same. After I patched that, four paint seeds and two shape seeds all still painted the frying pan white, like a plate. The paint model's caption did nothing either; the texel under my pin came back identical with and without it. Weighting the front view four times as heavily moved that texel by 2 levels out of 255.

The result of that day was a rule: the paint has to come from the generator, and the way to get better paint is to draw again with another seed.

Pose

Hunyuan builds objects standing up. The huitlacoche's cob lies at 49.6 degrees. Four seeds failed on it; the first came back as a green lump with an outline match of 0.61.

The fix rotates the sticker until its long axis is vertical, generates the model upright, and turns the mesh back. The cob needed 40.4 degrees, and its outline match rose to 0.941. Balloons, which lean, went from 0.84 to between 0.95 and 0.97 on 8 of 12. It now runs on its own whenever a sticker's long axis is 1.4 times its short one and leans 8 degrees or more.

Layers

The most common geometric failure is layers. Stickers draw features flat on top of each other, and Hunyuan sometimes builds each feature as its own slab at its own depth. Loudly crying plus stars has its tears, eyes and mouth on three separate planes; melting face plus sun duplicates the melt on three layers; nauseated plus stars put the vomit in front of the teeth.

Every combination now gets two draws with different seeds, and the pipeline keeps the better one. The first draw of nauseated plus stars had the vomit in front of the teeth; seed 6 put it in the mouth, and I approved it.

Mouths

Hunyuan builds open mouths as bumps that stick out of the face. fix_mouth.py finds the mouth's vertices that face the camera and pushes them back along the depth axis into a bowl 0.26 times as deep as the mouth is wide. The front view stays the same. On heart eyes plus balloon, the critic's mouth indent went from -3.7% (sticking out) to 2.7% (set in), and the mesh went from 40,000 faces to 115,000.

The first run fixed 7 of 9 mouths; the other 2 flattened an eye, and a guard at the face's midline now prevents that. Carving fixes the mouth it targets and nothing else about a model.

Templates and composites

For faces, I also built the opposite approach: a template engine that puts the sticker's eyes and mouth, cut from the PNG, onto a production 3D head. Its outline match was 0.982 against 0.993 for the best generated face, and its color error was 1.6 against 35.1. I rejected all three of its faces anyway and went back to generation.

Crystal ball plus pizza came out with the pizza inside the ball, so the gallery version is a fitted glass sphere around Google's production 3D pizza. Two saucer composites, a generated passenger in the beam of the production saucer, came out wrong and stayed out of the gallery.

The critic

critic3d.py renders each model straight from the front at 320 px, color and depth, and compares it with the sticker. It rejects a model when the outlines overlap by less than 0.68, which catches the wrong pose; when the model has 3 or more separate parts or sharp depth cliffs over 4% of its area, which catches layers; and when the mouth sticks out by more than 1.5%. I set those thresholds on my first 70 verdicts.

It works as a gate. Its weighted score failed as a ranking: on 83 models it didn't separate my yes from my no. The gate sees geometry. It misses a missing mouth, the wrong expression, a highlight built as a raised patch, sunglasses with no lenses and a face built as a flat disc 0.31 units thick. The next step is a vision model as the judge.

Running it as a queue

The process now runs as a queue. I write a list of combinations, the cluster generates two seeds of each, and ingest3d.py on my Mac pulls the results, packs them, scores and gates them, keeps the better draw, carves mouths, rebuilds the gallery and pushes everything to its repositories.

The first queue was 200 combinations: 400 shapes and 400 paints in about 21 hours with no errors, about 6 minutes a combination. Ingesting took about 2 hours, and carved 106 mouths along the way. Between September 29 and October 11 the cluster's storage quota was full, and nothing ran.

Where it stands

The approved faces match their stickers' outlines at 0.97 to 0.997. Color is the open problem. Outline match is how much the model's silhouette from the front overlaps the sticker's. Color error is the CIE ΔE between the two images' colors; about 2 is the smallest difference most people can see. Hunyuan paints yellow faces tan orange, around 30 on most faces, and paints mouths olive where the stickers have dark brown. Red balloons come out closest, at 12.3 to 15.4.

The gallery's 44 models are 31 faces, 8 balloons, 4 objects and 1 composite, 174 MB in all. The next three queues are written: the 200 most used faces, 40 popular combinations the pipeline is likely to get wrong, and 100 faces and objects ranked by how often they're used.