Toy robot arm simulator for on-the-fly training & learning about small models

AMP-Robot

A browser-only demo where you type or say an instruction and a very small AI model controls a toy robot arm. The model was trained on an everyday PC workstation. Its small, but it can follow these simple instructions (most of the time)!

The demo includes a model viewer that I’ve wanted for a while. You can fly through model layers in 3D, styled (Gibson) view! It’s fun, but I’ve found it useful for understanding real models. The code extends to all pytorch and LeRobot models so SmolVLA and Cosmos3 work, and seeing SmolVLA’s repeated passes laid out in space made flow matching easier to understand.

Background: We’re a small company building modular robot hardware. Robots that run AI on board need small models, and even fine-tuning the big ones was painful on our workstations. The standard robotics datasets also filled up my disk!

The project started by building a minimal on-the-fly dataset generator, which was coded by hand to check if I could still do 3D trigonometry (nope!), later it was passed to Qwen3.8 27B running locally, and finished off with Claude Opus 5.5.

It generates images and language instructions on the fly. That gives you a cheap way to check whether a model can learn a simple task before you spend days training on real data. The included 6.5M parameter model gets about 95% on pick-and-move instructions in unseen scenes.

The data generator and model are open. Constructive feedback and comments welcome: which models you want to see and what would make it more useful?

These bits are Apache licensed: On the fly data generator: https://github.com/Adaptive-Machine-Patterns/amp-toy-robotic… - Model: Adaptive-Machine-Patterns/amp-tadpole · Hugging Face

1 Like