The researchers additionally expanded the system’s skills by including texture and colour choices. For instance, utilizing an look immediate like “Electrical guitar in metallic purple,” LegoGPT can generate a guitar mannequin, with bricks assigned a purple colour.
Testing with robots and people
To show their designs labored in actual life, the researchers had robots assemble the AI-created Lego fashions. They used a dual-robot arm system with drive sensors to select up and place bricks in keeping with the AI-generated directions.
Human testers additionally constructed a few of the designs by hand, displaying that the AI creates genuinely buildable fashions. “Our experiments present that LegoGPT produces secure, various, and aesthetically pleasing Lego designs that align intently with the enter textual content prompts,” the crew famous in its paper.
When examined in opposition to different AI programs for 3D creation, LegoGPT stands out by means of its concentrate on structural integrity. The crew examined in opposition to a number of options, together with LLaMA-Mesh and different 3D era fashions, and located its method produced the very best proportion of secure buildings.
A video of two robotic arms constructing a LegoGPT creation, supplied by the researchers.
Nonetheless, there are some limitations. The present model of LegoGPT solely works inside a 20×20×20 constructing house and makes use of a mere eight commonplace brick sorts. “Our technique at the moment helps a set set of generally used Lego bricks,” the crew acknowledged. “In future work, we plan to increase the brick library to incorporate a broader vary of dimensions and brick sorts, resembling slopes and tiles.”
The researchers additionally hope to scale up their coaching dataset to incorporate extra objects than the 21 classes at the moment obtainable. In the meantime, others can actually construct on their work—the researchers launched their dataset, code, and fashions on their mission web site and GitHub.
