New physics sim trains robots 430,000 times faster than reality
According to reports, the AI-generated world will include realistic physics, camera movement and object behavior, all from text commands. The system then creates physically accurate ray-traced movies and data for the robot to use for training. Of course, we haven’t tested this yet, so these claims should be taken with a grain of salt for now.
An example of a “4D dynamic and physical” world created by Genesis based on text prompts.
This prompt-based system allows researchers to create complex robotic testing environments by entering natural language commands instead of manual programming. “Traditionally, simulators require a lot of manual work from artists: 3D assets, textures, scene layout, etc. But every component in the workflow can be automated,” Fan wrote.
Using its engine, Genesis can also generate character movements, interactive 3D scenes, facial animations, and more, which may allow for the creation of artistic assets for creative projects, but may also lead to more realistic AI-generated games and videos in the future, built from data A simulated world rather than operating on the statistical appearance of pixels like a diffusion model for video synthesis.
An example of character motion generation in Genesis, using prompts such as: “A miniature Wukong with a stick in his hand sprints across the table for 3 seconds, then jumps into the air, swinging his right arm downward as he lands.”
Although the build system is not yet part of the currently available Code on GitHubthe team plans to release in the future.
Training tomorrow’s robots today (using Python)
Genesis is still under active development on GitHubthe team accepts community contributions.
The platform stands out from other 3D world simulators for robot training by using Python as its user interface and core physics engine. Other engines use C++ or CUDA for low-level operations while wrapping them in a Python API. Genesis takes a Python-first approach.
Notably, the non-proprietary nature of the Genesis platform allows any researcher to conduct high-speed robot training simulations for free through simple Python commands that run on ordinary computers with off-the-shelf hardware.
Fan said in a post announcing Genesis that running robot simulations previously required complex programming and specialized hardware, but that shouldn’t be the case. “Robotics should be a moonshot project owned by all of humanity,” he wrote.
2024-12-20 10:20:17