MuJoCo is a free and open source physics engine that aims to facilitate research and development in robotics, biomechanics, graphics and animation, and other areas where fast and accurate simulation is needed.
MuJoCo offers a unique combination of speed, accuracy and modeling power, yet it is not merely a better simulator. Instead it is the first full-featured simulator designed from the ground up for the purpose of model-based optimization, and in particular optimization through contacts.
Agilex achieved Mobile Aloha simulation in Mujoco and will share the instructions in this topic.
Build environment
The system version of this project is Ubuntu20.04
Install MuJoCo
-
Unzip the downloaded
mujoco210to~/.mujoco/mujoco210.
Install and use mujoco-py
For more details: GitHub - openai/mujoco-py: MuJoCo is a physics engine for detailed, efficient rigid body simulations with contacts. mujoco-py allows using MuJoCo from Python 3.
To install and use mujoco-py, please make sure the below libraries were installed.
sudo apt install libosmesa6-dev libgl1-mesa-glx libglfw3
If you have the above libraries installed but still see -lGL not found errors, you most likely need to create the symbolic link directly:
sudo ln -s /usr/lib/x86_64-linux-gnu/libGL.so.1 /usr/lib/x86_64-linux-gnu/libGL.so
To include mujoco-py in your own package, add it to your requirements like this:
mujoco-py<2.2,>=2.1
Enter the Aloha simulation environment
The directory structure is as follows:
aloha_mujoco
└── aloha
├── CMakeLists.txt
├── meshes_mujoco
│ ├── aloha_v1.xml
│ └── meshes_mujoco
├── package.xml
└── scripts
├── aloha_ctrl.py
└── aloha_ctrl_test.py
Among them,meshes_mujoco mainly stores stl files and description xml files of mujoco.
Open the aloha_v1.xml file in mujoco:
~/.mujoco/mujoco210/bin/simulate /<your path>/aloha_mujoco/aloha/meshes_mujoco/aloha_v1.xml
The following interface will be displayed:
Mujoco interface introduction
Right column
Joint displays the angle of each joint in radians.
Control is used to control joint angles. Unit: radians.
Drag the slider in
control to control the movement of a certain joint of the robotic arm.
Scroll down the left column to see the
Rendering tab. Click it and slide down. After the Camera option lights in the Model Elements tab, you can see the views of the three cameras:
Enter Aloha simulation with python files
1. Run ‘test’
Note: Please make sure you installed mujoco_py, and add below in bashrc:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin
export LD_PRELOAD=$LD_PRELOAD:/usr/lib/x86_64-linux-gnu/libGLEW.so:/usr/lib/x86_64-linux-gnu/libGL.so
Run the aloha_mujoco/aloha/scripts/aloha_ctrl_test.py file in the folder. Seven windows as below will pop up, but three of them have no pictures because the camera pictures are sent through another window created by OpenCV.
The robotic arm will move every 2 seconds, and you can see that the camera image changes accordingly.
2. Let two robot arms in Mojuco follow the movement of the main robot arm in the real world
(1) Firstly, open a new terminal and runroscore
(2) Secondly, run the master robotic arm node. Open both robotic arm nodes, and check whether the data of /master/joint_left and /master/joint_right are normal.
(3) Finally, run the aloha_ctrl.py file. If there is no problem with the operation, seven windows will pop up as above, and you can move the main robotic arm to control the robotic arm in the simulation environment.
Troubleshooting
Missing GLFW
Common error during installation is:
raise ImportError("Failed to load GLFW3 shared library.")
This happens when the glfw python package cannot find the GLFW dynamic library.
MuJoCo has a copy of this library that can be used during installation.
Add the path to the mujoco bin directory to the dynamic loader:
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco210/bin