DEV Community: Simone Zandara

Robotics on WSL2 using ROS, Docker and Unity 3D (Part II)

Simone Zandara — Tue, 16 Mar 2021 19:13:58 +0000

Not that this is the second part on how to setup ROS to work with WSL2 (Part I). However, the upcoming section would work in a full Linux setup as well.

Robotics for Unity

There are several ways of using Unity for robotics. Robotics for Unity is a new field and support is still custom made. Most of the applications out there are proprietary software created by private companies and not easily accessible or configurable by others. ROS support has been recently introduced by Unity itself, and it is a fork from a previous library from Siemens' RosSharp. In my setup, I used the Unity ROS library which works with the latest ArticulationBody types.

For this demo project I have chosen a cute robotic manipulator which is open sourced: Reachy from Pollen Robotics. To use a robot with ROS in Unity you need an existing URDF (robot file descriptor) which is usually shipped by the robot manufacturer. In specific I have decided to work on a static robotic manipulator and operate it with MoveIt!.

There are already quite few tutorials on how to do that in Unity, however, I took the opportunity to create a MoveIt! connector for Unity which allows to control the robot using the MoveIt! MoveGroup API rather than creating custom services as done by other tutorials. I find this solution easier to plug to other robots since it uses all standard ROS messages and does not require an additional service.

Both Unity and ROS share the same copy of the robot descriptor URDF. Unity will serve as main source of the joint state of the robot. MoveIt! will use the information streaming from Unity to plan the route and send the arm trajectory back to Unity which will actuate the joints.

MoveIt! will communicate the trajectory to Unity using a custom ROS topic on which a custom created Unity MoveIt Handler publishes the trajectory to execute.

The custom code for the project has 3 main components which are in Github:

unity_connector_manager.cpp which sends the trajectory to Unity after calling execute() on a planned MoveIt! trajectory.
unity_tcp_server.py which acts as proxy between ROS and Unity
ROSMoveItControllerGeneric.cs which controls the robot in Unity and communicates with ROS.

ROS

Let's now install our ROS distribution. I am using ROS1 because lots of necessary libraries are missing in ROS2. ROS will only be installed inside the docker even though I have a ROS version also installed in my WLS2 for quick tests.
Along with ROS, the docker image will contain few libraries which are available in Github and we will use for our demo.

First, create a workspace folder in your WSL home (or wherever you please)

zandara@XXX:$ cd ~
zandara@XXX:$ mkdir unity_demo
zandara@XXX:$ cd unity_demo
zandara@XXX:$ git clone https://github.com/szandara/unity_reachy_tutorial
zandara@XXX:$ cd unity_reachy_tutorial

We will now build the Docker image which contains the custom code and the ROS distribution. Run ./build_docker and go grab a tea.

Next, we will collect the code for Reachy in ROS. This code is outside of the Docker and will be mounted when the Docker is started.

zandara@XXX:$ cd src
zandara@XXX:$ git clone https://github.com/szandara/reachy_description_ros1
zandara@XXX:$ git clone https://github.com/pollen-robotics/reachy_moveit_config

I am using a fork of Reachy's URDF for ROS1 since they recently moved to ROS2.

Now all is setup to start using our ROS system!

Unity 3D

Let's now setup Unity to work with ROS over Docker. But first, let's install Unity3D, if you still do not have it. https://unity3d.com/get-unity/download. I am working with the community version (2020.2.5f1).

Let's create a project called reachy_demo
Open Unity and go to the Package Manager. Window -> Package Manager -> + sign -> * Add package from git URL.
Add the following packages.

https://github.com/Unity-Technologies/ROS-TCP-Connector.git#v0.1.2 
https://github.com/Unity-Technologies/URDF-Importer.git#v0.1.2

Make sure they compile without errors. The two packages allow Unity to:
- Import URDF files for robot models (which is the ROS standard).
- Communicate with a ROS proxy (which will run on the docker).
Keep the ROS/Unity settings in Robotics -> ROS Settings as default (unless you need to change it for some reason)

Finally open the TCP 5005 port for ROS to talk to Unity. It's fairly straight-forward as explained here. See Part I for the reasons.

Setup Reachy in Unity

Go to GitHub and download https://github.com/szandara/reachy_description_ros1 as a zip file.
Go to your Unity Asset directory and create a subfolder called URDF.
Copy and extract the downloaded reachy_description.zip into Assets/URDF/reachy
Copy the file Assets/URDF/reachy/reachy_description_ros1/reachy.URDF to Assets/URDF/reachy/reachy.URDF
Right click on reachy.URDF and Import Robot from selected URDF. A window will popup giving you few options. Change Mesh Decomposer to Unity unless you have VHACD.

Select reachy from the GameObject list on the left. In the inspector on the right, find Controller (Script) and set the following values.

Note that these are arbitrary values which I manually tuned. Accurate values could be provided by the manufacturer.

Select torso from the GameObject list on the left and tick Immovable in Articulation Body

We are done with importing Reachy. We should now see it in our simulated world!

Setup MoveIt! in Unity

Here we setup a script to control Unity's ArticulationBody objects with MoveIt! messages from ROS. I have uploaded the script in this repository https://github.com/szandara/unity_moveit_connector/blob/master/Editor/ROSMoveItControllerGeneric.cs

Copy paste the content of the file directly into a file named Assets/moveit/ROSMoveItControllerGeneric.cs make sure Unity compiles it without errors.
Finally, add the script to Reachy by selecting r_shoulder then Add Component -> Scripts -> ROS Move It Controller Generic
Add the parameters as seen in this example
- Joint State Topic Name: joint_states
- Move It Move Group: unity_right_arm_controller
- Joint Group: Drag and Drop the r_shoulder joint from the Unity tree as visible in this screenshot.

We are done! We have setup Reachy to talk to ROS!

Combining all

As last step, we need to start the system. Let's dive right into it.

First, let's start Unity if it's not started already and open our project reachy_demo
Open a WSL shell and go to our project folder unity_demo
Edit the file unity_demo/src/unity_reachy_tutorial/src/unity_reachy_tutorial/launch/unity_connector.launch and set these parameters using the values that were configured in Unity. Note that UNITY_IP is set by Windows and it's the IP used to communicate with WSL2. You can find this IP from WSL by running grep nameserver /etc/resolv.conf

    <param name="ROS_IP" type="str" value="$(env ROS_IP)" />
    <param name="ROS_TCP_PORT" type="int" value="10000" />
    <param name="TCP_NODE_NAME" type="str" value="TCPServer" />
    <param name="TCP_BUFFER_SIZE" type="int" value="1024" />
    <param name="TCP_CONNECTIONS" type="int" value="10" />

    <param name="UNITY_IP" type="str" value="XXX.XXX.XXX.XXX" />
    <param name="UNITY_SERVER_PORT" type="int" value="5005" />

Run the following

zandara@XXX:$ ./launch_docker
root@docker:~# source /opt/unity_demo/unity_reachy_tutorial/devel/setup.bash 
root@docker:~# roslaunch unity_reachy_tutorial reachy_demo.launch

You should see a confirmation that you can start planning.

Open another WSL and go to our project folder unity_demo
Run the following

zandara@XXX:$ ./launch_docker
root@docker:~# rosrun moveit_commander moveit_commander_cmdline.py

You should see the MoveIt command line.

Go to Unity and start the game, there should be no errors in the console
Go to the MoveIt command line (in WSL2) and run the following

> use right_arm
> current

Make sure you see no errors. If you receive an error the problem could be related to the time offset of Windows vs WSL2. To fix it, open a WSL2 console and run the following

sudo ntpdate time.windows.com

If you have no errors continue planning the route in MoveIt

> rec c
> goal = c
> goal[0] = -2.1
> plan goal
> execute

You should see Reachy moving its right arm to the top! Congratulations in planning your first Unity simulation!

Conclusions

We have shown how to setup Windows to work as powerful Linux development environment.
We have shown how Unity can be used as simulation environment for robotics
We have shown how ROS and Unity exchange messages
We have shown how to control Reachy on Unity with MoveIt!.

I had lots of fun doing this project and I am going to pursue a personal project using this setup. However, the solution is far from perfect. The pitfalls experienced due to Windows and WSL2 not working well together is substantial and by no means ensure a relaxed development environment for professional use.

The lack of support for ROS2 by Unity has also lead to produce a probably soon outdated ROS project and an eventually painful migration. Unity has announced that ROS2 support is under development and I will make sure to update this article with the new ROS2 version.

Nonetheless, it was rewarding to know that I can now develop robotics comfortably from my Windows host!

Robotics on WSL2 using ROS, Docker and Unity 3D (Part I)

Simone Zandara — Mon, 15 Mar 2021 18:23:04 +0000

Have you ever wanted to just be able to write code on Windows but with the familiar tools you always use at work on your Linux station? Have you ever wanted to be able to just close your Cyberpunk 2077 quest and start tinkering with your latest robot project in ROS?

I have always developed under Linux but I recently decided to leverage my Windows laptop for my coding projects. I do occasionally game and use graphic applications so I did not really want to move to Linux and I was super lazy to partition the hard drive. Moreover lots of tools for sensors or robots are developed for Windows only and it's often inconvenient to switch back and forth.

I decided to give a try to the new Windows WSL feature. For those who don't know, WSL is a Windows feature which allows you to have a running Linux image in your Windows Desktop. Yes you heard correctly! No more need to partition your drive!
In this article I will go through how I set up my old laptop to run some cool robotics simulation.

Let's dive right into it!

Setup WSL (the right one!)

WSL is under quite some development and there are actually 2 versions around. The latest version WLS2 has much better support for GPU hardware and docker than its predecessor. To install WSL2 you need to have a developer preview of Windows which you can conveniently install from your Windows Update. After joining https://insider.windows.com/en-us/getting-started and updating your Windows to the latest build, follow the steps below
https://docs.microsoft.com/en-us/windows/wsl/install-win10
Once done, choose your Linux distribution and set it up the way you like. I chose Ubuntu 20.4.

Finalize by setting up your display in WSL2. Add this to your ~/.bashrc

export DISPLAY="`grep nameserver /etc/resolv.conf | sed 's/nameserver //'`:0"
export LIBGL_ALWAYS_INDIRECT=1

Note: In reality, I had a pretty hard time updating my outdated Windows version to the latest build. Updates kept on failing with weird reasons and I had to manually turn off several services for some updates to be installed. Good luck!

GPU & Docker

For the sake of this demo, having a working GPU is not necessary, however, if you have one, I strongly encourage you to setup your WSL system to use it so you will be ready to move your robot to the deep learning world. If you do not have a GPU skip to the Docker Only section.

Note: when installing Docker, use the same hard drive of the WSL installation or you will notice an extremely slow Docker building or running.

NVIDIA Docker

To setup GPU support for WSL2 follow the instructions as reported from the NVIDIA official page. https://docs.nvidia.com/cuda/wsl-user-guide/index.html. This guide will first help you to setup WSL2 for your CUDA drivers and then help you setup your Docker installation with CUDA support. You can stop at Simple CUDA Containers.

After the installation, if you are having troubles with permissions, make sure you set them up properly. Usually you have missed running this command

sudo usermod -aG docker $USER

Note: This Docker installation differs from Docker Desktop which is a similar initiative from the Windows developers. If you have Docker Desktop installed you might have hard time getting the Nvidia Docker installation to work. I have tried to use Docker Desktop but did not manage to get my GPU running even though the Windows developers claim it's ready (https://www.docker.com/blog/wsl-2-gpu-support-is-here/).

Docker Only (No GPU)

If you do not have a GPU I strongly recommend you to install Docker Desktop as it's very simple and can be done directly from Windows with a couple of clicks. https://docs.docker.com/docker-for-windows/install/

Last important adjustments

While setting up the rest of this tutorial I had lots of WTF moments which I listed below.

Firewall (Github issue)

WSL2 and Windows do not communicate through the same network interface but through a virtual network. WSL2 and Windows each share a different IP which you can see through the ifconfig. That said, I realized that Windows was blocking incoming connections from WSL2 and I had to add an exception to the firewall to allow ROS to communicate with Unity. How To

Time sync (Github issue)

WSL2 and Windows time will easily drift causing both systems to have different time. Some people experienced high discrepancies, however, I had only few seconds difference. Nonetheless, those few seconds were enough to cause ROS to reject the messages incoming from Unity. To fix the issue I have setup a continuous sync in WSL2.

We will go ahead and let WSL2 sync from Windows time each 10 minutes.

sudo crontab -e

Then add the following line

10 * * * * ntpdate time.windows.com

Service startup

WSL2 does not support system.d or any sort of service start. Among many different ways, I have chosen the simplest one to make it happen (https://www.how2shout.com/linux/how-to-start-wsl-services-automatically-on-ubuntu-with-windows-10-startup/)

Add the following lines to your /etc/sudoers file

%sudo ALL=(ALL) NOPASSWD: /usr/sbin/service docker *
%sudo ALL=(ALL) NOPASSWD: /usr/sbin/service cron *

Then go to your Windows search bar and type Run and inside it type

shell:startup

It will open a folder. Inside it, create a file wslservices.bat with this content

wsl.exe sudo service docker start
wsl.exe sudo service cron start

You can add other services if required.

Conclusions

You are now setup to work with WSL2 and do software development using the convenience of Linux while keeping your Windows system running! I have used this setup for other projects related to deep learning and it's extremely convenient if you like me don't like swapping OS often.

Continue to Part II where we will setup our demo to use Unity and ROS over WSL2 and Docker.