DEV Community

Masui Masanori
Masui Masanori

Posted on

2 2

[.NET 6][Uno Platform][Ubuntu 21.10] Using Leap Motion

#dotnet #csharp #leapmotion #ubuntu

Intro

In this time, I will try using Leap Motion Controller from the Uno Platform application on Ubuntu.

I use this project to add LeapSDK.

Environments

  • Ubuntu 21.10
  • .NET 6.0.101
  • LeapDeveloperKit ver.2.3.1

Using Leap Motion Controller on Ubuntu

To use Leap Motion Controller on Ubuntu I have to install tracking software.

After unpacking the tgz file, install a deb file.

sudo dpkg --install Leap-2.3.1+31549-x64.deb
Enter fullscreen mode Exit fullscreen mode

And I also install Mesa OpenGL.

sudo apt-get install libgl1-mesa-dri libgl1-mesa-glx
Enter fullscreen mode Exit fullscreen mode

After installing, I start the daemon of Leap Motion.

sudo leapd
Enter fullscreen mode Exit fullscreen mode

I can open a control panel.

LeapControlPanel &
Enter fullscreen mode Exit fullscreen mode

To open the control panel, I have to execute the command twice.
I don't know why :(

Controller was not working

Though I didn't get any errors, my machine hasn't recognized the controller yet.

According to this post, I could know I had to add a config file(leapd.service).

Now I can use the Leap Motion Controller on the control panel.

Add a LeapSDK library file in .NET 6 projects

In the tgz file, I can also get library files to control the Leap Motion Controller.
So I add a dll file for .NET Framework 4.0.

UnoSample.Skia.Gtk.csproj 

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType Condition="'$(Configuration)'=='Release'">WinExe</OutputType>
    <OutputType Condition="'$(Configuration)'=='Debug'">Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
    <LangVersion>latest</LangVersion>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup Condition="exists('..\UnoSample.UWP')">
    <EmbeddedResource Include="..\UnoSample.UWP\Package.appxmanifest" LogicalName="Package.appxmanifest" />
    <Content Include="..\UnoSample.UWP\Assets\StoreLogo.png" Link="Assets\StoreLogo.png" />
    <Content Include="Assets\Fonts\uno-fluentui-assets.ttf" />
  </ItemGroup>
  <ItemGroup>
    <UpToDateCheckInput Include="..\UnoSample.Shared\**\*.xaml" />
  </ItemGroup>
  <ItemGroup>
    <Reference Include="Leap">
      <HintPath>References/LeapSdk/LeapCSharp.NET4.0.dll</HintPath>
    </Reference>
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Logging" Version="5.0.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="5.0.0" />
    <PackageReference Include="Uno.UI.Skia.Gtk" Version="4.0.11" />
    <PackageReference Include="Uno.UI.RemoteControl" Version="4.0.11" Condition="'$(Configuration)'=='Debug'" />
    <PackageReference Include="Uno.UI.Adapter.Microsoft.Extensions.Logging" Version="4.0.11" />
  </ItemGroup>
  <Import Project="..\UnoSample.Shared\UnoSample.Shared.projitems" Label="Shared" />
</Project>
Enter fullscreen mode Exit fullscreen mode

libLeapCSharp can't be found

Though I don't get any compile errors and I can get static value like "Gesture.GestureType.TYPE_CIRCLE",
when I add classes for controlling the Leap Motion Controller, I get a runtime exception.

LeapCtrlListener.cs 

using Leap;

namespace UnoSample.Skia.Gtk.MotionControllers
{
    public class LeapCtrlListener: Listener
    {
        public LeapCtrlListener()
        {
            Console.WriteLine("LeapCtrlListener");
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Program.cs 

using GLib;
using Uno.UI.Runtime.Skia;
using Leap;
using UnoSample.Skia.Gtk.MotionControllers;

namespace UnoSample.Skia.Gtk
{
    class Program
    {
        static void Main(string[] args)
        {
            // OK. "Gesture: TYPE_CIRCLE"
            Console.WriteLine("Gesture: " + Gesture.GestureType.TYPE_CIRCLE);

            // Error
            var leapCtrlListener = new LeapCtrlListener();

            ExceptionManager.UnhandledException += delegate (UnhandledExceptionArgs expArgs)
            {
                Console.WriteLine("GLIB UNHANDLED EXCEPTION" + expArgs.ExceptionObject.ToString());
                expArgs.ExitApplication = true;
            };

            var host = new GtkHost(() => new App(), args);

            host.Run();
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Error 

Unhandled exception. System.TypeInitializationException: The type initializer for 'Leap.LeapPINVOKE' threw an exception.
 ---> System.TypeInitializationException: The type initializer for 'SWIGExceptionHelper' threw an exception.
 ---> System.DllNotFoundException: Unable to load shared library 'LeapCSharp' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libLeapCSharp: cannot open shared object file: No such file or directory
   at Leap.LeapPINVOKE.SWIGExceptionHelper.SWIGRegisterExceptionCallbacks_Leap(ExceptionDelegate applicationDelegate, ExceptionDelegate arithmeticDelegate, ExceptionDelegate divideByZeroDelegate, ExceptionDelegate indexOutOfRangeDelegate, ExceptionDelegate invalidCastDelegate, ExceptionDelegate invalidOperationDelegate, ExceptionDelegate ioDelegate, ExceptionDelegate nullReferenceDelegate, ExceptionDelegate outOfMemoryDelegate, ExceptionDelegate overflowDelegate, ExceptionDelegate systemExceptionDelegate)
   at Leap.LeapPINVOKE.SWIGExceptionHelper..cctor()
   --- End of inner exception stack trace ---
   at Leap.LeapPINVOKE.SWIGExceptionHelper..ctor()
   at Leap.LeapPINVOKE..cctor()
   --- End of inner exception stack trace ---
   at Leap.LeapPINVOKE.new_Listener()
   at Leap.Listener..ctor()
   at UnoSample.Skia.Gtk.LeapMotions.LeapController..ctor() in /home/example/Documents/workspace/UnoSample/UnoSample.Skia.Gtk/LeapMotions/LeapController.cs:line 7
   at UnoSample.Skia.Gtk.Program.Main(String[] args) in /home/example/Documents/workspace/UnoSample/UnoSample.Skia.Gtk/Program.cs:line 13
Enter fullscreen mode Exit fullscreen mode

Add link to shared object file

To use the library file, I have to add link to "libLeapCSharp.so" on my machine.

I can execute this command before run the project.

export LD_LIBRARY_PATH="$Home/home/example/Documents/leap_developer_kit"
Enter fullscreen mode Exit fullscreen mode

(In "leap_developer_kit" folder, there are "libLeapCSharp.so", "libLeap.so", and so on)

But because I don't want to do this everytime.

So I add "leap_developer_kit.conf" into "/etc/ld.so.conf.d".

leap_developer_kit.conf 

/home/example/Documents/leap_developer_kit
Enter fullscreen mode Exit fullscreen mode

After I execute "sudo ldconfig", I can watch the link of the files by "ldconfig -p".

...
  libLeapCSharp.so (libc6,x86-64) => /home/example/Documents/leap_developer_kit/libLeapCSharp.so
    libLeap.so (libc6,x86-64) => /home/example/Documents/leap_developer_kit/libLeap.so
...
Enter fullscreen mode Exit fullscreen mode

(Sample) Getting hand count

According to the sample code in the tgz file, I add classes and get tracking hand count.

Program.cs 

using GLib;
using Uno.UI.Runtime.Skia;
using UnoSample.Skia.Gtk.MotionControllers;

namespace UnoSample.Skia.Gtk
{
    class Program
    {
        static void Main(string[] args)
        {
            using(var motionController = new MotionController())
            {
                ExceptionManager.UnhandledException += delegate (UnhandledExceptionArgs expArgs)
                {
                    Console.WriteLine("GLIB UNHANDLED EXCEPTION" + expArgs.ExceptionObject.ToString());
                    expArgs.ExitApplication = true;
                };
                var host = new GtkHost(() => new App(), args);
                host.Run();
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

MotionController.cs 

using Leap;

namespace UnoSample.Skia.Gtk.MotionControllers;
public class MotionController: IDisposable
{
    private LeapCtrlListener leapCtrlListener;
    private Controller controller;
    public MotionController()
    {
        this.leapCtrlListener = new LeapCtrlListener();
        this.controller = new Controller ();
        this.controller.AddListener(this.leapCtrlListener);
    }
    public void Dispose()
    {
        this.controller.RemoveListener(this.leapCtrlListener);
        this.controller.Dispose();
    }
}
Enter fullscreen mode Exit fullscreen mode

LeapCtrlListener.cs 

using Leap;

namespace UnoSample.Skia.Gtk.MotionControllers;
public class LeapCtrlListener : Listener
{
    public LeapCtrlListener()
    {
        Console.WriteLine("LeapCtrlListener");
    }
    public override void OnInit(Controller controller)
    {
        Console.WriteLine("Initialized");
    }
    public override void OnConnect(Controller controller)
    {
        Console.WriteLine("Connected");
        controller.EnableGesture(Gesture.GestureType.TYPE_CIRCLE);
        controller.EnableGesture(Gesture.GestureType.TYPE_KEY_TAP);
        controller.EnableGesture(Gesture.GestureType.TYPE_SCREEN_TAP);
        controller.EnableGesture(Gesture.GestureType.TYPE_SWIPE);
    }
    public override void OnDisconnect(Controller controller)
    {
        //Note: not dispatched when running in a debugger.
        Console.WriteLine("Disconnected");
    }
    public override void OnExit(Controller controller)
    {
        Console.WriteLine("Exited");
    }
    public override void OnFrame(Controller controller)
    {
        var leapFrame = controller.Frame();
        var handList = leapFrame.Hands;
        Console.WriteLine($"OnFrame Hand: {handList.Count}");
    }
}
Enter fullscreen mode Exit fullscreen mode
profile
Hostinger
 Promoted

Hostinger image

Get n8n VPS hosting 3x cheaper than a cloud solution

Get fast, easy, secure n8n VPS hosting from $4.99/mo at Hostinger. Automate any workflow using a pre-installed n8n application and no-code customization.

Start now

Top comments (1)

Collapse
 
insight_it_891bf8cd689700 profile image
Insight IT •

Nice blog and informative content,
We are providing Best Dot NET Training in Hyderabad,
Thanks for sharing with us,
DOT NET Training in Hyderabad
DOT NET Online Training in Hyderabad

profile
Sentry
 Promoted

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

Read next

cnr_br profile image

Como o .NET Gerencia sua Memória (e por que você não deve interferir).

Cristiano Rodrigues -

mirnes_mrkaljevic profile image

Design Patterns: Strategy

Mirnes -

sertxudev profile image

Change keyboard to Spanish distribution on Ubuntu Server

Sergio Peris -

mirnes_mrkaljevic profile image

Composition Over Inheritence

Mirnes -

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay