DEV Community

jeikabu
jeikabu

Posted on • Originally published at rendered-obsolete.github.io on

Nupkg Ergonomics or, The ongoing saga of C# bindings to NNG native library

#dotnet #nuget #csharp #native

The package we created previously was painful to use:

  • The nupkg runtimes/ folder containing platform-specific binaries doesn’t get copied somewhere that’s obvious. On installation everything goes into the nuget cache; on OSX ~/.nuget/packages/subor.nng.netcore/, on Windows %USERPROFILE%/.nuget/packages/subor.nng.netcore/
  • In Visual Studio, if Properties->Build->Platform target is set to x86 or x64 the correct shared library will get copied to the output folder

Additionally, we had problems with the nupkg not working because of a disconnect between the development and the nupkg environments. We want to develop using the same folder structure as what nuget uses.

Putting the “Run” in Runtime

Adding a .props/.target file is probably unavoidable. When Nuget installs a package it will <Import> files in the package’s build/ folder to the project; .props at the top of the project and .targets at the bottom.

Our package is Subor.nng.NETCore, so create build/Subor.nng.NETCore.targets:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <Content Include="$(MSBuildThisFileDirectory)..\runtimes\**">
      <Link>runtimes\%(RecursiveDir)%(Filename)%(Extension)</Link>
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </Content>
  </ItemGroup>
</Project>
Enter fullscreen mode Exit fullscreen mode

This copies all the runtimes/ files to the output directory.

To include the .targets file in the nupkg, in nng.NETCore.csproj:

<ItemGroup>
  <Content Include="build\**">
    <PackagePath>build\%(RecursiveDir)%(Filename)%(Extension)</PackagePath>
    <Visible>false</Visible>
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </Content>
</ItemGroup>
Enter fullscreen mode Exit fullscreen mode

Now, any project that consumes our package will <Import> the .targets and copy our platform-specific runtimes/ files to its output folder.

Several online sources mention creating install.ps1, but that was deprecated in Nuget 3.1.

Multi-Targetting

Requiring .NET Standard 2.0 is potentially a steep requirement. With some well-placed #if blocks it is easy to reduce the requirement to .NET Standard 1.5. But, this necessitates our package supports multiple target frameworks.

The Wrong Way

Created additional projects like nng.NETCore.15.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netstandard1.5</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="..\nng.NETCore\ **\*.cs" Exclude="..\nng.NETCore\obj\** \*.cs" />
  </ItemGroup>
Enter fullscreen mode Exit fullscreen mode

The Exclude property is to avoid the AssemblyInfo.cs that is generated and will result in:

obj/Debug/netstandard1.5/nng.NETCore.15.AssemblyInfo.cs(10,12): error CS0579: Duplicate 'System.Reflection.AssemblyCompanyAttribute' attribute [/Users/jake/projects/nng.NETCore/nng.NETCore.15/nng.NETCore.15.csproj]
Enter fullscreen mode Exit fullscreen mode

If we had pursued this approach, should probably move the AssemblyInfo meta-data into the project.

However, the output assembly bin/$(Configuration)/netstandard1.5/nng.NETCore.dll isn’t included by dotnet pack- it’s not a dependency of the main project.

So, we created a project just for packing:

<PropertyGroup>
    <PackageId>Subor.nng.NETCore</PackageId>
    <!-- SNIP -->
</PropertyGroup>
<ItemGroup>
    <ProjectReference Include="..\nng.Shared\nng.Shared.csproj" />
    <ProjectReference Include="..\nng.Shared.15\nng.Shared.15.csproj" />
</ItemGroup>
Enter fullscreen mode Exit fullscreen mode

But this resulted in “multiply-defined” symbols:

Bus.cs(9,29): error CS0433: The type 'Defines' exists in both 'nng.Shared.15, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' and 'nng.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' [/XXX/nng.NETCore/nng.NETCore/nng.NETCore.csproj]
Enter fullscreen mode Exit fullscreen mode

NB: we later found something that might fix this.

Time to head to the Internet. We need a way to include unrelated assemblies in a nupkg. Turns out this is a re-occurring pain point:

The available solutions boil down to:

  1. Use TargetsForTfmSpecificBuildOutput (yikes)
  2. Use a nuspec file
  3. One nupkg per dependency (the current recommendation)

Tried to add a nuspec file to what we already had:

<PropertyGroup>
    <PackageId>Subor.nng.NETCore</PackageId>
    <!-- SNIP -->
    <RepositoryUrl>https://github.com/subor/nng.NETCore</RepositoryUrl>
    <!-- Added the below -->
    <NuspecFile>nng.NETCore.nuspec</NuspecFile>
  </PropertyGroup>
Enter fullscreen mode Exit fullscreen mode

dotnet pack yields the mysterious error:

/XXX/.nuget/packages/nuget.build.packaging/0.2.2/build/NuGet.Build.Packaging.targets(377,3): error : Path cannot be the empty string or all whitespace. [/XXX/nng.NETCore/nng.NETCore/nng.NETCore.csproj]
Enter fullscreen mode Exit fullscreen mode

Apparently you can’t mix a nuspec file with the PackageReference format. In fact, no matter what, we couldn’t get dotnet pack to use the nuspec with NET Core 2.1.302. But it doesn’t matter because we were barking up the wrong tree anyway.

The Right Way

.NET has a mechanism to multi-target frameworks: <TargetFrameworks> (note the “s”).

Change the project to:

<PropertyGroup>
    <TargetFrameworks>netstandard1.5;netstandard2.0</TargetFrameworks>
</PropertyGroup>
Enter fullscreen mode Exit fullscreen mode

And dotnet pack… yielded a mysterious error:

/XXX/nng.NETCore/nng.Shared/nng.Shared.csproj : error MSB4057: The target "_GetBuildOutputFilesWithTfm" does not exist in the project.
Enter fullscreen mode Exit fullscreen mode

This had us stumped for a while:

  • Google doesn’t turn up anything helpful
  • Installed .NET Core 1.1
  • <TargetFramework>netstandard1.5 (no “s”) worked
  • Tested on a Windows 10 box with Visual Studio 2017, same error
  • A new, empty project worked

Bafflement. Turns out it was self-sabotage. <TargetFrameworks> was broken by:

<PackageReference Include="NuGet.Build.Packaging" Version="0.2.2" />
Enter fullscreen mode Exit fullscreen mode

No idea why that was added, but removing it made dotnet pack generate a nupkg targetting both frameworks:

| ____ lib
| | ____ netstandard2.0
| | | ____ nng.NETCore.dll
| | ____ netstandard1.5
| | | ____ nng.NETCore.dll
| ____ build
| | ____ Subor.nng.NETCore.targets
| ____...
Enter fullscreen mode Exit fullscreen mode

Oh… now the project dependency, nng.Shared.dll, isn’t included. Apparently NuGet.Build.Packaging had been doing it this whole time.

And now we come full circle; need multiple assemblies in a nupkg using one of the solutions we outlined above. Ironically, we inadvertantly rediscovered something I already knew. Two months ago I read (and commented on) a post about the same thing.

Project Structure

We have two managed assemblies:

  • nng.NETCore.dll
  • nng.Shared.dll

Consumers of the package need a reference to nng.Shared which then loads nng.NETCore at runtime (which subsequently loads the platform specific native library).

We will create two nupkgs:

In nng.NETCore.csproj:

<PropertyGroup>
    <OutputPath>runtimes\any\lib\</OutputPath>
    <!-- Including assembly as part of runtimes/ so don't want it placed in lib/ -->
    <IncludeBuildOutput>false</IncludeBuildOutput>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\nng.Shared\nng.Shared.csproj">
      <Private>false</Private>
    </ProjectReference>
  </ItemGroup>

  <!-- Must be run after build so output assembly is in runtimes/ -->
  <Target Name="Runtimes" AfterTargets="Build">
    <ItemGroup>
      <Content Include="runtimes\**">
        <PackagePath>runtimes</PackagePath>
        <Visible>false</Visible>
      </Content>
    </ItemGroup>
  </Target>
</Project>
Enter fullscreen mode Exit fullscreen mode

<OutputPath> is runtimes/any/lib/ so our development path is the same as when packaged. netstandard2.0/ or netstandard1.5/ will be automatically appended since <AppendTargetFrameworkToOutputPath> and <AppendRuntimeIdentifierToOutputPath> are enabled by default.

<ProjectReference> to nng.Shared will be turned into a package reference; adding Subor.nng.NETCore package to a project automatically adds Subor.nng.NETCore.Shared as well. <Private>false</Private> ensures nng.Shared.dll doesn’t get copied to nng.NETCore output; once packaged they won’t exist in the same folder.

<Target Name="Runtimes" AfterTargets="Build"> forces the <Content> to get packaged after the build. Without this nng.NETCore.dll won’t get copied along with the rest of runtimes/.

Nothing special is done in nng.Shared.csproj.

Multi-Platform Multi-Targetting

It’s possible to also target .NET Framework:

<TargetFrameworks>netstandard1.5;netstandard2.0;net461</TargetFrameworks>
Enter fullscreen mode Exit fullscreen mode

Building this on OSX fails with:

/usr/local/share/dotnet/sdk/2.1.302/Microsoft.Common.CurrentVersion.targets(1179,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.6.1" were not found. To resolve this, install the SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend. [/Users/jake/projects/nng.NETCore/nng.NETCore/nng.NETCore.csproj]
Enter fullscreen mode Exit fullscreen mode

Can use a conditional PropertyGroup (from Cross-platform TargetFrameworks):

<PropertyGroup Condition=" '$(OS)' == 'Windows_NT' ">
    <TargetFrameworks>netstandard1.5;netstandard2.0;net461</TargetFrameworks>
</PropertyGroup>
<PropertyGroup Condition=" '$(OS)' != 'Windows_NT' ">
    <TargetFrameworks>netstandard1.5;netstandard2.0</TargetFrameworks>
</PropertyGroup>
Enter fullscreen mode Exit fullscreen mode

Also need conditional PackageReference because System.Runtime.Loader is not available in .NET Framework:

<ItemGroup Condition="'$(TargetFramework)' == 'netstandard1.5' or '$(TargetFramework)' == 'netstandard2.0' ">
      <PackageReference Include="system.runtime.loader" Version="4.3.0" />
  </ItemGroup>
Enter fullscreen mode Exit fullscreen mode

And then in source files:

#if (NETSTANDARD1_5 || NETSTANDARD1_6 || NETSTANDARD2_0)
//...
#endif
Enter fullscreen mode Exit fullscreen mode

With .NET Standard 2.1 right around the corner this is becoming unwieldly.

It looks like it could be simplified with Property Functions. And eventually came across a decent solution from this SO. In the project:

<PropertyGroup Condition="'$(TargetFramework)'=='netstandard1.5' or '$(TargetFramework)'=='netstandard1.6' or '$(TargetFramework)'=='netstandard2.0' or '$(TargetFramework)'=='netstandard2.1'">
    <DefineConstants>NNG_NETSTANDARD1_5_AND_UP</DefineConstants>
</PropertyGroup>

<!-- Conditional PackageReference -->
<ItemGroup Condition="$(DefineConstants.Contains('NNG_NETSTANDARD1_5_AND_UP'))">
    <PackageReference Include="system.runtime.loader" Version="4.3.0" />
</ItemGroup>
Enter fullscreen mode Exit fullscreen mode

In C#:

#if NNG_NETSTANDARD1_5_AND_UP
//...
#endif
Enter fullscreen mode Exit fullscreen mode

Far more manageable.

Odds and Ends

Miscellaneous nupkg notes that don’t really warrant their own posts.

Testing

Microsoft docs on adding a local package source.

Clearing local nuget cache:

nuget locals all -clear
Enter fullscreen mode Exit fullscreen mode

This SO on adding a package source to a project:

<PropertyGroup>
  <RestoreSources>$(RestoreSources);/XXX/projects/multitargettest;https://api.nuget.org/v3/index.json</RestoreSources>
</PropertyGroup>
Enter fullscreen mode Exit fullscreen mode

When to Not Multi-Target

We’ve been trying to slim down sln files by moving from direct inclusion of source code projects to nuget packages. One of these was Apache Thrift. Thrift contains 3 projects: 2 targetting .NET Framework 3.5 and 4.5, and one targetting .NET Standard 2.0.

We use .NET Framework 3.5 library in our Unity 3D SDK, and use .NET Standard 2.0 library in the rest of our code base (which targets either .NET Framework 4.6.1 or .NET Standard 2.0).

We created a nuspec to package the 3 unrelated projects without touching the original projects:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <!-- ... -->
  </metadata>
  <files>
    <file src="thrift\lib\netcore\src\bin\Release\Thrift.dll" target="lib/netstandard2.0/Thrift.dll" />
    <file src="thrift\lib\csharp\src\bin\Release\Thrift.dll" target="lib/net35/Thrift.dll" />
    <file src="thrift\lib\csharp\src\bin\Release\Thrift45.dll" target="lib/net45/Thrift.dll" />
  </files>
</package>
Enter fullscreen mode Exit fullscreen mode

But after adding the nuget package to some projects we got compiler errors. In obj/project.assets.json:

"Subor.Thrift/0.11.0.3": {
        ...
        "compile": {
          "lib/net45/Thrift.dll": {}
        },
        "runtime": {
          "lib/net45/Thrift.dll": {}
        }
      },
Enter fullscreen mode Exit fullscreen mode

Our .NET Framework 4.6.1 projects opted to use the 4.5 assembly instead of the .NET Standard 2.0 assembly like we wanted.

Out of curiosity, we tried removing the 4.5 assembly from the package. When faced with the choice between NF 3.5 and NS 2.0, .NET Standard wins out. But, in the end we decided it was probably best to separate the .NET Framework and .NET Standard assemblies so the end-user can make the choice between NF3.5/NF4.5/NS2.0.

ContentFiles

While perusing Stack Overflow, came across this issue about “ContentFiles”.

Was intrigued because it was an aspect of nupkg we’ve yet to use. Found the following helpful resources:

Top comments (0)

Read next

chinonsoike profile image

Reading Configuration Values in Your ASP.NET Application

Chinonso Ikewelugo -

pradeepradyumna profile image

Build your first Hangfire job .NET8 with PostgreSQL

Pradeep Pradyumna -

nikiforovall profile image

A Guide to OpenAPI Client Generation with Kiota. Introduction (Part 1)

Oleksii Nikiforov -

rahulpnath profile image

A .NET Programmer's Guide to CancellationToken

Rahul Nath -