DEV Community

Cover image for The DLL Hell of Windows Containers: Debugging "Silent" Failures
Maykol Alfaro
Maykol Alfaro

Posted on

The DLL Hell of Windows Containers: Debugging "Silent" Failures

#containers #docker #microservices #dotnet

In our previous posts, we talked about some techniques to dramatically lower your final image size, the experience working with Windows Containers, and why it was the most viable bridge to the cloud when trying to modernize your applications (.NET Framework). Today, we're moving from the theory of "Lift and Shift" and diving into how to solve the "it works on my machine" nightmare.

If you've spent any time with Windows Containers, you know how it feels when you run docker build, it finishes successfully, then you run docker run, and... nothing. The container exits immediately. No stack trace, no event log, just a Exit Code 1.

Debugging these type of errors require moving from the basic Docker commands and diving into the internals of the Windows kernel and subsystem.

The Entrypoint Trap

When a Windows Container crashes on startup, the logs are often lost because the container object is destroyed before the buffer flushes. To fix this, you must hijack the entrypoint.

Instead of letting your Dockerfile run its ENTRYPOINT, run it manually:

docker run -it --entrypoint powershell your-image-name:latest
Enter fullscreen mode Exit fullscreen mode

Once inside the PowerShell prompt:

  1. Navigate to your app directory (e.g., C:\inetpub\wwwroot or C:\app).
  2. Manually execute your .exe or start the IIS service.
  3. Monitor the console. Here is where you'll likely see a "missing DLL" or "access denied" error that the Docker daemon was hiding.

Hunting the "Ghost" DLLs and Exceptions

Windows Server Core for example is a "refactored" OS. To keep the size down, Microsoft stripped out UI components, fonts, and specific legacy APIs. If your app or one of its third-party dependencies calls one of these missing APIs, it will crash with a DllNotFoundException.

Trying to find these types of errors might be overwhelming, but there are some tools you can download and use directly in your Windows Containers. When the docker logs command is not enough, you can use something like Log Monitor, a Microsoft-supported opensource tool that bridges Windows application logs to STDOUT/STDERR. And it's configured via a config file.

As the Microsoft's documentation says, Log Monitor can either be used in a SHELL usage pattern:

SHELL ["C:\\LogMonitor\\LogMonitor.exe", "cmd", "/S", "/C"]
CMD c:\windows\system32\ping.exe -n 20 localhost
Enter fullscreen mode Exit fullscreen mode

Or an ENTRYPOINT usage pattern:

ENTRYPOINT C:\LogMonitor\LogMonitor.exe c:\windows\system32\ping.exe -n 20 localhost
Enter fullscreen mode Exit fullscreen mode

One tool that also helped me find hidden application specific exceptions was ProcDump. This is a command-line utility from Microsoft Sysinternals designed to monitor applications for CPU spikes, memory spikes, or unhandled exceptions and automatically generate crash dumps (DMP files). Later, you can analyze this generated files with other tools such as WinDbg which offers more modern visuals to see where your application/container is failing.

There are many other tools that you can use to generate these types of logs and try to find "Ghost" DLLs or exceptions that might be causing your containers failure, but at least these are the tools that helped me troubleshoot these scenarios. And at least all these 3 tools are from Microsoft, in case your company worries about using third-party tools that don't align with their security policies.

The Kernel Mismatch (Isolation Modes)

In Linux, the container shares the host kernel. In Windows, the "Kernel" is much more rigid. If your host is Windows 11 and your container is built on ltsc2019, they speak a slightly different "language."

The Symptom: You get an error like "The container operating system does not match the host operating system."

The Fix: Strategic Isolation
As an engineer, you need to understand the two isolation levels:

  • Process Isolation: The container shares the host kernel directly. It’s fast and has low overhead, but the versions must match (e.g., Host 2022 -> Container 2022).
  • Hyper-V Isolation: The container runs inside a tiny, optimized VM. It’s slower to start but allows you to run older containers on newer hosts.

Command Tip:

For local development try to force Hyper-V isolation, that way you can test your Windows Containers on different OS versions and don't worry about these kernel mismatches.

docker run --isolation=hyperv your-image-name
Enter fullscreen mode Exit fullscreen mode

The "Invisible Text" Mystery

One of the most common issues you'll get after a "successful" deployment is: "The app runs, the PDF generates, but all the text is missing or replaced by weird squares (▯▯▯)."

This happens because in Windows Server Core Microsoft removed nearly every font you take for granted to reduce its size. If your .NET Framework code asks for "Arial" and it's not there, GDI+ won't always tell you, it will just fail silently or fall back to a font that doesn't support your character set.

On a desktop, you just drag a .ttf into C:\Windows\Fonts. In a container, that does nothing. You can't just COPY the files; you have to register them in the Windows Registry so the GDI+ engine knows they exist.

To solve this, we had to modify our Dockerfile to copy the missing fonts and register them in the Windows Registry.

 # Step 1: Copy your fonts into a temporary folder
 COPY ./my-fonts/ /fonts/  
 # Step 2: Register each font in the registry
 RUN powershell -Command \  
    $fontPath  =  'C:\fonts'; \
    $registryPath  =  'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts'; \ 
    Get-ChildItem -Path  $fontPath  -Filter *.ttf | ForEach-Object { \  
        $fontName  =  $_.BaseName; \ 
        Copy-Item  $_.FullName -Destination  'C:\Windows\Fonts'; \ 
        New-ItemProperty -Path  $registryPath  -Name  $fontName  -Value  $_.Name -PropertyType String -Force \
    }
Enter fullscreen mode Exit fullscreen mode

Final Thoughts

Nobody told us dealing with Windows Containers was going to be a bit frustrating sometimes, but not imposible. At that point we didn't know what else we could do to monitor/debug our containers trying to find these errors. Just to finally see a "missing DLL" or just a "missing font" error. And "tutorials" or these guides on the internet to help you start with Windows Containers don't talk about these nuances, that's why I decided to start writing this series and help some people out there dealing with these types of issues and if you're starting, at least let you know what you will/can encounter during your containerization process.

Top comments (0)