DEV Community: Bob Rundle

Finding Types at Runtime in .NET Core

Bob Rundle — Sun, 29 Aug 2021 14:34:46 +0000

One of the best features of .NET has always been the type system. In terms of rigor I place it midway between the rigidness of C++ and the anything-goes of JavaScript which in my view makes it just right. However one of my frustrations over the years has been finding types at runtime.

At compile time you find the integer type like so…

            Type t00 = typeof(int);

But at runtime this doesn't work…

            Type t01 = Type.GetType("int");  // null

You need to do this…

            Type t02 = Type.GetType("System.Int32");

Similarly for other system types such as DateTime…

            Type t10 = typeof(DateTime);
            Type t11 = Type.GetType("DateTime"); // null
            Type t12 = Type.GetType("System.DateTime");

Let's say you created your own local type…

    public class ClassA : IClassAInterface
    {
        public string Hello()
        {
            return "In main program";
        }
    }

Which references this interface in a separate assembly…

namespace ClassAInterface
{
    public interface IClassAInterface
    {
        public string Hello();
    }
}

Again it is simpler to find it at compile time than run time.

            Type t20 = typeof(ClassA);
            Type t21 = Type.GetType("ClassA"); // null
            Type t22 = Type.GetType("TypeSupportExamples.ClassA");

To find it at runtime you need to specify the full name of the type which includes the namespace. This seems wrong because the code in this case is being executed in the namespace which contains the type.

Finally if the user defined type you are looking for is defined in a different assembly you need to provide the assembly name…

            Type t30 = typeof(IClassAInterface);
            Type t31 = Type.GetType("IClassAInterface"); // null
            Type t32 = Type.GetType("ClassAInterface.IClassAInterface"); //null
            Type t34 = Type.GetType("ClassAInterface.IClassAInterface, ClassAInterface");

What is happening of course is that the reason the compiler can find types so easily is because of the using statements at the top of the file…


    using ClassAInterface;
    using System;

The using statements provide a scope that guides the compiler to finding the right type. No such scoping mechanism exists at runtime. Instead, at runtime, scoping is provided by the container the type is in. Types are contained in assemblies which in turn are contained within load contexts which in turn are contained within app domains. This strict top down hierarchy is not required of namespaces which can span multiple assemblies. The same type name might be used in multiple assemblies and the same assembly name might be used in multiple load contexts.

Even though the same type name might be used in multiple assemblies they are seen by the runtime as distinct types even though they might be identical. I explored the ramifications of this in my previous post https://dev.to/bobrundle/forward-and-backward-compatibility-in-net-core-3c52 .

A review of type names. There are 3 for each type: simple name, full name and assembly qualified name…

            // 3 Names of a type

            Console.WriteLine(t22.Name);
            Console.WriteLine(t22.FullName);
            Console.WriteLine(t22.AssemblyQualifiedName);

ClassA
TypeSupportExamples.ClassA
TypeSupportExamples.ClassA, TypeSupportExamples, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

I set out to make finding types at runtime easier, so I explored implementing a kind of runtime "using" statement. First I create a global dictionary of types, GlobalTypeMap, that allowed simple names for built-in types and system types and requires full types names for all others.

            Console.WriteLine();
            var globalTM = new GlobalTypeMap();
            Type t0 = globalTM.FindType("int");
            Console.WriteLine(t0.FullName);
            Type t1 = globalTM.FindType("DateTime");
            Console.WriteLine(t1.FullName);
            Type t2 = globalTM.FindType("TypeSupportExamples.ClassA");
            Console.WriteLine(t2.FullName);

System.Int32
System.DateTime
TypeSupportExamples.ClassA

Then create a child type map, ScopedTypeMap, where I apply using statements.


            var scopedTM1 = new ScopedTypeMap(globalTM);
            scopedTM1.UsingNamespace("TypeSupportExamples");
            Type t3 = scopedTM1.FindType("ClassA");
            IClassAInterface d0 = Activator.CreateInstance(t3) as IClassAInterface;
            Console.WriteLine();
            Console.WriteLine(d0.Hello()); // In main program

In main program

If new assemblies are loaded, the global type map is updated and the change is reflected in the scoped type map.

            var scopedTM2 = new ScopedTypeMap(globalTM);
            string apath0 = Path.Combine(Directory.GetCurrentDirectory(), "AssemblyA.dll");
            Assembly a0 = AssemblyLoadContext.Default.LoadFromAssemblyPath(apath0);
            scopedTM2.UsingNamespace("NamespaceA1");
            Type t4 = scopedTM2.FindType("ClassA"); // This is NamespaceA1.ClassA in AssemblyA
            IClassAInterface d1 = Activator.CreateInstance(t4) as IClassAInterface;
            Console.WriteLine(d1.Hello());

This is NamespaceA1.ClassA in AssemblyA

The global type map also works across multiple load contexts…

            var scopedTM3 = new ScopedTypeMap(globalTM);
            string apath1 = Path.Combine(Directory.GetCurrentDirectory(),@"AssemblyB.dll");
            AssemblyLoadContext alc0 = new AssemblyLoadContext("alc0");
            Assembly a1 = alc0.LoadFromAssemblyPath(apath1);
            scopedTM3.UsingNamespace("NamespaceB1");
            Type t5 = scopedTM3.FindType("ClassA"); // This is NamespaceB1.ClassA in AssemblyB
            IClassAInterface d2 = Activator.CreateInstance(t5) as IClassAInterface;
            Console.WriteLine(d2.Hello());

This is NamespaceB1.ClassA in AssemblyB

Finally we might apply namespace scope to types that have identical simple names. This is also supported.

            var scopedTM4 = new ScopedTypeMap(globalTM);
            scopedTM4.UsingNamespace("TypeSupportExamples");
            scopedTM4.UsingNamespace("NamespaceA1");
            scopedTM4.UsingNamespace("NamespaceA2");
            scopedTM4.UsingNamespace("NamespaceB1");
            Type[] tt = scopedTM4.FindTypes("ClassA");
            Console.WriteLine();
            foreach (var t in tt)
                Console.WriteLine(t.FullName);

NamespaceA1.ClassA
NamespaceA2.ClassA
NamespaceB1.ClassA
TypeSupportExamples.ClassA

Summary and Discussion

What I have demonstrated is a runtime type facility to allow types to be more easily found. All the code for this facility including the examples above can be found at https://github.com/bobrundle/TypeSupport

The reason I built this facility is that I want to use it for serializing types in a very lightweight way. This type serialization mechanism will be the subject of a future post.

This runtime type facility is definitely not lightweight. A Hello World program contains over 2000 types. For certain applications, however, I think it will be useful.

I did not support all of the capabilities of the .NET type system. For example load contexts can be unloaded and this properly should remove all the relevant types from the global type map. I will add that later if I need it.

I did not address generics but the runtime type facility will support them. You simply need to understand how the grammar of the type name system works. This is documented in https://docs.microsoft.com/en-us/dotnet/framework/reflection-and-codedom/specifying-fully-qualified-type-names.

I did struggle with the issue of ambiguous types. At compile time if you try to use a simple type name that is scoped in multiple namespaces, you get an ambiguous type error. For the scoped type map, I considered throwing an exception if you tried to find a single type by name and there were more than one defined. In the end I decided not to throw an exception and to simply return the first type in a sorted list. The sort for the type list moves types in the default load context to the front of the list. Perhaps I will change my mind on this later.

I hope this post is useful. .NET types should be thoroughly understood and I was surprised how much I learned about various aspects of the type system that I already thought I thoroughly understood.

Forward and Backward Compatibility in .NET Core

Bob Rundle — Sun, 22 Aug 2021 03:40:18 +0000

I only recently discovered load contexts in .NET Core having been using app domains up to now. The load context, I learned, is designed to solve the problem of isolating code using different versions of the same assembly. This was exciting news to me! If you have ever worked on large applications that load a lot of 3rd party code you invariably run into this problem. Most of the time this problem can be solved by binding to the highest version of the assembly used. Most of the time however is not "All of the time". The moment a higher version of an assembly contains breaking changes then this strategy goes out the window. The larger the application you are working on, the more likely you will be outside the window looking in. I'm old enough to remember "DLL Hell". If you are of a certain age you remember that assemblies were supposed to solve DLL Hell. What we have all learned since is that Assembly Hell is ten times hotter than DLL Hell.

So I decided to write some code to check out this capability. Alas! I was disappointed to discover that load contexts solve backward compatibility but not forward compatibility. Let me explain.

Let's say we have an interface…

namespace AssemblyAInterface
{
    public interface IClassAv1
    {
        public string OriginalMethod();
    }
}

With this implementation

namespace NamespaceA
{
    public class ClassA : IClassAv1
    {
        public string OriginalMethod() { return "this is the original method in v1"; }
    }
}

We've shipped this code some time back and now want to make some improvements. Best practice is to not change the interface but create a new one.

namespace AssemblyAInterface
{
    public interface IClassAv2
    {
        public string OriginalMethod();
        public string NewMethod();
    }
}

We update our implementation to support both the old and the new interface…

namespace NamespaceA
{
    public class ClassA : IClassAv1, IClassAv2
    {
        public string OriginalMethod() { return "this is the original method in v2"; }
        public string NewMethod() { return "this is a new method in v2"; }
    }
}

So now old code can use the v1 interface while new code can access the v2 interface. Voila!

Well…

Let's write some code to load both the new and the old implementations. This was possible using app domains but very difficult and as far as I know no one ever used app domains to solve this specific versioning problem. However with load contexts, the solution is straightforward…

namespace LoadContextBench
{
    class Program
    {
        static void Main(string[] args)
        {
            // Our versioned assemblies are in separate directories

            string apathv1 = Path.Combine(Directory.GetCurrentDirectory(), @"..\v1\net5.0\AssemblyA.dll");
            string apathv2 = Path.Combine(Directory.GetCurrentDirectory(), @"..\v2\net5.0\AssemblyA.dll");

            // Separate load contexts for the different versions.

            AssemblyLoadContext alcv1 = new AssemblyLoadContext("v1");
            AssemblyLoadContext alcv2 = new AssemblyLoadContext("v2");

            // Load v1 and v2 of the same assembly

            Assembly av1 = alcv1.LoadFromAssemblyPath(apathv1);
            Assembly av2 = alcv2.LoadFromAssemblyPath(apathv2);

            // Look for our types in the various contexts.  Doesn't exist in default context.

            Type t0 = Type.GetType("NamespaceA.ClassA"); // null in default context.
            Type tv1 = av1.GetType("NamespaceA.ClassA");
            Type tv2 = av2.GetType("NamespaceA.ClassA");

            // Create v1 and v2 objects.  Reference interfaces.  Call methods.

            try
            {
                var dv1 = Activator.CreateInstance(tv1);
                var dv2 = Activator.CreateInstance(tv2);
                IClassAv1 cv1 = (IClassAv1)dv1;
                IClassAv1 cv1a = (IClassAv1)dv2;
                IClassAv2 cv2 = (IClassAv2)dv2;
                Console.WriteLine(cv1.OriginalMethod());
                Console.WriteLine(cv1a.OriginalMethod());
                Console.WriteLine(cv2.NewMethod());
            }
            catch(Exception ex)
            {
                Console.WriteLine("Exception: " + ex.Message);
            }
        }
    }
}

This results in the following output…

this is the original method in v1
this is the original method in v2
this is a new method in v2

This demonstrates backward compatibility: new code calling old code. It also demonstrates executing both old and new versions of the same code simultaneously which is very special.

Note that I had to reference the new interface code in order to reference the v2 interface. Now let's examine forward compatibility. To demonstrate forward compatibility we need to show old code calling new code. So in the main program we reference the v1 interface instead of the v2 interface. Some code changes need to be made since the v2 interface does not exist in the old code.

                var dv1 = Activator.CreateInstance(tv1);
                var dv2 = Activator.CreateInstance(tv2);
                IClassAv1 cv1 = (IClassAv1)dv1;
                IClassAv1 cv1a = (IClassAv1)dv2;
//                IClassAv2 cv2 = (IClassAv2)dv2; // v2 only
                Console.WriteLine(cv1.OriginalMethod());
                Console.WriteLine(cv1a.OriginalMethod());
//                Console.WriteLine(cv2.NewMethod()); // v2 only

This doesn't work. This statement returns null…

Type tv2 = av2.GetType("NamespaceA.ClassA");

This surprised me greatly I expected the type to be resolved. After all the v2 interface assembly is in the v2 directory along with the v2 implementation of the interface. Looking closer it seems that the interface type must be resolved against the interface in the default load context and while v1 resolves, v2 does not. Forward compatibility is simply not supported.

I could not accept this result. Surely there was a way to call v2 from v1. I decided to implement a custom load context, the idea being to explicitly load the dependencies.

namespace LoadContextBench
{
    public class MyLoadContext : AssemblyLoadContext
    {
        private AssemblyDependencyResolver _resolver;
        public MyLoadContext(string folder)
        {
            _resolver = new AssemblyDependencyResolver(folder);
        }
        protected override Assembly Load(AssemblyName assemblyName)
        {
            string assemblyPath = _resolver.ResolveAssemblyToPath(assemblyName);
            if (assemblyPath != null)
            {
                return LoadFromAssemblyPath(assemblyPath);
            }

            return null;
        }
    }
}

The custom loader forces the loaded assembly to load dependencies from the folder in which the assembly resides

This custom loader is referenced like so…

            // AssemblyLoadContext alcv1 = new AssemblyLoadContext("v1");
            // AssemblyLoadContext alcv2 = new AssemblyLoadContext("v2");
            AssemblyLoadContext alcv1 = new MyLoadContext(apathv1);
            AssemblyLoadContext alcv2 = new MyLoadContext(apathv2);

So here was another surprise. The forward compatibility code got a lot further. The types were found and instantiated. But both casts to the v1 interface failed.

                IClassAv1 cv1 = (IClassAv1)dv1;
                IClassAv1 cv1a = (IClassAv1)dv2;

It seems that now, not only can old code not call new code, but old code cannot even call old code anymore! This was truly shocking. The v1 interface assembly loaded in both the default and the v1 context are identical…identical meaning they are exactly the same assembly. One is loaded into the default context and one is loaded into the v1 context. Yet the interface cannot be cast from the v1 to the default contexts. This seems very wrong. The only way it seems to be able to cast interfaces is when the exact same assembly is shared between load contexts.

Switching the old code back to the new code by referencing the v2 interface in the main program and still using the custom load context yields a similar problem. This time the new code can call neither the old code nor the new code.

Summary and Discussion

What I have illustrated here with examples is how load contexts support backward compatibility but fail completely to support forward compatibility. I spent quite a bit of time trying to get forward compatibility to work….after all the overarching promise of managed code was an end to the brittleness inherent in unmanaged code. What I have discovered to my great disappointment is that brittleness is still with us in managed code. Assembly hell is with us forevermore.

My search for a solution to forward compatibility in .NET Core came to end when I discovered this line in https://docs.microsoft.com/en-us/dotnet/core/compatibility/categories:

Maintaining forward compatibility is not a goal of .NET Core.

Really? Why not?

Why Forward Compatibility is Necessary

But isn't backward compatibility all we need? We have a long experience with backward compatibility and expect it in every bit of software we use. Our typical experience with compatibility is not from code to code, but from code to data.

So for example whenever we update a desktop app to a new version we expect all the files created by the old version to be read by the new version. This is backward compatibility and no application would survive long without supporting it.

Forward combability is never an issue as long as we are an island: that is we read only files we have created and we update apps only we use. This was good enough until about 1995. Once we entered a networked world where we needed to share files among a large set of colleagues running different versions of the same software. Like a convoy that moves only as fast as its slowest boat, no one who needed to share documents could upgrade their applications until the last colleague (who was invariably both the most backward and the most likely to be able to cause trouble for everyone) updated theirs. As a result everyone's software was 5 years out of date.

So of course you are thinking that this is the problem that SaaS solves. Everyone is forced to use the latest software and so forward compatibility is no longer an issue. Indeed this solves the problem for the end user, but the general need for forward compatibility is not eliminated…it is simply moved to a different constituency. Which explains why your SaaS software stack is a pit of forward and backward compatibility horrors. From this observation we might derive a general principle:

No problem in software is ever solved…it simply becomes the responsibility of people who are paid more and care less.

Why Forward Compatibility is Hard

The reason that forward compatibility is rarely supported is that it is hard. For backward compatibility to work all you need to do is make sure that all the old code and data you created in the past is still working. Old code and data is something you know completely.

Forward compatibility means that all the code and data you write in the future will work with code you are writing today. The code and data of the future is solving problems you haven't encountered yet and have no idea how to solve today. The solutions you put in place today to solve forward compatibility might be totally inadequate when it comes time to address the problems which need to be solved in the future. Indeed this happens quite often.

Signoff

I hope this excursion into forward and backward compatibility and the associated rants have been useful and entertaining. All the code used in this post can be found at

https://github.com/bobrundle/LoadContextBench

I invite comments and critiques. Perhaps here is something I am missing here. Perhaps there is a magic .deps.json file that will solve the forward compatibility problem in the same way there was a way to do forward compatibility in the .NET Framework using an app.config file with assembly redirects that essentially defeated the versioning system entirely. But you have to convince me that this is really better.

Sure I can have forward compatibility by instantiating the class in the future code base and using GetMethod() and Invoke() to find and call methods. Is this really what we want? Is this what you would call elegant?

These are the enduring questions.

Value Conversion using String Extensions in C#

Bob Rundle — Sun, 08 Aug 2021 16:45:23 +0000

One of the perennial frustrations I have with C# is the poverty of string to value conversions. There are plenty of course, but plenty never seems to be enough as I find myself continually writing new ones. Furthermore what is there seems inelegant. One idea is to use string extension methods to improve this situation.

If you have never used extension methods they can be a valuable tool for strapping new functionality on built-in types. On the downside, it can be disconcerting to see, through IntelliSense, new methods suddenly appear on types that you thought you thoroughly understood.

Let's take a simple string to value conversion and see how it might be done with an extension to the string class.

    string s = "123";
    int i = int.Parse(s);

The extension is defined in a static method using "this" as the first argument.

    public static class StringExtensions
    {
        public static int GetValue(this string s)
        {
            return int.Parse(s);
        } 
    }

We invoke the extension like this…

    String s = "123";
    int i = s.GetValue();

So far, nothing has been improved. But now let's redefine GetValue() as a generic method and use reflection to find the static Parse() method on the type and invoke it. We end up with this…

        public static T GetValue<T>(this string s)
        {
            MethodInfo mi = typeof(T).GetMethod("Parse", new Type[] { typeof(string) });
            if (s == null)
            {
                throw new ArgumentNullException("s");
            }
            else if (mi != null)
            {
                return (T)mi.Invoke(typeof(T), new object[] { s });
            }
            else if (typeof(T).IsEnum)
            {
                if(Enum.TryParse(typeof(T), s, out object ev))
                    return (T)(object)ev;
                else
                    throw new ArgumentException($"{s} is not a valid member of {typeof(T).Name}");
            }
            else
            {
                throw new ArgumentException($"No conversion supported for {typeof(T).Name}");
            }
        }

Note that we are looking for a Parse() method with a single string argument. A lot of types such as float, double, int, long, DateTime have this. Enum is a special case. Enum always seems to be a special case however we really want to support it. There seems to be a fundamental truth revealed as the enumeration type seems to create special cases in every programming language in which it appears.

Now we can do things like…

    String si = "123";
    int i = si.GetValue<int>();
    String sf = "1.234";
    float f = sf.GetValue<float>();
    String sd = "8/7/2021";
    DateTime d = sd.GetValue<DateTime>();

We want the value conversion to work both ways so I've added another static method to the extension class to convert back…

        public static string SetValue(object value)
        {
            if (value == null)
            {
                return null;
            }
            else if (value is float)
            {
                return ((float)value).ToString("R");
            }
            else if (value is double)
            {
                return ((double)value).ToString("R");
            }
            else if (value is DateTime)
            {
                return ((DateTime)value).ToString("O");
            }
            else
            {
                return value.ToString();
            }
        }

Note the use of MakeByRefType() to search for the out parameter for TryParse();

With the additional of the default value for our value conversion we now have something that delivers. But here is the real bonus. GetValue() is not limited to built-in types. It will work for any type of user defined class or struct that we define Parse() and TryParse() on. For example….

    public struct BoardSize
    {
        public int Width { get; set; }
        public int Height { get; set; }
        public override string ToString()
        {
            return $"({Width},{Height})";
        }
        public static bool TryParse(string s, out BoardSize bs)
        {
            string[] ss = s?.Split(new char[] { '(', ',', ')' });
            if (ss.Length == 4)
            {
                bs = new BoardSize()
                {
                    Width = ss[1].GetValue<int>(40),
                    Height = ss[2].GetValue<int>(40)
                };
                return s == bs.ToString();
            }
            else
            {
                bs = new BoardSize() { Width = 40, Height = 40 };
                return false;
            }
        }
        public static BoardSize Parse(string s)
        {
            if (TryParse(s, out BoardSize bs))
                return bs;
            else
                throw new FormatException();
        }
    }

The primary use case for value conversions is serialization. Let's say you are building a game want to load a config file on startup and save it on shutdown

    public class ConfigFile
    {
        public BoardSize BoardSize { get; set; } = new BoardSize() { Width = 40, Height = 40 };
        public int HighScore { get; set; } = 0;
        public DateTime HighScoreDate { get; set; } = DateTime.MinValue;
        public enum Difficulty {  Easy, Medium, Hard, Impossible };
        public Difficulty DifficultySetting { get; set; } = Difficulty.Medium;
        public float ScalingFactor { get; set; } = 1.0f;
        public string GameFolder = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "MyGame");
        public const string GameConfigFile = "MyGame.ini";
        public string GameConfigFilePath => Path.Combine(GameFolder, GameConfigFile);
        public ConfigFile()
        {
            Load();               
        }
        public void Load()
        {
            try
            {
                string contents = File.ReadAllText(GameConfigFilePath);
                string[] elements = contents.Split('|');
                BoardSize = elements[0].GetValue<BoardSize>(new BoardSize() { Width = 40, Height = 40 });
                HighScore = elements[1].GetValue<int>(0);
                HighScoreDate = elements[2].GetValue<DateTime>(DateTime.MinValue);
                DifficultySetting = elements[3].GetValue<Difficulty>(Difficulty.Medium);
                ScalingFactor = elements[4].GetValue<float>(1.0f);
            }
            catch { }
        }
        public void Save()
        {
            Directory.CreateDirectory(GameFolder);
            File.WriteAllText(GameConfigFilePath, ToString());
        }
        public override string ToString()
        {
            return StringExtensions.SetValue(BoardSize)
                + "|" + StringExtensions.SetValue(HighScore)
                + "|" + StringExtensions.SetValue(HighScoreDate)
                + "|" + StringExtensions.SetValue(DifficultySetting)
                + "|" + StringExtensions.SetValue(ScalingFactor);
        }
    }

This will produce a config file with contents similar to this…

(100,50)|0|0001-01-01T00:00:00.0000000|Hard|1

Summary and Discussion

In this post I have defined a set of string extensions that assist in value conversion of not only built in types but also user defined structs and classes. The primary use case is wherever serialization of objects is needed. There are, of course, plenty of approaches to serialization of .NET objects, including JSON, XML, ISerializable and many others. The string extensions defined here have the advantage of being very light weight and suitable for many use cases other than serialization.

Performance might be an issue with this code as it relies heavily on reflection. Performance can be improved by explicitly implementing the built-in type conversions. I'll do some performance modeling when I get a chance.

Are these extensions more elegant then the available Parse() and TryParase() methods? I'll admit this is debatable for the built-in types but where the extensions shine is the ability to extend the GetValue/SetValue pattern to user defined types and structs.

All the code for this post can be found at https://github.com/bobrundle/stringextensions

More .NET Core Apps on Linux

Bob Rundle — Sun, 13 Jun 2021 14:48:34 +0000

In my first post on this subject, I demonstrated how to develop a simple console app on windows and get it running on Linux with unit tests. In this post I will expand on the previous post and show how Linux-specific and Windows-specific code can be developed and unit tested. In addition I will take a look at xUnit test fixtures, Windows ACLs and Linux file permissions.

As in the last post, the approach is to develop code on Windows and test in Linux containers. So on your windows dev box the set up you need is Hyper-V and Docker.

Also needed are the dotnet CLI and VS Code for this optimum (in my view) setup.

All the code for this tutorial can be found at https://github.com/bobrundle/dotnettolinux2

What I want is to create an API to test read and write access of files and folders. This functionality is missing from the .NET API in my view. The only way to test that a file or folder is readable is to try to read it. If it is unreadable, an exception will be thrown. Likewise for writing an existing file. To verify that a folder is writable, that is, a file can be created in it, you must try to create a file and see if an exception is thrown. Catching exceptions might be fine for a lot of use cases but I keep running into use cases where I really want to know whether a file or folder path is readable or writable before actually doing file operations…for example for validating user entry. To do that, I need to check ACLs on Windows and File Permissions on Linux. So here we go

Add a set of stubs to define our API.

using System;

namespace FileSupport
{
    public static class FileAccess
    {
        /// <summary>
        /// This method determines whether the file indicated by the path argument is writable. To be writable in this context
        /// means that the file can be either modified, appended to, overwritten or deleted.  If the file does not exist, it can
        /// be created.
        /// </summary>
        /// <param name="path">The path to the file to test for writability.  The path may be either relative or absolute.</param>
        /// <returns>True, if the file is writable.</returns>
        public static bool IsFileWritable(string path)
        {
            return false;
        }
        /// <summary>
        /// This method determines whether the folder indicated by the path argument is writable. To be writable in this context
        /// means that files or folders can be added or removed from the folde.
        /// </summary>
        /// <param name="path">The path to the folder to test for writability.  The path may be either relative or absolute.</param>
        /// <returns>True, if the folder is writable.</returns>
        public static bool IsFolderWritable(string path)
        {
            return false;
        }
        /// <summary>
        /// Determines whether the file indicated by the path is readable.  To be readable means that the file can be opened to an input stream and bytes can be
        /// read from the stream.
        /// </summary>
        /// <param name="path">The relative or absolute path to the file to be tested.</param>
        /// <returns>True, if the file is readable.</returns>
        public static bool IsFileReadable(string path)
        {
            return false;
        }
        /// <summary>
        /// Determines whether the folder indicated by the path is readable.  To be readable means that the folder contents can be
        /// listed.
        /// </summary>
        /// <param name="path">The relative or absolute path to the folder to be tested.</param>
        /// <returns>True, if the folder is readable.</returns>
        public static bool IsFolderReadable(string path)
        {
            return false;
        }
    }
}

The API is very simple. Just 4 top level methods each taking a path and returning a Boolean. Note that this API has an expansive view of writability. If I can write a file, I should also be able to read and delete it. Perhaps there is a use case for being able to write a file without being able to read it, but I can't think of it and am not interested in it. This in general is the problem with working with privileges and permissions…the number of corner cases is huge. At every turn we will seek to simplify and limit the number of permission combinations we will accept.

Filling out the stubs…

using System;
using System.IO;
using System.IO.FileSystem.AccessControl;

namespace FileSupport
{
    public static class FileAccess
    {
        /// <summary>
        /// Determines whether the the file indicated by the path is readable.  To be readable means that the file can be opened to an input stream and bytes can be
        /// read from the stream.
        /// </summary>
        /// <param name="path">The relative or absolute path to the file to be tested.</param>
        /// <returns>True, if the file is readable.</returns>
        public static bool IsFileReadable(string path)
        {
            if (File.Exists(path))
            {
                return  IsNormalFile(path) && HasFilePermission(path, FileSystemRights.Read);
            }
            return false;
        }
        /// <summary>
        /// Determines whether the the folder indicated by the path is readable.  To be readable means that the folder contents can be
        /// listed.
        /// </summary>
        /// <param name="path">The relative or absolute path to the folder to be tested.</param>
        /// <returns>True, if the folder is readable.</returns>
        public static bool IsFolderReadable(string path)
        {
            if (Directory.Exists(path))
            {
                    return HasFilePermission(path, FileSystemRights.Read);
            }
            return false;
        }
        /// <summary>
        /// This method determines whether the file indicated by the path argument is writable. To be writable in this context
        /// means that the file can be either modified, appended to, overwritten or deleted.  If the file does not exist, it can
        /// be created.
        /// </summary>
        /// <param name="path">The path to the file to test for writability.  The path may be either relative or absolute.</param>
        /// <returns>True, if the file is writable.</returns>
        public static bool IsFileWritable(string path)
        {
            bool result = false;
            if (File.Exists(path))
            {
                result = IsNormalFile(path) && !IsReadOnly(path) && HasFilePermission(path, FileSystemRights.Modify);
            }
            else
            {
                result = true;
            }
            return result && IsFolderWritable(path.GetDirectoryName(path));
        }
        /// <summary>
        /// This method determines whether the folder indicated by the path argument is writable. To be writable in this context
        /// means that files or folders can be added or removed from the folde.
        /// </summary>
        /// <param name="path">The path to the folder to test for writability.  The path may be either relative or absolute.</param>
        /// <returns>True, if the folder is writable.</returns>
        public static bool IsFolderWritable(string path)
        {
            if (Directory.Exists(path))
            {
                return HasFolderPermission(path, FileSystemRights.Modify);
            }
            return false;
        }
    }
}

I added the nuget package System.IO.FileSystem.AccessControl. This is Windows only code. I will address the Linux part of the code later.

Need to add code now for several utility functions we added. IsNormal() verifies that we are reading and writing only "normal" files…that is, files that are not directory, system, hidden, encrypted or temporary files. IsReadOnly() tests the read only file attribute of the file.

Finally HasPermission() checks the ACL for the file (More properly the DACL, the discretionary access control list) . ACLs are the primary means for determining whether the caller has read or write access. The read only file attribute must still be checked, because even if the ACL allows write the read only file attribute will prevent it. An ACL is a list of ACE's (access control entries), each of which denies or allows access for a particular right and a particular user or group.

ACLs are Windows only so the HasFilePermission() and HasPermission() functions are protected by the SupportedOSPlatformAttribute(). ACLs are also supported on some Linux systems, but they are not often used and there is no support in .NET for Linux ACLs.

using System;
using System.IO;
using System.Linq;
using System.Runtime.Versioning;
using System.Security.AccessControl;
using System.Security.Principal;

namespace FileSupport
{
    public static class FileAccess
    {
        /// …
        /// <summary>
        /// Determines whether the file indicated by the path argument is a normal file.  To be a normal file in this context
        /// means that the file is not a directory and is also not encrypted, hidden, temporary or a system file.
        /// </summary>
        /// <param name="path">the path to the file to be tested.</param>
        /// <returns>True, if the indicated file is a normal file.</returns>
        [SupportedOSPlatform("windows")]
        public static bool IsNormalFile(string path)
        {
            FileAttributes fa = File.GetAttributes(path);
            FileAttributes specialFile = FileAttributes.Directory | FileAttributes.Encrypted | FileAttributes.Hidden | FileAttributes.System | FileAttributes.Temporary;
            return (fa & specialFile) == 0;
        }
        /// <summary>
        /// Determines if the read-only file attribute is set.
        /// </summary>
        /// <param name="path">The path to the file</param>
        /// <returns>True, if the file has the read only attribute set.</returns>
        public static bool IsReadOnly(string path)
        {
            FileAttributes fa = File.GetAttributes(path);
            return (fa & FileAttributes.ReadOnly) != 0;
        }
        /// <summary>
        /// Determines whether a file has the indicated file system right.
        /// </summary>
        /// <param name="path">The path to the file</param>
        /// <param name="right">The file system right.</param>
        /// <returns>True, if the file has the indicated right.</returns>
        [SupportedOSPlatform("windows")]
        public static bool HasFilePermission(string path, FileSystemRights right)
        {
            FileSecurity fs = new FileSecurity(path,AccessControlSections.Access);
            return HasPermission(fs, right);
        }
        /// <summary>
        /// Determines whether the indicated file system security object has the indicated file system right.
        /// </summary>
        /// <param name="fss">The file system security object.</param>
        /// <param name="right">The file system right.</param>
        /// <returns>True, if the indicated file system security object has the indicated file system right.</returns>
        /// <remarks>The current Windows user identity is used to search the security object's ACL for 
        /// relevent allow or deny rules.  To have permission for the indicated right, the object's ACL
        /// list must contain an explicit allow rule and no deny rules for either the user identity or a group to which
        /// the user belongs.</remarks>
        [SupportedOSPlatform("windows")]
        private static bool HasPermission(FileSystemSecurity fss, FileSystemRights right)
        {
            AuthorizationRuleCollection rules = fss.GetAccessRules(true, true, typeof(SecurityIdentifier));
            var groups = WindowsIdentity.GetCurrent().Groups;
            SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
            FileSystemRights remaining = right;
            foreach (FileSystemAccessRule rule in rules.OfType<FileSystemAccessRule>())
            {
                FileSystemRights test = rule.FileSystemRights & right;
                if (test != 0)
                {
                    if (rule.IdentityReference == user || (groups != null && groups.Contains(rule.IdentityReference)))
                    {
                        if (rule.AccessControlType == AccessControlType.Allow)
                        {
                            remaining &= ~test;
                            if (remaining == 0)return true;
                        }
                        else if (rule.AccessControlType == AccessControlType.Deny)
                        {
                            return false;
                        }
                    }
                }
            }
            return false;
        }
    }
}

Let's build what we have…

$ dotnet build -c Release
Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(22,71): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Read' is 
only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(36,28): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(22,47): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(36,52): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Read' is 
only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(22,25): warning CA1416: This call site is reachable on all platforms. 'FileAccess.IsNormalFile(string)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(70,48): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Modify' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(70,24): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(52,69): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(52,93): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Modify' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(52,26): warning CA1416: This call site is reachable on all platforms. 'FileAccess.IsNormalFile(string)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
  FileSupport -> D:\rundle\dev.to\2021-06-05\FileSupport\bin\Release\net5.0\FileSupport.dll

Build succeeded.

D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(22,71): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Read' is 
only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(36,28): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(22,47): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(36,52): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Read' is 
only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(22,25): warning CA1416: This call site is reachable on all platforms. 'FileAccess.IsNormalFile(string)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(70,48): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Modify' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(70,24): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(52,69): warning CA1416: This call site is reachable on all platforms. 'FileAccess.HasFilePermission(string, FileSystemRights)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(52,93): warning CA1416: This call site is reachable on all platforms. 'FileSystemRights.Modify' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
D:\rundle\dev.to\2021-06-05\FileSupport\FileAccess.cs(52,26): warning CA1416: This call site is reachable on all platforms. 'FileAccess.IsNormalFile(string)' is only supported on: 'windows'. [D:\rundle\dev.to\2021-06-05\FileSupport\FileSupport.csproj]
    10 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.58

We have a number of warning related to the windows-only code. We'll address these later. First let's build some unit tests to run on Windows.

Add project reference..

For our units tests we will need a test fixture. To test the various access methods we will need files and folders set to various combinations of read and write permissions. The test fixture will create these files and folders before the unit tests run and then remove them when the unit tests are complete. The test fixture must be robust, removing any test files left over from previous runs before adding new ones. Also an environment variable is checked before removing files. This is important for a debug scenario where we want to examine the created test files.

using System;
using System.IO;
using System.Linq;
using System.Runtime.InteropServices;
using System.Security.AccessControl;
using System.Security.Principal;
using Xunit;

namespace FileSupportTests
{
    public class TestFileFixture : IDisposable
    {
        public string TestDir { get; }
        public const string InvalidFilePath = " <|>";
        public const string NullFilePath = null;
        public const string EmptyFilePath = "";
        public const string UnwritableFileName = "Unwritable.txt";
        public const string UnreadableFileName = "Unreadable.txt";
        public const string WritableFileName = "Writable.txt";
        public const string HiddenFileName = "HiddenFile.txt";
        public const string ReadOnlyFileName = "ReadOnly.txt";
        public const string NonExistentFileName = "NonExistent.txt";
        public const string UnreadableFolderName = "UnreadableFolder";
        public const string UnwritableFolderName = "UnwritableFolder";
        public const string ReadableFolderName = "ReadableFolder";
        public TestFileFixture()
        {
            TestDir = CreateTestDirectory();
            Directory.SetCurrentDirectory(TestDir);
            DeleteTestFiles();
            CreateTestFiles();
        }
        public void CreateTestFiles()
        {
            CreateUnwritableFile();
            CreateUnreadableFile();
            CreateWritableFile();
            CreateHiddenFile();
            CreateReadOnlyFile();
            CreateUnreadableFolder();
            CreateUnwritableFolder();
            CreateReadableFolder();
        }
        protected string CreateTestDirectory()
        {
            string testbasedir = Path.GetTempPath();
            string testDirPath = Path.Combine(testbasedir, "FileSupportTests");
            Directory.CreateDirectory(testDirPath);
            return testDirPath;
        }
        private void CreateReadOnlyFile()
        {
            string path = ReadOnlyFileName;
            CreateFile(path);
            FileAttributes fa = File.GetAttributes(path);
            File.SetAttributes(path, fa | FileAttributes.ReadOnly);
        }
        protected void CreateFile(string path)
        {
            using (FileStream fs = File.Create(path))
            {

            }
        }
        private void ClearReadOnlyAttribute(string path)
        {
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileAttributes fa = File.GetAttributes(path);
                File.SetAttributes(path, fa & (~FileAttributes.ReadOnly));
            }
            else
            {
                FileInfo fi = new FileInfo(path);
                fi.IsReadOnly = false;
            }
        }

        private void CreateHiddenFile()
        {
            string path = HiddenFileName;
            CreateFile(path);
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileAttributes fa = File.GetAttributes(path);
                File.SetAttributes(path, fa | FileAttributes.Hidden);
            }
        }

        private void CreateWritableFile()
        {
            string path = WritableFileName;
            CreateFile(path);
        }

        private void CreateReadableFolder()
        {
            string path = ReadableFolderName;
            Directory.CreateDirectory(path);
        }

        private void CreateUnreadableFile()
        {
            string path = UnreadableFileName;
            CreateFile(path);
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Read, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = 0;
            }
        }

        private void CreateUnwritableFile()
        {
            string path = UnwritableFileName;
            CreateFile(path);
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Write, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = FileAccessPermissions.OtherRead | FileAccessPermissions.GroupRead | FileAccessPermissions.UserRead;
            }
        }

        private void CreateUnwritableFolder()
        {
            string path = UnwritableFolderName;
            Directory.CreateDirectory(path);
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Write | FileSystemRights.Modify, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = FileAccessPermissions.OtherRead | FileAccessPermissions.GroupRead | FileAccessPermissions.UserRead;
            }
        }

        private void CreateUnreadableFolder()
        {
            string path = UnreadableFolderName;
            Directory.CreateDirectory(path);
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Read, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = 0;
            }
        }
        public void ClearDenyACEs(string path)
        {
            FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
            AuthorizationRuleCollection rules = fs.GetAccessRules(true, true, typeof(SecurityIdentifier));
            SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
            AuthorizationRuleCollection newRules = new AuthorizationRuleCollection();
            FileSystemSecurity fssNew = new FileSecurity();
            foreach (FileSystemAccessRule rule in rules.OfType<FileSystemAccessRule>())
            {
                if(rule.IdentityReference == user && rule.AccessControlType == AccessControlType.Deny)
                {
                    fs.RemoveAccessRule(rule);
                    FileInfo fi = new FileInfo(path);
                    fi.SetAccessControl(fs);

                }
            }
        }
        public void DeleteTestFiles()
        {
            if(File.Exists(UnwritableFileName))File.Delete(UnwritableFileName);
            if(File.Exists(UnreadableFileName))File.Delete(UnreadableFileName);
            if(File.Exists(WritableFileName))File.Delete(WritableFileName);
            if(File.Exists(HiddenFileName))File.Delete(HiddenFileName);
            if(File.Exists(ReadOnlyFileName))
            {
                ClearReadOnlyAttribute(ReadOnlyFileName);
                File.Delete(ReadOnlyFileName);
            }
            if(Directory.Exists(UnwritableFolderName))
            {
                if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
                {
                    ClearDenyACEs(UnwritableFolderName);
                }
                Directory.Delete(UnwritableFolderName);
            }
            if(Directory.Exists(ReadableFolderName))Directory.Delete(ReadableFolderName);
            if(Directory.Exists(UnreadableFolderName))Directory.Delete(UnreadableFolderName);
        }
        public bool NeedsCleanup()
        {
            var flag = Environment.GetEnvironmentVariable("NOCLEANUP");
            return flag != "1";
        }
        public void Dispose()
        {
            if(NeedsCleanup())
            {
                DeleteTestFiles();
            }
        }

    }
}

Now our unit tests. We not only use the API to check file access but we also try to actually access the file to ensure the fixture is creating the file properly.

using System;
using System.Security.AccessControl;
using Xunit;
using FileSupport;
using Xunit.Abstractions;
using System.Runtime.InteropServices;

namespace FileSupportTests
{
    public class FileSupportTests : IClassFixture<TestFileFixture>
    {
        private TestFileFixture _fixture;
        public FileSupportTests(TestFileFixture fixture)
        {
            _fixture = fixture;
        }
        [Fact]
        public void IsWritableTest()
        {
            Assert.False(FileAccess.IsFileWritable(TestFileFixture.UnwritableFileName));
            Assert.Throws<UnauthorizedAccessException>(() => { System.IO.File.WriteAllText(TestFileFixture.UnwritableFileName,""); });
            Assert.True(FileAccess.IsFileWritable(TestFileFixture.WritableFileName));
            Assert.True(FileAccess.IsFileWritable(TestFileFixture.NonExistentFileName));
            Assert.True(FileAccess.IsFileWritable(TestFileFixture.InvalidFilePath));
            Assert.False(FileAccess.IsFileWritable(TestFileFixture.NullFilePath));
            Assert.False(FileAccess.IsFileWritable(TestFileFixture.EmptyFilePath));
        }

        [Fact]
        public void IsFolderReadableTest()
        {
            Assert.False(FileAccess.IsFolderReadable(TestFileFixture.UnreadableFolderName));
            Assert.ThrowsAny<UnauthorizedAccessException>(() => { System.IO.Directory.GetFiles(TestFileFixture.UnreadableFolderName); });
            Assert.True(FileAccess.IsFolderReadable(TestFileFixture.ReadableFolderName));
            Assert.False(FileAccess.IsFolderReadable(TestFileFixture.InvalidFilePath));
        }
        [Fact]
        public void IsFolderWritableTest()
        {
            Assert.False(FileAccess.IsFolderWritable(TestFileFixture.UnwritableFolderName));
            Assert.Throws<UnauthorizedAccessException>(() => { System.IO.File.WriteAllText(System.IO.Path.Combine(TestFileFixture.UnwritableFolderName,"1.tmp"),""); });
            Assert.False(FileAccess.IsFolderWritable(TestFileFixture.UnreadableFolderName));
            Assert.True(FileAccess.IsFolderWritable(TestFileFixture.ReadableFolderName));
            Assert.False(FileAccess.IsFolderWritable(TestFileFixture.InvalidFilePath));
        }

        [Fact]
        public void IsNormalFileTest()
        {
            Assert.False(FileAccess.IsNormalFile(TestFileFixture.HiddenFileName));
            Assert.False(FileAccess.IsNormalFile(_fixture.TestDir));
            Assert.True(FileAccess.IsNormalFile(TestFileFixture.WritableFileName));
        }
        [Fact]
        public void IsReadableTest()
        {
            Assert.True(FileAccess.IsFileReadable(TestFileFixture.UnwritableFileName));
            Assert.True(FileAccess.IsFileReadable(TestFileFixture.WritableFileName));
            Assert.False(FileAccess.IsFileReadable(TestFileFixture.UnreadableFileName));
            Assert.Throws<UnauthorizedAccessException>(() => { System.IO.File.ReadAllText(TestFileFixture.UnreadableFileName); });
            Assert.False(FileAccess.IsFileReadable(TestFileFixture.InvalidFilePath));
        }
        [Fact]
        public void HasFilePermissionTest()
        {
             Assert.True(FileAccess.HasFilePermission(TestFileFixture.WritableFileName, FileSystemRights.Write));
             Assert.False(FileAccess.HasFilePermission(TestFileFixture.UnwritableFileName, FileSystemRights.Write));
        }

        [Fact]
        public void IsReadOnlyTest()
        {
            Assert.False(FileAccess.IsReadOnly(TestFileFixture.WritableFileName));
            Assert.True(FileAccess.IsReadOnly(TestFileFixture.ReadOnlyFileName));
        }

    }
}

Build our unit tests

We get a large number of warnings related to windows-only code. These warning will get tedious and they are not telling us anything we don't know already so let's disable them by adding in every source file…

#pragma warning disable CA1416

Let's run our unit tests.

We've got our windows only code well in hand. Now let's turn to the Linux side of things. On Linux we use file permissions. .NET Core does not have native support for Linux file permissions but we can use a Mono package to get these. For those unfamiliar with Mono, the short history is that Mono is a predecessor to .NET Core, the first .NET that ran on Linux.

We also add it to the FileSupportTests project. In FileSupport we'll add Linux specific code. So for each of our 4 top level API functions…

       public static bool IsFileReadable(string path)
        {
            if (File.Exists(path))
            {
                if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
                {
                    return  IsNormalFile(path) && HasFilePermission(path, FileSystemRights.Read);
                }
                else
                {
                    return IsNormalFile(path) && HasFilePermission(path, FileAccessPermissions.UserRead | FileAccessPermissions.GroupRead | FileAccessPermissions.OtherRead);
                }
            }
            return false;
        }

        public static bool IsFolderReadable(string path)
        {
            if (Directory.Exists(path)) 
            {
                if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
                {
                    return HasFilePermission(path, FileSystemRights.Read);
                }
                else
                {
                    return HasFilePermission(path, FileAccessPermissions.UserRead | FileAccessPermissions.GroupRead | FileAccessPermissions.OtherRead);
                }
            }
            return false;
        }

        public static bool IsFileWritable(string path)
        {
            bool result = false;
            if (File.Exists(path))
            {
                if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
                    result = IsNormalFile(path) && !IsReadOnly(path) && HasFilePermission(path, FileSystemRights.Modify);
                else
                    result = IsNormalFile(path) && HasFilePermission(path, FileAccessPermissions.UserWrite | FileAccessPermissions.UserRead 
                        | FileAccessPermissions.GroupWrite | FileAccessPermissions.GroupRead
                        | FileAccessPermissions.OtherWrite | FileAccessPermissions.OtherRead);
            }
            else
            {
                result = true;
            }
            return result && IsFolderWritable(Path.GetDirectoryName(path));
        }

        public static bool IsFolderWritable(string path)
        {
            if (path == "") path = ".";
            if (Directory.Exists(path))
            {
                if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
                {
                    return HasFilePermission(path, FileSystemRights.Modify);
                }
                else
                {
                    return HasFilePermission(path, FileAccessPermissions.UserWrite | FileAccessPermissions.GroupWrite | FileAccessPermissions.OtherWrite);
                }
            }
            return false;
        }

We need Linux specific code in the IsNormalFile() method.


        public static bool IsNormalFile(string path)
        {
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileAttributes fa = File.GetAttributes(path);
                FileAttributes specialFile = FileAttributes.Directory | FileAttributes.Encrypted | FileAttributes.Hidden | FileAttributes.System | FileAttributes.Temporary;
                return (fa & specialFile) == 0;
            }
            else
            {
                var fi = new UnixFileInfo(path);
                return fi.FileType == FileTypes.RegularFile;
            }
        }

Finally we need a new overload of HasPermission() to handle Linux file permissions. Note that I am using UnsupportedOSPlatform("windows") instead of SupportedOSPlatform("linux") because I want this code to run for macOS as well, even though this won't be tested.

        [UnsupportedOSPlatform("windows")]
        public static bool HasPermission(UnixFileSystemInfo fi, FileAccessPermissions fap)
        {

            var effective = fi.FileAccessPermissions & fap;
            var user = UnixUserInfo.GetRealUser();
            if(user.UserId == fi.OwnerUserId)
            {
                return (effective & FileAccessPermissions.UserReadWriteExecute) == (fap & FileAccessPermissions.UserReadWriteExecute);
            }
            else if(user.GroupId == fi.OwnerGroupId)
            {
                return (effective & FileAccessPermissions.GroupReadWriteExecute) == (fap & FileAccessPermissions.GroupReadWriteExecute);
            }
            else
            {
                return (effective & FileAccessPermissions.OtherReadWriteExecute) == (fap & FileAccessPermissions.OtherReadWriteExecute);
            }
        }

We'll need some Linux specific code in the test fixture.

        private void ClearReadOnlyAttribute(string path)
        {
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileAttributes fa = File.GetAttributes(path);
                File.SetAttributes(path, fa & (~FileAttributes.ReadOnly));
            }
            else
            {
                FileInfo fi = new FileInfo(path);
                fi.IsReadOnly = false;
            }
        }

        private string CreateHiddenFile()
        {
            string path = HiddenFileName;
            CreateFile(path);
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileAttributes fa = File.GetAttributes(path);
                File.SetAttributes(path, fa | FileAttributes.Hidden);
            }
            return path;
        }

        private string CreateUnreadableFile()
        {
            string path = UnreadableFileName;
            CreateFile(path);
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Read, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = 0;
            }
            return path;
        }

        private string CreateUnwritableFile()
        {
            string path = UnwritableFileName;
            CreateFile(path);
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Write, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = FileAccessPermissions.OtherRead | FileAccessPermissions.GroupRead | FileAccessPermissions.UserRead;
            }
            return path;
        }

        private string CreateUnwritableFolder()
        {
            string path = UnwritableFolderName;
            Directory.CreateDirectory(path);
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Write | FileSystemRights.Modify, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = FileAccessPermissions.OtherRead | FileAccessPermissions.GroupRead | FileAccessPermissions.UserRead;
            }
            return path;
        }

        private string CreateUnreadableFolder()
        {
            string path = UnreadableFolderName;
            Directory.CreateDirectory(path);
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                FileSecurity fs = new FileSecurity(path, AccessControlSections.Access);
                SecurityIdentifier user = WindowsIdentity.GetCurrent().User;
                FileSystemAccessRule r = new FileSystemAccessRule(user, FileSystemRights.Read, AccessControlType.Deny);
                fs.AddAccessRule(r);
                FileInfo fi = new FileInfo(path);
                fi.SetAccessControl(fs);
            }
            else
            {
                var fi = new UnixFileInfo(path);
                fi.FileAccessPermissions = 0;
            }
            return path;
        }

Also needed is a little bit of Linux specific code in the unit tests since there is no such thing as a hidden file on Linux. Also we are using a different HasPermission() overload on Linux.

       [Fact]
        public void IsNormalFileTest()
        {
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
                Assert.False(FileAccess.IsNormalFile(_fixture.HiddenFilePath));
            else
                Assert.False(FileAccess.IsNormalFile("/dev/null"));
            Assert.False(FileAccess.IsNormalFile(_fixture.TestDir));
            Assert.True(FileAccess.IsNormalFile(_fixture.WritableFilePath));
        }

        [Fact]
        public void HasFilePermissionTest()
        {
            if(RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                Assert.True(FileAccess.HasFilePermission(TestFileFixture.WritableFileName, FileSystemRights.Write));
                Assert.False(FileAccess.HasFilePermission(TestFileFixture.UnwritableFileName, FileSystemRights.Write));
            }
            else
            {
                Assert.True(FileAccess.HasFilePermission(TestFileFixture.WritableFileName, FileAccessPermissions.UserWrite
                        | FileAccessPermissions.GroupWrite
                        | FileAccessPermissions.OtherWrite));   
                Assert.False(FileAccess.HasFilePermission(TestFileFixture.UnwritableFileName, FileAccessPermissions.UserWrite
                        | FileAccessPermissions.GroupWrite
                        | FileAccessPermissions.OtherWrite));
            }
        }

Build for Linux…

As in the 1st .NET Core to Linux post, create a dockerfile to run the tests on Linux

FROM mcr.microsoft.com/dotnet/sdk:5.0

WORKDIR /bin
COPY bin/Release/net5.0/linux-x64 .

CMD ["dotnet","vstest","FileSupportTests.dll"]

Build docker…

Running our unit test on Linux…

This is surprising. Our unit tests are failing on Linux. What is going on? Let's run an interactive docker session to find out. Turn on our NOCLEANUP flag and examine the files created by the test fixture.

So here is the problem. Our API for IsFileReadable() properly returns false for Unreadable.txt, but when we open it, no exception is thrown. This is because, by default, we are root in the docker container. There is no stopping root from reading everything. So clearly to get our unit tests working we need to run the tests as an ordinary user. In the docker file we add a new user "tester"…

FROM mcr.microsoft.com/dotnet/sdk:5.0

SHELL ["/bin/bash","-c"]
RUN adduser --gid 100 tester
WORKDIR /home/tester
COPY bin/Release/net5.0/linux-x64 .

CMD ["dotnet","vstest","FileSupportTests.dll"]

Then when we run it we use the "-u" switch to specify tester as the current user. Note that no passwords are needed to do this.

Much better! Let's go interactive and see how the environment has changed.

Summary and discussion

To recap:

A simple file access API library was built.
Unit tests were written for the library which included a test fixture that created and deleted test files.
The library was tested on Windows.
Linux-specific code was added to the library and unit tests.
The Linux code was tested in docker Linux containers.

This is my 2nd post on .NET Core Apps for Linux. There have been some improvements over my 1st post. In particular I figured out how to unit test in Linux without building source. The key to this is the dotnet cli command vstest

$ dotnet vstest FileSupportTests.dll

Now I can simply move the cross compiled binaries to the container and run the tests. This is a big improvement over the 1st post where I moved source and compiled the source. The problem with compiling source in the containers is that it gets harder as the code gets bigger, more complex with more dependencies. Binaries are much easier. All the dependencies are in one place. This is still not ideal however, because I still need the .NET SDK to run the tests. I would prefer to run in plain vanilla Ubuntu to ensure that I have all the needed dependencies. Note that the library built in this example would fail in plain vanilla Ubuntu because, as discussed in the first post, .NET 5.0 requires ICU by default and it is not available by default on Linux. The 1st post indicates how to address this problem.

The following discussion points come to mind…

100% code coverage is not a panacea. In the 1st post I claimed that 100% code coverage was the "gold standard" of unit testing. Well…let's not get carried away. The Windows specific code in this example was covered 100% by the unit tests run created for the Windows-only code. However 100% coverage does not begin to address all the possible test cases for Windows ACLs. Even with this very limited set of test cases, the unit test code, including the test fixture, has 3 times the number of lines of code as the code under test. I have the idea that people think that amount of unit test code you write is much less than the actual code. This example shows how the reverse can be true. Certain functionality, like Windows ACLs, have a depth of complexity that causes the number of possible test cases to explode.

Just good enough unit testing. The Linux specific code in this example does not have 100% code coverage. To fully cover the HasPermission() method would need two more users added to the docker container and files in the test fixture created by these two other users in order to test the group and everyone file permissions. Not sure how to do this in the container. The point is that writing unit test code does not have a uniform level of difficulty as most people seem to think. Getting to 100% code coverage sometimes requires very challenging approaches that in turn suck up a lot of developer time. This is sometimes not worth it. I have manually tested the uncovered code sections and then declared the units tests "good enough". People want perfection in unit tests only when they are not paying the bills.

More corners than sides. The term "corner case" is curious because it implies there can be no more than 4. Yet it turns out that for certain problem spaces, corners dominate. File access is one of these. Look at the huge corner case we found during Linux testing. The API we wrote indicated a file as unreadable when in fact the file could be read because we were acting as root. Note that we simply ignored this corner case and changed the test to use a normal user. Is this right? Shouldn't the code be changed to return true when the user is root? After some thought, I decided no. Our API is evaluating the permissions of the file and not considering special privileges of the user. In the same way the "ls -l" command reports file access permissions independent whether user is root. On a different day I might have decided yes. This is the nature of many corner cases…they are subtle with no clear cut solutions. There is another corner case in the Linux version of HasPermission(). I am checking user permissions before group and before everyone. This is means if user has no access, then HasPermission() will return false even if everyone has access. This must be wrong. But no! Test it for yourself…

So it seems that the Linux file permission 044 implies "everyone but owner can read." Find a description of Linux file permissions that explains this subtlety. You can't do it. A better name for corner case might be "between the lines case" because the situation is one where the documentation utterly fails you. So there are at least two big corner cases in the Linux code in this example. Linux file access permissions, however, are much simpler to understand than Windows ACLs. The Window ACL code in this example has even more corner cases and I don't want to spend the 5 paragraphs it would take to enumerate them. The point is that I believe people underestimate the importance of corner cases. They are thought to be a small part of the problem of writing working code when in fact for certain problem spaces the corner cases form what might be called a Super Pareto Principle…where 2% of the problem requires 98% of the effort.

I hope people find this post useful. You don't see a lot of examples devoted to ACLs, file permissions, Linux mixed with Windows specific code so I thought I would add to this oeuvre.

.NET Core Apps on Linux

Bob Rundle — Thu, 20 May 2021 02:22:17 +0000

One of the great benefits in working with .NET Core is knowing that your code will be cross platform. In particular it will run on Linux. This opens up a lot of possibilities. But does it really run on Linux if you have never seen it run? But even if you have seen it run, is it really working if you have never run the unit tests on Linux? In my way of looking at the world…no.

So in this post I will lay out how to get your cross platform .NET Core apps running and tested on Linux in the most straightforward and efficient way.

The approach is to develop code on Windows and test in Linux containers. This is the best combination in my view. So on your windows dev box the set up you need is Hyper-V and Docker. Getting this setup right is not without its challenges which I will not get into here, but I am pleased to report that once you get this working it stays working and I have had this setup working for years now through all manner of Windows and Docker updates.

Also needed are the dotnet CLI and VS Code for this optimum (in my view) setup.

All the code for this tutorial can be found at https://github.com/bobrundle/dotnettolinux

I'll start by creating a simple console app that adds the numbers that appear as arguments.

In VS Code…

Build and run on windows…

Here is where it gets interesting. Create a dockerfile for Linux deployment…

Build a Linux docker image

Let's try running it…

Oops. ICU stands for Internationalization components for Unicode which is used to handle culture dependent APIs. .NET 5.0 requires ICU by default and it is not available by default on Linux. For a simple app such as ours, the easiest thing to do is disable globalization support.

To disable globalization support we need to add another property to our add.csproj project file…

Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
    <InvariantGlobalization>true</InvariantGlobalization>
  </PropertyGroup>

</Project>

Now lets build and run again…

Now let's add unit tests. You test-first wingnuts will be very disappointed that I didn't write these first, but I am simply not a test first guy. I could say more but need to stay focused.

Need to add a project reference to add.csproj…

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="1.3.0">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="../add/add.csproj"/>
  </ItemGroup>

</Project>

Our unit tests…

using System;
using Xunit;
using add;
using System.IO;

namespace AddTests
{
    public class ProgramTests
    {
        [Theory]
        [InlineData(new string[] {}, "0",0)]
        [InlineData(new string[] {"1","2","3"}, "6",0)]
        [InlineData(new string[] {"1","2","a"}, "",1)]
        [InlineData(new string[] {"1.1","2.2","3.3"}, "6.6",0)]
        [InlineData(new string[] {"-1e6","1e6"}, "0",0)]
        public void MainTest(string[] args0, string r0, int e0)
        {
            string outfile = Path.GetTempFileName();
            var outstream = File.CreateText(outfile);
            Console.SetOut(outstream);
            int e1 = Program.Main(args0);
            Console.Out.Close();
            string r1 = File.ReadAllText(outfile);
            Assert.Equal(e0, e1);
            if(e0 == 0)
            {
                Assert.Equal(r0 + Environment.NewLine,r1);    
            }
        }
    }
}

Build the unit tests and run them…

To run the unit tests in Linux we need to more than move binaries…we have to setup a development environment and build the code before running the tests. To do this we need a Docker file in the parent directory to both code and test folders.

The Dockerfile…

FROM mcr.microsoft.com/dotnet/sdk:5.0

WORKDIR /src
COPY /add add
COPY /addtests addtests
WORKDIR /src/addtests

CMD ["dotnet","test"]

Build and run on Linux…

Summary and Discussion

To recap:

A simple Windows console app was created, built and run on Windows.
The console app was built for Linux on Windows and run in a Linux container.
A xUnit testing library was created to run tests against the console app. It was built and run on Windows.
The source for both the console app and the xUnit tests were built and run in a Linux container.

The following questions about this approach come to mind...

Why are you not a test-first guy? My answer is too long to be considered here.

Your "unit tests" are actually integration tests! This is semantics. What we can agree on is 100% code coverage is the gold standard of automated testing and this has been achieved in this example.

What about macOS? You cannot run macOS containers on Windows. You can only run macOS containers on Macs. There might be a way to test all 3 platforms (Windows, Linux, macOS) on a Mac with containers. I will experiment when I get a chance.

Why build in the Linux container? Why not simply use a test runner to run the binaries? Indeed, this is a good idea. I simply don't know how to get this to work with xUnit.

Why not construct a CI/CD pipeline to build and test on Windows and Linux in the cloud? Indeed, the next logical step. However, you still cannot reach macOS in the cloud.

I hope what I have done is useful and addresses some questions you might have. I spent about a day researching the various aspects of this problem. I came to this issue when I was designing a command line tool for Windows and came to realize that the tool would be useful on Linux. Then I began to look into building and testing on Linux and discovered the approach was not well documented and not straight-forward and so suggested a post to capture the learnings.

How to fix WordPress "Error establishing a database connection"

Bob Rundle — Sun, 15 Nov 2020 03:46:35 +0000

This problem is easily fixed. There are only 2 steps: (1) get the detailed error message; (2) fix the problem indicated by the error message. So ignore all the advice you have received so far. There are not 5 easy steps…there are just 2….find out more…fix the problem. It is simply idiotic to start changing database credentials, repairing the database, restoring the default WordPress files, etc., etc., etc. All of these shotgun techniques are unlikely to work. When the check engine light come on in your car do you simply start replacing parts? Of course not. You get the diagnostic code behind the idiot light (they are called idiot lights for a reason) and proceed accordingly. The same approach applies here. The famous "Error establishing a database connection" is simply an idiot light.

Step 1: Get the Detailed Error Message

For this you will need to modify wp-config.php in the root directory of your site.

Find this line…

define( 'WP_DEBUG', false );

and change it to

define( 'WP_DEBUG', true );

Now restart the site and you will see a detailed error message.

Step 2: Fix the problem indicated by the error message

Just like the check engine light in your car, any of a large number of issues may have caused the WordPress idiot light to come on. Below you will find a list of possible error messages and the indicated fixes. This list is by no means exhaustive.

Problem 1: SSL Connection required

Warning: mysqli_real_connect(): (HY000/9002): SSL connection is required. 
Please specify SSL options and retry. 
in /home/site/wwwroot/wp-includes/wp-db.php on line 1635
SSL connection is required. Please specify SSL options and retry.

This was the issue with my site. I was deploying WordPress to Azure in a Web App with the database in MySQL for Azure. MySQL for Azure requires a secure connection and by default WordPress uses an unencrypted connection.

The fix for this one is to add the following line of code to wp-config.php…

define ( 'MYSQL_CLIENT_FLAGS', MYSQLI_CLIENT_SSL);

This issue is a great example of why you need to get the detailed error message. You can search the hundreds of answers claiming to know how to fix the WordPress database connection problem and never find this issue. Without detailed error messages you would be flailing around for days before giving up completely.

Problem 2: Bad host name or DNS lookup issue

Warning: mysqli_real_connect(): (HY000/2002): php_network_getaddresses: getaddrinfo failed: No such host is known. 
in D:\rundle\WordPress\wordpress-5.5.3\wordpress\wp-includes\wp-db.php on line 1635
php_network_getaddresses: getaddrinfo failed: No such host is known.

This means the DNS lookup failed for the hostname you have provided. This might be because the hostname is misspelled or it might mean the DNS server that your site uses has no entry for the host. Fixing a spelling error is easy. The other problem is harder but at least you can talk intelligently about it to people that know how to fix DNS lookup problems.

Check the spelling error in wp-config.php….

/** MySQL hostname */
define( 'DB_HOST', 'localhost' );

Problem 3: Database not running or not accessible

Warning: mysqli_real_connect(): (HY000/2002): No connection could be made because the target machine actively refused it. 
in D:\rundle\WordPress\wordpress-5.5.3\wordpress\wp-includes\wp-db.php on line 1635
No connection could be made because the target machine actively refused it.

This means that your database is not reachable. It might not be running. There might be a firewall issue. The default port might have been changed. There are a lot of things to look into, but the problem has been narrowed considerably.

Problem 4: Invalid user name

Warning: mysqli_real_connect(): (HY000/2054): The server requested authentication method unknown to the client 
in D:\rundle\WordPress\wordpress-5.5.3\wordpress\wp-includes\wp-db.php on line 1635
The server requested authentication method unknown to the client

This cryptic message means the user is not defined. With any set of credentials, WordPress will try a few authentication methods before giving up. WordPress assumes the credentials are valid, but simply doesn't have the proper authentication method defined.

Check the user name in wp-config.php…

/** MySQL database username */
define( 'DB_USER', 'username_here' );

Problem 5: Invalid password

Warning: mysqli_real_connect(): (HY000/1045): Access denied for user 'wpuser'@'localhost' (using password: YES) 
in D:\rundle\WordPress\wordpress-5.5.3\wordpress\wp-includes\wp-db.php on line 1635
Access denied for user 'wpuser'@'localhost' (using password: YES)

Your password for the MySQL database is wrong. In this case the username is wpuser. Check the password in wp-config.php…

/** MySQL database password */
define( 'DB_PASSWORD', 'password_here' );

Problem 6: Database name wrong or database does not exist or user does not have access

Access denied for user 'wpuser'@'localhost' to database 'wpbench2'

This means that the database name is wrong or does not exist. WordPress can create the database tables from an empty database but it cannot create the database itself.

Check the database name in wp-config.php…

// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
define( 'DB_NAME', 'wpbench2 );

If the name is right, then you need to log in to the database using a tool like MySQL Bench to see if the schema exists.

If the schema exists then the next issue to determine is if your database user has access to the schema.

With all schema privileges, the WordPress site will be able to create all the tables from an empty schema. It is not good practice to allow all privileges to the database user, however if the database is initially empty then you will need more privileges then if the schema is properly initialized. Determining the minimum set of privileges required is a topic for another post.

Finally…

So once you get your site working and go from 😭 to 😎 it is easy to simply forget everything and move on. But wait! Before you do that you need to turn off diagnostics…

In wp-config.php, change…

define( 'WP_DEBUG', true );

back to

define( 'WP_DEBUG', false );

There is a reason that WordPress does not, by default, give you detailed error messages. It is because the error messages can display sensitive information such as server names and user names which can be used to compromise your system.

So after getting your connection to work…turn the idiot light back on!