Automate AVDL to C# Generation

In the previous post, you saw how to take an Avro IDL file and manually turn that into a C# class.

For a small number of files, this is fine but as soon as you start dealing with a lot of them it'll be something you'll want to automate. I've written a PowerShell script to do this will cover how to use it + what it does.

Prerequisites

• Java
• .NET Core SDK
• Apache.Avro.Tools
• PowerShell Core

Folder Structure

Create a folder called Avro and within that:

• Add a file called Generate.ps1
• Add a folder called AVDL and place any .avdl files in there

It should look something like:

.
└── Avro
    ├── AVDL
    │   └── ExampleMessage.avdl
    └── Generate.ps1        

The Generate.ps1 Script

The PowerShell script is what does all the work. The contents can be copied from this gist:

What it does:

• Downloads avro-tools to $HOME/.avro-tools if it's not there already
• Scan the contents of the AVDL folder
• For each .avdl file found, generate the .avsc and from that, the C# class
• All generated files are written to the Avro/_generated folder

Running

To run the script just do:

pwsh Generate.ps1

The results should end up like:

.
└── Avro
    ├── AVDL
    │   └── ExampleMessage.avdl
    ├── Generate.ps1
    └── _generated
        └── avro
            ├── classes
            │   └── example
            │       └── avro
            │           └── ExampleMessage.cs
            └── schema
                └── ExampleMessage.avsc

The script also does some basic change detection so it'll only generate the files if something has changed within the AVDL folder. It's just a simple check of last modified dates, so it's not perfect (i.e. doesn't work with deletions) but it works for most cases.

Summary

We took the manual steps from the previous post and automated it using PowerShell. In the final post, we'll look at adding this to a project so it gets executed as part of the build process.