DEV Community: Rafael

Controlling My AC Unit With nanoFramework

Rafael — Mon, 17 Jul 2023 05:18:39 +0000

This blog post first appeared on my blog

Why

My room's AC unit (Panasonic) is quite old. It does a great job cooling/heating the place after all these years, but also lacks any "smart" capabilities found on modern units.

Personally, I am not a fan of making everything at my home internet-connected for reasons I won't delve into now, and I certainly don't want my AC unit to be internet-connected either. However, I want to be able to expand on its functionality as needed.

The unit has a basic set of features:

Set mode (Cool, Heat, Dry, etc...)
Set temperature
Swing
Fan speed
ON/OFF Timer

Using the timer functionality, I can set a time for it to switch itself ON and another time for it to switch OFF. This is helpful, but is quiet limited as it only accepts time as a trigger and doesn't allow specifying a temperature/mode other than the one it was set to before the On/Off timer is set.

I'd like it to work based on the room temperature instead.
In winter, it should turn on in HEAT mode when the room temperature drops below a specified threshold and off when it goes above another threshold. In summer, it should turn on when the temperature is above a certain threshold and off when it gets below another threshold.

Yes, it has a thermostat but it is not very good. In summer, it makes the room too cold and in winter, it gets too hot. I suspect that if there was a temperature sensor at the other end of the room, it would help.

Additionally, Australian weather can be very humid. The AC unit has a "Dry" mode which activates its built-in dehumidifier but there is no built-in way to activate it based on air humidity levels.

Being a programmer, naturally I thought I could build something to automate the functions of the AC unit based on temperature and humidity parameters. And while I am at it, I could have it report the state to the isolated HomeAssistant instance on my network.

So to summarize, the goals of this project are:

Turn the AC unit on/off and set mode & temperature based on room temperature.
Turn the AC unit on/off and enable the dehumidifier when humidity goes above a set threshold.
Nice to have: Report the AC unit status to the internal HomeAssistant instance on my network.

This post is about my journey to achieve this and all the problems/hurdles I ran into. Spoiler: I actually did it at the end.

How

After I had the goals of the project defined, I started thinking about how I could achieve that. The first problem I had to solve was:

Communication

Like I mentioned above, the AC unit is old and has no smart features or APIs I could hook into. I didn't want to solder wires into a PCB deep into its guts. I wanted this solution to be completely unobtrusive.

That left me with the remote control and infrared (IR) channel as the communication interface. My thinking was that if I could somehow capture a signal from the remote, I could then replay it at the AC unit.

In theory, I could capture an "ON" signal, an "OFF" signal, a "COOL/HEAT" signal, and, somehow, a signal for temperature control. At this point, I only knew how remote controls worked at a high level (Remote has IR LED, IR LED sends signal, IR Receiver receives signal and device does something). I soon learned there is a lot more to IR communication than just that. A lot more.

For now, the solution was to somehow send commands through the IR channel.

Tech stack

I am a .Net dev. I love .Net and last year I was introduced to nanoFramework. This is an open-source project to bring C# and .NET to the world of embedded systems. There are various reasons to choose nanoFramework:

Easy to get started with as a beginner in the world of IoT and embedded system programming.
It is a serious, open-source project used in personal and commercial solutions and not only to blink LEDs.
As a developer, you can be extremely productive with .NET and C# when compared to the traditional C/C++.
You can also use Visual Studio with nanoFramework to write, deploy, and debug code directly on the target device. It is very easy to place breakpoints, execute the program and step through the code line by line while inspecting variables and other state. All within the Visual Studio IDE.
Dependency management is available through NuGet where nf and the community publish a lot of libraries.
nanoFramework is also very easy to extend and add new stuff or improve what's already there.
There's a great community that's very friendly and supportive.
But most importantly, it is just so damn fun to work with!

So you can see why nf is my choice for this side project. Keep in mind that there are other projects to bring C# & .Net to the micro-controller world such as Meadow & TinyCLR by GHI but I don't have much experience with them at this time.

The next thing I had to figure was probably the easiest in this project:

Sensing temperature & humidity

In my previous post, I used the DHT22 sensor to get the room temperature and humidity level.

DHT22 is simple to use, requires 3-5V power, and provides the measurements using a digital signal on a single pin. Perfect for use with a Micro-Controller Unit.

Ok, so I know what to use to get the temp & humidity. How can I talk to the DHT22 module?

MCU

ESP32 is a very good choice for this project. It has support for IR communication using its built-in RMT module. It is also compatible with nanoFramework making it ideal for what I am doing here.

The Journey

The first thing I needed to do was to build something that can capture signals from the AC remote control and store them somehow for further analysis and replay.

ESP32 has an RMT (Remote Control Transceiver) module and the folks on nanoFramework's lively discord server pointed me towards a nanoFramework library that exposes the RMT hardware to the managed C# code. The library can be found here.

I wired up an IR Receiver to my ESP32 dev board as follows:

With that in place, the next step is to write some code to capture incoming signals from the AC remote. nanoFramework has a sample project showing how the RMT nuget library can be used. Based on that sample, I wrote something like this:


var rxChannel = new ReceiverChannel(RxPinNumber);
rxChannel.ClockDivider = 80; // 1us clock ( 80Mhz / 80 ) = 1Mhz
rxChannel.EnableFilter(true, 100); // filter out 100Us / noise 
rxChannel.SetIdleThresold(40000);  // 40ms based on 1us clock
rxChannel.ReceiveTimeout = new TimeSpan(0, 0, 0, 0, 60); 

rxChannel.Start(true);

while (true)
{
    var recivedSignal = rxChannel.GetAllItems();

    Debug.WriteLine($"Received a signal with {receivedSingal.Length} items:");

    foreach (var pulse in receivedSignal)
    {
        Debug.WriteLine($"Pulse: {pulse.Level0} {pulse.Duration0} - {pulse.Level1} {pulse.Duration1}");
    }

    Thread.Sleep(1000);
}

What this does is it initializes a new RMT Receiver Channel and sets some reasonable defaults as per the comments. Then it enters an infinite loop to listen for incoming infrared signals.

ESP32 RMT module has a simple way of representing the signal. You can read about it in details here. The main thing to know is that a signal is made of a collection of RMT Items. Each item represents a single "pulse" and is made of 2 pairs of values. Each pair consists of the duration in ticks and the signal level (HIGH vs. LOW).

The above code should've outputted something like:

Received a signal with 64 items:
Pulse: true 500 - false 1000
Pulse: true 250 - false 500
etc...

And here, I ran into the first set of issues:

There was a lot of random, 1-item signals being picked up. (Interferance?)
The remote signal seemed to have a variable length. Sometimes it would be 48 pulses long, sometimes only 35 (and other "random" numbers) even for the same button press.

I wasn't sure why I was getting a lot of 1-pulse signals in my output window. Increasing the EnableFilter duration parameter improved things but it still let some 1-pulse signals through frequently.

Doing some research helped me learn that there is usually a lot of infrared interference around us. Sunlight is a big source but there are others. It is why IR Signals are modulated. Manufacturers typically use a 38KHz carrier wave but it can range from 36KHz to 46KHz.

This didn't explain why I was seeing those short signals. My theory at this point was noise from the circuit and I opted to write some basic code to filter out such signals. After all, this was a temporary device put together just to record some signals for analysis and it was going to get disassembled afterwards. No reason to invest more time into this, right?

That solution worked and now I was left with the second problem: Is my AC unit remote really sending variable length signals for the same button presses? This made absolutely no sense and it took a while to get this sorted out.

Variable length signal? No way!

There are various IR protocols out there. Each manufacturer usually uses one of them for their products. These protocols define some signal characteristics including but not limited to:

Signal carrier frequency
Header pulse length
Logical 0 and 1 pulse length
If the signal should repeat and how

It is up to the manufacturer to define what data to transmit. Some simple remote controls such as TV remotes could have a specific command for every button, while other remotes (like my AC unit's one) send out the entire state of the remote on every button press. More on that later.

For example, the most popular IR protocol is NEC. This protocols specifies that a signal:

consists of 8 bit address & 8 bit command
uses pulse distance modulation to represent digital data
has a carrier frequency of 38kHz
pulse is always 560µs

Logical 1 & 0 are represented as follows:

(Source: NEC Protocol)

But what about my AC Unit's remote control? Is it actually sending variable length signals even for the same button?

Well, no. According to some arduino IR libraries, Panasonic AC remotes send 216 bits of data. This is not even close to what I was getting above. This led me to believe that there was a problem with my setup: either hardware or software.

In terms of hardware, there wasn't much going on. An IR receiver connected directly to the MCU. So I decided to turn my attention to the software side of things.

My theory as to why I was seeing a variable number of pulses was due to some timing issues. I played around with SetIdleThreshold and found that setting it to 12000 ticks worked best. This is equal to waiting for 12ms for edges. If none are detected, then the RMT module would stop listening for more signals and return what it had already detected.

This change gave me a stable number of pulses. At this point, I was constantly getting 64 pulses on every button press.

But 64 != 216 bits. So what is going on here?

nanoFramework's relationship with ESP IDF

nanoFramework integrates with the ESP IDF framework. This integrations allows nanoFramework to communicate with the various hardware functions that ESP32 MCUs offer (GPIO, RMT, I2C, Wifi, Bluetooth, etc...).

ESP IDF methods are then exposed by nanoFramework and made available to the C# managed code. Libraries such as the nanoFramework ESP32 RMT one make use of these native methods.

This fact led me to consult with the ESP IDF documentation where I found this:

And more clearly, on the web version:

What this means is that ESP allocates 8 blocks of 64x32bit of internal RAM shared by the 8 RMT channels. Each channel gets 1 block (64x32bits).

This perfectly explained why I was now constantly getting 64 pulses.

Luckily, the docs mention that the number of allocated blocks to a channel can be increased using:

esp_err_t rmt_set_mem_block_num(rmt_channel_t channel, uint8_t rmt_mem_num)

This is great! I went looking for an equivalent in nanoFramework's RMT library but sadly, it was not implemented.

nanoFramework's RMT implementation exposed only some of the capabilities of the ESP RMT module and it lacked many options already available in ESP IDF. This was an opportunity for me to contribute an improvement to the nanoFramework project!

Initially, I thought about adding support only for the method mention above. This would've been a quick contribution and would get me back to working on my project sooner. But it didn't feel right and the more I looked at the implementation the more I leaned towards a simple API refactor and exposing ALL the RMT configs.

Updating the nanoFramework RMT library

As mentioned earler, the nf library relies on native ESP IDF methods exposed by nf. This meant that any changes and improvements I make to the RMT library would first start in the nanoFramework firmware itself.

The nanoFramework discord is pretty amazing. I had a thread running with a few people (Including the founders of nf) discussing questions I had and how to approach this improvement/refactor. I made it clear that I had no experience with C/C++ code but I was keen on doing what is needed to fix the issue at hand. I was pointed towards the nanoFramework dev container approach which makes use of the dev containers feature in Visual Studio Code to setup a fully functional nf development environment within Docker.

It didn't take much effort to get that setup and I started diving into the codebase and making changes to the RMT implementation.

The result of the refactoring and improvement work was:

On the native side (nanoFramework Firmware) most of the changes were about introducing new C methods exposing RMT configs. Additionally, some changes were to add C#/Managed code-compatible types as needed.

On the managed side (nanoFramework RMT nuget library), the changes were as follows:

Introduce a new RmtChannelSetting containing various config options for RMT channels. This is an abstract class with 2 classes inheriting from it:
- ReceiverChannelSettings for receiver-specific channel configs.
- TransmitterChannelSettings for transmitter-specific channel configs.
Expose all ESP IDF RMT Configs through the various *ChannelSetting classes.
Introduce a new RmtChannel abstract class exposing common functionality and properties used by receiver and transmitter channels. It has 2 inheritors:
- TransmitterChannel
- ReceiverChannel

These changes meant bumping the library version number from 1.x to 2.x as it introduced breaking changes. However, migrating to the new APIs was very straightforward. We have the following code snippit on the RMT library which shows how easy it is:

V1.x API Surface:


// creating a transmit channel
var txChannel = new TransmitterChannel(TxPinNumber);
txChannel.ClockDivider = 80;
txChannel.CarrierEnabled = false;
txChannel.IdleLevel = false;
txChannel.AddCommand(new RmtCommand(20, true, 15, false));
// add more commands...

txChannel.Send(false);

// creating a receive channel
var rxChannel = new ReceiverChannel(RxPinNumber);
rxChannel.ClockDivider = 80; // 1us clock ( 80Mhz / 80 ) = 1Mhz
rxChannel.EnableFilter(true, 100); // filter out 100Us / noise 
rxChannel.SetIdleThresold(40000);  // 40ms based on 1us clock
rxChannel.ReceiveTimeout = new TimeSpan(0, 0, 0, 0, 60); 
rxChannel.Start(true);

In V2.x the above must be re-written as:

var txChannelSettings = new TransmitChannelSettings(-1, TxChannelPinNumber)
{
  ClockDivider = 80,
  EnableCarrierWave = false,
  IdleLevel = false
};

var txChannel = new TransmitterChannel(txChannelSettings);
txChannel.AddCommand(new RmtCommand(20, true, 15, false));
// add more commands...

txChannel.Send(false);


var rxChannelSettings = new ReceiverChannelSettings(pinNumber: RxChannelPinNumber)
{
  EnableFilter = true,
  FilterThreshold = 100,
  IdleThreshold = 40_000,
  ReceiveTimeout = new TimeSpan(0, 0, 0, 0, 60)
};

using var rxChannel = new ReceiverChannel(rxChannelSettings);
rxChannel.Start(clearBuffer: true);

As you can see, the big change is around declaring the channel configs in a separate class and passing that instance as a parameter to the channel object constructor.

Some of the new configurations that were added to the library as part of this were:

NumberOfMemoryBlocks: set the number of memory blocks that a channel can own. This is what I needed.
EnableDemodulation: enable/disable signal de/modulation.
CarrierWaveFrequency: set the carrier wave frequency used in modulating/demodulating the IR signal.
CarrierWaveDutyPercentage
EnableLooping: allows to repeat the signal sent
LoopCount
EnableSignalInverter enables/disables a built-in ESP32 features that inverts the signal using hardware.

Once these pull requests were merged in, I was back working on my AC automation project.

Decoding the Panasonic AC Unit Remote Signal

Through work done by other people in the open source community (referenced at the end) I learned that AC unit remote controls generally transmit the entire state of AC unit that is stored in the remote control.

This is generally true for AC remotes that have an LCD such as this one:

What this meant is that my plan to simply record a signal from button press and play it back would not work. I need to set the mode & temperature at the same time and in order to do that, I have to replicate the state of a remote control in my code, manipulate it to set the parameters as I need and then transmit that state data via IR.

The plan has now shifted towards decoding the remote control signal rather than simply replaying it.

Luckily, I didn't have to start from scratch. The same work I referenced above proved to be a great starting point. It listed out some important facts about the contents of the signal and some details that saved me a lot of time.

The signal is made of 2 frames of data:

Frame 1:
- 8 bytes long.
- Constant value equal to 0x4004072000000060.
- Sent in MSB order.
Frame 2:
- 18 bytes long.
- Transmitted as LSB8 (more info on this later).
- First 5 bytes are equivalent to bytes from Frame 1 but in LSB8.
- The remaining bytes transmit the state of the remote.
CRC:
- Calculated by adding all 18 bytes of Frame 2.
- Transmitted as LSB8.
- Last byte in the transmission.

In order to transmit data on this Panasonic Protocol, the follow timings must be adhered to:

Type	Time (micro-second, approx.)
Header	3500
Header Space	1750
Pulse	435
'Zero' Bit Space	435
'One' Bit Space	1300
Modulation	38Khz
Tolerance	Max. 30%

With this info, I only had to focus on the second frame of the signal.

I started this by writing an "IR Decoder" program that is deployed to my ESP32 that I connected to an IR receiver (as per the fritzing diagram above).

The complete utility program can be found on github here. Deploying it to the ESP32 dev board results in the following when a signal is received from the remote control:

Frame 1:
=== RAW === | === HEX ===
 0100 0000  |     0x40
 0000 0100  |     0x04
 0000 0111  |     0x07
 0010 0000  |     0x20
 0000 0000  |     0x00
 0000 0000  |     0x00
 0000 0000  |     0x00
 0110 0000  |     0x60
Frame 2:
=== RAW === | === HEX === | === LSB8 === | === MSB HEX ===
 0100 0000  |     0x40    | 0000 0010  |    0x02
 0000 0100  |     0x04    | 0010 0000  |    0x20
 0000 0111  |     0x07    | 1110 0000  |    0xE0
 0010 0000  |     0x20    | 0000 0100  |    0x04
 0000 0000  |     0x00    | 0000 0000  |    0x00
 0001 1100  |     0x1C    | 0011 1000  |    0x38
 0000 0100  |     0x04    | 0010 0000  |    0x20
 0000 0001  |     0x01    | 1000 0000  |    0x80
 1000 1100  |     0x8C    | 0011 0001  |    0x31
 0000 0000  |     0x00    | 0000 0000  |    0x00
 0000 0000  |     0x00    | 0000 0000  |    0x00
 0111 0000  |     0x70    | 0000 1110  |    0x0E
 0000 0111  |     0x07    | 1110 0000  |    0xE0
 0000 0000  |     0x00    | 0000 0000  |    0x00
 0000 0000  |     0x00    | 0000 0000  |    0x00
 1000 0001  |     0x81    | 1000 0001  |    0x81
 0000 0000  |     0x00    | 0000 0000  |    0x00
 0000 0000  |     0x00    | 0000 0000  |    0x00
 0111 1110  |     0x7E    | 0111 1110  |    0x7E      <<< CRC

Having the signal output formatted like this was very important. I was then able to use WinMerge I could easily see which bits flipped with each button press and how the state was being represented.

For example, this shows the difference between sending an OFF command (LEFT) vs. sending an ON command (RIGHT):

You can see how bit 0 in byte 13 indicates what state (ON or OFF) the AC should be on: 0 == OFF, 1 == ON.

I used this appraoch for the other buttons and functions on the remote and my results matched what I had seen in other resources online. Here's the breakdown of all the commands and their representation in the data frame:

Byte #	Function	Description / Notes
0	Static	First 8 bytes are always the same
1	Static	This is the device/manufacturer ID
2	Static	Always equal to `0x4004072000000060`
3	Static
4	Static
5	Static
6	Static
7	Static
FRAME 2	====================	====================================
8	Static	The first 5 bytes in frame 2 always
9	Static	match the first 4 bytes from frame 1
10	Static	and equal to 0x4004072000 but in the LSB8 format
11	Static
12	Static
13	Mode, On/Off Switch and Timer	Bit 0 is On/Off (1/0). Bits 4..7 are mode. Bit 3 is special. More info below.
14	Temperature	Divide by 2 to get actual temp in C.
15	Unknown	0x80
16	Swing & Fan	The 4 MSB bits = FAN / 4 LSB bits = Swing. Refer to table below for codes
17	Unknown	0x00
18	Timer (ON)	Starting value: 0x00
19	Timer (ON/OFF)	Starting value: 0x08
20	Timer (OFF)	Starting value: 0x80
21	Profile	Refer to profile table for info (WIP/Not 100% correct)
22	Unknown	0x00
23	Model (Potentially)	0x80 (Static?)
24	Time (Part 1)	The value in this part is equal to the time in minutes. Max 255 minutes.
25	Time (Part 2)	The value in this part is how many 256 minutes are in the current time.
26	CRC	Calculated by adding all the bytes in frame 2.

On/Off, Mode, and Timer Function

Byte #13

This byte packs a lot of info:

Bit #0: On/Off command. 0 = Turn off AC unit. 1 = Turn on AC unit.
Bit #1: Timer option. 1 = Enable ON Timer option.
Bit #2: Timer option. 1 = Enable Off Timer option.
Bit #3: Execute command. 1 = AC unit will execute the command immediately. 0 = AC unit will receive the command but will not execute it.
Bit #4 to #7 indicates the operation mode as per below table:

Bit Mask	Mode
0110	FAN
0010	DRY
0011	COOL
0100	HEAT
0000	AUTO

Swing & Fan

Byte #16. The possible values as per the below tables:

Swing

4 LSB bits on Byte #16

Bit Mask	Meaning
1111	AUTO
0001	Horizontal (1)
0010	2
0011	3
0100	4
0101	Vertical or Ground (5)

Fan

4 MSB bits on Byte #16

Bit Mask	Meaning
1010	AUTO
0011	Slowest (1)
0100	2
0101	3
0110	4
0111	Fastest (5)

Current Time

The time is represented in number of minutes since midnight (00:00) by 2 bytes: #24 & #25.

Byte #24 represents the minutes since midnight. But, because the max value of a byte is 255, byte #24 can't go higher than that and so byte #25 will increment by 1 each time we reach the limit in byte #24.

So to read the time, do this:

Read byte #25.
Multiply its value by 256 (255 = 0xFF in byte #24 but when the value is 256, byte #24 becomes 0x00 and byte #25 becomes 0x01).
Read byte #24.
Add the 2 values.
Use TimeSpan (from the .Net BCL) to get the time in hour:minute format.

Examples:

Time is 00:02

byte	value
24	0x02
25	0x00

Time is 04:43

byte	value
24	0x1B
25	0x01

Timer

On/Off Timer

To enable/disable, set bit #1 in byte #13 to 1 (ON)/0 (OFF). This is for ON Timer. Off timer is the same but the state is stored in bit #2.

Time is stored in byte #18, #19, and #20 (Yes, this took a while to figure out) in simiar fashion as how Current Time is represented:

Byte #18 is number of minutes in increments of 10 (0x10, 0x14, etc...) and used for the ON timer.
Byte #19 is made of 2 parts:
- The lower 4 bits increment by one when byte #18 overflows. Each increment equals 256 minutes.
- The upper 4 bits represent the continuation of the Off timer value from byte #20.
- The starting value of byte #19 is 0x08.
Byte #20 is made of 2 parts as well:
- The lower 4 bits are basically the upper 4 bits of the minutes (upper 4 bits of byte #19). You take these lower 4 bits then put the upper 4 bits next to it to get the value. This is for the OFF timer.
  - Example: (Byte #20 lower 4 bits) (Byte #19 upper 4 bits)
- When minutes go beyond 256, the upper 4 bits increment by 1.
- The starting value of byte #20 is 0x80.

Examples:

ON Timer: 00:00
OFF Timer: 00:00

Byte	Value
18	0x00
19	0x08
20	0x80

ON Timer: 05:00
OFF Timer: 04:20

Byte	Value	Explanation
18	0x2C	0x2C = 44. So 44 Minutes on ON Timer
19	0x49	Starting value is 0x08. Ignore the upper 4 bits and value would be 0x09. 0x09 - 0x08 = 0x01 = 256 minutes. Add this to minutes from byte #18 and you have ON Timer.
20	0x90	0x90 - 0x80 = 0x10. Take the result and append the upper 4 bits of byte 18 to get the off time in minutes. So 0x104 = 260 (4hrs and 20 minutes)

Programmer Guidance:

For byte #19, to get the upper 4 bits, right shift by 4:

var upper4Bits = 0x49 >> 4; // result: 0x04

To get the lower 4 bits, you can isolate them using an & operator:

var lower4Bits = 0x49 & 0xF; // result: 0x09

// this is equal to writing it like: 0b_0100_1001 & 0b_0000_1111

Temp

Read byte #15 as int8 (byte in C#) and divide by 2 to get temp in Celsius.

Profiles

This is a work in progress as I don't know how the remote is structuring the data when a profile key is pressed. This is not important for my project so I haven't spent much time on it yet.

Bit Mask	Profile
0001 0000	Normal
0001 0001	Boost
0011 0000	Quiet

LSB8 vs MSB vs LSB Tranformation

Here's an example showing the difference between LSB and LSB8 when encoding data:

MSB: 01 02 03 (0000 0001 0000 0010 0000 0011)
LSB: C0 40 80 (1100 0000 0100 0000 1000 0000)
LSB8: 80 40 C0 (1000 0000 0100 0000 1100 0000)

Clean and Valid Starting State

The following byte array has a valid starting remote state that can be transmitted as is. I used this to observe the AC unit acknoledging the command properly.

public static byte[] StartingData = new byte[]
{
    0x40, 0x04, 0x07, 0x20, 0x00, 0x00, 0x00, 0x60,
    0x02, 0x20, 0xE0, 0x04, 0x00, 0x38, 0x20, 0x80,
    0x31, 0x00, 0x00, 0x0E, 0xE0, 0x00, 0x00, 0x81,
    0x00, 0x00, 
    0x7E
};

CRC Calculation

Here's how to calculate the CRC:

private static byte CalcCrc(byte[] data, int start, int end)
{
    byte crc = 0x00;
    for (var i = start; i < end; i++)
    {
        crc += data[i];
    }

    return crc;
}

Automating my AC Unit

With all that done, I was finally able to write some code to automate my AC unit using the IR signal. You can find the full source code here.

Here's how the ESP32 is wired up to the DHT22 and IR sender module:

As for the code, here's the important parts:

I started by defining the GPIO pins, the GPIO Controller, the DHT22 driver, and finally, the RMT channel I'll use:


public static void Main()
{
    var dhtEchoPin = 33;
    var dhtTriggerPin = 32;
    var dhtWakePin = 26;
    var irWakePin = 15;
    var irSignalPin = 16;

    using var gpioController = new GpioController();
    using var statusLed = gpioController.OpenPin(statusLedPin, PinMode.Output);
    using var dhtWaker = gpioController.OpenPin(dhtWakePin, PinMode.Output);
    using var irWaker = gpioController.OpenPin(irWakePin, PinMode.Output);

    using var dhtSensor = new Dht22(dhtEchoPin, 
                    dhtTriggerPin, 
                    gpioController: gpioController);

    // wake the DHT22 now. This should give it enough time to "warm up"
    dhtWaker.Write(PinValue.High);

    // while the DHT is waking up, I connect the ESP32 to Wifi and to my MQTT instance on my network. This is used to send room temperature and humidity levels to my home assistant instance. Code omitted for brevity.

    var attemptNumber = 0;
    while (attemptNumber < 5)
    {
        var temp = dhtSensor.Temperature;
        var humidity = dhtSensor.Humidity;
        if (!dhtSensor.IsLastReadSuccessful)
        {
            // log error to serial output if enabled and then try again
            // logging code here
            Thread.Sleep(1000);
            attemptNumber++;
            continue;
        }

        // very basic automation implementation (WIP)
        if (temp.DegreesCelsius > 25)
        {
            irWaker.Write(PinValue.High);
            PanasonicIRController.TurnOn(16, PanasonicACMode.Cool, irSignalPin);
            irWaker.Write(PinValue.Low);
        }
        else if (temp.DegreesCelsius < 20)
        {
            irWaker.Write(PinValue.High);
            PanasonicIRController.TurnOn(25, PanasonicACMode.Heat, irSignalPin);
            irWaker.Write(PinValue.Low);
        }
        else
        {
            irWaker.Write(PinValue.High);
            PanasonicIRController.TurnOff(irSignalPin);
            irWaker.Write(PinValue.Low);
        }
    }
}

// code continues by reporting temperature and then putting the ESP32 to sleep for 15 minutes.

The PanasonicIRController class contains all the code required to manipulate the clean starting state shown above and sending the signal via the RMT module:

internal static class PanasonicIRController
    {
        private const ushort HeaderPulse = 3468;
        private const ushort HeaderSpace = 1767;
        private const ushort Pulse = 432;
        private const ushort ZeroSpace = 432;
        private const ushort OneSpace = 1296;
        private const ushort PauseSpace = 10000;

        private const int OnOffModeByteIndex = 13;
        private const int TempByteIndex = 14;
        private const int ProfileByteIndex = 21;

        private static RmtCommand Header = new RmtCommand(HeaderPulse, true, HeaderSpace, false);
        private static RmtCommand ZeroBit = new RmtCommand(Pulse, true, ZeroSpace, false);
        private static RmtCommand OneBit = new RmtCommand(Pulse, true, OneSpace, false);
        private static RmtCommand Pause = new RmtCommand(Pulse, true, PauseSpace, false);
        private static RmtCommand End = new RmtCommand(Pulse, true, 0, false);

        public static void TurnOn(byte temp, PanasonicACMode mode, int irChannelPinNumber)
        {
            var commandData = GetStartingCommandData();

            // set on flag
            commandData[OnOffModeByteIndex] |= 0x01;

            // set mode
            commandData[OnOffModeByteIndex] |= (byte)mode;

            //set temp
            commandData[TempByteIndex] = (byte)(temp * 2);

            SendIRCommand(irChannelPinNumber, commandData);
        }

        public static void TurnOff(int irChannelPinNumber)
        {
            var commandData = GetStartingCommandData(); // starting command data is by default OFF
            SendIRCommand(irChannelPinNumber, commandData);
        }

        private static TransmitChannelSettings GetTransmitChannelSettings(int irChannelPinNumber)
            => new(irChannelPinNumber)
            {
                ClockDivider = 80,

                EnableCarrierWave = true,
                CarrierLevel = true,
                CarrierWaveFrequency = 38_000,
                CarrierWaveDutyPercentage = 50,

                IdleLevel = false,
                EnableIdleLevelOutput = true,

                NumberOfMemoryBlocks = 4,
                SignalInverterEnabled = false,
            };

        private static byte[] GetStartingCommandData()
            => new byte[]
        {
            /*  Byte 0 - 7   */ 0x40, 0x04, 0x07, 0x20, 0x00, 0x00, 0x00, 0x60, // frame 1: static

            //  frame 2
            /*  Byte 8 - 12  */ 0x02, 0x20, 0xE0, 0x04, 0x00, // this is static

            /* Byte 13       */ 0x38, // On/Off + Mode
            /* Byte 14       */ 0x20, // Temperature
            /* Byte 15       */ 0x80,
            /* Byte 16       */ 0x31,
            /* Byte 17       */ 0x00,
            /* Byte 18       */ 0x00,
            /* Byte 19       */ 0x08,
            /* Byte 20       */ 0x80,
            /* Byte 21       */ 0x00, // Profile
            /* Byte 22       */ 0x00,
            /* Byte 23       */ 0x81,
            /* Byte 24       */ 0x00,
            /* Byte 25       */ 0x00,
            /* Byte 26       */ 0x7E // crc
        };

        private static void AddCommands(this TransmitterChannel txChannel, byte[] data, int start, int end)
        {
            for (var i = start; i %3C end; i++)
            {
                byte b = data[i];
                for (byte mask = 0x01; mask > 0x00 && mask < 0xFF; mask <<= 1)
                {
                    if ((b & mask) > 0)
                    {
                        txChannel.AddCommand(OneBit);
                    }
                    else
                    {
                        txChannel.AddCommand(ZeroBit);
                    }
                }
            }
        }

        private static byte CalcCrc(byte[] data, int start, int end)
        {
            byte crc = 0x00;
            for (var i = start; i < end; i++)
            {
                crc += data[i];
            }

            return crc;
        }

        private static void SendIRCommand(int irChannelPinNumber, byte[] commandData)
        {
            // setup an IR trasnmitter channel
            var irTxChannelSettings = GetTransmitChannelSettings(irChannelPinNumber);
            using var irTxChannel = new TransmitterChannel(irTxChannelSettings);
            irTxChannel.ClearCommands();

            // add command data to the tx channel buffer
            irTxChannel.AddCommand(Header);
            irTxChannel.AddCommands(commandData, 0, 8); // frame 1
            irTxChannel.AddCommand(Pause);
            irTxChannel.AddCommand(Header);
            irTxChannel.AddCommands(commandData, 8, commandData.Length - 1); // frame 2 without CRC

            // calculate the crc
            var crc = CalcCrc(commandData, 8, commandData.Length - 1);
            irTxChannel.AddCommands(new byte[] { crc }, 0, 1); // CRC
            irTxChannel.AddCommand(End);

            // finally, send the command
            irTxChannel.Send(waitTxDone: true);
        }
    }

    internal enum PanasonicACMode : byte
    {
        // bits already left shifted by 4 (<< 4)

        Auto = 0x00,

        Fan = 0x60,

        Dry = 0x20,

        Cool = 0x30,

        Heat = 0x40,
    }

    internal enum PanasonicACProfile : byte
    {
        Normal = 0x10,

        Boost = 0x11,

        Quiet = 0x30
    }

And with that, I now have a functioning device to automate my Panasonic AC unit. The automation "algorithm" isn't the most sophisticated there is but it is a great starting point as I go and add more scenarios to it.

If you have read this far, thank you. I hope you enjoyed the post and found it helpful.

Github repos:

References & Literature

Some of this was confirmed by this project. It is for an different (older or newer) revision of the AC remote so there are some changes.
Panasonic AC Protocol Library
IR Protocols Database (by John S. Fine)
Info-rich page about IR protocols
IR-Send library intro post
IR Remote Control Theory

Uwp IoC Support

Rafael — Sat, 16 Nov 2019 09:53:02 +0000

This blog post first appeared on my blog

I have been developing UWP applications for a few years now and I truly enjoy it. It's been the platform where I experiment with and used it to learn a lot about programming.

In my day job I develop web applications on the ASP.NET MVC framework and one of the things it does right is the native support for IoC containers out of the box.

When a HTTP request arrives to an action method in a controller, MVC will initialize the controller, inject all dependencies in its constructor and have it ready to handle the request.

I find MVC and MVVM to be comparable as a software architectures. However, the support for IoC Containers in Microsoft's UWP platform is surprisingly lacking. When navigating to a new page in UWP you are forced to manually initialize the view model and provide all the required dependencies to it.

There is no native support in UWP for developers to bring in an IoC container and have it inject dependencies in the page constructor.

This has always been annoying for me personally and when I tried to find a workaround this, it seemed the most popular solution was to rely on the Service Locator pattern in the form of a ViewModel locator.

Some people consider the Service Locator to be an anti-pattern. I agree with that, but UWP seems to be forcing us to go down that path.

Recently I had an idea to overcome this limitation and it has worked surprisingly well at least for me. So this post is an invite for other UWP developers to look at this solution, use it, evalaute it, and decide if it is a better replacement for service locators in UWP.

How does it work?

It works by providing a modified Frame class that provides a custom handler for the Navigated event. In the custom handler, this frame attempts to perform a property injection on the new page before the page instance is loaded and displayed.

I have packaged up this solution in a nuget package. It is still a pre-release at version 1.0.0-alpha.

How to use it?

Setup

Start by installing the nuget package in the UWP application:

Install-Package UWPIoC -Version 1.0.0-alpha01

... and then make the App.Xaml.cs look like this:

public static Host ApplicationHost; // (1)

/* ... */

protected override void OnLaunched(LaunchActivatedEventArgs e)
{
    IServiceProvider serviceProvider = /* ... */ // (2)
    ApplicationHost = new Host(serviceProvider);

    var rootFrame = Window.Current.Content as Frame;

    // Do not repeat app initialization when the Window already has content,
    // just ensure that the window is active
    if (rootFrame == null)
    {
        rootFrame = ApplicationHost.CreateNewHostedUwpFrame(); // (3)

        /* ... */

        // Place the frame in the current Window
        Window.Current.Content = rootFrame;
    }

    /* ... */
}

Here's what the code above is doing:

(1) Initializes a new instance of the Host class. This class holds a reference to a IServiceProvider instance and Frame instance.

(2) Depending on what IoC container being used, this line gets a reference to its service provider which is used by IoCFrame.

(3) Requests a new instance of IoCFrame from the Host instance. This is done when the window has no current frame instance (this is always true when the app has just been launched).

Now, whenever a call is made to Frame.Navigate from any page, the IoCFrame will take care of injecting the new page dependencies.

Dependency Injection in pages

So how does a page code-behind file looks like when using IoCFrame to inject properties?

Here's an example:

public sealed partial class MyPage : Page
{
    [ViewModel] // (1)
    public MyPageViewModel ViewModel { get; set; }

    [Dependency] // (2)
    public ILogger<MyPage> Logger { get; set; }

    public MyPage()
    {
        /* ... */
    }

    /* ... */
}

That's it! Really. That's all there is to it!

Here's what's going on in this snippet:

(1) This the public auto-property holding a reference to a view model instance. the [ViewModel] attribute tells the IoCFrame this is a view model and that it should manually construct an instance because it won't be registered in the IoC container. IoCFrame will use a bit of reflection to find the view model's dependencies, request them from the IoC container and then create the instance.

(2) The [Dependency] attribute tells the IoCFrame that this is a typical service/dependency that is registered in the IoC container. IoCFrame will then request an instance of this service type from the IoC container and inject it in this property.

As you can see, the constructor has no custom code, no additional work from you as a developer is required here.

Now, when some code makes a call to Frame.Navigate(typeof(MyPage)) the app will navigate to the new page, and all of its dependencies will be there and ready to be used as expected.

Conclusion

While this is still not constructor injection, I'd say that in this case, property injection still works just fine and allows you as a developer to focus on building good UI/UX rather than writing biolerplate code for every new page you create to get instances of services/ViewModels you need.

Create a Dotnet Framework and Node Build Environment in Docker

Rafael — Tue, 01 Oct 2019 14:09:34 +0000

This blog post first appeared on my blog

Story time

At the company I currently work for, our CI pipelines run on a dedicated server that we own and manage on premise. This build server is using Atlassian Bamboo and is configured to run the builds using agents running directly inside the host OS which means builds share and depend on components installed on the host OS.

This configuration has been working fine for us and we rarely run into issues with it. Last week, however, one of the CI pipelines started to fail suddenly and at the worst time as it was a day before a hard deadline. We didn't know what went wrong with the build server. We had no idea whether someone made a change to the host OS causing our build to throw this random error and we had no time to fully investigate the issue.

In the interest of time and to deploy the site before the deadline I used my co-workers dev machine to run the same CI commands we use on the build server in order to deploy the site. This is not great. Trust me, I know. But we didn't have the luxury of time to come up with a more elegant solution. We literally had to fallback to an almost manual deployment.

This is obviously not great. Having multiple CI pipelines running on a single server is OK. What's not OK is having them share the host OS as this introduces the danger of a new pipeline being created breaking other CI pipelines accidentally.

So I decided it is time to start containerizing our builds.

Why

By containerizing our builds we can be sure that any CI pipeline we have with no matter what kind of configurations it needs, will never mess up other pipelines as each one is running in its own container separately from the others.

This means that I can run my build knowing that no matter how bad my configs are, they will never affect others. And by containerizing the pipeline, I can store the config files in the git repo and have those configs versioned alongside of the project code-base.

What I'll cover

This post will cover creating a build environment image in docker and how to use the image to build your code base locally on your own machine. Hopefully in a next post, I will cover how to use this with Atlassian's Bamboo.

Building our custom docker image

I thought docker image registry would have a pre-made image ready that fits my requirements:

Windows Based
Has DotNet Framework 4.X SDK
Has Node and NPM 10.X

As far as I can tell, there is no such image on the official docker registry. I don't know if I just didn't look hard enough or because I was a bit lazy. Turns out that creating my own image for this is quite easy.

Requirements

Obviously you'll need docker installed on your machine. You can use the community edition of docker for Windows.

Make sure that your docker installation is switched to Windows Containers. The reason for this requirement is DotNet Framework 4.X requires a Windows host and the official SDK image from Microsoft that's hosted on Docker's official registry doesn't run on Linux Containers. To switch your docker instance to Windows Containers, right click on the docker icon in your task bar and then select "Switch To Windows Containers". The Docker engine will restart during this process and will take a minute or so.

I am using this image. This is the Official .Net Framework SDK Container Image from Microsoft.
This image is based on the Windows Server Core and has the SDK installed on top of it. It also contains nuget and Visual Studio Build Tools (MSBuild).

What it doesn't have is NodeJS and I need it as the site I'm trying to build requires a build step to run some NPM commands responsible for building the UI assets.

So how can we modify that image?

Technically, we can't. Docker only allows us to build new ones. However, the above image will be our base. So we will just add NodeJS on top of it.

To build your own image, you'll need to create a DockerFile. Here's the DockerFile for the build environment image I created:

# Specify a base image. In this case, I'm using the .Net SDK image from MS
FROM mcr.microsoft.com/dotnet/framework/sdk AS DOTNET_SDK

# Tell Docker that I want to use PowerShell to run my commands
SHELL ["powershell"]

# Install Scoop (Windows Package Manager) from Scoop.sh (This command is on their homepage)
RUN iwr -useb get.scoop.sh | iex

# Tell Scoop to download and install NodeJS
RUN scoop install nodejs

# Set a working directory for us on the root drive
WORKDIR /app

# DONE
RUN exit

Ok, so what happened here? The base image I'm pulling has everything I need to build the BackEnd code of the site. However, to build the Front-End assets, I need NodeJS. The easiest way I could think of to add NodeJS to the image was to use Scoop.

Next step is to actually build the image. To do this save the above file and run this command:

docker build --tag=my-image-name --file path\to\dockerfile .

This will take some time to finish as Docker will have to download the SDK image which is ~1.5GB.

When done, we can run a quick test to make sure that the image we created has everything we need. To do this, we are going to run a command that will launch a container based on our image and then "SSH" into it:

docker run --rm -it my-image-name

--rm tells docker to remove the container once we exit it.

-it makes this container an interactive process that accepts input from us and display output directly in our shell.

When you run that command, your shell will look like this:

Microsoft Windows [Version 10.0.18362.356]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\app>

If you type in MSBuild and hit enter, you will see MSBuild execute on an empty directory and complain about it.

Do the same for nuget and you'll get the help output.

Finally, type node and you will start a new NodeJS session.

At this stage, we have successfully created a Docker image with all the tools we need to build an ASP.NET MVC Project and all of the Front-End assets using NodeJS and NPM.

In the next post, I'll show how to actually compile some code in that container and grab the output from it.

Dotnet Core Webapi With Mssql in Docker Part 2

Rafael — Sat, 28 Sep 2019 00:08:44 +0000

This blog post first appeared on my blog

In the previous post I showed how to setup a Dotnet Core WebApi project to run inside docker.

In part 2, I'll go over what is needed to containerize an SQL server and connect the WebApi to it.

What do you need?

This post assumes you have Docker Community Edition installed on your machine. If not, you can grab a copy from here.

Containerizing Microsoft SQL Server

Thankfully, Microsoft has created a base image for the SQL server which I will be using in this blog post. The image is available on the official docker registry. You can find it here.

We will start by creating a docker compose file. Docker Compose is a tool in docker used to define the configurations of your app's services and then create those services and start them up with a single command. More info can be found in Docker's official documentation page.

Name the file

docker-compose.yml

. The file should look like this:

version: "3"
services:
networks:

As you can see, we are using Version 3 of the docker compose file format.

Next, we need to define and configure the required services for out application. Here, Application encapsulates everything that must be up and running so the WebApi can return results as expected. In this case, we will have the following services:

WebApi: the dotnet core web api project
Sql-Server: an instance of Microsoft SQL server

Great! so now that we know what services to define, let's go ahead and define the first one in the docker compose file. We will start with the WebApi service.

We already have the DockerFile from the previous post which created the WebApi image. Let's see how to use that image to create the first service:

version: "3"
services:

  # webapi serves as the name of the service
  webapi:
    # The image name which this service will be based upon.
    image: MyWebApiProject

    # This defines the ports which should be exposed by this service's container to the outer world
    ports:
      - "5000:80"
      - "5001:443"

    # Make this service part of a network called webnet
    networks:
      - webnet
networks:
  webnet:

With this service, we are using the image name MyWebApiProject which was created by the WebApi DockerFile which we created in the previous post.
And now that we have defined the first service that will be created and launched, we move on to the next service to define: Microsoft SQL Server.

We don't have a DockerFile for MSSQL and we don't need one. We can specify an image hosted on Docker's official registry and docker compose will pull that image if it is not cached on the machine already. Let's see how we create the MSSQL service in docker compose:

version: "3"
services:

  # sql-server is the name of the service and will also be the name of the server on the network
  sql-server:

    # this is the image docker compose will pull to create a MSSQL service instance
    image: mcr.microsoft.com/mssql/server

    # this defines the ports to expose
    ports:
      - "1433:1433"
      - "1434:1433"

    # this put the MSSQL service instance on the same network as the WebApi service
    networks:
      - webnet

    # these are required by the image as per the MS documentions
    environment:

      # This is the password we will use for this server (passwords must pass the password rules of SQL server)
      SA_PASSWORD: "YG4ZkkGrn7QSz5&"

      # Required to accept the End User License Agreement
      ACCEPT_EULA: "Y"

      # This defines which edition of the server we want to run. We can also use "Express"
      MSSQL_PID: "Developer"

  # webapi serves as the name of the service
  webapi:
    # The image name which this service will be based upon.
    image: MyWebApiProject

    # This defines the ports which should be exposed by this service's container to the outer world
    ports:
      - "5000:80"
      - "5001:443"

    # Make this service part of a network called webnet
    networks: 
      - webnet
networks:
  webnet:

We gave the MSSQL service a name. In this case it is sql-server. This is what we will use in the WebApi connection string to connect to the containerized SQL server. So the connection string in the appsettings.json will look like this:

  "ConnectionStrings": {
    "Default": "Server=sql-server;Database=YourDatabaseName;User Id=sa;Password=YG4ZkkGrn7QSz5&"
  }

Also notice the user credentials. As per the MSSQL base image documentations, we will use the SA account to login to the server, and the environment variable we defined earlier in the docker compose file: SA_PASSWORD sets the password for that account which is what we are also using in the connection string.

Once you save the docker compose file and update the connection string in the WebApi configuration file, then re-build the WebApi image as shown in the previous post. This is so we have an up to date image of WebApi that uses the new connection string.

Next, we need to make the machine a manager of a docker swarm. To do this, in a command line, execute this command:

docker swarm init

This will create a docker swarm and make the current machine a manager for the swarm. In docker, a swarm is the cluster management feature which is required to deploy our multiple services. To read more about docker swarm, start here.

Once we have swarm created, we will then deploy our services. The command for this looks like:

docker stack deploy -c docker-compose.yml MyDockerApp

This might take a few minutes to finis if it is the first time running it as you probably don't have the MSSQL server image cached on your machine. But once done, you can run this command to see all your services created and running inside their containers:

docker stats

If you have SQL Server Management Studio on your machine, you can connect to the containerized sql server by using localhost,1434 as the host name and same credentials used in the connection string.

Go ahead and call your WebApi endpoints using localhost:5000 or localhost:5001.

Initializing a database

You might already be thinking that the SQL Server we created has no databases and that your WebApi won't work because the specific database you want to connect to does not yet exist.

Luckily, we can do something about that. We can have docker run a few commands for us as it is creating the MSSQL service. The commands we will write next will use 2 sql scripts to create the database and all of its tables.

How do we do that?

Let's start by creating an SQL file which will create the database:

CREATE DATABASE YourDatabaseNameHere;
go

This is a very simple script and will be the first to run against the SQL server.

Next, we will use Entity Framework Core to export the migrations as an SQL script. To do this, run the following command:

dotnet ef migrations script -o migrations.sql

Once that is done, create a Bash script file that will take care of executing those 2 scripts against the SQL server within the docker container:

#!/bin/bash
database="YourDatabaseNameHere"
username="sa"
password="YG4ZkkGrn7QSz5&"

# wait until the SQL server instance is ready
while ! /opt/mssql-tools/bin/sqlcmd -S 0.0.0.0 -U $username -P $password -d master -Q "select 'hello world'"
do
echo "Waiting for SQL server connection..."
sleep 1

done;

echo "SQL server ready!"

# this creates all databases specified in the databases.sql script
/opt/mssql-tools/bin/sqlcmd -S 0.0.0.0 -U $username -P $password -i databases.sql
# this runs the ef core migrations script
/opt/mssql-tools/bin/sqlcmd -S 0.0.0.0 -U $username -P $password -d $database -i migrations.sql

Place the .sh and the .sql files in a sub-directory called data. We will share this directory later with the sql server container.

Ok, now that we have everything ready, we need to get docker-compose to run the bash script for us. In the docker-compose.yml file, find the sql-server service definition and add the following to it:

volumes:
      - ./data:/init/scripts
    working_dir: /init/scripts
    command: sh -c 'chmod +x ./init-database.sh; ./init-database.sh & /opt/mssql/bin/sqlservr'

This is telling docker-compose to share the data folder with the container and place the content under this path: init/scripts. It then tells it to use the init/scripts as the working directory. The command part is overriding the default command defined by the base image. It is updating the file permissions for our bash script to allow execution, then it executes it and runs the sql server instance. While the sql server instance is being initialized and getting ready, the script would wait for it by attempting to login and query something and sleeping for 1 second.

Now, when we deploy the docker stack, we should have a database ready in the sql server instance.

You can deploy the stack again and then attempt to connect to the sql server using the SQL Server Management Studio. You should be able to see the database and the tables inside.

The Web Api should also be able to access that sql server instance.

Next steps

There is a lot more to docker than what I showed you in the last 2 parts and every project requires a different set of steps to containerize. You should familiarize yourself with the different things docker is capable of and read their documentations.

This ends this 2 part post series, but there will be more posts in the future with tips and gotcha's as I try more things with my own applications in docker.

Dotnet Core Webapi With Microsoft SQL Server in Docker - Part 1

Rafael — Sun, 25 Aug 2019 04:17:59 +0000

This blog post first appeared on my blog

I have been hearing a lot of people talking about docker and all of the benefits it introduces and they got me interested in the technology. However, I only recently managed to get some time to play around with Docker.

What I attempted to do was to take a DotNet Core WebApi project that I'm building and containerize it. The WebApi project was build initially with no intention of containerizing and it was surprising how easy it is to get it up and running in docker.

The project also uses a Microsoft SQL database and for me, this was a bit challenging to setup and get the WebApi to communicate with it properly.

In this post, I'll explain the steps I took to set it up with sample dockerfiles and docker compose configurations.

What do you need?

To get started, you'll need to install docker on you development machine. I'm using a Windows 10 Machine. You can read the install instructions here

Creating the DockerFile for the WebApi Project

The dockerfile will take care of setting up the image for the WebApi project. If you don't know what docker images are, I strongly suggest reading the Docker Getting Started section here.

In this dockerfile we will execute the same commands we do on our machine when building and publishing any dotnet core application via the dotnet cli.

Usually we do this (at the root of the project):


> dotnet restore
> dotnet publish -c Release -o out

then when we inspect the out directory in the project root, we will find the compiled assemblies of our project and we can then simply execute this command to run the application/webapi:


> dotnet MyProjectEntryPointAssembly.dll

But there are a few extra command we will execute in the dockerfile to setup the base image and the entrypoint for the resulting image.

First, let's look at my project's folder structure:

/root
- src
- webapi
- dockerfile
- docker-compose.yml

Let's look at my dockerfile and I'll explain what every command is doing:

FROM mcr.microsoft.com/dotnet/core/sdk as build
WORKDIR /app
COPY src/. .
RUN dotnet restore
RUN dotnet publish -c Debug -o out
FROM mcr.microsoft.com/dotnet/core/aspnet as runtime
WORKDIR /app
COPY --from=build /app/webapi/out ./
EXPOSE 80
CMD [ "dotnet", "webapi.dll" ]

Ok, let's start with the first line:

FROM mcr.microsoft.com/dotnet/core/sdk as build

This is telling docker that we are going to use the DotNet Core SDK image from microsoft as our base image. You can find the image here in Docker Hub.

The ... as build is an alias given to a build stage. This makes it easier when you want to reference a build stage from another one as it is more friendly to work with names. By default docker doesn't name build stages. You can still reference build stages that don't have an alias defined by using then integer number which starts from 0 for the first build stage.

The next step in the dockerfile is to set a working directory within the image we just pulled from docker hub. The working directory name can be anything and docker will create that directory if it doesn't already exist in the base image. I chose to name my working directory as App but feel free to use whatever name you like. To set a working directory, we execute this command:

WORKDIR /app

Next, we need to copy our project files from our local dev environemnt to the base image. You can think of the base image we just pulled as a Virtual Machine (they aren't actually VMs but this might help when working with them) that docker will initialize and run so it won't have access to our files. We copy the files to the base image by executing this comand:

COPY src/. .

This is copying everything inside the /src directory into the working directory we specified earlier.

Ok, so now that we have all the source code in the base image we're ready to build it. Since we are still inside the DotNet Core SDK image, the dotnet command is available in the shell, so we will tell docker to execute a couple of dotnet cli commands for us to build the project:

RUN dotnet restore
RUN dotnet publish -c Debug -o out

The RUN keyword here is telling docker to execute the following commands in the shell of the base image. So this will restore nuget packages and then performs the publish command and outputs the results to the out directory. Remember, we are still in the work directory we specified so everything is happening under /app, our code is in /app/webapi/. and the outout folder will be in /app/webapi/out/.

Awesome! now that we have successfully got our code to compile and publish (inside the base image) we will move to the next build stage.

The reason we have multiple stages in this dockerfile and the reason to use them (at least in this case) is to avoid creating big images.

The Microsoft DotNet Core SDK base image is about 400MB. While we can build the project and also run it inside this image, using ~400MB + ~75MB for my webapi project seems like a waste when we can instead use the Microsoft DotNet Core Runtime base image which is much smaller in size to run my webapi.

To start a new stage in the dockerfile we use the FROM statement. For this stage, we will pull the Microsoft DotNet Core Runtime image found here by executing this command:

FROM mcr.microsoft.com/dotnet/core/aspnet as runtime

we name this stage as runtime and we set its working directory also to /app:

WORKDIR /app

Next step we need to grab the build outout files from the previous stage and copy it over to the currect stage. We do this by executing this command:

COPY --from=build /app/webapi/out ./

The --from=[stage name] is specifying the source of the files to be from another image/stage. /app/webapi/out is the source of the files in that step and ./ is the destination (the root of the working directory in this step).

Once the copy is complete, and since this is a webapi project that will run on port 80, we need to tell docker that we port 80 should be expesed to incoming network traffic:

EXPOSE 80

Simple, way too simple!

The final command we run is literally running our webapi. To do this we execute:

CMD [ "dotnet", "webapi.dll" ]

Now that we have a complete dockerfile we can test it by using the docker cli to build the final image and running it.

To do that, open a terminal, and set the working directory to the root of you project (where the dockerfile is located) and run:

docker build --tag=MyWebApiProject .

the build command will execute the commands in the dockerfile we created. --tag=[name] gives the resulting image a name. the period after the command sets the build context to the current directory. docker will attempt to find the dockerfile in the current context and all the commands inside the dockerfile will run within that context. This is important for when we copy the files to the SDK image as we are using relative paths. All relative paths will rely on the build context we set.

If your build is successful, you will have a docker image stored on your machine.

Use docker image ls to list the images on your machine. You can see the image you created and it's size.

On my machine, the SDK image is 1.74GB but the webapi image is only 283MB because it is using the runtime image as per the multi-stage dockerfile above.

Now it is time to run the containerized application and this is as simple as:

docker run -p 5000:80 MyWebApiProject

docker run is the CLI command to run a containerized application's image.

With -p [OS port]:[Container port] I'm are binding a port on my operating system (in this case port 5000) to the port we exposed earlier in the dockerfile (port 80). This binding will allow us to use HTTP REST Clients such as Insomnia to send HTTP requests to the webapi application in the container.

That's it! This is how I took an existing and under development DotNet Core WebApi project and containerized it in docker.

In the next part, I'll explain how to setup a containerized MS-SQL server and create a docker swarm to have my WebApi project talking to the containerized MS-SQL server.

DEV Community: Rafael

Controlling My AC Unit With nanoFramework

Why

How

Communication

Tech stack

Sensing temperature & humidity

MCU

The Journey

Variable length signal? No way!

nanoFramework's relationship with ESP IDF

Updating the nanoFramework RMT library

Decoding the Panasonic AC Unit Remote Signal

On/Off, Mode, and Timer Function

Swing & Fan

Swing

Fan

Current Time

Timer

On/Off Timer

Temp

Profiles

LSB8 vs MSB vs LSB Tranformation

Clean and Valid Starting State

CRC Calculation

Automating my AC Unit

Github repos:

References & Literature

Uwp IoC Support

How does it work?

How to use it?

Setup

Dependency Injection in pages

Conclusion

Links

Create a Dotnet Framework and Node Build Environment in Docker

Story time

Why

What I'll cover

Building our custom docker image

Requirements

Next

Dotnet Core Webapi With Mssql in Docker Part 2

What do you need?

Containerizing Microsoft SQL Server

Initializing a database

How do we do that?

Next steps

Dotnet Core Webapi With Microsoft SQL Server in Docker - Part 1

What do you need?

Creating the DockerFile for the WebApi Project