DEV Community: klyse

What do you think of ASP.net core minimal apis?

klyse — Thu, 18 Aug 2022 16:04:43 +0000

I’ve been working with asp.net core since dotnet core 2.2. Wrote many different services with it.
Lately I’ve been trying out the “new” minimal apis and I’m very happy about them. What’s your opinion?

By the way:
There is a great video from Nick Chapsas about how to get started: https://m.youtube.com/watch?v=eRJFNGIsJEo

Dopper + direnv == "<3"

klyse — Wed, 17 Aug 2022 06:18:00 +0000

I'm using React to build cl3rk.io. The SPA needs some configuration like OAuth Client ID, Auth0 Domain, and backend URL. I try not to commit any configuration in the git repo so I have two options:

create a .env file on every machine I'm working on
outsource the configuration to a configuration provider

There are plenty of configuration providers out there like Azure App Configuration. A few months ago I found Doppler, which by definition is not a configuration manager BUT a secret store with some neat features. In this post, I'm showing you how I use doppler with direnv.

Requirements:

This will only work on Linux/Mac OS
Doppler shell
direnv

Doppler

Doppler has a generous free offering (5 users for free!🎉) - that's very important for me as I'm using doppler for cl3rk which is not yet making any money as well as personal projects.

Noticeable in the top left I'm in the cl3rk workspace. Each user can have multiple workspaces. I have a workspace for each product or prototype I'm working on.

Each Workspace has multiple projects. I use projects in two ways:

Make configuration available to other projects via secrets referencing. In the screenshot, the azure project is an example of this. It holds the credentials to talk to azure.
Per logical unit of the project: frontend and potentially backend / API

In each project, there are stages to represent different instances of the configuration for different deployments.

Each of the stages contains its own set of secrets and configurations.

There is much more to explore so head over to Doppler and try it out.

direnv

The link from the Doppler config and React is still missing and I'm using direnv to do that.

With direnv you can place an instruction file (.envrc) in each folder and as soon as the terminal enters the folder the .envrc is executed. This is helpful to set additional env variables or to add executables to the path.

example .envrc to read the .env file and adds its content to the environment variables:

dotenv
export Hi="dev.to"

Shell output when entering the directory / applying direnv allow .:

direnv: loading ~/projects/cl3rk/.envrc                                                        
direnv: export +DOPPLER_TOKEN +Hi
~/projects/cl3rk main* ❯

Two vars are being set DOPPLER_TOKEN comes from the .env file and Hi from the .envrc

Whenever we leave the folder all added variables are removed again:

direnv: unloading

Here is a real world example:

.
│   .envrc
│   .env                 # contains the doppler access token    
│   ...
└───IaC                  # contains all code for Infrastructure as Code
│   │   .envrc           # updated .envrc file to point to the correct Doppler project (infrastructure) and stage (dev by default)
│   │   main.tf
└───Web_Frontend
    │   .envrc           # updated .envrc file to point to the correct Doppler project (backend) and stage (dev by default
    │   package.js

./.envrc:

dotenv # only read the .env file

./.env:

DOPPLER_TOKEN="dp.pt.***"

IaC/.envrc:

source_up                               # also scan top-level folders for .envrc files and run them as well
export DOPPLER_PROJECT=infrastructure   # use the infrastructure project
export DOPPLER_CONFIG=dev               # and the dev stage
# download all the secrets to your shell and remove them
export $(doppler secrets download --no-file --format env-no-quotes)

Whenever you lose your notebook just regenerate your Doppler token and no secret is left behind on your pc. On top of that you can share the configuration easily with colleagues without copying the .env file. As soon as you enter one of the directories the .envrc code gets executed and Doppler injects secrets as environment variables which you can use in your projects. Pretty neat. What do you think?

The problem(s) with Azure Functions

klyse — Tue, 16 Aug 2022 19:43:00 +0000

I’m a curious person. Trying out new things makes me happy. I strongly believe in learning by doing. That’s the only way for me to make progress and learn new programming languages, technologies or frameworks. One might say I love to overengineer things ;). I've been working on cl3rk.io for a while and spend most of the time researching technologies and trying out new stuff. A few weeks ago I decided to take the project a bit more serious and am now working with warp speed on getting the first version out there.

Before that, I spend a ton of time testing Azure Functions and stumbled over a few ~~rocks~~ mountains 🗻.

I wrote already a couple of Azure Functions in the past to interact with Telegram bots for example and was always happy with them. So I immediately started my first PoC with C#. Here is why I ended up moving away from Azure Functions:

1. Authentication and Authorization

Functions can be triggered in multiple ways: HTTP, Queue, Db, Blob Storage Change and many more. More on this can be found here. This was one of the major reasons why I thought Azure Functions are great. I mostly needed the HTTP and Queue trigger. With HTTP functions comes also the requirement for Authentication and Authorization. I'm using Auth0 as an authentication provider. The implementation is usually straightforward. The frontend obtains an access token and the API validates the token and authenticates the request. OpenID connect is well documented and somewhat easy to use in asp.net core for example. Not with Azure Functions I googled for days, opened an issue and tried everything I could think of and came to the conclusion: Microsoft doesn’t provide you with proper SDKs to handle authentication adequately.

Expected:

I add the [Authorize] attribute on an Azure Function and configure the validation in the startup method for example. This is the well-known way to use OpenIDConnect in asp.net core apps. The user context is then injected via HttpRequest and accessible when the code needs it:

[Authorize]
[FunctionName("CreatePersonalTask")]
public static IActionResult Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post")]
    HttpRequest req,
    [CosmosDB(
        "personal_tasks",
        "personal_tasks",
        Connection = "CosmosDBConnection")]
    out PersonalTask personalTask,
    ILogger log)
{
    log.LogInformation("Create Personal Task");
    personalTask = new PersonalTask
    {
        Id = Guid.NewGuid().ToString(),
        Name = "yolo",
        UserId = // get userid from HttpRequest
    };

    return new OkObjectResult(personalTask.Id);
}

Reality:

You can authenticate on a service level (Azure takes care of OpenID connect and validates the access token before the function is executed) BUT this only works when the function is deployed. Testing locally becomes difficult as the local runtime does not come with this authentication layer.
The Azure Function docs do not show how to overcome this problem. I ended up creating a Github issue to understand if I'm the only guy with this problem or if it's just not well supported. Turns out the latter one is true😢
There are some libraries that workaround this problem BUT they are not well maintained and potentially use preview code that might never make it to the new Out Of Process runtime.
You can write the validation logic yourself and call custom code at the beginning of your function BUT who wants to maintain that?

[FunctionName("CreatePersonalTask")]
public static IActionResult Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post")]
    HttpRequest req,
    [CosmosDB(
        "personal_tasks",
        "personal_tasks",
        Connection = "CosmosDBConnection")]
    out PersonalTask personalTask,
    ILogger log)
{
    // validate the token in every function. This is messy if you have many functions

    log.LogInformation("Create Personal Task");
    personalTask = new PersonalTask
    {
        Id = Guid.NewGuid().ToString(),
        Name = "yolo",
        UserId = // get userid from HttpRequest
    };

    return new OkObjectResult(personalTask.Id);
}

Its a real bummer that authentication with JWT tokens does not work out of the box. This is a dealbreaker to me💁🏼‍♂️

2. CI/CD

I work as a DevOps guy so I strive to automate as much as I can to avoid errors and free up my time for development. The Github Action was working before I realized the authentication debacle. With the CD pipeline I quickly realized one flaw: whenever a new function version is deployed a zip file is uploaded to the mandatory Storage Account on Azure. Soon I had a few dozens of zip files in the storage account and had no overview of what's deployed and what is not deployed. That's messy if you need to debug or roll back a version.

This is not a major problem but worth keeping in mind when using dotnet and Azure Functions.

3. In-process or Isolated process

If all of that was not enough Azure Functions can be written in two flavours: in-Process or isolated process. Basically, this is how functions are hosted. More info on that can be found here. The in-process way has better binding support and a faster startup time. Isolated process hosting is newer and allows you to target a different dotnet version than the function runtime itself. It also has less dependency collisions as your app is separated from the function runtime. I decided to roll with in-process for my PoC because of the awesome binding support. I wanted to use dotnet 7 when released but I feared that might only work when using isolated process Azure Functions (Unless Microsoft offers the in-process runtime with dotnet 7 too). Unfortunately, some code is different between in-process and out-of-process so one may not easily switch between the two hosting options.

Conclusion:

Azure Functions are great if you have a small project that does not need authentication. Make sure to carefully decide between in-process and isolated process hosting. If you have a larger project you can check out Azure Container Apps as an alternative. That's what I'm using right now and I'm very impressed with it so far. I hope to share some details on how I'm doing that soon in another post.
All in all I can say that I learned a lot when working with Azure Functions and I'd love to use them in the future again. But only when I find a great use case without authentication ;)

cl3rk

cl3rk is a new, innovative and easy-to-use document management app. We all know document management is a pain. Have you tried finding the documents you need for tax declaration? I struggle with that every year. cl3rk's goal is to take this pain away from every user with AI. Keeping the documents secure cl3rk aims to provide reasonable reminders on chores like renewing car insurance each user has to stay on top of. This way nothing will get forgotten.

Let me know what you think of Azure Functions and cl3rk. If you want you can signup the email list (no spam - I promise) or reach out at klaus@cl3rk.io

Advent Of Code 2020

klyse — Wed, 18 Nov 2020 12:38:48 +0000

A couple of days ago the Advent of Code website switched from the 2019 edition to the 2020 edition. Finally 🎉

What is Advent of Code?

Every day from December 1st to December 25th a new coding challenge is released. Solve it ASAP to help the Elves save Christmas. Here is the challenge from 1st December 2019 to give you a better idea about the challenges. Usually they get more difficult with each day. What I like about AoC is that the challenges only rarely rely on math IMHO. Instead you are required to demonstrate an open mindset, debugging skills, some logic and a lot of googling to solve the problem.

Why should I join?

Advent of code is a great way to challenge your collogues and improve your coding skills.

What programming language should I use?

That's completely up to you. I personally use C#. Here is my GitHub repo of last years event:

klyse / AdventOfCode2019

Repository for advent of code

Advent Of Code 2019

This is my repository containing the alogithms for AoC 2019.

I use unit tests to debug the solution. I was able to obtain a total of 30 stars :)

Authors

Klyse: @Klyse

View on GitHub

Do I need to solve each days challenge?

Nope you don't. You solve as many challenges as you can / want. Be warned, some of them are really difficult! Don't give up if you find no solution for one day, the next day might be completely different and more accessible for you.

Private Leaderboard

Don't forget to create your private leaderboard. This way you can challenge your friends and compare the coding results.

There is only one last thing to say:
Code humans, Code!

🎄🎄🎄🎄🎄🎄🎄🎄🎄🎄

Where would you use the new C# 9 Record Type

klyse — Wed, 18 Nov 2020 07:09:54 +0000

I found a great post by wrijugh.

Wriju's BlogFollow

Cloud Architect and a Software Developer

Wriju explains the new C# Record Type.

C# 9.0 - Record Type

Wriju's Blog ・ Nov 13 '20

#csharp9 #net5 #programming

I am excited about this feature, but I'm not 100% sure when I should use it and when I should stick to a class. Mads Torgersen provided some insights there in his presentation on C# 9 during the .net conference but I'm actually more interested in the practical side.

How do you plan on using this new feature? Do you think there are issues one has to think about when one uses the new Record Type? Do you think Record Types should be used for deserialization of a json string? Are you going to migrate from classes to Record Types?

Let me know in the comments :)!

My G815 Keyboard is on Steroids (Shortcut helper)

klyse — Sat, 29 Aug 2020 17:11:48 +0000

Logitech G815 Shortcut Helper

I wrote a small shortcut helper in GoLang that helps me to remember the most useful shortcuts for IntelliJ based products tested with Rider and GoLand and a QWERTZ keyboard.

Intro

I recently bought a Logitech G815 which turned out to be a great decision for multiple reasons. The obvious one being that it's a great keyboard ✨. The other one is that I found out Logitech provides an SDK for the keyboard => 👌.

RGB Keys

The keyboard supports RGB background lighting for each key. There are plenty of integrated functions such as a color wave which looks great🌟. The keyboard is made for gaming purposes and yes it does integrate with a lot of games but I didn't try any myself🙈.

SDK

The Logitech SDK can be found on this website: https://www.logitechg.com/en-us/innovation/developer-lab.html.

I was particularly interested in the LED Illumination SDK.

The SKD is written in C++ and provides the .lib, .h, .dll, and some examples and is windows only.

Integration with GO

I'm currently learning GoLang and I wanted to program this in go and not in C++ or C#.

Calling native C++ functions from GO

I found a great way to call native C++ functions from Go using the sys module.

Original repo: https://github.com/golang/go/wiki/WindowsDLLs

type LogiKeyboard struct {
    dll                             *windows.LazyDLL
    ledInit                         *windows.LazyProc
    // ...
}

func Create() *LogiKeyboard {
    p := new(LogiKeyboard)

    // initialization of the dll
    p.dll = windows.NewLazyDLL("LogitechLedEnginesWrapper.dll")

    // assign the function
    p.ledInit = p.dll.NewProc("LogiLedInit")
    //...
    return p
}

// wrapper for the init function
func (v LogiKeyboard) Init() {
    ret, _, _ := v.ledInit.Call()

    if ret != 1 {
        log.Fatal("Already initialized")
    }
}

Additionally, I had to translate the .h file in go syntax. This helps to have all the constants (like defines) in one place and reduce code duplication. Here is the file.

Integration with Windows

I needed a slim and easy way to catch Windows keystrokes. Turns out there is a great module for that as well: github.com/moutend/go-hook.

main.go:

if err := run(shortcuts); err != nil {
    log.Fatal(err)
}

func run(shortcuts []Shortcuts.Shortcut) error {
    // Buffer size is depends on your need. The 100 is placeholder value.
    keyboardChan := make(chan types.KeyboardEvent, 100)
    if err := keyboard.Install(nil, keyboardChan); err != nil {
        return err
    }

    signalChan := make(chan os.Signal, 1)
    signal.Notify(signalChan, os.Interrupt)
    for {
        select {
        case <-time.After(5 * time.Minute):
            fmt.Println("Received timeout signal")
            return nil
        case <-signalChan:
            fmt.Println("Received shutdown signal")
            return nil
        case k := <-keyboardChan:
            // check if shortcut modifiers are pressed and set RGB color
        }
        continue
    }
}

Those few lines of code are everything you need to create a keylogger in go. Isn't that crazy?

Shortcuts

Shortcuts can be modified in main.go:

var shortcuts = []Shortcuts.Shortcut{
    // if the left shift key is presed:
    // F6 will start blinking red
    // F9 will change the color to blue
    Shortcuts.CreateWithKey([]types.VKCode{types.VK_LSHIFT}, []Shortcuts.ShortcutKey{
        Shortcuts.CreateKeyColorEffect(LogiKeyboardTypes.F6, 100, 0, 0, Shortcuts.Blinking),
        Shortcuts.CreateKeyColor(LogiKeyboardTypes.F9, 0, 0, 100),
    }),
    //...
    }

Integration with Logitech GHub

The keyboard has five G keys and each key can be assigned with a different task. One task is to launch an application => great :). The downside of this function is that an application can be started but not stopped. I wanted to start and stop the shortcut helper with the same key just like a toggle.

As always there is a workaround in PowerShell😂:

Set-Location -Path C:\Projects\LogitechKeyboardLED

$ProcessActive = (Get-Process logitechKeyboardLed -ErrorAction SilentlyContinue).Id
if($null -eq $ProcessActive)
{
    Start-Process ./logitechKeyboardLed.exe -NoNewWindow
}
else
{
    Stop-Process $ProcessActive
}

Configure the G key using the Launch Application option:

Shortcuthelper in action

Github Repo

klyse / LogitechKeyboardLED

Logitech G815 / G915 LED control written in GO

Logitech G815 ⌨️ Rider and GoLand Shortcut helper

This is a little shortcut helper written in GO 👨‍💻. The software can be started using one of the G keys. Once started the process profiles every keystroke made on the keyboard. If a certain combination ex. SHIFT+CTRL is pressed the most important shortcuts for Rider / GoLang are visualized on the keyboard using the RGB key background lightning 💡.

Here's a quick example (sorry for the bad quality, the keys are shining bright and clear in reality🙈):

Here's another one:

How to install 💻

Clone the repo
Build the source code go build -o LogitechKeyboardLED.exe ./ (win only)
Navigate to the GHub
Add a new launch command with the following params:

The script checks the state of the process and starts it if the process is stopped, and stops it if the process is running. This way…

View on GitHub

Thanks for reading :)

The miracle tools that help me to develop software

klyse — Thu, 16 Jul 2020 16:22:09 +0000

I love to solve problems by writing some code and in my opinion, great software starts with the right tools. So here is a list of tools I use professionally to write (hopefully 🙀) great software:

Some might be pretty obvious ;)

Git https://git-scm.com

version control

GitHub https://github.com

CI/CD
Issues
great octocat images:

AzureDevOps https://azure.microsoft.com/en-us/services/devops

CI/CD
Boards https://azure.microsoft.com/en-us/services/devops/boards

JetBrains Suite https://www.jetbrains.com/products.html

Toolbox

update and install JetBrains products
launch projects from the "Projects" tab 🌟

Rider

for C# development
(really, give it a try. It's Visual Studio on steroids)

PHP Storm

you guessed it, its made for PHP

GoLand

for go (what a surprise)

DataGrip

for SQL connections to various databases (MySQL, MongoDB, MsSql, MariaDB, MySql)
learn one tool => rule them all

Visual Studio https://visualstudio.microsoft.com

C++ development (I have some issues with CLion here)

GitKraken Git GUI https://www.gitkraken.com/git-client

It's great UI and easy structure help me get my head around git pretty well :D

Postman https://www.postman.com

REST requests
documentation
- by importing the swagger OpenAPI Specification 3.x <= this is love 💗

NOSQLBooster for MongoDB https://nosqlbooster.com

MongoDB Compass https://www.mongodb.com/products/compass

I use multiple tools to access MongoDB because I haven't (yet) found the tool with all features I want :D

Windows Terminal https://www.microsoft.com/en-us/p/windows-terminal/9n0dx20hk701

restored my love for Windows
PowerShell 7 is also very nice! https://github.com/PowerShell/PowerShell

PowerToys https://github.com/microsoft/PowerToys

PowerToys Run `ALT` + `SPACE` opens a run window https://github.com/microsoft/PowerToys/wiki/PowerToys-Run-Overview

FancyZones

to arrange windows on my 21:9 Philips Monitor (yep, the wide screens are worth the price!) https://github.com/microsoft/PowerToys/wiki/FancyZones-Overview

LockHunter https://lockhunter.com

A small tool that shows me what process locks a file if I want to delete it.

Docker https://www.docker.com/

With WSL 2 enabled => Developer Love https://docs.microsoft.com/en-us/windows/wsl/wsl2-index

VS Code https://code.visualstudio.com

Markdown editing, file editing

nomacs | Image Lounge https://nomacs.org

Great image viewer, definitely better than the windows default image viewer.

SpeedCrunch https://speedcrunch.org/

a great alternative for the windows calculator

Winmerge https://winmerge.org

compare different files (old, but gold)

Grammarly https://app.grammarly.com/

because my grammar sucks

Dev.To

Because I ❤️ the community 🧍🧍‍♀️

And now it's your turn. What tools enable you to write great software?
What tools do you particularly like and how do you use them?
Let me know in the comments!

How to handle BigData?

klyse — Thu, 29 Aug 2019 19:33:27 +0000

Abstract

I'm currently working on a new (exciting 🤩) project. The project contains data storage💾 and analyzation. One of the most important characteristics of this BigData project is data integrity.
Here are the hard facts:

I'm using MongoDB as data storage
asp.net core as web service framework
.net core to analyze/collect the data

There are a few so-called collectors that harvest information from different sources. Those collectors are written in c# and send data to the so-called Raw.Api🌐. The Raw.Api handles the information and takes care to make inserts into the MongoDb collections. Since data integrity is crucial to this project I devided the operations in three steps:

Transaction scopes

The first transaction scope is: every collector must never send incomplete information. As soon as the data is transferred to the API all information must be written all at once into the database collections. Partial inserts are never allowed.
The second transaction scope is: as soon as the data is dumped into a temporary collection a background worker starts and writes information to the final collection(s). This must again happen in a DB transaction. Partial updates are not allowed. MongoDB 4.2 supports ACID across multiple collections as far as I know.
The third transaction scope is: the analyzer must read all data at once and work on the information. A partial refresh is not allowed because the data might have been updated in the meantime and this would lead to inconsistent information.

Handle the data⚙️

There are multiple collectors and every collector sends information every now and then. For example every 5 minutes. One transaction of data consists of approximately 40mb. 40mb x 12 transactions per hour * 24 hours * 365 = ~4tb. That's too much data📚. Therefore every dataset has a specific Data Save Interval for example daily, hourly, etc.
Every new request replaces all the data from the previous request as long as the request is within the same Data Save Interval.
Every document is stored with a "Data Scope" and a collection date time stamp. A "Data Scope" is, for example, a set of data that can be uniquely identified. This could be a domain (www.google.com). The first step of writing information from the temporary collection to the history collection is to delete every document in the same "Data Scope" and in the same "Data Save Interval". This guarantees that if less information was transmitted by the collector (because someone deleted data which is an allowed scenario) the "old" information is deleted. Then every document is inserted into the History collection. The same happens for a Current collection to make sure that access to the current information is faster.

Analysis

The analysis happens on a base of rules and I'm not yet sure how to do that yet. Is there a great tool or programming language to archive that?

Any ideas or suggestions for this project? I'm happy to discuss 😜