DEV Community: ik_5

10 Things I Learned from Vibe Coding (or: How I Let an AI Agent Write My Audio Library)

ik_5 — Fri, 16 Jan 2026 09:40:24 +0000

10 Things I Learned from Vibe Coding

TL;DR — The finished result is here: https://github.com/ik5/audpbx.
I hope it helps someone else too.

I recently finished a small side project and decided to handle one required feature differently than usual.

The task was simple on the surface: accept an uploaded audio file in almost any format, then resample and convert it to PCM 16-bit signed, mono, 8 kHz WAV.

In the telecom world this ultra-low-spec format is still the standard you must support — most class-4 and class-5 PBXs expect (or silently convert everything to) exactly G.711-compatible PCM: 8000 Hz, mono, 16-bit linear. Anything else usually gets transcoded anyway, or the call just fails.

Normally I would just shell out to ffmpeg and be done with it. This time I wanted pure native Go code — no external binaries.

A few years ago I wrote a tiny Go package that took raw PCM samples and wrapped them into a .wav file. That was it — no decoding, no resampling, no format conversion. When I tried to extend the idea to real resampling and multi-format input I quickly ran into trouble.

After a week of one-hour evening sessions the code was still:

crashing on several inputs (bad buffer logic)
producing noticeably worse audio quality than ffmpeg or Audacity (naive linear resampling)
supporting only PCM .wav files (no RIFF metadata handling, no other containers)
taking forever to make progress

At that point I decided to see whether free non-agent AIs could help. I tried:

🟡 Claude
🔵 Grok
🟢 GitHub Copilot
🟣 ChatGPT
🟠 Gemini

Copilot gave by far the most useful starting point. It produced code that could:

read and write RIFF WAVE files
decode Ogg Vorbis
decode MP3

…all attempting to share roughly the same interface.

Still — lots of crashes, memory corruption, infinite loops, bad buffer management, and the interface was mostly ignored in practice.

I cleaned it up by hand until I reached this early commit — still very fragile, but at least the abstractions were starting to hold.

On January 9th I switched to Claude with agent/computer-use capabilities — and that made a huge difference. The rest of this post is what I learned during that second, agent-driven phase.

What I learned

1. Be clear

Modern AIs — even agents — almost never tell you “this design is wrong” or suggest a completely different approach unless you explicitly ask them to critique it.

If you don’t already have a mental model of what you want (boundaries, failure modes, safety, layering), the AI will happily generate code that compiles but misses the point.

Solution: start with tiny, very concrete tasks.
Example: instead of “fix this crash”, I started asking “look at this backtrace — why do you think it crashed, and what would you change to prevent it?”.

Very often the first answer was wrong (e.g. “the buffer isn’t initialized” when the real issue was missing bounds checks). I had to stop the agent many times because it was heading in a useless direction.

2. Always check what was offered

Claude frequently wrote outdated or non-idiomatic Go:

interface{} instead of any
old-style for i := 0; i < n; i++ instead of for i := range n
Python-like patterns shoehorned into Go

I gradually pushed back toward better patterns:

package-level var ErrXXX = errors.New("…")
avoiding fmt.Errorf when a static error value carries enough meaning (still work in progress)

Good error handling in Go makes flow readability much easier later. Not every io.EOF is a failure — sometimes it’s the expected way to exit a loop.

3. Test the implemented code

The agent generated plenty of unit tests — which is nice.
But passing unit tests only proves the cases you thought to check. It says almost nothing about real-world files or unintended usage.

That’s why I created the examples/resampler program early on.
It acts as an integration test: feed it dozens of different audio formats → run them through the whole package → produce PCM 16-bit mono 8 kHz WAV → compare with ffmpeg/Audacity output when in doubt.

4. Don’t ask the AI to do things you already understand

AI agents will cheerfully agree with you after you point out an obvious mistake (“You’re right, I should have…”).
If you already know the safer/clearer/faster way — just write it.

I fixed >90% of the memory leaks, slice shadowing bugs, uninitialized values, and off-by-one errors myself — usually faster than waiting for another iteration.
Only a few times was I actually wrong, and even then the agent’s version was usually over-complicated.

5. Write code as if a human will have to maintain it later

Avoid “write-only” code — even when the first draft comes from an AI.
KISS still applies. Clear names > clever tricks.
Future readers (including yourself in six months) will thank you.

6. Demand proper unit tests and benchmarks — with modern Go idioms

Early on the agent wrote:

benchmarks named TestSomething instead of BenchmarkSomething
unit tests without the Test prefix
classic for i := 0; i < b.N; i++ instead of for b.Loop() { … } (Go 1.24+)
no t.Parallel() even when it made sense

I had to keep correcting it until the generated test code followed current Go conventions.

Later I started asking for tests on one specific function/file at a time with specific conventions — much better results.

7. Documentation inside the code

Don’t waste comments on the obvious.
Do explain non-obvious decisions and magic numbers.

Example: why 32768.0?
If you know 16-bit signed integer range it’s obvious — but most readers won’t instantly see it. A one-line comment saves confusion later.

8. Write examples

My original goal was always to expose a very simple high-level API — ideally one or two friendly functions that “just work” for the most common case (convert anything → 8 kHz mono PCM16 WAV).

At the same time I wanted clean low-level building blocks underneath so the package could be reusable in other contexts.

When I asked the agent to write a collection of usage examples — not just the happy path, but showing different ways to combine the low-level pieces — I was surprised.

The low-level API turned out to be much more powerful than I had realized.
There were capabilities and combinations I had never thought about and never intended to build — yet they emerged naturally from the design and suddenly looked very useful.

Seeing those examples helped me understand (and appreciate) my own library much better.

9. Good documentation, bad documentation, and outright lies

My native language isn’t English, and writing clear technical docs doesn’t come naturally to me.

The agent produced documentation that was:

grammatically correct
properly structured
better spelled and phrased than I would usually manage

I also learned Go doc tricks I hadn’t known before — like using heading markers to create sections in godoc.

10. “Code Monkey”

The classic definition still fits best:

A pejorative term for programmers employed to write simple or repetitive code.

That’s basically what a strong coding agent is — an extremely fast, very knowledgeable code monkey.

It can produce huge amounts of code quickly.
It cannot take architectural responsibility.
It won’t warn you about future maintenance pain unless you force the topic.
If you don’t steer tightly, you get working-but-awful code.

Use it like a very junior-but-fast pair programmer: you supply the thinking and the taste, it supplies the typing.

And just for fun — here's the song that gave the section its name:

(Code Monkey — the unofficial anthem of every developer who’s ever pair-programmed with an AI 😄)

Happy coding — and don’t trust the robot too much. 😉

Have you had similar (good or terrible) experiences vibe-coding with AI agents?
Drop a comment — I’d love to hear your war stories!

The Perils of Over-Engineering in Technology

ik_5 — Sun, 20 Jul 2025 05:54:20 +0000

In today’s world, technology often gravitates toward unnecessary complexity, with bloated software and intricate systems like AI dominating development. I believe we should prioritize simpler methods that empower humans, avoiding overcomplication that can lead to inefficiency, loss of control, and even economic disruption.

Most of us view technology as computers or electronics, but even a chair, glasses, and the stairs we climb are technologies that aim to address human needs.

Several years ago, during a hiking trip, I had a fascinating conversation with an engineer about the fax (facsimile). While fax machines are often mocked, their underlying technology offers a powerful lesson in designing simple, human-centered tools.

The Fax Machine: A Model of Simplicity

A Brief History

In 1846, Alexander Bain invented a chemical device that reproduced graphical shapes via the telegraph, laying the groundwork for fax technology. By 1865, after improvements by other inventors, the first commercial telefax service connected Paris and Lyon. Nearly a century later, in 1964, Xerox introduced the modern fax machine—a device many of us love to hate, especially if you’re old enough to recall its quirks.

Why the Fax Machine Matters

The fax machine, in both analog and digital forms, embodies simplicity. It scans a page, attaches metadata, and transmits it using basic signals like radio waves or sound to produce an identical copy elsewhere. This process doesn’t require complex systems like AI, relying instead on straightforward electrical wiring or software.

Digital fax machines use a Digital Signal Processor (DSP), a microprocessor that converts analog signals to digital and vice versa. Using simple copper wires and telephony filters, it translates human-audible sounds into metadata and shapes. Software-based solutions, like SpanDSP and protocols like T.38 for Fax over IP (FoIP), further streamline this process.

Consider this: with basic electrical or audio signals, we can achieve reliable results without overthinking the design. Once fax machines became commercially available, anyone who could use a printer or dial a phone could operate one—provided the machines and supplies were affordable—hence it’s an intuitive system.

The Consequences of Overcomplication

I’m old enough to remember floppy disks, which stored multiple applications in minimal space. These programs used little RAM yet delivered impressive functionality. Even complex software managed with multiple disks, swapping data as needed, without demanding excessive resources other than storage space for its resources.

These resource-efficient designs highlight the drawbacks of today’s complex systems, which carry serious consequences.

Modern technology often abandons simplicity, with serious consequences. For example, containers like Docker or Podman require complex network stacks, registries, proxies, gateways, and often privilege escalation. A simple .tar.gz file with statically compiled binaries or a .rpm/.deb package with clear dependencies could suffice. Ecosystems like Kubernetes amplify this complexity, increasing development time, maintenance costs, and error risks, while distancing users from intuitive control. Developers often need to manage multiple database or server instances or build custom dependency instructions. This can slow execution or create security risks, such as running containers as ‘root’ or ‘Administrator’ when a ‘nobody’ user with Linux capabilities would have sufficed.

Similarly, AI is often treated as the ultimate tool, overshadowing simpler alternatives. Its complexity—while powerful—shouldn’t dominate every solution. The “all-or-nothing” mindset, where AI is seen as the only path forward, risks sidelining human labor entirely. This could have dire economic consequences: if AI automates most work, people may lose access to jobs and income, potentially rendering currency obsolete as economic power shifts to machines. Instead of complementing human efforts, AI becomes a replacement, eroding the human-centered purpose of technology.

A Call for Simpler, Human-Centered Design

Overcomplication doesn’t just make technology harder to use; it risks inefficiency, user alienation, and economic instability. Just because we have abundant computing resources doesn’t mean we should bloat software or rely solely on AI. Technology should empower humans, not overwhelm them or render their contributions obsolete.

The fax machine reminds us that simplicity can be powerful. By prioritizing straightforward, human-centered design, we can create tools that enhance our lives, preserve human agency, and maintain economic stability. Let’s think simpler, ensuring technology serves people first.

What I've Learned About My Editing Skills

ik_5 — Tue, 10 Dec 2024 17:19:32 +0000

I experimented with an impressive text editor called Helix.

I believe the work on this editor is fantastic, and I love its feature set.

However, I have one “small” issue with it — it operates very differently from (neo)vim key bindings.
I tried to reconfigure it to be as close to vim as possible, but I couldn't achieve 100% vim-like key bindings.

If you think I'm criticizing the editor, think again. It's awesome, and you should give it a try (at least for a month).

This post is about what I learned about my skills while trying the editor.

Some History

In the year 2000, I started using Unix systems (AIX, Solaris, and FreeBSD) as a user for specific services, and I liked them much more compared to Windows. By the end of the year, someone offered me to try something new called Linux.

He gave me a CD named Mandrake Linux, and when I booted my machine, I saw a nice GUI installation screen.

Back then, I was a Windows-based software developer (Delphi, VB(A), and some basic yucky C++), so it was very different for me.

A friend showed me a cool editor named vim and some tricks he did with text on it.

In 2003, Microsoft decided to obsolete a library that a startup I worked for used, and I decided to move from the MS world to Linux.
I think it was the best decision of my life.

I wanted to understand how to do the cool stuff I saw with vim, so I wrote a guide on how to do some basic work in VIM (in my native language) to help me learn vim.

The VIM Language

One big misunderstanding about vim for non-vim users is the lack of randomness when using keys.

You have a count of something, an action to do, and a movement of some kind.

What does it mean?

3yw means to copy 3 boundaries of words (including spaces) forward.
3ye means to copy 3 boundaries of words (not including the last space) forward.
3yb means to copy 3 boundaries of words (including the first space) backward.
3dw means to delete 3 boundaries of words (including spaces) forward.
3cw means to change 3 boundaries of words (including spaces) forward.

Can you see a pattern?
That's the vim language, but it's more than that. You can do it on paragraphs, parameters, and more.

What does “more” mean? You have something called “text objects” that you can define, for example, “class,” "method," etc.

Skills, Damn Skills, and Muscle Memory

One of the issues I have with vim (I actually use neovim for several years now) is that I do not remember most of the language.

Even while writing this blog post, I needed to test what I've written regarding the vim language because I do not remember it by heart.

So, how do I use vim? Muscle memory.

I just do it, and it works for me for some strange reason.

And that's my issue with the Helix editor — it uses similar but different key bindings for doing the same things.
While I can find a way to re-learn it, I still need vim when I'm inside servers (for example).

But I can't figure out how to have two types of muscle memory based on the application I'm using. It's the same for me when I'm writing software.
I'm polyglot and use around 3 programming languages at the moment, and in each language, I sometimes write the wrong syntax at the start.
Also, it happens to me when I'm using different human languages that are close to each other (e.g., Spanish, Portuguese, and Italian) while my native language is not Latin-based.

But why the vim way and not something word processing-like (nano, pico, emacs, VSc, etc.)?

First, I live in the terminal. I love it, and for most stuff, it is enough for me, so Zed or VSc are not that.
Second, I find myself much more productive while using the vim language over word processing shortcuts.

So now I'm in a deadlock situation where an amazing text editor is unusable for me simply because it offers a means that makes me work well but with a different language.

Where Do I Go From Here?

Maybe I'll try Evil Helix: "A soft fork of Helix which introduces Vim keybindings and more."

But I think I wil keep myself at the moment in (neo)vim, until my skills becomes better.

You (probably) do not understand UDP

ik_5 — Sat, 03 Aug 2024 11:18:51 +0000

On my day to day work, I find that few types of IT equipment cannot properly support anything but TCP transport layers, even when their specification explains that it does.

I also read rants such as this regarding implementing something in UDP where TCP would have been better.

So this post is about giving a small idea of what is UDP, and I hope it will shed more light on what is missing in the understanding of the transport protocol.

TLDR

You and I (probably) do not (fully) understand UDP.

TCP and UDP have different usages and should be compared based on the end application needs rather than "you do not have a 3-way handshake so it sucks" like rants.

Most of the time, TCP does not require you to do anything regarding transport instructions besides the port number regarding L7 (there are very few exceptions to this rule that this post never touches and blatantly ignores).

UDP is a lightweight transport protocol that forces handling needed transport logic at the application layer (L7), while TCP does most of it on its own almost auto-magically (when you just handle L7) for you.
This post tries to provide examples of such usage.

Network story

Let's talk about networking (almost) without talking about the transport layer.

There is a need for some sort of way to deliver data (e.g., RJ45-based Ethernet cable).

Then there are some physical hardware (e.g., Wifi, Ethernet, token-ring [not really], Bluetooth, Cellular, modem, FXO/FXS modem, etc.
A.K.A. "Physical layer" or Layer #1.

Then there is some sort of firmware and device driver at the software level that can read stuff on operating systems such as Windows/Linux/MacOS/... and provides a low-level API for the OS to do stuff that makes it unified for user-space stuff.

From Layer 2 up to and including Layer 7, everything is software-based.

Each layer is encapsulated backward than what the numerical layer should be, so application level (L7) data is encapsulated first, then L6..L5 it goes down to Layer 2, and then hardware layer to pass data around in a Big-Endian way.

The order of each "capsule" exists, so the first thing that gets accessed will get the data it knows how to parse, leaving everything else to the next level in the chain to handle until it's our program that handles an application layer.

Let us network a bit

To figure out what the size of encapsulation is, there is a measuring unit named MTU (Max Transmission Unit)—the biggest payload size that can send/receive (on normal networking, the value can be only up to 1500 bytes) with the entire encapsulation of everything.

If our payload is bigger than the MTU size, there is a requirement to fragment the payload into smaller chunks and send each of them on their own.

The part that might do that is either our software or transport layer depends first on our application and only then on the transport layer (when supported by it).

The application can decide to perform fragmentation of the payload without considering the transport layer. HTTP has a nice usage for L7 fragmentation using HTTP range requests, for example.

What did you say?

Let's take a hardware tool named "hub".
That tool is an array of RJ45 Ethernet cards that are only separated by actual position, but all data is available at the same time by all used cards.

On that hub, let's connect a single point (that is, a machine = computer).

Let's ask using the "arp" protocol, who knows some sort of hardware address, to return its IP address.
While doing that, let's do some tcpdumping magic and sniff our requests.

We will find out very soon that now and then we cannot see our request after it exists from our machine.

Even if we add another machine and sniff the traffic using the new machine, the result will be the same.

Sometimes we are losing packets!
Why? Gremlins love bits on the net, that's why!

So some transport layers also handle the loss of data.

But if there are a lot of requests (or heavy traffic, or slower processing, etc..), sometimes the order of the packets does not go by the order the L7 wanted them to go, so packet #10 arrives before packet #4, for example.

So some transport layers also handle the assembling of the packets based on the order number inside that layer.

And by talking about "some", it is usually about TCP (in our case).

TCP (and) UDP

TCP

TCP is a complex transport protocol that does so many things for you (almost in an automagically manner), such as:

Open a communication path to the endpoint, preparing it to accept your data (3-way handshake) and also alert that the entire payload was sent.
Understand for you how to fragment a payload and number it.
Understand the timeout and missing fragment, and send the missing fragment again (retransmission).
Handle congestion of the network by using a sending window handler.
Can send "ping" like to keep alive the path and endpoint of the connection.

Sounds amazing, but it comes with a few costs, such as:

Heavy traffic (e.g., 3-way handshake, retransmission of fragments, and keep-alive).
If you have network issues, you'll lose your connection to the endpoint when timers are timed out.
Big header to include many features = smaller payload (MTU).
The speed of data is much slower than UDP (for example) due to so many things that are going on that need to be verified.
Your TCP stack implementation handles for you the congestion, and if it supports using the wrong algorithm for your needs, you need mostly to suck it up.
When a reply is too slow for your timers (L7/L4), it will re-transmit the same fragment many times even though the endpoint did receive it but was slow to respond.

UDP

UDP is a simple lightweight (compared to TCP) transport protocol.
It does not assume anything besides Port (mostly) and lets your application do all that it needs as business logic.

Does it sound bad compared to TCP?
Well, no, because it exists for different uses.

Examples

VPN

Let's say I create a site-to-site VPN—that is, a VPN as infrastructure—that provides new network legs rather than two machines connected.

If an application creates a VPN using TCP transport, every packet will take a long time to work.
It never knows when something will be sent, and the traffic will create congestion in the network.

So to mitigate some of the issues, things are more relaxed, thanks to UDP^*.

The connection can be once every 10 hours, but in theory, there is a connection that will transport that content and is "alive" and kicking.

^{*There are some TCP-based VPNs, but they are not suited for site-to-site, to my knowledge.}

ssh

Normal ssh is TCP-based.
But there is also a different implementation named mosh.

What happens if your network connection is unstable?
Well, the connection will be lost at some point, no matter how stable the protocol is trying to be (even with keep-alive), the end server will disconnect you when the client is up again.

For the above reason mosh is using UDP.
Even when your connection was lost for 10 hours (don't you love my magic numbers in this post?), you can still connect and continue from the point it ended.

Streaming

When using communication protocols (it does not have to be telephony), there is a Real Time Protocol (RTP) that sends chunks of media (audio and/or video) for this example.

That media is broken out into "frames", and there is a buffer (jitter buffer) that collects X amount of media chunks and sends it to be processed by a device (such as audio devices).

Such media is part of a codec (encoder-decoder) that stores the data in a specific format instead of its raw form.
Such codec defines (or provides a range of) several frames that need to be collected to be proper media.
Some codecs also contain a variant of frames that change based on the bandwidth and/or performance (such as OPUS).

When there are fewer frames of a defined amount of frames available at the buffer level, audio (up to a point) sounds more like a metallic sound (the issue named jittering), and video will not be updated.
Also, each implementation needs to define a sweet spot between waiting for data and releasing the current buffer, and that also can create the jittering.
Older frames that arrive late are thrown away if the buffer was already released (or the re-ordering of data will cause a big delay).

But there is no need to collect older frames that arrived late because it misses the point of streaming; there is a need to continue with the newer frames, ignoring older ones.

But how can a program know how much was arrived in total and how much was lost if there isn't any L4 implementation?!

There is another "nice" protocol named RTCP (RTP Control Protocol)
that sums up every X amount of time how much data (frames) were sent, and now there are also statistics and the ability to change variants of codecs or even negotiate new ones based on that information.

Such decisions usually use table scores such as MOS to help understand packet loss of media and its effect over time.

Conclusion?!

Well, to explain UDP, I'll require several boring blog posts (even more than this one), so I decided to explain the way networking works and provide examples of using UDP and why TCP might not be suited for the task.

I hope that by doing so, I was able to explain a bit more why there is a need for UDP and why it is so different from TCP.
It's not about "what L4 protocol is better", but "what L4 protocol suits my needs", and what is required to be made for that.

About me

I'm writing telecommunication software in my day-to-day work.

My day-to-day work forces me to interact with many networking protocols, and many OSI network layers, including three transport layers:

TCP.
UDP.
SCTP.

My work also forces me to work with so many IT-related stuff, such as:

To name a few...

Not a malware

ik_5 — Wed, 07 Apr 2021 14:01:16 +0000

For the past year or so I wrote several Golang based applications for the Windows environment.

The pain of writing software for Windows in non Microsoft tooling is a whole set of posts that I can create, but one thing is worst.

Many applications that are written using Golang are considered to be a Virus, or a maleware of some kind simply because they are written in Go.

When you use tools such as Virus Total, upload a Go based application that you wrote and see how many results of Anti Viruses think that you are trying to create a malware of some kind.

The malware I wrote is just an HTTP server with support of TLS certificate and JWT, registry access for some information, Windows service(ing) to run as one, loading dynamically a .dll file that I wrote (using c++) and few additional COM binding for my witchcraft mix.

That simple mixture was tagged as ransomware just because I had support for x509 inside my code and working as non GUI service.

An "expert" of the organization that the server was suppose to use it told me that I have "hidden code" because of the traces on x509 PKI implementation inside the .exe .

I know, I'm a bad person just because I use tech that can be used for evil.
Like if you use a knife you are an evil person because you can stab or just cut another person - that's why anti viruses consider my code to be a malicious one, right?

After investigation, I realized that I'm not the only one, look at the Golang FAQ:

This is a common occurrence, especially on Windows machines, and is almost always a false positive. Commercial virus scanning programs are often confused by the structure of Go binaries, which they don't see as often as those compiled from other languages.

If you've just installed the Go distribution and the system reports it is infected, that's certainly a mistake. To be really thorough, you can verify the download by comparing the checksum with those on the downloads page.

In any case, if you believe the report is in error, please report a bug to the supplier of your virus scanner. Maybe in time virus scanners can learn to understand Go programs.

If (almost) every program written in Go is considered a malware, what can a simple developer can do?!

I never ever wrote a malware in my life, never tried to do harm, but now if I use a term from my native language: "I need to prove that I do not have a sister" just because someone/thing decided that I do.

So what would you do?

First class citizen

ik_5 — Sat, 02 May 2020 08:09:09 +0000

In the past few months I find myself writing a Windows based service using the Go.

Long time ago (almost 20 years back) I used to be a Delphi and VB(A) developer, but I moved to Linux, and almost never looked back again.

I had one small project back in 2012 for a simple cross platform service (it needed to work both on Windows and Linux) and then I learned Go and created it by providing a single binary that "just works".

These months it's something different. I create an infrastructure for a big company that is going to work on over 40,000 machines just on Windows as a standalone servic, and it is an infrastructure for a big high demand service that the company provides to everyone and not just to their regular customers.

Go and Windows

The first thing you notice using Go is that it feels right at home when using it on Unix based systems (Linux in my case).

But Windows is not Linux or Unix for that matter. It does have some POSIX support but it's not a POSIX OS.

I tried to take a string holding environment variable and replace it with it's content, so I found os.Expand that exists for Windows as well, Great.
But when it execute in Windows it does not do a it's job at Windows.

Note that the link is for Windows related functions, but the implementation when you read the code is not for environment variables of Windows that uses percentages as variable marks, but rather for a dollar sign that is used mostly by Unix based shell(s).

Windows has it's own function to do the same action named ExpandEnvrionmentStringsW (or ends with S as well).

I would expected with Go's os package to create env_unix.go and env_windows.go to provide the same function, but internally do different work when having the Go method for doing things.

Citizen Go

I know what you are now thinking, Go is doing a bad work on providing a standard and extended libraries.

Well, sort of, but not really. Only incomplete work for that.

There is a thinking that was not invented by Go - "I" should provide the same syntax regardless of uniqueness of things unless there is absolutely no way in doing that, and then there is an exception for that rule. That is create something with the least common denominator.

Go implement that idea but not invented it. But it did something different then most languages. That is create their own "lingo" for the standard library, rather then using the lingo of a specific environment/operating system.

There are other languages such as Object Pascal and C#* that tries to provide the same standard library idea - emphasis on specific environment regarding the main libraries syntax it provides.

Both Object Pascal (Delphi/FPC) and C# provides Windows based API thinking for everything.

Both Object Pascal and C# run also on Unix based systems today, but still, the API is Windows based, rather then more generic way. It is important to note that C# does much better work then Object Pascal, due to CLI.

Go did the opposite - POSIX is the default (and most common denominator of OS based) rather then Windows, but also provide it's own syntax or "lingo" for that.

But it's not the only way of doing things.

A coin have infinite sides

All three languages: Go, C# and Object Pascal took a shortcut. Both Object Pascal and C# started with Windows, so that was the thinking when creating the language support.

Go started by a Unix Developers (Ken Thompson and Rob Pike), and most usage of Go is on Unix based systems mainly Linux.

But the there are other ways such as how Rust* made it.
Rust provides an OS specific implementation for everything that is not unified.

It makes few things a bit harder to do though (the support for generics didn't help there btw).

For example Unix provides a mask for a file or directory/library - The permission. So a mask for owner to read, write and execute (enter) a directory while group and others can only read and execute (enter) is 0644.

It does not exists for specific file systems, and by default Windows (unless it supports a file system that does support it) does not support it as well.

Using the Rust way will have me check both OS and file systems together.
For example both Linux, Unix and raw have metadata for their own uniqueness, but based on documentation, Windows does not have it.

So at the end developers will create a higher level class/function that tries to answer that need, or will create two implementations:

With masks.
Without masks.

And if you do it for compile time, and for example Windows 10 added Ext4 file system for example, you will not be able to create or change mask.
If it's on runtime, you will not have any support for it.

Let's C

C also has it's own way of doing things, but it's Unix based because C was invented for creating Unix back in the 70's.

The C standard library is actually a POSIX standard, and as such it is very Unix based that every implementation must support a syntax that might not be native to that operating system, and might require additional libraries that will help in that matter.

The best way is...

I do not really know what is the best way for solving this issue.

I find myself less opinionated. I do not think that there is a single way of doing it "right", but only the least worst.

The effort and the result must matter in the path.

Important note: I never wrote programs on C# or Rust, only investigated it.

Design it simple, stupid

ik_5 — Sun, 01 Mar 2020 08:07:09 +0000

I love some programming principles such as as KISS - Keep it Simple Stupid, or even Ockham razor when debugging my code.

But somehow I do not practice them as much as I wish to see them on my daily work.

For example, I designed a powerful syntax the other day for a new customer, but the person that needed to code it didn't understand how simple it is (in my eyes at least).

When another fellow developer cannot see the simplicity of things I design, well, it is not simple as I see it!
It's on me, not on the fellow developer.

Design Complexity

The fast thinkers probably thinking "you are mixing between understanding and design".

I used to think that the two are different, but no longer.

As an example, lets say I provide you with two oval shapes. Then I'm explaining to you that the system must have oval shapes in order for it to work.

It is simpler to understand when everything is designed to help you understand that only oval shaped objects for example can get inside the system.

If you place a rectangle, well, the entry will not fit. The opening contain oval cutting, and the entire way of the system enforces the oval shape.

When my design helps at very fast glance to understand things I wanted them to understand, then my design is good.

When my design does not help that, and a lot of effort is taken place to figure it out, then my design is bad.

Testing a design

I know what you are thinking, let's create some unit testing for my theory.

Well there is a problem with that.

Things that are easy for me to understand and figuring it out, are not the same as what is simple and easy for others.

I need to generate something in a cultural agnostic way that can be understood by many.

That's why Emoji for example works. Emoji started with few symbols, and then incremented over the years.

The shapes are simple to understand most of the time - Happy face, Sad face, Angry face, Laughing face etc...

Then some less understood faces appeared, and we misused them. And that was the time that the design was over complicated.

The shape of the Emoji was too confusing.
On that point, many icons were re-shaped to be much better and much clearer for us to understand.

If something that supposed to be simple for humans to do (well, it is actually very complicated) became so hard, it's not the people that misunderstand it, it's the design that has a flaw in it.

Taking responsibility

I'm taking full responsibility when something is not that clear for others.

If there are people that it's clearer for them without me helping them, and others that it's unclear for them, then I'm trying to figure out what is the places people misunderstand the most.

If one or two people misunderstand, then I do not know if I can simplify it further (that is the effort for it, rather then imagination).

But if most or all people misunderstand something...
It's time to re-group and re-think on a better way of designing the same thing.

Learn from what makes people misunderstand the design, rather then blame the people, and you'll become a much better KISS'er then me ;)

Quick and dirty Audio playing in Golang on Windows

ik_5 — Thu, 20 Feb 2020 15:50:17 +0000

Over 20 years ago I used to write Delphi and VB in the MS Windows world.

A decision by MS to remove something from Windows libraries helped me switching to Linux, and I never really looked back.

But now I have a client that wish for my service to work on their Windows (instead of Linux).

I started looking at the Windows API again (hello old MSDN), and understood how much have changed since I last touched the API.

Most documentation are for the C# dev's, but I never even written an Hello world with it, and found out how much effort does the working with IMMDevice requires from me, and all my client requires from me here is to play a wav file for the default Audio device.

So I started thinking how to do that. Google reminded me by accident the "good" old world of waveform and PlaySound function.

After several hours, I came up with a working copy of Go over Windows (my development is still on Linux, and cross compiled to Windows).
I do not own a Windows copy, only Linux(es) for so many years now, so my tests are on my Client's Windows machine, while I'm connected using RDP (the pain).

Binding ...

Windows usually provides C based ABI for API calls (unless you are using .NET apps, and then you are using CLR based virtual machine's ABI).

There are two ways for me to load it..., and I decided to bind myself on run-time, using Go's syscall.DLL.

First step

My first step was going over the mmsystem.h header file, just to find out that it's a meta header for including many headers.
Quick grep (thanks Google for that command btw) I found that all I need is under the playsoundapi.h header.

I copied all the integer based constants and just made it Go syntax friendly (but not Linter friendly).

...
const (
    SND_SYNC      uint = 0x0000 /* play synchronously (default) */
    SND_ASYNC     uint = 0x0001 /* play asynchronously */
    SND_NODEFAULT uint = 0x0002 /* silence (!default) if sound not found */
    SND_MEMORY    uint = 0x0004 /* pszSound points to a memory file */
    SND_LOOP      uint = 0x0008 /* loop the sound until next sndPlaySound */
    SND_NOSTOP    uint = 0x0010 /* don't stop any currently playing sound */
...

Initialize `syscall.DLL`

I can implement some nice functions after having the constants.

After some thinking, I decided to create a wrapper using a more Go friendly function that calls the ugly Windows API functions.

I started by defining a call to the .dll file:

...
var (
    mmsystem = syscall.MustLoadDLL("winmm.dll")
...

syscall.MustLoadDLL loads a .dll file to memory, and if fails the loading will panic my process.

mmsystem is now a pointer of struct syscall.DLL.

Note:

Using Go, every function with Must will panic if there is an error.

Memory address for functions

Now that I have the .dll file all loaded an warm up for me (thanks Go), I can get do some stuff with it.

In this case, I need some API's function love.

...
    playSound     = mmsystem.MustFindProc("PlaySound")
    sndPlaySoundA = mmsystem.MustFindProc("sndPlaySoundA")
    sndPlaySoundW = mmsystem.MustFindProc("sndPlaySoundW")
)
...

The old world of Windows (maybe also in the new?) the NT/server based run-time supported Unicode (UTF-16BE), while home/pro etc.. editions supported mostly extended ASCII with some code page encoding for human languages.

The suffix of A and W provides support for the two types.
The A suffix is for ASCII and the W suffix is for Wide char encoding - meaning multi-byte encoding.

I decided to support both of them in order to learn how Go works with both of them.

Wrapping and gifting

I loaded the functions that I wish to use.

Now it's time to wrap the API functions in order to have a nice Go like code rather then repeating my code usage each time.

When I used MustFindProc, I actually got a new pointer to a struct named Proc.

Proc have one interesting function: Call. It actually execute the arguments we need (up to 18 arguments), using syscallXX for us instead of us writing this ugly code.

Call returns three arguments, but I ignored them (bad dev, bad dev) - the last argument is an error that might have happened.

...
// PlaySound play sound in Windows
func PlaySound(sound string, hmod int, flags uint) {
    s16, _ := syscall.UTF16PtrFromString(sound)
    playSound.Call(uintptr(unsafe.Pointer(s16)), uintptr(hmod), uintptr(flags))
}

// SndPlaySoundA play sound file in Windows
func SndPlaySoundA(sound string, flags uint) {
    b := append([]byte(sound), 0)
    sndPlaySoundA.Call(uintptr(unsafe.Pointer(&b[0])), uintptr(flags))
}
...

On PlaySound I converted the Go string (UTF-8) to syscall's UTF16BE (c-)string.

On SndPlaySoundA I converted Go string into ASCII, using slice of byte, and added a null terminated at the end of the slice to let C based code when the string needs to end.

On all functions (PlaySound and SndPlaySoundX) there is some ugly code, so the wrapper hides it, giving us a nice Go-ish syntax instead.

When using unsafe, it is important to note that CPU's endian, and so does memory addresses are not cross platformed as much as other code.

So it is important to understand that usage of code that touches unsafe contain some possible pitfalls that will be discovered on run-time only, and on some machines only. FUN!

The full Code

Warning

On a real production code, there are more steps to take - Always check and validate errors, never ignore them as I did

Golang and shared objects Part 1 - The background

ik_5 — Fri, 10 Jan 2020 17:29:52 +0000

When creating an application using Golang, it is usually a monolithic executable that holds "everything" is inside.

When there is a need to separate some logic for execution, I find that many software architects prefer actors/model or workers for that.

I do think that there is room for having actors/model or workers, and I personally do use them, but as software developers we have so many tools for our disposal that I feel like they are not in use or taken in consideration on places that they can be helpful, and that is the trigger for the current series of posts.

One tool that is over looked is shared object a.k.a. shared library.

The following series of blog posts are about shared objects/libraries. On my next posts I will use Golang (and sometimes even C) for explaining how things are done, but the basics are the same in their concept for most programming languages - even most dynamic languages (such as Python and Ruby), uses the same concept but with different syntax or tool set.

What is it all about

Shared Object

In a nutshell:

Shared Object/Library is a binary file with dynamic/static loading table for functions.

There are two types of shared objects:

Dynamic.
Static.

As I wrote at the beginning of the post - the static version is usually what Go is using. Static library compiles functions inside the executable causing the function to be part of the program inside the executable as it was written by us for that program.

Dynamic library is actually the same as an executable file on most systems and formats, but without the execution part, only dynamic memory table that holds function addresses that can be used when looking for a function.

We know dynamic libraries as .dll, .so and .dylib.

On this post, I'll only talk about Dynamic Library. You can find more information about static library on this link.

Loading of a library

A library can be used in two main ways:

Static loading
Dynamic loading

Static Loading

The main usage of a library is to provide reusable functions.
The functions do have memory address, but it is built to be as it was part of the application that calls it.

In order to do that, a programming language requires to share the function name, and it's arguments.

A function name (as well as some additional data structures) are called symbol. The symbols hold a unique identifier for accessing, and contain "human readable" value.

In order to access a symbol, there is a need to know the memory order of argument both for input and output values of the function.

The signature and arguments order have the name of ABI - Application Binary Interface.

Once a developer knows all of that information, it can instruct a linker on how to connect a library and it's function(s) to the executable or another library.

It happens on build time (after compilation), when the executable is ready, there is an instruction for the Operating System (OS) on how it should be loaded with the process as part of it dependencies.

A library is loaded by the OS when it is executing the process (program) as part of the memory address as the rest of the process, having the memory address of the library as something virtual rather then physical.

A library can be loaded many times on the entire system, based on the number of processes or other libraries that uses it and are loaded by the OS, but each copy has it's own memory address that is private and not accessible by non related processes and shared libraries.

On Unix system, it is easy to see the static linking by using the ldd command:

$ ldd main
        linux-vdso.so.1 (0x00007fffed2c9000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f5321c21000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f5321e74000)

Dynamic Loading

On static loading the linker and OS connects a library function to memory address, but it requires to bind a shared library to a process or another library. It is limiting when writing extensions for example.

Another way to connect such function is on run-time.
Run-time means that the process is loading the library to memory and requires to handle the entire ABI on run-time, including freeing the memory that is allocated and any issues that can happen by going that way.

Due to the fact that the information is dynamic, it is impossible to know beforehand what library will be linked, unlike the static loading.

Loading of a function

As mentioned above, a function on a shared library/object is called a symbol.

A symbol is a unique identifier for function, variable, constants etc...
Something that the execution process knows how to access when it finds the name, and can find it's memory using Symbol table.

The symbol table holds a given name, the type of what the symbol is, memory position in order to attach to it's execution and ABI.

Based on the format, it can hold additional information, or even a bit different method of getting such information, but that does not matter for us on this post.

When a function is called, it is translated to a symbol name, and the symbol is looked upon and is used based on the memory address and usage.

There are two ways in doing so:

Static loading
Dynamic loading

Static loading

For static loading of a function, the main code need to have a function with the same ABI, and the function will hold the memory to the function at the library.

It is done after "compile time" - The linker places the memory address of the library symbol to the function at the executable (or other library that uses it).

Using the command objdump on Linux it is possible to see the linked symbols and their usage (written by me in C):

$ objdump -t ./main

./main:     file format elf64-x86-64 

SYMBOL TABLE:
...
000000000000000       F *UND*  0000000000000000              printf@@GLIBC_2.2.5    
0000000000000000       F *UND*  0000000000000000              __libc_start_main@@GLIBC_2.2.5                     
0000000000000000       F *UND*  0000000000000000              g_uri_parse_scheme        
...

Dynamic loading

Dynamic loading of a function is as of dynamic loading of a library, and actually is part of it.
The dynamic loading happens on run-time - The linking is done dynamically at the code level rather then at linkage time, providing more flexible way on expanding a process capabilities when needed.

A function or memory address a.k.a pointer (depends on the programming language) is assigned to hold the loaded symbol.

On Unix system the main library to use commands is named libc or C Standard Library. libc contains a set of commands to load libraries dynamically, however it is not part of the main libc, but an extension, because it is part of an operating system and not a standard way of dynamic loading a symbol out of a shared object.

On most Unix systems, the API for loading symbols dynamically are under dlsym extensions.

On MS-Windows there is a different but similar implementation named libloaderapi, contains additional functionality that Windows provides.

Summary

On this blog post I have provided information that explains what is a shared object that is also known as shared library.

I explained that there are two types of shared objects:

Static shared object.
Dynamic shared objects.

I have explained in a nutshell the differences, and focused on dynamic shared objects.

Explained that there are two methods of using them:

Static linking.
Dynamic linking.

And in a nutshell what it means.

What comes next?

On the next part I will start writing some code, providing examples on how to write shared library both using C and Go.

And on the 3rd part how to use them inside both C and Go.

The 4th part will explain how to create special cases of Shared libraries (e.g. Go's plugins).