DEV Community: STrRedWolf

[Boost]

STrRedWolf — Wed, 02 Jul 2025 11:42:35 +0000

Digital Detoxification: My Journey Back to Real Life

Aditya ・ Jun 29

#discuss #beginners #healthydebate #devdiscuss

The Art of Defensive Programming

STrRedWolf — Thu, 26 May 2022 13:31:16 +0000

The Art of Defensive Programming

As a programmer, there are times where you have to dig deep to find a simple solution for a complex problem.

For instance, my boss had come to me, pointing to a problem on a customer-facing web portal. I did deep into the code for it, pulling out error messages logged in various systems... and I see something odd in one table. So I ask if something was added... and there was. But it wasn't fully added... and expectation on if some of the data in the table had to be there or not was faulty. In this case, it was a description of a new location the company had developed. Add a description in... and the portal problem resolved itself.

In times like these, I am reminded back to an article written by David Small in STart Magazine back in the summer of 1986. The magazine was for Atari 16-bit computer enthusiasts, starting with the Atari 520ST. The article was titled "Voodoo Computing: The Pragmatic Art of Defensive Programming".

Nearly 40 years later, a programmer myself, I feel it's time to give that old article a refresh with my own perspective. I borrow a bit heavily from that article, because many of the points made still hold true today.

But like Mr. Small has done before, we must for the moment examine a typical modern computer... and how truly we're lucky it works at all.

Refresh my memory on this

The first really big thing to realize is that memory (unlike storage) is as Mr. Small said, "a frighteningly transient thing." Most every computer, from your PC and the servers you're talking to down to your phone and tablets, use what is called "dynamic RAM". DRAM allows us to pack a lot of memory into a small package, but at the trade-off of having to refresh it with additional circuitry -- at millions, if not billions, of times a second.

It used to be that the CPU inside the computer had to do this, but now, it has dedicated circuitry called a memory controller that does it for the "core" of the CPU that does the actual work. The CPU also has it's own internal RAM stores, usually smaller "static RAM" that doesn't need refreshing and are used for caches -- namely because reaching out to the memory controller and out to that fancy DDR5 uber-clocked DRAM is still damn slow!

It's also still susceptible to power failures, lightning strikes frying the circuitry, cosmic rays, and even a rouge magnet coming around and flipping a few bits of data. It's just that... well, actually, most of the time the RAM is being used. If not for actual programs and their data, but as disk caching.

Intel back in the day thought that error-checking RAM was too expensive, and made it optional... and when it was revealed that it happened, there was a bit of an uproar because such RAM would stop crashes.

Think of it this way: It's over 16 billion dominoes stacked on top of each other. One push, one external influence, and down it goes.

And then there's storage.

Do you have a backup?

You got two types of storage: hard disk drives and solid state drives. Hard disk drives store data by employing tiny tiny magnetic fields on spinning disks, contained in a case in an attempt to stabilize them as much as possible, even though they're spinning tens of thousands of revolutions a minute. Any vibration, even from another drive or you just walking in, can send a shock wave that knocks the head off kilter and the drive has to slow down, recover, and reread (or re-write) a good chunk of data, slowing it's response down. Even then, the drive's discs can lose those magnetic fields, lose the capability to host those magnetic fields, or even have the read-write head used take a nose-dive right into the disc.

Solid state drives fare differently. They remember the data for a long time but every write damages the flash chips used. Too many writes, and the chip is dead. This is why SSDs use techniques such as "write leveling" and "sector remapping" to try to extend the life of the drive as much as possible. Sure, reading is fine... but do you want to trust your only copy of a project to something that can turn from read-write to write-once-read-never?

There are other things that Mr. Small does talk about that are still relevant, but we tackle by adding more computers into the mix. Keyboards, for instance, have a tiny CPU in it, a microprocessor, to handle each key press "bouncing" and sending signals erroneously as the key settles back into it's upright (and hopefully not locked) position. CPUs themselves run "microcode" and some even have tiny CPUs that can monitor and correct errors. Monitors have CPUs now. Some mice even have them.

It's turtles all the way down. And occasionally, something goes wrong, through no fault of your own. Take Andy Ihnatko's common comment that humans "are but imperfect vessels for the perfection of the universe", add that computer demand perfection, and you end up what Mr. Small calls the "ultimate question of the Frustrated Programmer": Is it my code or my PC?

Mr. Small laments that "something wonderful, childlike, and native dies in a programmer the first time a computer genuinely screws up. Up to that point, you are sure the problem is yours, somehow, somewhere. Afterwards, you are never the same. It's like learning the truth about Santa."

So what can we do? What can we follow as sane people?

Mr. Small says to treat working with computers as like performing sorcery. Programming is like casting spells. Take defensive measures to not let the demon out as you bind it to do your bidding, and you'll go far. To that, he describes ten maxims to which I'll expound upon:

Maxim #1: When you're having a bad day, stop working.

I've had many of a time when working on a project involving various Microsoft projects where I know I've coded a routine right. I could find no bugs, even from the examples provided... but somehow, it wasn't working.

But if I put it aside, get up, get a snack or something to drink, and come back to it... it's working. I didn't change a thing. Some cache must of gotten flushed or the system decided to actually run the code I had written instead of a different version five changes back.

The thing here is to take a break. Stop hammering the system. Let it do some idle maintenance. Save your work, save a backup, and go do something else or something useless. I hear social media is a useless thing...

Maxim #2: Comment your code to death

I've had too many times where I was hyper-focused on one thing... and then get interrupted and lose the train of thought. How can you recover from an interruption quickly?

Comment your code. Even a high-level set of steps help. It takes a minute more but saves tons of time, not only for you but the next person who takes over your project after you've left it.

Maxim #3: Steal from the best

Corollary: Googling a problem to find some code that works is good.

Lets face it. A lot of times, you're writing code but using a library that handles the heavy lifting of interfacing with some site. Other times, it's a problem that you're getting frustrated over... but someone's solved it already.

Good programmers know this. They're happy to share their code, and look for others code to pick up their tricks. Maybe there's a better way that's faster.

So let other people look at your code, and take in their input. You may find other bugs that way.

Maxim #4: Use the best tools, and be prepared to pay for them

Seriously. Your time is valuable, and good tools are worth it.

Back in 1986, Mr. Small was making an Atari ST think it was a Macintosh. It was 7000 lines of Motorola 68000 assembly language, or about 120 letter sized pages. It's not a job for a slow printer... nor a job for a slow floppy-disk based compiler. Hard drives and RAM drives (virtual drives in RAM) were a must back then.

Now, we have Visual Studio with how it type-checks on the fly and catches common mistakes. Rarely have I had to go back and correct a mistake where I capitalized a variable in one place when I didn't some place up. We have plugins for Visual Studio that even do deeper analysis. That is well worth it.

Maxim #5: Backup your work.

I've had too many times where I had to save off what I was doing, switch to a different version of some code that had to go in the exact same place in order for it to compile, then switch back. Some of this code can't be put into a repository for some reason or another (maybe even too soon), so... I ZIP up a copy beforehand.

And then you have the time that your PC dies. I had that at a previous job where the hard drives were "just a bit" flaky. Ether I rebuild my work on a new hard drive or I try a recovery program called SpinRite. The latter worked, but afterwards, I backed up everything to a network drive.

Back it up often. You'll going to need it later.

Maxim #6: Keep three backups.

In this hyper-connected day and age, losing your work is essentially doing it twice. You don't have time to spend manually recovering your work, especially to folk who encrypt it for ransom and happen to be on your network because Bob in Accounting thought that cutie in his email was "safe".

No. Save a copy off on your network file share and make sure there's a copy of that off-site. Working on it off of a Git repo? Commit a copy on a remote server. It'll be needed.

Maxim #7: Don't be clever.

Seriously. You don't have time to be clever. The compiler takes care of that for you in most cases, but only if you let it.

For instance, C# has the "var" declaration, shoving any type-checking to run-time. Sure, you can "var" here and "var" there to have it assume the type of a returning function, but that returning type can change and if it's not what you were expecting (because you hid that from yourself by using "var")... well, there's your problem.

Besides, half the time you won't remember that one clever trick anyway... and that trick may not work in the future when someone has to debug your code because something changed. You did comment it, right?

Maxim #8: If it works, don't fix it.

The compiler doesn't care how it's formatted. If it's a kludge but it works, leave it alone and do something else.

Do you need to sweat the indenting, the pretty printing? A modern IDE like Visual Studio will do that for you for most languages, and some like Python they'll assist you on getting it right. Let the computer be the neat freak. If you insist on being the neat freak, stick to Python, which mandates it.

Maxim #9: Write defensive code.

Mr. Small says "Always give your code the maximum chance to work. Or: It'll always think of something you don't."

Remember my example at the top of this article. Other programmers looked at the table in the past and saw the description always had data in it. But the program that manages the table allows it to be flagged as void of data. That flag is called "NULL", and no matter what programming language you're in that has such a concept, it's always something you have to pay special attention to.

Those other programmers assume it would always have a value. When it didn't it threw an error that resulted in a bad user experience.

Using variable? Initialize it when you declare it. Watch if you reuse the variable for something other than what you intend for it. And watch for special values (like NULL).

Maxim #10: Keep it simple, stupid.

Mr. Small said "Structured programming is useless in the real world. You don't need to program in a structured way."

To be honest, I think that falls a bit more under an old Navy saying: Keep it simple, stupid. UNIX philosophy follows this with "Do one thing, but do it well."

I see a violation of this numerous ways. For instance, a simple "pull data from a vendor server and store it in a database" program. Do you need to wrap the core of the program in one object library while calling it from a runner program? Or how about doing the pulls and pushes as calls through a set of web services when you have direct access to both ends? How about calling an endless set of routines down to only make a web service call to a second back-end server that pulls the data itself, but is only behind a firewall?

Do you really need to spend all that time with all that "plumbing?"

The more complicated the program is inside, the more problems you'll encounter, and the more time you (or whomever is debugging it in the future) waste when something does go wrong. Keep it simple and straight-forward.

In conclusion

Thus ends my update. If you've learned something from this, you'll already be on your way to a better programmer... or at least make sure the next one won't be trying to track you down and strangle you for how bad your code is.

KISS my blog! (part 1)

STrRedWolf — Sat, 26 Oct 2019 14:15:41 +0000

Sometimes, you get tired enough that you might as well do it yourself.

I want my own blog software. I want something light and simple, but able to be updated on the road without too much hassle. I want something that's secure and relatively bug free. Something that doesn't get a ton of press for being repeatedly buggy (I see you, WordPress!).

Why not make it open source (GPLv2) and develop it here on dev.to? It'll be fun. We'll name it "Nothing" (Seinfeld reference). I'll throw the code up on Github at https://github.com/STrRedWolf/Nothing .

Philosophy

Before we get too gun-ho about things, I want to bring in the Navy mindset:

Keep it simple, stupid.

We should not add anything that makes it more complicated and bloated. The more that there is, the more that can go wrong. Maxim 61: Don't bring big grenades into small rooms.

BTW, you should read The Seventy Maxims of Maximally Effective Mercenaries ether online or get the book itself. I'll be dropping related maxims here and there because they do relate to what we're doing.

With that in mind...

Requirements.

Use Github-flavored Markdown for posts. It seems to be the best way of writing stuff out.
Use only what is necessary to do the job. That means being light on the libraries on the server, and eschewing Javascript frameworks. Maxim 30: A little trust goes a long way. The less you use, the further you'll go.
Use a database to keep things tidy, but don't overload it. Using views is fine.
Have users with privilege levels, icons, etc.
Optional: Have "attachments" that can be inlined, so you don't have to SSH-copy stuff onto the server.

The project will be inspired by LiveJournal, because those avatar icons can be switched to indicate various moods and such of the user.

For fun, I turned a Raspberry Pi 4 into a work server with lighttpd, PHP, and MariaDB. Just a 1GB version will do, and a Pi 3B or 3B+ will do the job as well. For development work, you don't need much.

First stab at the SQL

CREATE TABLE posts (
        post_id         INTEGER UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT,
        title           VARCHAR(255) NOT NULL,
        content         TEXT NOT NULL,
        author_id       INTEGER UNSIGNED NOT NULL,
        dtstamp         TIMESTAMP,
        has_attach      BOOLEAN NOT NULL
);

CREATE TABLE authors (
        author_id       INTEGER UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT,
        username        VARCHAR(64) NOT NULL,
        fullname        VARCHAR(255) NOT NULL,
        passhash        VARCHAR(1024) NOT NULL,
        saltine         VARCHAR(1024) NOT NULL,
        blurb           TEXT NOT NULL
);

CREATE TABLE attachments (
        attachment_id   INTEGER UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT,
        title                   VARCHAR(255) NOT NULL,
        disk_location   VARCHAR(255) NOT NULL,
        link_location   VARCHAR(255) NOT NULL
);

CREATE TABLE post_attach (
        post_id                 INTEGER UNSIGNED NOT NULL KEY AUTO_INCREMENT,
        attachment_id   INTEGER UNSIGNED NOT NULL
);

CREATE VIEW postview AS
        SELECT P.post_id, P.title, P.content, P.dtstamp, A.username, A.fullname
        FROM posts P
        INNER JOIN authors A ON P.author_id=A.author_id;

CREATE VIEW postattach AS
        SELECT F.attachment_id, F.title, F.link_location
        FROM attachments F
        INNER JOIN post_attach PA ON F.attachment_id=PA.attachment_id;

I wanted to start out simple and small, although I did get a bit ahead than intended. Still, if we can make it easier for ourselves to program it and it's cheap to do, do it.

Note the authors table. We'll use that to indicate posting users. I haven't thought about commenting users at this time. Yes, we're going to salt the hashes.

Comments and suggestions are welcome. I intend on posting more as I work on this. However, remember to heed your moderators. Maxim 8: Mockery and derision have their place. Usually, it's on the far side of the airlock.

The next post? The main index view.

Question pivoting for faster results

STrRedWolf — Fri, 26 Jul 2019 20:35:53 +0000

In my job, as the subject matter expert of the monitoring system we've brought on and use, I get the odd question from our support team. Some are obvious to me, some are worthy of an article in an internal knowledge base, and some... well...

"Hey Red, maybe this will be quicker. I need a list of all Linux servers that have under four gigs of RAM and MySQL installed. Can you query the monitoring system database quickly?"

I said I'll check into it. With some time on my hands I did...

And this is where I curse H.P. Lovecraft and fight great old eldritch horrors.

You see, the monitoring system's database layout isn't all that great. It's optimized for fast writing, a choice the vendor of said system made a long time ago. Plus, their reporting features only get you so far, and the use of MySQL in the past and MariaDB now makes me want to file bug reports for features in MS SQL Server (at least MariaDB has "WITH alias AS (SELECT statement)" now!). Plus, data I need isn't in a normal structure -- no, it's captured by convoluted means and placed in the fast write structure tables.

Which means to get the list, I can't really use MySQL or MariaDB's SQL language. It's not that capable. It took me 2.5 hours of frustration to realize that.

I give up, talk to the guy who asked the question... and realized an easier way. I was initially looking for Linux servers with 2 gigs of RAM. I could of said "What Linux servers have MySQL on them?" The monitoring system has a fast query page that lets me ask that, without going through the database or tearing my hair out. I was able to say "Okay, do this, you get the list of 130 servers to check the RAM on." The support guy said "I'll take that!" and can easily dig through them to get a shorter list.

The point here is that sometimes, you have to reorder the question and do a bit of optimization yourself. SQL servers do this all the time, gathering some info on the query, reordering the searches needed, and producing a fast execution path.

The case against "natural/localized" sorting

STrRedWolf — Sat, 29 Jun 2019 12:17:11 +0000

So my favourite text editor on the iPad updated -- Textastic went version 8, and one of the features is choosable sorting, by name, extension, date, and size. All fine and dandy, I thought.

I then went to find a file... and it wasn't where I was expecting it.

Here I am, on a commuter train going to Baltimore, pulling up my phone and franticly searching for a file I knew I created and named. Not in Textastic. Not in Dropbox, where it's synced. Not in the regular places...

I then relaunch Textastic... and find it... in the wrong spot. Same directory -- they changed how it's sorted.

It's only ASCII! It's so simple!

Let me put this into context. I'm writing a novel, mixing a weird bit of furry culture with the Station to Station nomadic train, called Throng: Going Station to Station. PLUG WARNING You can see the story of a multi-furry band tour by train at my Patreon PLUG WARNING.

The novel is written in sections, each between 5-10K in size. Chapters are at most 8 sections long, so to keep them in some order, I name them with a section number prefix... that's in hexadecimal.

Here's a small example:

01.Boston-1.md
02.Boston-2.md
03.Boston-3.md
04.Boston-4.md
05.Boston-5.md
06.NewHaven.md
08.NewYork.md
09.Secaucus.md
0A.Trenton.md
0B.Trenton-2.md
0C.Trenton-3.md
10.Philly.md
11.Philly-2.md
18.Baltimore.md
19.Baltimore-2.md
20.WashDC.md
24.WashDC.md
28.Pittsburgh.md
30.Cleveland.md
37.Gary.md
38.Chicago.md
39.Chicago.md
3A.Chicago.md
3B.Chicago.md

As the story progresses through various cities along existing real life rail lines, I'm naming the sections after them.

So what does GTK's file picker, MacOS Finder, and Textastic do? It treats the first number as that, a number, and ignores any front-facing zeros. This is so you can have this:

file 8
file 9
file 10
file 11
file 12

Okay, that's fine... but what about my naming convention? Thankfully, Linux's "ls" requires you to give it the "-v" switch.

$ ls -v1 *.md
0A.Trenton.md
0B.Trenton-2.md
0C.Trenton-3.md
01.Boston-1.md
02.Boston-2.md
3A.Chicago.md
3B.Chicago.md
03.Boston-3.md
4A.Dallas.md
4B.Ft_Worth.md
4C.Ft_Worth.md
04.Boston-4.md
5A.Tuscon.md
05.Boston-5.md
06.NewHaven.md
7F.SanFrancisco.md
08.NewYork.md
09.Secaucus.md
9F.Metroburg-Home.md
10.Philly.md
11.Philly-2.md
18.Baltimore.md
19.Baltimore-2.md
20.WashDC.md
24.WashDC.md
28.Pittsburgh.md
30.Cleveland.md
37.Gary.md
38.Chicago.md
39.Chicago.md

And that's what I'm seeing in MacOS, in GTK's file picker, and now Textastic.

Why hate on strcmp?

It's UTF-8, aka Unicode. I'm running the "en_US.utf8" localization, mainly because it's default. MacOS and iOS uses it. GTK uses it. And now Textastic picks it up, because it's using iOS' NSString.localizedStandardCompare routine... and it sorts by context instead of byte-wise. And it's mandated by Unicode's spec.

That's the thing. It may be mandated but it's not right in all cases. I have the breaking case.

What can be done?

As a programmer, give users the option to turn off "natural" sorting for "literal" sorting. The more options, the better.

As a user, file bugs. I did so for Textastic and GTK/XFCE.

Resist the urge to work-around. Linux has it right at the command line -- make it optional.

The right hash(s) for the job?

STrRedWolf — Sun, 01 Oct 2017 12:39:52 +0000

Once again, I'm finding myself deduplicating a repository of files, and trying to do it right. Yeah, I could use a dedicated tool... but I'm not sure they're doing it correctly.

You see, they hash the file contents and try to match on that, assuming that they're going to be unique for each file. But in reality, it doesn't work that way. You're going to collide.

How do I know this? I wrote my own deduplication tool.

Right now it's using SHA-256 to generate the hash digests, going over the entire file. It stores them in a MariaDB database and compares them to all the other files as it steps through each file. If there's a hash collision, it breaks out the venerable cmp tool and verifies that they're byte-to-byte duplicates.

The result?

In a hand-deduped archive of 69248 files, 420 were false positives.

My question now is: Is SHA-256 the right choice? Which would be better, or should I be using multiple hash algorithms to reduce that false positive number? Speed boosting here is gravy at this point.

Baltimore at 4:45am

STrRedWolf — Thu, 10 Aug 2017 20:04:18 +0000

Something I wrote back in Summer of 2016:

Unlike most folk my age, I am cursed with a job that requires the occasional odd-hour service. A function I do not like but carry it out dutifully.

I'm a programmer. I solve digital problems. Like how are you going to release and test a new set of software and web site that the world has never seen except in previews for a major Internet Services company, with an overseas team at the same time? Oh, and you can't drive and you need to be in the office to do it?

You overnight in Baltimore and get into the office by 5am.

I stay at a half-decent hotel for the night, a few blocks away from the job site. Check in, sleep fitfully, check out.

Baltimore at 4:45am. It is quiet. It is dark. There are few people around, namely waiting at bus stops to be picked up by the first runs. Blacks. No hookers. No nightlife. No owls. One looking out around a set of tour buses blocking a stop for the 3. I go up and make my right onto Baltimore Street. A lone Asian, walking alone, destination unknown. I don't stop to ask. I don't need to know. I have to get to work.

The subway isn't open, and I doubt the light rail is ether. I know there's no MARC service. It's why I overnighted in Baltimore. The air is crisp and dry and cool, verging on cold. I haul my backpack and overnight bag down the street. Around the closed station. Blacks waiting. Not many, but keeping to themselves. A bus rumbles up Charles. Waiting, waiting, waiting. I hit the intersection and cross with the light.

Nothing is open. Chinese place is lunch and dinner only. Rebooted New York buffet and grill doesn't open until 6. Coffee shop doesn't open until 7. I keep moving. Starbucks isn't open. The McDonalds closed while construction was going on next door. It can go. I cross Baltimore and then St. Paul. I can use coffee. I have to be at work at 5am. They have coffee.

I make it up to the office at 4:54am, and two coworkers are there with the boss. The boss passes a piece of paper around for breakfast orders. I scratch down my order after looking it up online. Everyone puts theirs in. I hunker down with a cup of decent coffee and say a silent prayer to St. Offrans.

We let the site fly. The core is good and proven, but the visible parts fumble and falter. I push a change through. Off shore team pushes changes as well. Test, fix, push, retest. It stabilizes around 2:30pm. We call it a release and go home at 3.

I grab the first train home. It is light. I get off at the airport, meet my ride, and get a late lunch. I am tired. I am hungry. I chow on Burger King and suck down Coke. Forget beer. That is for tomorrow. Today I am spent. Tomorrow, fire fighting. I collapse into bed early. A drone of a fan that is constant, never changing, never giving up or giving in. Never taking orders from something too close to be useful.

Baltimore can have 4:45am.

A fuzzball of an introduction.

STrRedWolf — Mon, 22 May 2017 15:17:49 +0000

What can I say... Well... I've been programing in various languages (C, multiple dialects of Basic, C++, Perl, PHP, Python, Informix 4GL, bash) for over 30 years now. Slowly I'm tossing some code up on my GitHub repo.

Mainly, though, I'm a generalist programmer. Toss me at some project, and I'll research it and get it done. I like my programming languages to resolve integer 0 to false and my code well formatted yet compact.

I work for Edge Hosting, LLC and take the MARC train into Baltimore every day. There, I'm mostly writing up C# and Python.

There's much that I didn't know about, but Dev.To's been very useful for some coding here and at home. Plus, the "books" have been useful (O'RLY? YA RLY!).

For the name and avatar pog? Well, that's a bit of history that may be divulged later on...