Any engineer knows to listen to their intuition. I know I can point to a method in IDE or a line on an Excalidraw diagram and say to my friend “I can’t articulate it, but this spot gives me an ick”, and after staring there for 10 seconds they will nod “I know what you mean”. It is important, of course, to not trust it blindly, but to try and figure out exactly what is your 6th sense is hinting at. Most often it is a tech debt risk, a race condition, a probable edge case. Lately I’ve been getting one particular ick, and it’s been a challenge to diagnose and give it a coherent definition. What I have cooked up so far: “Do not write algorithm to control the human, but to be controlled by the human.” Sometimes, what your intuition hints at has nothing to do with the code quality.
I’ll try to sell you both ethics and shaky speculative math of it.
Software engineering is glorified plumbing: directing, splitting and joining data flows, and as such any considerations of ethics come to it as an afterthought on a Saturday podcast, or a two paragraph piece on Wired. Some professions over few hundreds of years built up monastic orders around themselves, which aggregated ethics, philosophy, and good disciplines into one consistent framework, and follow the tradition to this day, as far as to have internal legal code, rituals and relics. They combine intuition and wisdom about both quality and ethics of how things to be done. Notably, professions that have the semi-religious reverence and order tradition are the ones where social good is an important metric not fully overshadowed by chasing profits. Jurisprudence, medicine, law enforcement. The two French l’Ordre des médecins and - des avocats - for doctors and lawyers respectively. Canadian - most close to us of all available - Ordre des ingénieurs du Québec (which I’ve been told includes even chemists, but not programmers) goes as far as to issue rings to the members. Software engineering being young and STEM-pilled is missing it. Of course things like TDD or clean code philosophies are treated with militant reverence, but the spirituality/ethics optimize for social good at the cost of convenience and earnings.
The plight of modern tech can be diagnosed as algorithms commanding humans, not the other way around. Many if not most modern tech concerns can be reduced under this lens to algorithm becoming the master. It pollutes the space of our relationships and politics through social media algorithms, it pollutes our natural environment through LLM compute farms. In fantasy media magic schools of control and manipulation are considered dark magic comparable to necromancy, (afaik Crucio is the second most condemned spell after Avada Kedavra). Tell me you don't feel at the back of your mind a prick of conscientiousness discussing "user engagement and retention" at work. Don't get me wrong, I do enjoy the small power trip implementing UX/UI tricks to raise the chances of a visitor clicking the button I want, or trying to figure out most addictive game loop for an odd pet project at night, but ultimately I do feel like like I'm engaging in something profane and ethically questionable.
However, the ethics are an afterthought justification. Sometimes the fault in question doesn’t even trace to ethics. There comes a delusional minute few times a week, when I see myself as more than a glorified plumber: reducing local space entropy and wrangling algorithms. We build abstract systems to execute human will, and most everything has real world effects. While some humility would do me good, I don’t feel shame for tapping into this feeling - it’s precisely what gives me the high of the flow state - not typing green letters in a black terminal window like a secondary Matrix character, nor dreaming of building next Facebook for money and glory. Minecraft’s Redstone or Factorio's belts give that flow feeling all the same. All jobs shape physical reality in one way or another, and for us - logic is the element we tamed. All jobs come with professional deformation - and for us it is viewing the world through the mathematical lens.
So to win back engineers’ attention, I want to present the faulty systems we are building as a positive feedback loop. To plug a user into the algorithm, he needs to command it by at least dragging a thumb across the screen. But when the chain of method calls you wrote leads to more thumb drags - you’ve created positive feedback. It drives the system out of equilibrium with ever rising speed, and when the function consumes finite external resources - clean water, teenager mental health, society cohesion - it erodes the space it taps into. That is to say, at the foundation, stripped from definitions down to declarations, such design is evil magic. If we ever get to found “The Chamber of Programmers”, I’d make sure to put this into the scripture.
The tool enables man to conquer. But, man, dost thou not know there is no more victory which is thy victory? The victory of our days belongs to the tool. - Jacques Ellul
Top comments (0)