DEV Community

Cover image for The First Rule of Shadow Tooling
Ben Link
Ben Link

Posted on

The First Rule of Shadow Tooling

Long ago, I was a new(ish) IT Professional working on an Application Support Team. It was old-school traditional support work - always undermanned, underequipped, and just trying to keep our heads above the flood of help tickets.

Along the way, I realized that if we were ever going to get ahead of the curve, we needed a couple of tools that weren't available to us. I also knew that everything we needed to create those tools existed, but... we were a traditional support team. We weren't authorized to build things, that would be work for real developers.

...I did it anyway.

Shadow I.T.

I must have broken just about every architectural rule in the company... Unapproved development work. Unauthorized use of server hardware. Unapproved connectivity. Unapproved use of free/open source software packages. But I built a small suite of tools because my team needed them, and gave my team access to them.

Things changed... rapidly, I might add!

Using our new tools, we were able to complete tasks faster. That meant we were able to spend more of our time writing docs, and permanently solving problems rather than just reapplying bandages. We had a little margin in our lives now, and that margin was enough to completely revolutionize how our team supported our systems.

To the Company's credit, they didn't reprimand me (or fire me) for all the rule-breaking. In fact, as word got out about the existence of our little toolkit, other teams began using it. Before long, hundreds of technicians and developers were daily using these tools and the quality of life for our engineering teams was improved across the board.

If you can't do something smart, do something right

These tools remained in use long after their intended lifespans - some of them even to this day. There have been multiple efforts to try and replace them with "official" systems, built within our rules, but each of these efforts has met with one failure or another that's left them somehow inferior to the "Shadow" product.

And That Got Me Thinking

The people who write strict architectural rules say that they're doing it for multiple reasons:

Business Continuity

If you write some critical code in some obscure language, no one else might understand how it works, and then we have unmaintainable systems!

My friends: Are we software engineers, or aren't we? If some code is useful and helpful and beneficial... it deserves to be documented and canonized rather than isolated and treated with contempt. Perhaps it's not in the "standard stack"... but I know a lot of software engineers, and giving them a challenge like "maintain this system in some language you've never used before" can be absolutely enticing if presented in the right way.

Licensing Concerns

We have to have strict control on our licensing costs. You can't just bring anything and everything you want into your app!

On the surface, you might think this one is a slam-dunk case, BUT:

  • Are you telling me that we have absolutely ZERO licensing issues across ALL of our software and hardware today?
  • Are you also telling me that you're sure I haven't thought of ways to mitigate licensing risks myself?
  • Are you willing to shut down all the productivity improvements for the sake of avoiding a little paperwork?

Process Control

We bill our customers based on how you log your time for the day. If you were doing Development work and charging it as Support work, Accounting will get twitchy.

I spent a little time writing some code that ended up making my entire IT division faster at their jobs. Do you really want me to do the cost/benefit math on this?

But We Have People for That

We created an "Innovation Team" who's supposed to go and figure out how to optimize all these things. We don't need people doing off-the-books stuff!

You've fallen prey to a fallacy: it's simply not possible to centralize creativity. You can only spot it and capitalize on it. When you find creativity happening, you don't squash it and tell it to fill out a form! You nurture it, you give it what it needs to grow, and you celebrate the heck out of it when it blooms!


The Shadow I.T. Problem

If your organization has a "Shadow I.T. Problem", this post is for you. Because friend, Shadow I.T. isn't a problem to be solved, it's a signal that you're ripe for some focus on Developer Experience.

Think about it: someone on your team found a PAIN POINT in your Developer Experience, and FIXED IT FOR YOU. (And hopefully fixed it for the rest of their teammates, too!) They actually cared enough about their day-to-day work to stop doing the expensive, slow, manual thing and made it cheaper, faster, and automatic.

If you interpret that as an organizational threat... stop reading now and get help. I can't do anything more for you.

Samwise Gamgee saying You're Hopeless


What Now? Actions for the Brave

Look for the Pain, Not the Process

Instead of asking “Are you following procedure?”, start asking “What’s slowing you down?” Then trust that the people closest to the problem might already have a better answer than your standard stack.

Reward, Don’t Reprimand

If someone built a thing that made life better — even if it was “off the books” — treat that as a sign of leadership, not rebellion. Find ways to elevate those efforts, not bury them.

Canonical Doesn’t Mean Conformist

If a tool is effective, don’t dismiss it because it’s built with “the wrong stack.” Standardization should serve outcomes, not stifle them. Bring useful tools into the light and document them.

Loosen the Gate, Strengthen the Guardrails

Creativity thrives with autonomy, but autonomy without support is chaos. Shift from rigid rules to helpful guardrails — guide, don’t block. Make it easy to do the right thing.

Princess Leia saying The more you tighten your grip the more star systems will slip through your fingers

Track Developer Friction Like You Track Uptime

Your systems are reliable — but are your developers happy? Empowered? Unblocked? Developer Experience is a leading indicator of long-term velocity. Start measuring it.


The Wrap-up: If You’re a Leader, Get Curious

The next time you hear about a Shadow I.T. effort, you're likely going to mentally default to asking “Well why didn’t they follow the process?” Instead, try these questions:

  • What problem were they solving that we didn’t see?
  • What sucks about our process that prevented them from wanting to follow it?
  • Is there anything about my leadership that made them want to avoid bringing their concerns to me?

Top comments (2)

Collapse
 
joelbonetr profile image
JoelBonetR 🥇

I usually agree with what you say and I think lots of us have seen the evolution of the software since at least 15 years ago, but in this one, and even though I myself was responsible for "shadow IT" in the past... nope.

I understood with the time that having a nice place to work is not only a top-down responsibility and some rules apply to all of us.

Of course the CTO won't know all the details of your daily chores and thus it's difficult to understand key improvement areas. Now if you communicate properly about these improvements and the top of the chain is not stupid... they'll give room for these improvements, and this "shadow" thingy should not happen.

If you have to wait till the next quarter for the green light so be it, we can wait we're grown ups, right? Now if you get a red light or get ignored, then it's time to search a new place to work or create your own.

As tempting as it may be, solving the issues of a company which exec board does not see value fixing, not only enables shitty behaviour at the top but also does it with you (or the team) as the responsible faces whenever it goes south.

Processes are not a one-off job but evolving, necessary parts of every organisation, we call it organisation for a reason, if everyone does whatever they feel like it's disorganisation.

To the questions above:

Are you telling me that we have absolutely ZERO licensing issues across ALL of our software and hardware today?

I can ensure you I have zero licensing issues in the projects I work with and I can ensure the same for all my maintained side-projects and hardware. it's not that crazy to accomplish, most of the time you just look at the license before using something and you're good to go.
Licenses may change but no one is doing the upgrades for you and, in case you want a cool thing to do... you can code a bot to check whether the licenses remain the same before upgrading. Instead of avoiding the process, automate it!

Are you also telling me that you're sure I haven't thought of ways to mitigate licensing risks myself?

Judging by the previous question I am sure, yes.

Now if the company was yours would you ignore licensing risks by allowing anyone to do that judgement? I won't.

Are you willing to shut down all the productivity improvements for the sake of avoiding a little paperwork?

No, I would establish guardrails and flexible processes to give room for improvement avoiding chaos. If that hurts the productivity a bit so be it, the fastest way to go fast is to go well.

I know a lot of software engineers, and giving them a challenge like "maintain this system in some language you've never used before" can be absolutely enticing if presented in the right way.

Yeah! almost got me with this one but no. Even nowadays with AI being able to close the language gap... nope. The kind of profiles that do that will leave the company in months anyway and it will be the responsibility of the rest of the team, adding to the burnout rather than anything else.


I trust in responsible leadership, giving everyone a voice, and accounting for some room to analyse the concerns, planning improvements and move towards the same goals as an organised cohesive team. I trust in accountability, including the accountability of a bad leadership, even if that means leaving a workplace you like due to other factors.

I do not trust in that everyone going rogue with what they want is acceptable and I won't buy it even with the "delivery improvements" flag, I've been involved with companies of all sizes long enough to understand this is a trap and will materialise as sticks in the wheels of the company later on.

Even I disagree with this one thanks for sharing, it was a nice read 😄

Collapse
 
linkbenjamin profile image
Ben Link

I appreciate the point of view... helps me tighten my arguments for next time, and gives me pause to think from a different perspective. It's a major part of why I write these, thanks for being part of the Adventure!

One thing I probably should have framed better is that I absolutely DO have a desire for this Shadow Work to be "above board". The goal of the post was to encourage leadership to think about how they respond to Shadow IT when it happens... because I believe it's somewhat inevitable that makers are going to make. And (especially in large corporate enterprise) they're likely going to do some things off the books. It just happens... but rather than attacking it, good leadership should say "hey that's really innovative, let me help you get this back on the rails." Your response about establishing guardrails is EXACTLY the kind of leadership I want, but too often they find something unofficial, cry foul, and have a good idea completely wrecked in a knee-jerk reaction - now the improvement is taken away from the users, the person who built it was punished for making things better, and everyone learns that it isn't safe to do something.

I trust in responsible leadership,

One common thread in my posts is that while I very frequently don't trust leadership, I DO want to help them earn my trust. I hope some of those leaders are the ones who read this, taste a bit of hyperbole, and get curious.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.