I agree with this. We can't always know how much time to put in to this kind of stuff, but it's the kind of thing we want to keep iterating on with the DEV team.
Custom tooling requires a lot of good communication because the folks who need the tools most are not necessarily the ones who have the time to build them.
Thoughts on this?
Top comments (17)
This may be a bit off topic, since the question is more about actual tooling that's worked into the dev process and maintained by the team, but this makes me think about the tiny automation scripts I write to avoid having to manually perform tedious tasks.
I'm always doing this sort of thing at work: writing userscripts to fix time-wasting bugs in corporate web applications, scripting tasks I don't want to do (or do too often), and trying to share this stuff around as much as I can. I might occasionally spend too much time on it... but I learn stuff, others sometimes benefit from it (though the majority of what I write is only ever used by me), and it's fun 😁. It's that classic hacker mindset of "why do it by hand when I can write a script to do it for me?". Even when writing the script takes longer, it's more enjoyable, and then if it ever comes up again, boom, already got the script!
For instance, last Tuesday my team lead noticed that our remote git repo was becoming clogged with old, merged branches that hadn't been deleted. He asked each of us devs to go through the branches we had created, check whether the associated tickets were closed, and clean them up. I had like 40 branches to check and clean up. I could have manually checked each and banged the git commands into the keyboard one at a time, and it probably would have taken like twenty minutes. But I decided to script the whole thing. It took me at least 20 minutes to figure that out, probably more like 30 (mostly because of checking the ticket statuses), but now I've got it done, and I have a feeling that this will come up again in a few months. And I may be able to share it around if someone else wants it.
But even if I never look at it again, it was still good practice, and I learned some neat things about our ticketing system,
gitcommands, andxargsthat will be useful anyway. And I mean, come on, that manual process would have been super boring. This was way more fun 😎💻I had exactly the same issue with old branches in a project I worked on.
It was maybe two years of development without deleting most of the merged branches.
A Dev also build a script similar to yours ;)
Hello, I'm someone :D
Ha, I was more referring to my teammates, but I can share some of it. The bit involving checking branches against our ticketing system (aka the interesting part) is very tailored and wouldn't be useful outside of my office, but the bit that actually deleted the branches is just a little
git+xargs:Investing in custom tools is good, but sometimes it can prevent people to think about changing the software/design for improvements that can't be fixed by just custom tooling 🤷♂️
So if the people agree that there is currently nothing that can be done in short term (always iterate instead of investing too much time), then %100 spend sometime now and save a lot later 👍🏽
I like your shaded answer, because I think this is our daily struggle as developper: do we invest time in "absorbing" a new entity (e.g. library), even if it does not fits 100% our needs, and work it out around, or do we put some time to create our own.
It's always a matter of trade offs and bet. The best product is where you made the best medium-long term choices in term of tools.
I would personnaly go for it if I know this is the kind of tool that will serve my needs a lot of times (like I did with my service worker library). This is a bet, maybe service workers will be superseded by another tech, but I feel in my gut it will be worth for now.
Interesting point about choices, yes especially in frontend world, it is never easy to predict, but it should be relatively easy to design for a couple of years ahead, even though you're not sure about the choice of lib/framework/tool
I love this sentiment. Nothing beats a good custom built tool!
My dream job is working for a company and building internal devtool type things to make everyone in the org faster and/or better!
I like building things, and it's awesome when the 'customer' of what I write is other developers!
Me too... in a way. I sort of know I don't really have the patience for doing this all the time.
I think my dream job is a situation where I get to work with a dedicated team that I have great rapport with. Occasionally I'd get to come in and hack on dev productivity tooling but when I'm not doing that I know there are folks working on it without cycling off to new things too often.
When my company Monsterbox was a web-dev firm churning out web-apps it was mandatory on paid time to contribute to internal tools.
We had our own project manager which generated invoices, based on associating git commits with tickets (3poch)
We had our own web-framework which was a high level dsl that allowed you to use either Angular 1 or Mithril as the engine (monster-mithril, monster-javascript)
We had our own secret tool which allowed you to generate our entire applications using a yaml file (think CloudFormation but for web-apps).
That's just to name a few.
The problem was I never found developers who actually wanted to work on these internal custom tools, even though the business outcome was extraordinary and return to developers who have been less work, larger contract, larger salaries.
Since the motivation was not there, these tools couldn't keep pace which the changing landscape of the industry, never created the documentation or the marketing effort.
They are still amazing tools but sit in my graveyard.
So I disagree with this line:
"Custom tooling requires a lot of good communication because the folks who need the tools most are not necessarily the ones who have the time to build them."
Not that good communication isn't awesome and of course we should strive for it. But the sentiment then is that you have a group of devs who are maxed out who say "I'd like tool X", and a separate team dedicated to building those tools.
In my area we've found that you need to make developers suffer the pain points and allow them the slack to go and fix them. Our mantra is "smallest thing first". If your release process sucks, improve it a tiny bit, write an internal blog about what you've done, and iterate. Other people will join in and help if they're suffering the pain and have the capacity to help. If they don't have the capacity then you need to create it for them.
Not all tools will succeed and many may get rewritten multiple times, but we're a few years down the road in our journey and have an absolute ton of custom tooling which allows the devs to go at lightening speed with minimal interference. The pay off from giving people the capacity to increment the tooling as a side-of-desk project has been many fold.
A somewhat recent episode of JavaScript Jabber had Anatoliy Zaslavskiy on as a guest and he talked about his goal of building great tools for developers as a way of having a positive impact on them.
I think it's easy to forget about all the open source tools we use every day that make our lives easier.
My apps are built with VS Code, Prettier, TSLint, npm CLI, Karma, ect... as much as they are built with Angular.
The libraries help me get work done, but the tools help me get work done without losing my mind.
I moved into a technical product management role in order to facilitate these kinds of projects across software development and infrastructure groups. There are a lot of pitfalls to making a team(s) working on this effective. Most businesses are better off focusing on their customers primarily despite not making developers as effective as they could be. I think there is a certain size and amount of developers required before this can have people dedicated to it. Otherwise the team can become a catchall for random work and not stewarded properly.
Yes! Maybe not for startups and smaller companies but for medium / big companies or projects it can bring a lot of value.
As you grow, It's natural that you will start to find more bottlenecks and very specific use cases that might not fit existing tools.
Having some people or team more focused on doing internal tools can have a great impact on the teams productivity and morale and be a great help to keep the processes running smoothly, while other people can focus on building features and delivering value to the end customer.
And can be simple things as some script to automate some boring task you have to do constantly, a chatbot to alert you of some important metric or a more complex application to manage your deploys.
I can't say I've ever worked at a place where this attitude was encouraged. I still wrote custom tooling, but I had to do it in secrecy or I'd get in trouble for over-complicating things and taking too much risk. It takes effective leadership to foster an environment where you get real innovation...I probably could have come up with some really awesome tools if I wasn't being micromanaged to death. I definitely see the risks where I can get caught up working on something that is cool but provides no discernible return on investment, but some of the more pointless projects (from my manager's perspective) taught me the most.
For me the way we build software are completely rotten, cobol has departed from fortran to make programming more accessible and inclusive and it seems things have regressed to become more and more complicated. So yes coders have to build custom tools to help them stay productive. But it doesn't really help achieve the productivity as a whole, nor automation nor "agile" process because problems are always at the boundaries of system and that is fractal: at organisation human department levels, at software architecture levels, ... at components levels. Neither UML nor BPMN (derived from UML for Business side) did a good job, but the king is still nude. So for me global productivity have still to be solved. And that's what I'm working on ;) But due to huge inertia in development field, I expect people will take between 7 and 15 years to accept my solution so I'm not pushing for it - hey when node came out, it got only 8 stars and stabbing on hackernews and it's only after about 7 years it becomes accepted :)
3 years ago while learning a bit C I build a small templating-tool, which just included files into other files with a special inclusion-syntax, and piped the result to STDOUT.
I wanted to use it to use it for building static HTML-pages for a project at work. I tried to build the front-end without npm and any bundlers. I set up some command-line-scripts for building JS, CSS and asset-files to a dist-folder.
And wooooosh it was quick! And a great learning-experience, too.
BUT: Then the need for some convenience arose. Modern JS-syntax? Babel is needed. Snippets and Layouts for the templates? Would have been pretty much work. SCSS? Have to introduce SassC. Sourcemaps? Local server with file-watcher? All of that supporting Steve, Bill and Linus? 😩
I wanted to get started with the project ASAP, not working days and days on the darn toolchain. I bit the bullet, added Yarn and replaced my custom toolchain with Node-Sass, Assemble and Babel/Webpack. No more cross-platform-headaches. All the features.
Yes the build-time wasn't sub-second anymore, but still around 5 seconds after 3 years, so I can cope.
That's why Angular-CLI, CRA etc. are so good: You get started quicker. They are battle-tested and cross-platform.
I want to work on the product. Others will rather do tools for developers, which is awesome! I'm not that person.