DEV Community: Leon Adato

Software migration: Pain vs Risk

Leon Adato — Mon, 07 Sep 2026 14:00:00 +0000

“And the day came when the risk to remain tight in a bud was more painful than the risk it took to blossom.” – Anais Nin

Over the course of my career, I’ve experienced my fair share of software upgrades, migrations, and replacements. Some have gone smoothly. Many have not. But none of them could be described as “easy”. The reason none were is is because, to paraphrase a famous sword-swinging Spaniard, “…that word doesn’t mean what you (non-IT people) think it means.”

In tech, and especially on the subject of system migrations, “easy” connotes “without significant unexpected issues”. It rarely – possibly NEVER – means “without effort.”

Nevertheless, the suggestion to upgrade (or migrate, or replace) is often thrown around by certain folks (usually business leaders and vendor sales) without a trace of understanding of what that truly entails. As a boots-on-the-ground IT practitioner, hearing it used in this way is nothing short of infuriating.

Upgrades are such a monumental task that teams will go to almost any length to avoid them unless absolutely necessary. “Almost any length?” you ask. A recent LinkedIn post* illustrates just how bad it has to be for a customer to consider migrating:

* I recognize readers might read this particular post and read into it a certain venting of grievances on my part, as a former SolarWinds employee. I want to be clear (and I spell it out below) that this is not the case because MANY vendors are making changes to pricing, causing MANY customer to feel similarly to the original poster. This post is just the most recent one to have crossed my timeline.

SolarWinds, a major player in the technology infrastructure monitoring space, is attempting to force our company into a three-year deal that represents a 543% price increase from our 2023 costs. To put this in perspective. they’re charging 5.6 times the inflation rate. […] I’m now being asked to pay more to monitor our company’s infrastructure than I pay to support that infrastructure itself.

If there is a lesson in this that avoids both showcasing the pain of others (in this case, Mr. DeMersseman) or pointing fingers at any specific vendor (because SolarWinds isn’t the only culprit here by a long shot), it’s that customers – despite the lack of meaningful updates or new capabilities; despite the jacked-up pricing and removal of flexibility; despite almost open hostility from the vendor C-suites – despite all of that and more, STILL haven’t migrated to another tool.

Why? Why would they do that? Because migrating is HARD. In this blog I want to outline just how much risk there is in “blossoming” (as a reference to the quote at the top of this post, by which I mean “migrating to something new”). So much that customers are willing to tolerate this level of pain to remain “tightly within the bud” of the current monitoring solution. I want to outline exactly what you* are asking of your team, when you are being courted by a new software vendor who is promising they can do everything the incumbent solution does (and more, and better) if only you’d switch to their solution. Making matters worse, you* are probably laboring under the belief that “we have too many tools already” (which is, to be honest, probably not entirely wrong.) and so you’ll only consider “…putting a brick in if we can also take a brick (or 2, or 3) out.”

* I’m assuming that YOU, dear reader, are part of the decision makers who buy the software. If you’re not, then I’m writing this to be shared with said decision makers. You’re welcome.

Let’s break down exactly what a software migration looks like – at least in my experience and in my not-so-humble opinion.

First, let’s be clear: I’m talking about the experience of an boots-on-the-ground IT practitioner who has found a tool that does something both important and essential. So much so that they’re willing to attempt the “business justification two-step”.

Imagine being that regular IT pro walking into a room of executives asking questions like the ones Keith Townsend (CTO Advisor) describes in this LinkedIn post. It’s not fun.

Tech people hate (and therefore usually suck at) speaking in terms of the business, so these types of meetings usually devolve into the IT person passionately citing technical stats until the executive shuts them down or caves. For the sake of argument, let’s pretend they cave (spoiler: they never do).

I’ll also remind readers that “I’m adopting a new tool” isn’t seen by most managers as a hall pass to get out of doing your regular job (which is typically 3-jobs-in-a-trench-coat that consumes 50+ hours a week).

The ADDITIONAL work on top of the normal work includes:

Creating the forms for a largely un-necessary RFP.
Working with companies who don’t have a free tier to get a free trial going without making it sound like buying is a sure thing.
Apologizing to the CFO when an over-enthusiastic salesperson skips 2 or 3 levels and calls them directly to see if they can “push” the sale through.
- Yes, that happens. 2 different monitoring solution vendors have done it to me.
Begging for cash and scrounging spare parts to build a lab because nobody will (justifiably) let you test in production.
Setting up the lab yourself. Even the people who would benefit from the new tool usually can’t help because – SURPRISE – they are doing 3-jobs-in-a-trenchcoat too.
Doing the RFP – installing and configuring software for at least 3 companies plus the incumbent and doing detailed comparisons in an attempt to get an apples-to-apples understanding.

But, ok, that’s ANY migration. So let’s wave our magic wand again say we get past the WEEKS (if not months) of effort. AND we’ll wave our wands yet again to ignore the weeks of back and forth with purchasing, who won’t understand what the tool does and nor they think the company needs it because it’s “just another piece of software.”

Congratulations! You’re magically past those ulcer-inducing stages. NOW you – the beleaguered IT person – are on the hook to:

Inventory all the systems that will be part of the pilot program.
Set up a QA environment for the new software, against the QA environment of the new application.
Explain to the vendor that you actually need TWO copies of the software, not one, and no you don’t have budget to pay for it twice, you need a free copy because it’s just QA.

Once again, we’ll pretend the vendor just say “duhhhh okay!” (spoiler: they rarely say that). Next on the list is to:

Install and configure the new vendor’s software.
Add the QA environment systems into the environment.
Manage and monitor the test, showing results to the application owners to prove your new software won’t destroy their app.

Once you’ve passed those items, the work is not done. You now have to:

Set up the new vendor’s software in the production system.
Add (not migrate) the application(s) to the new system.
Run the new system in parallel with the old one for a number of weeks, reporting comparative results to everyone up to the CFO’s father’s, brother’s, nephew’s, cousin’s, former roommate about how it’s going.
After that period, you can start to turn off (not delete, just turn off) the old vendor’s software. Note that your company is still PAYING for the old software, so exactly ZERO savings have been achieved. Try explaining THAT to the CFO.
During all of this, you are also responsible for re-creating all of the tests, alerts, reports, and dashboards in the old system that people now “desperately need” (even though they haven’t touched them in 2 years.)

Scroll back and review all those bullets one more time. Ask yourself what it would take to make you want to do that on top of your existing work. Most IT folks go through that kind of experience ONCE, often because they don’t realize what they’re in for. After that, most learn their lesson and stop asking about (or for) new tools because very little would be worth all (or any) of that.

“But Leon, companies DO aquire new tools. All the time. How do you explain THAT?”

Final spoiler: We wait for something to explode. We wait for a system we know is unstable to experience a surface-of-the-sun hot nuclear melt down. Because at that point, all the hurdles and bullshit clear away. The executives who were so picky before are now screaming bloody murder. They’ll do ANYTHING, agree to ANYTHING, to make sure that EXACT PRECISE problem never happens again,

Then and only then can a tool be adopted quickly.

This is why we can’t have nice things. Or at least, this is why IT folks don’t jump up and down with joy when you suggest they migrate to a new monitoring tool, despite their frustration with the current one.

So don’t be surprised when they don’t.

Cribl: Telemetry Gatherers

Leon Adato — Mon, 31 Aug 2026 17:00:00 +0000

(CriblCon is a month away, so I wanted to share a post - NOT about Cribl, or telemetry, or tech - but rather about the ways we IT folks sometimes find to combine whimsy and IT)

Honestly, it started off as a silly, wild-assed idea.

"Wouldn't it be funny if we made some Magic the Gathering cards that use Cribl products as spells, and maybe some of the problems our users have as monsters! We could give them out for a laugh in the booth. “

The problem - if you can really call it a problem - is that everyone at Cribl is really proud of our geeky heritage, and that includes MTG. We couldn't just joke about that idea. We had to jump in with both feet, burning our evening hours creating an ENTIRE FSCKING PLAY-ABLE SET. With boosters. And specialty cards. And counters. And… stuff.

From that initial silly wild-ass idea, things just snowballed. We quickly realized Cribl products (and not JUST products - other aspects of the Cribl solution) weren't spells. They were artifacts! Lands! Enchantments! We had to come up with mechanics for pipelines and replay. And we needed to create artwork for the cards.

SO. MUCH. ARTWORK.

To speed things up, Justin des Lauriers (once of our engineers, of course) created a whole-ass Gemini "gem" that would streamline creating card mechanics from a simple prompt.

And of course, once we had them, we had to playtest them. A lot. In multiple sessions that ran long into the night. Because we care. About authenticity. About game balance. But mostly, we care about maintaining the highest level of geek credibility we can have, and if it means forgoing sleep or proper hygiene, so be it!

We had a metric crap-tonne (that is a very technical measurement, by the way) of fun.

But as we played, we realized these cards could also teach people how Cribl did.

It might not be as good as Cribl University (OK, it definitely isn’t. Check them out at https://cribl.io/university/) but if you want to get some concepts in your bones in a fun and off-beat way, you could do a lot worse.

This should come as no surprise to anyone who has taught professionally (massive props to all the teachers out there!). It’s been clear for a long time that we all - adults as well as kids - learn better and retain information longer when it’s presented and experienced as a form of “play”. We sing the ABC song to memorize the alphabet. We have blocks, legos, and erector sets to learn spacial skills. We play dodgeball to learn to avoid flying wrenches.

And now we can tell our loved ones that we have to play Magic the Gathering in order to build our skills in Cribl and uplevel our career. Which will earn us a raise. With which we can buy more MTG cards. It’s the circle of life, but for nerds.

My point in all of this is two fold:

You absolutely positively need Cribl’s Magic the Gathering card additions in your deck.
Having these cards indisputably proves your credibility as an IT practitioner, a Cribl expert, and a geek.

Where can you find them?

We’ll be handing them out at conferences, CriblConnects, and of course CRIBLCONNNNNNNN! We will be looking for even more ways to share the MTG joy with the world in upcoming weeks and months.

Monitoring versu… I mean AND Observability

Leon Adato — Mon, 31 Aug 2026 14:00:00 +0000

Chapter 3 of the O’Reilly book I’m co-authoring with Kentik CEO Avi Freedman is out today (you can download it here: https://www.kentik.com/go/ebook/oreilly-guide-to-modern-networking-telemetry/ and I wanted to share a bit that didn’t make it through the editing process, because even with the eBook edition, space (and reader attention) isn’t infinite.

Nevertheless, I thought the longer version was worth a look, for some of the points it made.

In the book, we wrote:

In blog articles you’ll see the phrase “reason about”, as in “its more data than one person can reason about”, which is a fancy way of saying “keep it all in your head enough to be able to find patterns”. I mention this because one differentiations between “observability” from “monitoring” is not just the sheer volume of data, but also the disparity of that day that makes it difficult, if not impossible, to reason about it.

But here’s the longer version that we had to cut down:

In order to really appreciate how Machine Learning (ML) and AI can improve observability, it’s necessary to (briefly) differentiate between it (meaning: “observability”) and “monitoring”.

Monitoring is mostly concerned with “known unknowns”, by which I mean the things we know COULD potentially fail, but we don’t know WHEN. It’s comparatively trivial to understand what data needs to be collected, reported, and alerted on because we know where to look. Monitoring isn’t concerned with how unique the data is (i.e. “cardinality”) – whether we’re talking about an individual data point or a collection/grouping/slice of that data. And because of the two previous points (known unknowns and cardinality), correlation is usually manual – meaning humans who understand the combination of elements that indicate something is wrong are needed to craft the trigger for an alert or other action.

Observability, on the other hand, is more biased toward “unknown unknowns” – we don’t know what could go wrong and we also have no idea when that failure might occur. That ties into observability’s focus on “high cardinality” data – when specific data points or various permutations have never (or rarely) been observed. To achieve this, massive amounts of data need to be collected, normalized, parsed, and pivoted to find those unique combinations. And because of those two things (unknown unknowns and cardinality), correlation cannot be trusted to a human. There is simply too much data, and too many possible combinations of that data, to “reason about” it (to use phrase popularized by Charity Majors and Liz Fong-Jones).

This is a comparison I make often when I’m giving conference talks – NOT because I think I have the One True Definition (far from it) but rather because I want folks to have the proper context.

One additional point I make on both sides is “signals” – the data elements used to determine if the object of monitoring is healthy or not.

In “traditional” monitoring, signals are domain-specific. The data points that tell you a network device is not performing well are measurably different from those that indicate a database is unhappy. Or a block of code. Or even a wifi environment.

Meanwhile the philosophy of observability tries to standardize on the “Four Golden Signals” as established in the SRE Handbook – namely Latency, Traffice, Errors, and Saturation.

To be clear: the title of this blog is a misnomer. It’s not (and never was) “versus”. Monitoring and Observability should be seen as a BOTH-AND rather than EITHER-OR proposition.

With all that said, here’s a screenshot of the slide I use in my presentations, which lays it out (without the talk track). I hope you find it useful.

HOW TO DEVREL: Content Funnel Done Right

Leon Adato — Mon, 24 Aug 2026 14:00:00 +0000

DISCLAIMER: No, I don’t believe I’m the universe’s gift to DevRel. I don’t have all the answers, all the skills, or even all the Pokemon. What I have is my experiences, collected over the 11 years I’ve been doing this work. That – my experiences and observations – are what I’m sharing in this series.

Sales. Demand Generation. Product Marketing. Technical Marketing. Plain old MARKETING-Marketing.

And then there’s Developer Relations.

Depending on the company, the leadership, and sometimes the season, DevRel might be slotted into any one of those silos. Or more than one. Or all of them. For my money, it kind of doesn’t matter – at least not nearly as much as how the functions of a DevRel advocate fit into the overall conversation with the target audience, and how those conversational moments are carried over from one to the other.

Maybe Demand Gen sets the direction, and Product Marketing (PMM), Marketing, and Sales follow their lead.

Maybe Marketing creates the message and the other groups build on it.

Maybe… well, maybe a lot of things. But I’ve been doing this a while and (with the caveat at the top of this post firmly in mind) I’ve hit upon a structure that allows each group to play to its strengths and more importantly creates a consistent and coherent “thread” of conversations with customers, prospects, and users.

Top of Funnel: DevRel

At the top of the conversational funnel – meaning the broadest approach, the most varied set of topics, and (perhaps most importantly) the most vendor-agnostic type of conversation – sits DevRel. Their job – my job (our job if you’re also in DevRel) – is to talk about the problems, issues, and experiences faced by IT practitioners.

The goal at this stage is to tell a story (not a fiction, but rather a narrative of an experience) that is so credible, so real-world, so on-the-nose that the reader has an almost visceral reaction:

OH MY DOG, that was ME last week! Were you looking in the window of my office?!? What did you do next????

At that point, I might simply focus on the problem, leaving the solution up in the air. I’ve written about this before as part of the Texas Sharpshooter technique.

Or I might write about a an actual solution, putting it into the most vendor-agnostic terms. Even if the screenshots show my employer’s software, there’s a bit of latitude because my toolbox has to have SOMETHING in it, and these are just the tools I happen to have at my disposal. The key is to talk about the problem, the solution, and the techniques without mentioning the tool.

Same Story, More Detail: PMM

Product Marketing Managers are – in my (not so) humble opinion – close relatives to the evangelist / developer relations advocate role described in the previous section. They should have just as much technical acumen, and should understand the problem space and the user personas just as well.

In a different world, the PMM could write the same story as the evangelist. In fact, that’s exactly what they SHOULD do, with one tweak.

“Last week, Leon wrote about (some particular problem). He brought up a lot of good points, but he wasn’t exactly specific about how to go about solving it in a particular tool – OUR tool, in fact. So today I want to revisit that blog but offer up some advice on how you can fix this issue, in XYZ’s software. Let’s get started…”

Presuming the reader is coming to this blog having read the first one (or if they take a moment to read it now), the PMM has effectively earned the privilege to be vendor-specific. By virtue of their click, the reader has bought into the idea that THIS blog (compared to the last one) is going to talk about a product, feature, or module.

Offer More Value: Demand Gen

So “Demand Generation” (or just “Demand Gen”) is, as the name would imply, the group that finds ways to get people into the sales funnel. While part of that job lies in promoting the content generated by the previous two sections, another action they can take is to track the folks who interact with those pieces and reach out to them directly, offering – no, NOT a sales pitch – but rather another “gift of knowledge”. That might be a whitepaper, related blog post or video, an ebook, an upcoming webinar, or something similar.

It builds on the understanding that

“You’ve engaged with us on a specific topic, here are some additional ideas, use-cases, and solutions that we help you with.”

At this stage, Demand Gen is both broadening the conversation and narrowing the funnel – helping the audience to self-select whether they are only interested in a single specific problem or use-case, or if their needs extend beyond that, into the realm of things a company would pay money to fix, if a solution was available.

Close The Conversation: Sales

“Sales”, as a concept, doesn’t require much elaboration. What I want to emphasize is that, as the final stage of the previous sequence, it is a lot less intrusive and a lot more natural than other types of sales interactions.

If the reader/audience has been engaged to this point, a call or email from sales makes sense. There’s a context to the conversation and, once again, the individual has more or less agreed that this is the next logical step. There’s no guarantee, of course, but there’s much less time that has to be spent on what the company does, what problems the prospective user is having, and so on. That all is a known quantity, because it was validated in the previous stages of this funnel.

The Mostly Unnecessary Summary

This framework is, (somewhat) obviously, not fool proof. It’s not the One True Sales Funnel that will win you the client every time. It is – much like the software your company sells – another valuable tool in your tool belt.

But if you believe in speaking to users with an authentic, human voice; in telling stories that are problem-forward and put the user’s needs first, then it’s a framework you probably ought to consider.

HOW TO DEVREL: The Most Un-Natural Act

Leon Adato — Mon, 17 Aug 2026 04:00:00 +0000

Before this blog gets de-listed for potentially purile content, let me explain up-front what “un-natural act” I’m referring to:

Getting conference attendees to try your software during the conference

Yeah, THAT un-natural act. Like… you’re at a conference, enjoying the talks, meeting new people, and having a great time. You decide to take a swing through the vendor space, mostly to see what combination of socks, t-shirts, pens, and charging cables you can snag for free.

Stopping at a booth, you’re chatted up by one of the staff, and they say… something clearly compelling. It might even border on mind-control. Your response is to pull out your laptop, head to the vendor’s website, start a free trial, and play around with their software.

This, my friends, is an absolute fever dream. I’m sorry, but I’m not giving up my time (to say nothing of my anonymity) for “a chance to win this Lego Millennium Falcon”. Anyone who thinks this would work has not hung around humans – and specifically humans working in the tech sector – enough. Short of something like “and you’ll immediately get a Lamborghini” (or 1 million dollars, or a lifetime supply of delicious chocolate cake), there’s nothing I can imagine a vendor saying to me that would get anyone to ACTUALLY do the deed.

Opening scroll

Or can you? It turns out, I’ve found a way to do just that. In fact, this technique – tested over 3 years at ~20 different events – consistently delivered a sign-up rate of 10% of the overall attendees. So if the event has 300 people, this technique got me ~30 sign-ups. But at a 3,000 person event, we saw 300 sign ups. Because you probably work in tech and love data, I will admit that there’s an upper-bound at around 5,000 people. Going to re:Invent (about 60,000 people) did NOT garner 6,000 sign ups. However I attribute that to the fact that all 60,000 people didn’t swing by the booth.

But the point stands – this technique gets a non-zero number of folks to take a moment, RIGHT NOW, and sign up. If you’ve been working the booth at conferences for any amount of time, you know that’s worth a lot of $5-a-pair branded aviator glasses.

Level 1: Make it a game

The first thing I did was take my company’s software (remember, I work in monitoring and observability) and incorporated into a game. I found a existing 90’s style side-scroller

and instrumented it with additional monitoring that collected everything from CPU, RAM, and network stats to the number of gems collected, the top level achieved, the speed of movement, etc.

Level 2: Make the back-end visible

Then I took all those stats and built a dashboard around it.

Level 3: Let attendees play AND see!

I built a portable display that allowed someone to play the game AND ALSO see the dashboard in my company’s portal that showed all the monitoring statistics.

CUT SCENE – in-booth discussions

Once this is up and running in the conference booth, people will naturally gravitated to it. A game display is just too enticing NOT to stop and check out. It allowed me to not only invite people to play (and even compete against each other) but I also used the game as a springboard for conversations – describing HOW each of the game elements was instrumented, collected, and displayed, which naturally led to discussing real-world use cases that were closer to home for the attendee.

The best part was that, because the example I used was a game, folks didn’t get hung up on “that’s not like my business” but also it got past the “what are you trying to sell me” resistance that everyone (myself included) feels when passing by a vendor booth at a conference.

But it gets much better than that.

Level 4: Pack it up

Packaging everything up to use in the booth is one thing. But what if you made it so the users THEMSELVES could download the game, pre-instrumented, and load up the dashboard in your portal?

That’s exactly what I did. First I made everything as simple as possible – bundling the game AND monitoring agent into a single downloadable bundle.

Level 4: Provide a walk-through

Next, I wrote up step-by-step instructions that talked folks through:

creating a free account in my company’s portal
downloading the game-and-monitoring bundle
making any configuration changes to connect the game bundle to the free account

The goal wasn’t to make any of this a guessing game or a quiz. If sample configs were needed, I gave them. If there were special add-ons needed for a specific operating system, I provided links. I wanted players to have the easiest, smoothest, fastest set-up experience possible.

Level 5: Offer the Quest.

After seeing the game in the booth, I would offer attendees the opportunity to download it for themselves and set up the exact same dashboard. That way they could enjoy the game AND see the real software (mine) in action.

But this, in fact, is the heart of the blog. THIS is the “most un-natural act”. Why on earth would they DO that??

Level 6: Collect Those Gems!

If you’ve played any game ever, you know that hard work needs to be rewarded. I offered a prize to anyone who came back to the booth and showed me the installed game with the dashboard up and running. The prize was significant. Not “here’s a PS5” significant, but compelling. Over the years, I’ve handed out everything from Ember mugs to Raspberry Pis to light-up LED D&D dice.

What I found is that the thing that compelled attendees to download and try the software wasn’t the expensiveness of the give-away, but the “cool” factor along with the guarantee that they’d get it. Nobody’s going to jump through hoops at a busy tech conference for a raffle ticket.

And that’s where the “10% of attendees” part fits in. That’s how many people, on average, were willing to take time (usually doing it while they were listening to one of the conference talks) and download the game bundle, set up the free account, and do the (minor, easy) configuration to connect the two things.

Bonus Level: Beat the BFG

We also had a big give away (yes, sometimes if actually was a PS5 or XBox), but I’m not a big fan of raffles. So we made it part of the entire engagement strategy.

IF the attendee downloaded the game bundle and set up the dashboard…

AND IF they showed us the dashboard (and got the cool prize).

AND IF if they actually played the game during the event, and had the highest score at the end,

THEN They’d win the prize.

How would we know? Their score in the dashboard, of course.

Hidden Level: The Kobiashi Maru

What I discovered the very first time I did this activity was that I needed TWO big prizes.

One for the person who played by the rules, who got the high score fair and square.

And one for the inevitable group of folks who found a way to tweak and change something to create an impossibly high score. Maybe they adjusted the math on the dashboard display. Maybe they added a multiplier in the data collection process. Maybe both.

Why? Why would I reward a cheater?

Let me reframe that for you:

What would you have to give someone at a conference to download a program, set up your software and a dashboard, and then EDIT AND MODIFY that setup to do something different? What’s the likelihood that ANYONE you asked for that kind of investment of time would do anything except walk away (possibly after laughing in your face)?

But here, someone did all of that without you even asking. In marketing and sales terms, this makes them the most qualified of leads.

The End Credits

The core of this idea is that I made a relatively boring, work-intensive piece of software more fun and engaging by associating it with something that was already fun and engaging, and then structuring the handouts and give-aways in such a way that rewarded the behaviors that helped me.

It’s not exactly an epic action that can only be done when you’ve collected 1,000 mana points. But, based on the fact that I’ve seen precious few other vendors do anything like this, it’s still one of those little known power moves that anyone could capitalize on without being seen as derivative.

Punchline, or Gut-Punch?

Leon Adato — Mon, 10 Aug 2026 14:00:00 +0000

I was excited and also humbled to be given the chance to deliver the closing keynote talk at PyOhio this year. You can see the video of that talk here. But a conversation afterward offered me a chance to reconsider and – hopefully – improve both myself and that talk for the future.

I set up a particular joke early in the talk: After establishing that “there really aren’t any poor technical choices”, I go on to qualify that statement with examples of demonstrably bad technical choices – things like not having backups, testing in production, etc. In that list I include the punchline:

Using Oracle for literally anything, ever.

Near the end of the talk I offer another list, this time technical decisions that sometimes leave us paralyzed with indecision. Near the end I ironically include the same line – “using Oracle for literally anything, ever”.

Both lines never fail to get a laugh from the audience. First, because any sufficiently large, complex, storied, and difficult to manage technology would fit the bill. I could have chosen from a range of programming languages, platforms, or solutions and gotten almost the same type of laugh. But second, because Oracle – the company, and certainly it’s CEO – have made itself an easy target.

And look, it’s a good joke, ok?

What I failed to consider is that plenty of thoughtful, dedicated, committed IT folks work with Oracle every day. Some might have had taken jobs that covered a wide range of solutions, Oracle included. And some might have seen where the money was – that internally the big projects and big funding was going to an Oracle-based project – and decided to go in that direction because of the potential for promotion, bonuses, and stability.

And in that moment, anyone in the audience who’d made that choice found themselves in an unintentional spotlight. Because, without my meaning to, I’d made them and their work the butt of a joke. At best, they now had to deal with folks winking at them and saying “using Oracle for literally anything, amirite?!?”. At worst, they had to explain why they had made a “poor technical choice”.

Moreover, unlike any of the other technologies I mention in that talk, I hadn’t offered an “out”. Every other poor technical choice is presented in a “That’s bad! No that’s good!” type of framing.

I wasn’t aware of any of this, although I’d given the talk several times, until someone at PyOhio stepped forward. Through the Code of Conduct grievance process (and massive props to the PyOhio team for having this process, and taking it seriously), they registered their discomfort, both in the moment and afterward. They related that, having built a career around their expertise with Oracle, they found themselves uncomfortable and conflicted about how to react during my talk. And after the talk was over they were uncertain about how (or whether) to mention their work to other attendees.

They asked the organizers to ask me to consider my talk in from their perspective, and reconsider how willing I was to put other people in the same situation, all for the sake of a momentary laugh.

It’s easy – too easy – to write off this one person as being overly sensitive, unable to take a joke, too self-absorbed or self-important to laugh at themselves. Upon even a moment’s reflection, it should be obvious that this would be both defensive and condescending. Who are we to normalize cruelty just or judge someone else’s response? “Nobody can take a joke any more.” Is (and, let’s be honest, always has been) the response of those who don’t want to bear responsibility for their words.

The truth is the person who filed the complaint is the epitome of bravery. It takes strength of character and conviction to stand up and tell someone that what they’re doing is wrong. I applaud anyone who takes that kind of risk. I’m deeply grateful to the person who did so in this case, even though I’ll never know their name.

SO… where does that leave me and my punchline? To be honest, I’m still not 100% sure. On the one hand, I could just modify the line to say “Oracle corporation”, separating the solution from the Law-firm-with-a-small-software-division that is the Oracle business. I could go searching from a different target that’s so old nobody would be offended – like Perl or XML. Or something that’s likely to be universally reviled – like Windows 8. Or I could cut the punchline out entirely. A conference talk doesn’t need to be a thinly veiled standup routine.

So I’m not sure of HOW it will change, but I’m absolutely certain that it WILL change. I was given feedback that was specific, personal, and urgent. To ignore it would be more than just a poor technical choice. It would be a poor choice as a human being.

HOW TO DEVREL: The 10x DevRel

Leon Adato — Mon, 03 Aug 2026 14:00:00 +0000

If you’ve been anywhere near the coding side of tech within the last 8 or so years (and that includes just sitting with a programmer in the lunch room) you have probably heard of the “10x Engineer”.

While the phrase is well known, the meaning remains somewhat murky. Does it mean an engineer who can develop 10x the volume of code? Develop an equal amount of code 10x faster? Create code that’s 10x higher quality? Or does 10x the number of functions?

The IMPLICATION is that this is a person who can provide 10x the “value” (however you personally define it) compared to a regular run of the mill software engineer. It builds on an earlier version of the same trope – the Rock Star Engineer. An equally impossible bar which itself builds on a more realistic observation: we’ve all worked with certain folks who just seem to have it all together. Who know what they’re doing, make fewer mistakes, and knock out the work.

The problem is that nobody is like that all the time. The people we remember as being brilliant made their fair share of mistakes – even on the days we thought they were knocking it out of the park. Rock stars hit bad notes sometimes. Sports legends miss the ball sometimes. 10x engineers get stuck.

More than sometimes, all of those folks have the same experiences we do. A lot.

So if they’re just regular mortals like us, what’s behind the myth?

On one hand, I think it has something to do with what those folks do NEXT. Having made the mistake, found their code to be buggy, realized they have no idea how to proceed, etc… they keep going. They ask for help. They recognize (and even embrace) the opportunity to learn. Hell, the best engineers I know revel in the opportunity to learn something new, or even learn a new way to do an old thing.

But there’s ONE aspect of the truly amazing folks that does qualify them as “10x”.

My friend Yechiel Kalmenson put it best in a blog post back in 2019

“A real 10x Engineer is an engineer who can boost the productivity of their team 10x. An engineer who mentors, uplifts, shares their knowledge, and ultimately brings ten other engineers up to their level.”

The only realistic way to get 10x more out of someone is to give them the space to replicate what they do in others. To allow their spark to ignite the same passion and drive in the people around them.

And so I finally come to the topic in the tile of this blog: the 10x DevRel.

Having read this far, it’s self-evident that even the most experienced, gifted, driven Developer Relations Advocate can’t write 10x more posts, record 10x more videos, engage 10x more audiences, build 10x more communities, run 10x more hackathons, and so on.

BUT… unlike the software engineers I described at the beginning of this post, DevRel folks are not usually surrounded by teams of other DevRel folks. There aren’t multiple teams of DevRels working in (and on) various aspects of the company’s offerings. In many cases, the DevRel you meet is the only one at their company. They are the proverbial “army of one”.

So how could it be possible for that lone voice in the wilderness to level up their game?

It starts by recognizing that everyone is the star of their own movie, everyone has a story to tell, and many of those folks would love a chance to tell that story but they don’t know how to get past the barriers in their way.

Those barriers might be workload, or language, or confidence, or technical skills, or simply not having a platform.

And our job as Advocates is to do internally what we already do for our customers: to listen to their stories. To encourage and uplift them. To help them recognize that their story is, in fact, worth telling.

And then to help fill in the gaps for them so that they can tell their story in a way, in a place, and at a time that enables them to shine.

Too much work? The Advocate can (and should) go to leadership and help explain how that story would attract new business. Is language an issue? The Advocate should encourage their colleage to tell their story in their words. The world is a multi-lingual place. The person doesn’t feel confident in their abilities? It’s the Advocates place to provide whatever support they need – co-authoring, co-speaking, editing, coaching, brainstorming, or simply offering an honest reflection of the work, but also promising not to stop helping until the story is ready to be told.

A 10x DevRel is the person who sees the stories in the people around them, and then creates avenues for those new storytellers to speak their truth.

A 10x DevRel is someone who takes the privileges they have – the credibility, the opportunity, the experience, the access – and uses it to open doors, not for themselves, but to then stand back and let those new voices step through and into the light.

So my question to you (if you’re a DevRel) isn’t “Are you a 10x DevRel”. My question is: “Why AREN’T you a 10x DevRel?”

All Bait and No Fish

Leon Adato — Mon, 03 Aug 2026 04:00:00 +0000

I’ve been helping a friend out with their job search for a couple of weeks, and got an email back from them the other day:

I applied for a few more with no response still. At this point it feels like throwing out a fishing line with no bait.

I wanted to share some of my responses to them, in case you are feeling the same way. A lot of it echos things I’ve posted in the past, and I will link back to those earlier blogs when appropriate.

First, to re-state what I said yesterday: Job hunting (especially these days) is a process that takes months, and your first response could take weeks, not days. I don’t like it, you don’t have to like it, but that’s the state of things irrespective of our feelings about it. The only way I’ve found to get past this is to just keep applying. The more things I shove into my end of the pipe, the more likely something comes out the end.

SECOND – and again this is one of those “especially these days” things – the effort you put into a job application can be (not “is”, it’s not guaranteed, but it definitely helps and certainly doesn’t hurt) directly proportional to the value of the result.

At one of the spectrum is LinkedIn’s “easy apply” button. It takes almost no effort at all, just a single click. And that’s just about the value you’ll get out of it: not much. You’re certainly not likely to get a callback, and even more likely the entire thing is just a process to feed the AI machines involved.

At the OTHER end of the spectrum is maximum effort: You find a job posting and you first research the company to see what they’re all about. Then you look through LinkedIn to see if you know someone who is there. Let’s say you don’t. You then dig deeper to see if you can find people at the company who have professional interests that match up with yours. You reach out to them and ask, not for a job, but for a conversation. You just want to hear what the company is about. The culture. The strengths and weaknesses. (assuming they agree) You use that conversation to get that person to know you – not as a resume, but as a person. After 1 or 2 conversations about the company, the work, etc you mention the job posting and see if that person is willing to make an internal referral. If they are, you have a solid leg up on the process. It’s STILL not a sure-thing, but it’s better than just sending in your resume and hoping for the best.

Now that (the second option) is A LOT of work. You can’t do that for every application. So how do you choose? You do some work. You look at the company (and the role) and see if it’s a place where you not only feel like you can do the work, but that you would WANT to be part of that organization. You believe in what they make, what they do, their mission in the world. And if those things are true, you make the extra effort and do everything you can to ensure your application has the best possible chance.

SO TO SUMMARIZE

You probably won’t hear responses from any job application for 2-3 weeks (and sometimes 5-6 weeks).
Having an inside contact at the company is a massive boost in the likelihood you’ll move to the next step
This is a volume game – you have to throw 100 resumes into the pipe to hear back from 1 or 2.
NONE of this is fun, nice, or enjoyable. It also isn’t the way tech has always worked. It’s also probably not the way it will always be

But it is the way it is now, and we have to work with what we’ve got.

Knowing What’s Under the Hood Helps

Leon Adato — Mon, 27 Jul 2026 14:00:00 +0000

I’m not a car guy. I can drive them, but I don’t know the first thing about fixing them, and not much about maintaining them. My son, on the other hand – who’s getting a degree in mechanical engineering – is most definitely a car guy. He’s learning a lot about cars, and he’s teaching me a lot about what I’ve been doing wrong, even with simple things like maintenance.

I was thinking a lot about that as chapter 2 of my O’Reilly book, titled “Wrangling Telemetry” was being written. What book, you ask?

A long, long time ago (it was April 21) in a blog post far, far away (you can find it here) I wrote about the O’Reilly book that Avi Freedman and I are writing about Network Observability.

The book title is “Practical Guide to Modern Networking Telemetry”. The raw, “lightly edited”, somewhat unexpurgated version of chapter 2 is available today.

You can find it here: https://www.kentik.com/go/ebook/oreilly-guide-to-modern-networking-telemetry/.

I will briefly point out that it’s REALLY worth your time to download this copy because the final version will be about 120-150 pages (those O’Reilly editors don’t work for free!), but the raw version clocks in at 82 pages, and that’s just up to the end of chapter 2. So you really are getting a whole lot of extra content right now – content that will be lost to what looks to be a machete-and-chainsaw editing process later this year.

But if those 80+ pages aren’t enough for you – if you want a little more behind-the-scenes extra content to close our your week – I wanted to expand on just one idea that I throw out there in the first couple of pages of chapter 2.

On page 42 I wrote:

“Yes, it’s true that vendor solutions do a lot of heavy lifting behind the scenes. But this book is about exposing and elaborating on what that behind-the-scenes activity is. Part of the reason is so you can be a more informed customer. Knowing what happens inside the black box of observability solutions, HOW you get from “data comes in the left side” to “pretty charts come out the right side” allows you to make educated decisions about everything from OpenTelemetry methodology to aggregation options to pipeline management.

No matter how well the amazing folks at O’Reilly edit down chapter 2, the question remains: “Why should I read an entire chapter about how the observability sausage is made? Why do I care about enrichment, normalization, pipelines, data storage, and the rest. Like Avi and Leon SAY, RIGHT THERE IN THE BOOK, the vendor handles that shi…tuff.”

Let’s go back to my car analogy. It seems I’ve been spending a lot – too much – on oil changes. I thought that natural oil was better than synthetic. It was also significantly cheaper. With his experience on two different teams that compete in Baja SAE he’s learned a lot. And – without getting into the nitty gritty (because I still don’t understand all of it, and this is a blog about observability not oil) the synthetic oil is better for the cars we have, and will last longer, and overall work better.

That’s information I didn’t have because I didn’t choose to think about how my car operated on more than a surface level. I was content to let the vendor (the quicky-oil-change place) make my choice for me, and to make decisions based on price rather than effectiveness.

I find – especially when it comes to monitoring and observability – that IT practitioners can fall into the same trap. We already have too much on our plate. We already carry a huge cognitive load. Learning about the internal process of a tool that – let’s be brutally honest for a moment – is often seen as little more than something we do to satisfy an audit doesn’t seem like a good use of our time.

But – like that oil change analogy – it can have massive long-term impact. Understanding how your monitoring tools “wrangle” the data can

help you avoid cost overruns
understand why data doesn’t always match (what you expect or what other tools are telling you)
use your tools both effectively and for the right use cases
avoid wasting money replacing tools “that suck” when in fact they don’t suck, you’ve just been using them incorrectly.
and more.

With all of that said, I don’t believe that knowing how to wrangle your telemetry is as useless as some people think. With the rise of Open Telemetry, those pipelines – and all the techniques to normalize, filter, sample, and transport data that they perform – is becoming more important than ever to understand.

So consider this chapter – in it’s full, lightly edited, unredacted form – to be Avi and my gift to you to become a better user of your monitoring tools.

Once again, you can find the book – all 80+ pages at this link: https://www.kentik.com/go/ebook/oreilly-guide-to-modern-networking-telemetry/.

HOW TO DEVREL: The Bechdel Test for Community

Leon Adato — Mon, 20 Jul 2026 14:00:00 +0000

For those unfamiliar with it, the “Bechdel test” is named for the American cartoonist Alison Bechdel. In one strip from her series “Dykes to Watch Out For“, a character described their 3 basic requirements to go see a movie. The movie had to have:

At least two women, who…
…talk to each other…
…about something other than a guy…

The point is that an embarrassingly small number of movies (not to mention TV shows, books, comics, etc) can clear that fairly low bar.

Equally embarrassing are the number of so-called “community forums” that fail to pass what I have termed my “Bechdel test for community”. In order to qualify as a real community, it would need to have:

at least two (non-employee) accounts with fully filled out profiles…
…talking with each other in a public area of the community forum…
…about something other than the company (or it’s products).

Meaning: A true community is a space where two people with common interests or goals talk to each other about things that matter to them. It might include the company who owns and runs the site, but it might not.

And a true community is one that not only permits, but encourages and facilitates those interactions.

Like the real Bechdel test, that bar seems low until you start surveying community sites. Most will have some form of a #random channel. Many will have other variations of #off-topic discussions. But those examples point to the flaw: often the things that permit a community to pass my test are the same things that are labeled “random” and “off-topic”, betraying the bias against having a true community.

If this hits uncomfortably close to home, don’t let that discourage you. Because improvement is within anyone’s ability.

Put Baby Product in a Corner

One of the things I find vendors fail to recognize (or know, but then quickly forget) is that their name, logo, and product are ALL. OVER. THE. PLACE. There’s need to put the brand front-and-center, nor to make the products the focus. This is because both of those things are intuitive to anyone arriving at “TechBrand.Community.com.”

Moving product-centric forums and their discussions – whether those are announcements or support discussions – into a corner effectively does the opposite of the #random channel I mentioned earlier. It makes a clear statement that “this space is for you, not for us.”

Make It a Sales-Free Zone

Closely related is a rule that I consider to be inviolable, both as a member of an online community and as someone closely involved with running them: A community space has to be completely free from sales activities.

Never in the history of ever has someone in a community forum lamented, “If only I could find a sales person to take my money!”

Like the point I made about product information, you have to recognize that community members will have no problem engaging with sales if they want it. There’s absolutely no need for sales to “check in” or “touch base” if they think a conversation is veering into sales-related territory.

I promise you that if there’s even a hit that sales folks are lurking on those channels, waiting to swoop in (either in the comments or out-of-band via email, phone calls, or whatever) your membership will shrink to zero faster than your platform’s software will be able to track.

Solicit Input, Not Free Labor

If you’re like most tech practitioners, you probably have enough work to keep you and 3 clones busy basically forever. The people in your online community are no different. If you want to demonstrate your respect for their time, not to mention your appreciation for being part of the community the last thing you should do is ask them for free labor.

That includes bug bounties, offering beta version (aka “free QA testing”), or community tech support. Even if they get points or swag.

Instead, the emphasis should be on encourating the community to share opinions, ideas, and experiences in ways that don’t look or feel like free labor.

That starts with problem-forward questions and discussions. Rather than asking how the audience uses a particular module or feature, center the conversation on “how many of you have experienced X?” (where x is a situation, issue, or problem). Or “How do you all deal with X?”

By focusing on the experience (hopefully something that happens to IT practitioners with regularity) you are making it clear your not trolling for customer reference stories.

Other conversation starters that might help:

What are some things nobody told you when you started working with (a certain technology); or in (a particular field)?
What’s a cool hack for solving X?
What issues slow you down every day that you wish you could make disappear?

…and so on.

From a company standpoint, let your community be a place where you open the door for members to participate in UX/UI sessions. These are designed to show early ideas about a new technique or technology, but in a way that allows community folks to offer input that can directly impact (and even change) the product. It’s a win-win that is still surprisingly rare in the tech world. Very few companies allow their community to get involved in a way that allows them to say “You see that option? That’s included because I told them how important it was!”

Reward Participation

That attitude – of valuing real engagement and contributions that benefit both the membership and you – should extend beyond your product to every aspect of the community. You should recognize and reward individuals any time they share their knowledge.

The place to start is recognizing and rewarding when some ASKS a question, not just when they answer them. Asking good questions is actually the hardest and most valuable element of building a vibrant and engaged community. A good question is the thing that starts a conversation, or keeps it going, or makes the answer matter in the first place.

Yes, you should also reward members who share snippets of code, scripts, dashboards, reports, and any other portable (meaning import/export-able) assets. And of course you should reward people for answering questions. But that shouldn’t be the primary focus or else you will once again fall into the trap of making the community space biased toward free labor.

Feature Request System, as a Feature

From a vendor standpoint, the most important question you can ask is “What do YOU need the product to do?”

And, as I’ve mentioned earlier when talking about UX sessions, it has to be asked in a way and at a time when the feedback you receive actually makes a difference.

For this reason, I strongly suggest making a public feature request page part of your community. This is a place where ANYONE (well, any verified community member, at least) can request a new capability.

But not only that. It should be a place where people can comment on the request to add nuance or challenge assumptions. ask clarifying questions. It’s also a place where folks can up (or down) vote those feature request.

This voting capability is more than just a popularity contest. Remember, these are requests from the public who uses your software. They are taking their time to tell you what they need (and why). When an FR gets a significant number of up-votes, it should be seriously considered by your developers. More than that, the developer staff should have to justify why those FR’s are NOT on the list, each and every development cycle.

One piece of criticism I’ve heard about this idea is “what if they say, ‘make the product free’ or ‘buy me a pony’ ?” To clarify: I’m not saying you must slavishly execute on every FR. Of course there will be folks who put in requests as a lark or just to be funny. But the vast majority will be serious.

Some – while serious – will not be possible. And that’s OK. But it’s valuable insight to know that a majority of your user base wants (for example) dark mode. If that’s something your dev team you can’t deliver, then it’s at least something your marketing team should address.

Shall We Play a Game?

Underlying all of this is a truth about online communities: the best ones make engagement and participation fun. Part of that is the content and conversations, but another part is endorphin-boosting rewards. Having a “gamified” online system that assigns points to users who answer (and ask!) questions; who upload resources; who particpate in quizes and missions; who engage in scavenger hunts and other activities… having a system that automatically tracks and tally’s reward points for those types of behaviors is important.

Equally important is having something to DO with all those points. An online shop or other system that lets community members trade in points for swag may sound foolish, but it’s the sweet icing on the cake of community interactions for many of us. It helps give us a reason to hit YOUR site rather than one of the 3 dozen other options in front of us as we sit at our screens.

Communities Are Worth It

A vibrant, engaged, participatory community is more than just a gift of goodwill to the tech space at large (although it definitely is that). It’s also an engine for change, self-awareness, and profit for the sponsoring company. It is a valuable source of customer insight – from superfans to potential employees to canaries who give advance notice when something about your product has become toxic. In one company where I worked, over $1million of revenue per year could be directly attributed to community activities.

But they require work. That includes moderation, content curation, and a careful hand when it comes to participation by the platform owner (that’s you and your colleagues) but not domination.

Hybrid Cloud – the Worst Possible Option

Leon Adato — Mon, 13 Jul 2026 14:00:00 +0000

I’m excited to share that I’m in the process of writing a book for O’Reilly Media! You can check out the Introduction and first chapter of “A Practical Guide to Modern Networking Telemetry” now. That link leads to an early release version, which means it’s been “lightly edited”. If you have comments, corrections, questions, or even kudos, let our editor know: [email protected].

I’m co-authoring it with Avi Freedman, CEO of Kentik, and it’s a labor of love for both of us. As we wrote in the “About the Authors” section, the idea to write this arose first and foremost to get down on paper all of those samples, examples, stories, and lessons we usually share in at conferences, or in talks, or when helping customers; but also to provide a resource to the readers themselves: who might need to articulate those same lessons to colleagues, managers, or the odd (very odd) person at a dinner party.

There’s one comment from the introduction that I wanted to unwind a little bit in this blog. In the section titled, “What IS the Network, Exactly”, I wrote:

Back in 2023, in his book “The Ultimate Guide to Network Observability“, Avi wrote,

“People often refer to “the network” in their organizations, but in most cases the network isn’t one entity. It’s a complex, diverse, fragmented, and loosely interconnected set of physical and virtual links and equipment, and it’s housed in a variety of places, including data centers, corporate wide area networks (WANs), private and public clouds, the internet, container environments, and even inside hosts. Organizations own and control some of those resources but simply pay to use others.”

Since that time, things remain largely the same. Sure, the cloud has gotten more cloudy (meaning opaque to users) even as it has become more ubiquitous within organizations large and small. And at the same time corporate LAN and WAN environments have not, by and large, become less complex (or sprawling, or expensive). In fact, the opposite has been true. Even as company’s investment in cloud has increased, so too has their investment in on-premises networking gone up.

Why is that? Because the more we – businesses and IT professionals alike – use the cloud, the better we understand what the cloud should be used for; and conversely, where cloud should absolutely not be used, whether due to considerations of cost, complexity, or security. And in those cases, we have brought systems, applications, data, and solutions back into our data centers, and found new and creative ways to connect and combine it with the cloud infrastructure.

And that – the cursed franken-network of modern IT: infrastructure that spans both cloud and on-prem environments, offering all the complexity, cost, and issues of both, while offering very little of the benefits.

The sheer horrific nature of hybrid networking might not be intuitively obvious to folks reading this because we live and breathe in a technical context every day. So let me draw a parallel that might make it clearer: hybrid cars.

Blending a traditional gas-powered combustion engine with a modern fully-electric motor sounds like a great idea on paper, but in practice it’s a nightmare. The design burdens the user with all the quirks and complexities of both systems; it also creates a completely new set of quirks and complexities on top of those; and offers only modest benefits – which are usually completely outweighed by the negative aspects of smushing the two technologies into one form factor.

And that’s what hybrid networks are: the combined costs and complexities of on-premises and cloud networks, with a whole new layer of the same incurred by trying to make them play nicely together.

The big difference between hybrid cloud and hybrid cars is, of course, that we IT practitioners don’t have much of a choice. As much as we wish – hell, as much as we TRIED over the last decade to push environments to fully-cloud or fully-on prem, hybrid is the reality we’re going to live in for the foreseeable future.

And that means we – IT practitioners in general and folks who care about (and are responsible for) monitoring and observability in particular – have to improve the ways we ensure performance and reliability of those environments.

And here, approximately 700 words into this blog, I’ve finally gotten to the problem I hope “The Practical Guide” will help fix:

the folks who are often responsible for building the cloud-based side of those environments often know very little about networking
and meanwhile, the teams who are often responsible for building the on-prem side of those environments often know very little about modern observability.

The book offers a clear and complete explanation of everything from the types of devices that comprise a “network”; to the types of telemetry those devices emit; and continues from there to explain how to collect, transform, transmit, and use that telemetry.As I mentioned at the start of this blog, chapter 1 is available in its early release version now. Go ahead and give it a look, and let me know what you think – either in the comments below, or by emailing my editor Gary at [email protected].

Time Data Series: Written In Our Stars

Leon Adato — Wed, 08 Jul 2026 13:49:37 +0000

Once upon a time (i.e. way back in October 2024) I wrapped up a blog in this series with:

Believe it or not, there’s still more to cover. In the coming weeks whenever I get to it, I’d like to cover ways to leverage the built-in astronomy functions for time calculations. Believe it or not, it can really matter in terms of having truly accurate times for things like the start and end of Shabbat and holidays, along with other observances.

And here we are in 2025, when it’s finally time for me to fulfill that promise, and talk about how the KosherJava library (and by extension, the PHP Zmanim port I’ll be using in this blog) can help you calculate and display what amount to straight astronomy in your application or web page.

Once again I need to begin by expressing my deep gratitude to Zachary Weixelbaum and Eliyahu Hershfeld, who both continue to answer so many of my persistent (and often basic) questions with patience, care, and good humor.

The House of the Rising Sun

The core aspect of what I’m showing is fairly simple: For a specific location (latitude/longitude, plus timezone) and date, find the time of day that the sun will be at a given elevation point.

Or to be even more specific: At what time will the sun be at 20 degrees above the horizon on June 10, 2025 in Cleveland? So let’s break down the key elements of PHP Zmanim we’ll be focusing on today.

These two lines load the parts of the PHP library we’ll be using.

use PhpZmanim\Geo\GeoLocation;
use PhpZmanim\Calendar\ComplexZmanimCalendar;

Next, we set up a location object (using variables for location name, latitude, longitude, elevation, and time zone); and a calendar object (using the location object we just instantiated along with the year, month, and day):

$location = new GeoLocation($locname, $lat, $long, $elev, $tz);
$zmanim = new ComplexZmanimCalendar($location, $getyear, $getmonth, $getday);

Finally, the code that calculates and displays the results:

$sunrise = $zmanim->getsunrise();
$twenty = $zmanim->getSunriseOffsetByDegrees(70);
print("sunrise: $sunrise\n20 degrees: $twenty\n");

If playing along at home, I should explain that I grabbed both the regular sunrise simply as a reference to the time at which the sun is 20 degrees up in the sky.

I also need to explain that yes, I know that 70 is not 20, so how does THAT work out? The getSunriseOffsetByDegrees function “thinks” of the sun as being at 0 degrees when it’s at its zenith, and 90 degrees at the horizon line. So for the purposes of this function going up (higher in the sky) causes the degree value to go down.

The full code looks like this:

use PhpZmanim\Zmanim;
use Carbon\Carbon;
use PhpZmanim\Calendar\ComplexZmanimCalendar;
use PhpZmanim\Geo\GeoLocation;

# Set variables:
#41.4939407, -81.516709;
$locname = "Beit Knesset Chochmat Shlomo, Beachwood, OH";
$lat = 41.4939407;
$long = -81.516709;
$elev = 0;
$tz = 'America/New_York';
$getyear = "2025";
$getmonth = "06";
$getday = "10";

$location = new GeoLocation($locname, $lat, $long, $elev, $tz);
$zmanim = new ComplexZmanimCalendar($location, $getyear, $getmonth, $getday);
$sunrise = $zmanim->getsunrise();
$twenty = $zmanim->getSunriseOffsetByDegrees(70);
print("sunrise: $sunrise\n20 degrees: $twenty\n");
?>

When the Night Has Come, and the Land is Dark…

I’m sure there are useful reasons to know the time when the sun is at a certain point in the sky during the day, but from a religious perspective, there are far more reasons to want to know when the sun is below the horizon line. I say “religious” because I can think of at least two that care about the time of the sun’s position before sunrise: Judaism and Islam.

Both have a concept of dawn that indicates the very first visible lightening of the sky in the east. This time is known as fajr in Islam and alot hashachar in Judaism.

Because we’re talking about a matter of perception, it can be ferociously difficult to nail down, a matter not made any easier by the constant shifting of the sun over the year, and the differences due to latitude.

However – and bypassing a huge swath of religious debate – let’s accept the opinion that the first light is visible by perceptive humans on a clear day when the sun is 18

degrees below the horizon. How can we calculate that?

Honestly, it’s not hard now that you know the basic structure. In fact, we just have to change one thing:

$dawn = $zmanim->getSunriseOffsetByDegrees(90+18);

Yes, I could have just typed 108 but I wanted to make it explicit that we’re calculating 18 degrees before sunrise (90 degrees from the zenith).

Get Me to the Church Synagogue on Time…

To offer a more involved – not to mention specific – example of how this can be used, I’d like to explain one method of finding the time for “Tzeit haKochavim” (“nightfall”, which is more poetically described as the time when it’s dark enough to see 3 large stars in the sky). For this particular method, the following series of calculations are used:

Select the date of the equinox (spring or fall)
Get the time for sunset
Get the time when the sun is 3.86 degrees below the horizon (after sunset)
Get the difference in minutes between those 2 times
Select the current date
Calculate “seasonal minutes” – an even division of daylight 6.1) Subtract sunset from sunrise to get the total amount of daylight 6.2) Divide that time into 12 even “seasonal hours” 6.3) Divide 1 seasonal hour into 60 even “seasonal minutes”
multiply the clock minutes you got from the equinox day calculations and multiply them by seasonal minutes.
And finally, add THAT number of minutes to sunset, to get the time of “nightfall” / “Tzeit haKochavim”

Confused? Yep, me too. It took me a while to wrap my head around this. But here’s what it looks like in code:

First, let’s set our location and today’s date:

$locname = "Beit Knesset Chochmat Shlomo, Beachwood, OH";
$lat = 41.4939407;
$long = -81.516709;
$elev = 0;
$tz = 'America/New_York';
$getdate = date('Y-m-d');
$getyear = date('Y', strtotime($getdate));
$getmonth = date('m', strtotime($getdate));
$getday = date('d', strtotime($getdate));

Next, we’re going to create a location object, and use that location object to create a calendar object set to an equinox day (in this case March 20).

$location = new GeoLocation($locname, $lat, $long, $elev, $tz); 
$eqzmanim = new ComplexZmanimCalendar($location, $getyear, '3', '20');

Get sunset for that day, along with the time the sun is 3.86 degrees below the horizon

$eqsunset = $eqzmanim->getsunset();
$eqtzeit = $eqzmanim->getSunsetOffsetByDegrees(90+3.86);

Get the number of minutes between sunset and 3.86 degrees after sunset.

$tzeitmin = number_format((float)((strtotime($eqtzeit)-strtotime($eqsunset))), 2, '.', '')/60;

OK, now it’s time to get some calculations for TODAY’S date. We’ll create a new calendar object and grab sunrise and sunset:

$todayzmanim = new ComplexZmanimCalendar($location, $getyear, $getmonth, $getday);
$sunrise = $todayzmanim->getsunrise();
$sunset = $todayzmanim->getsunset();

Now we’ll divide the time between sunrise and sunset into 12 equal parts (a “sha’ah” in Hebrew), and then divide one sha’ah into 60 even parts. That will effectively give us the ratio of seasonal minutes to clock minutes

$stdshaamin = number_format((float)((strtotime($sunset)-strtotime($sunrise))/12)/60, 2, '.', '');
$stdzmaniot = $stdshaamin/60;

Now we multiply the number of minutes from our equinox day times the seasonal minute ratio.

$tzeitadjustment = $tzeitmin * $stdzmaniot;

And finally, we add that adjustment to today’s sundown (also note we have to multiply the adjustment * 60 to make the clock math work out).

$todaytzeit = date('Y-m-d G:i:s', strtotime($sunset) + $tzeitadjustment*60);

And let’s output it to see what we get:

print("sundown: $sunset\ntzeit: $todaytzeit\nadjustment: $tzeitadjustment\n");

For June 10, 2025, the output would look like this:

sundown: 2025-06-10 20:59:34
tzeit: 2025-06-10 21:19:59
adjustment: 20.4228

Good Night, My Someone

Believe it or not, we’re reached the end of my “Time Data Series”. I will, of course, keep adding things as I learn them myself. But – unless you have something specific you’d like to see (which, by all means, please tell me about in the comments), I’m going to turn out the lights, take a moment to admire the stars both for their beauty and the breathtaking wonder at the complexity of their movements, and head to bed.

Thank you for coming along on this ride with me.