DEV Community: Danielle Heberling

I Rewrote My Step Function as a Durable Function

Danielle Heberling — Sun, 01 Mar 2026 12:12:03 +0000

Ever since AWS announced Lambda durable functions at re:Invent 2025, I've been wanting to try them on a real project. Not a cookie cutter tutorial, but something where I could actually compare the two approaches. I already had the perfect candidate: my weather site.

Quick Background

If you haven't read my original blog post, here's the short version. I built isitsnowinginhillsboro.com because the existing Portland snow site wasn't accurate for my area. It's a serverless workflow that checks the OpenWeatherMap API every 10 minutes and updates a static S3 site with a YES or NO answer. The original uses a Step Function triggered by EventBridge Scheduler.

Why Rewrite It?

I think a lot of us have been wondering when you'd pick durable functions over Step Functions. This project was small enough to rewrite quickly, but complex enough to exercise real patterns like API calls, branching logic, parallel execution, and multiple AWS services. Building the same thing both ways seemed like the best way to form my own opinion.

The durable function version is here on GitHub.

How the Two Approaches Compare

The workflow logic is identical in both versions. Same end result. But the developer experience is pretty different.

Step Functions

You define your workflow as a state machine, chaining together states like LambdaInvoke, Choice, Pass, and direct SDK integrations in CDK. The visual representation in the console is nice for understanding the flow at a glance, and it feels very "managed" in the best sense. The service handles state tracking, retries, and transitions for you.

The tradeoff is that CDK code for Step Functions can get verbose. Passing data between states and setting up error handling means understanding how Step Functions manages its JSON payload. You're giving up some control for the convenience of the service doing the heavy lifting.

Durable functions

With durable functions, you write your workflow as plain TypeScript. Wrap your logic in ctx.step() calls, use regular if statements for branching, ctx.parallel() for parallel execution. Need to call an AWS service? Just use the SDK like you normally would inside a step.

Here's the thing that surprised me: this version felt more natural to write. I wasn't thinking about "states" or "transitions." I was just writing code. Since it's TypeScript with a thin SDK wrapper, it also feels more portable. If another cloud provider or open source project adopted a similar checkpoint/replay pattern, the mental model (and probably a good chunk of the code) would transfer over.

Side by Side

Concept	Step Functions	Durable functions
Workflow definition	JSON/YAML state machine (ASL)	Plain TypeScript code
State management	Managed by the service	Automatic checkpointing via SDK
AWS service calls	`CallAwsService` task or direct integration	Regular AWS SDK calls inside `ctx.step()`
HTTP calls	`HttpInvoke` task + Connection resource	Standard `fetch()` inside `ctx.step()`
Conditional logic	`Choice` state	Plain `if` statement
Parallel execution	`Parallel` state	`ctx.parallel([...])`
Scheduling	EventBridge Scheduler -> Step Function	EventBridge Scheduler -> Lambda
Infrastructure	State Machine + Connection + Lambda(s)	Single Lambda function
Debugging	Step Function execution history (visual)	Durable execution history

The Gotcha That Got Me

Durable functions require a qualified ARN for invocation, meaning a published version or alias. When I first wired up EventBridge Scheduler, I was using the unqualified ARN and couldn't figure out why it wasn't working.

The issue? The CDK LambdaInvoke scheduler target uses functionArn under the hood, which for a plain Function is the unqualified ARN — no version or alias suffix. Durable functions reject unqualified ARNs entirely.

What ended up working was creating a Lambda alias in CDK and passing that to the scheduler target instead. Since an alias implements IFunction, the construct picks up its qualified ARN automatically.

If you're getting started with durable functions and things aren't behaving as expected, check your invocation ARN first.

So Which One Should You Use?

In my opinion, both approaches are solid for a workflow like this. It really comes down to developer experience.

If you prefer writing workflows as code , durable functions are going to feel great. The code reads like a normal application, the infrastructure footprint is smaller, and the patterns are more transferable if you ever need to move beyond AWS.

If you value visual debugging and a fully managed experience , Step Functions has the edge right now. The console gives you a step-by-step view of every execution with clear status indicators. Durable functions have a visual for execution history too, but I prefer how Step Functions handles it.

If you need Distributed Map for massive fan-out scenarios (up to 10,000 concurrent executions) or rely heavily on Step Functions' 220+ native service integrations, Step Functions is the better option.

AWS also has official guidance. One thing I found interesting is they mention that hybrid architectures are totally valid, using durable functions for application-level logic while Step Functions handles higher-level cross-service coordination.

Wrapping Up

If you're curious about durable functions, I'd encourage trying it with a project you already have. You'll learn way more than a tutorial can teach you.

Both repos are open source:

Step Functions version: github.com/deeheber/weather-site
Durable functions version: github.com/deeheber/durable-function-weather-site

I'd love to hear from anyone else who's been experimenting with durable functions. Let me know what your experience has been! 🚀

Let an AI Agent Do Your Job Searching

Danielle Heberling — Sun, 08 Feb 2026 09:12:03 +0000

Photo by Markus Winkler on Unsplash

Quick disclaimer before I get into this: I'm no longer actively job hunting. I started building this project during my last job search and decided to finish what I started. Partly because I wanted to learn more about agents and partly because I think it might be useful for someone else out there. With that out of the way, here's the story.

The Problem

Ever since I started my last job search, one thing kept bugging me. I had a list of "dream companies" that I wanted to work at, and I'd manually go check their career pages every few days to see if anything new popped up. Sometimes daily. Sometimes I'd forget for a week and miss a posting entirely.

Here's the thing: job searching is already stressful enough without having to remember to check 10+ career pages on a regular cadence. I'd open a browser tab, search around, get distracted by Slack or email, and then forget where I left off. Multiply that by several companies and it becomes a real time sink.

I kept thinking there has to be a better way. What if something could just watch those companies for me in the background and send me an email when there's an opening that matches what I'm looking for?

That's when I decided to build it myself.

What I Built

The job-search-agent is an AI agent built with Strands Agents and deployed to Amazon Bedrock AgentCore Runtime. You give it a company name (and optionally a job title or location filter), and it searches the web for open positions at that company. Then it returns what it finds with links to the actual job postings.

You can also set it up to run on a schedule and optionally send you email alerts when a company is hiring. That's the part I was most excited about.

Design Decisions

Here's how the architecture works at a high level:

EventBridge Scheduler kicks things off on whatever cadence you configure. This triggers the agent running in AgentCore Runtime , which is where the Strands agent lives. The agent does its thing (searching for jobs at the companies you've specified), and if you've provided one or more email addresses in the environment variables and there is a job posting, it sends the results via SNS as an email notification.

The infrastructure is all defined in CDK (TypeScript), and the agent code is Python. I talked about why I use that particular combo in my strands-agent-template post, but the short version is that each framework is strongest in its native language right now.

Is this a bit over-engineered for what it does? Most likely. But I wanted to use the opportunity to build something I'd actually use while learning about agents and AgentCore. In my experience, the best way to learn new tech is to solve a real problem with it, even if the solution is fancier than it needs to be.

Challenges

When I first started building this, I took what felt like the simplest approach. I put instructions in the agent's system prompt to construct the career page URLs for each company. Something along the lines of "for company X, try going to company.com/careers and look for job postings."

This didn't work well at all.

The agent would guess at URLs that didn't exist, hallucinate job listings, and burn through a ton of tokens trying to figure out where the careers page actually was. Some companies have their postings on Greenhouse. Others use Lever, Ashby, or their own custom systems. There's no standard for this, and asking an LLM to guess the right URL is a recipe for frustration.

So I decided to build a tool for the agent to use instead. During that research, I came across Tavily, which is a search API built specifically for AI agents. Even better, Strands already had a pre-built Tavily tool available. I plugged it in, and the difference was night and day. Instead of guessing URLs, the agent now searches the web for actual career pages and job boards, then extracts the relevant information from those results.

This was a good reminder for me that agents are only as useful as the tools you give them. The LLM is great at reasoning and formatting responses, but it shouldn't be doing the heavy lifting of web searching through prompt instructions alone.

There was one more quirk worth mentioning. EventBridge Scheduler supports universal targets, which let you call pretty much any AWS API action directly without needing a Lambda function in between. You just give it the service ARN and the request payload, and it makes the API call for you. There's a list of unsupported action prefixes in the docs (things like get, describe, list). Interestingly, invokeModel is explicitly blocked, but invokeAgentRuntime is not.

So I set up EventBridge Scheduler to call the Bedrock AgentCore invokeAgentRuntime action on a schedule. And it works. The agent gets invoked, does its thing, and returns results. But for some reason, EventBridge Scheduler thinks the invocation failed. Every single time.

If you leave retries enabled, the scheduler will retry the invocation multiple times even though the first call succeeded. That means your agent runs (and costs you money) three or four times instead of once. Also you may get duplicate email notifications. Not great.

My workaround was to disable retries on the scheduler and set up a Dead Letter Queue (DLQ) to capture these "failed" invocations so I can monitor them. I'm honestly not sure if this is something I'm doing wrong or if it's a quirk on the AWS side that'll get fixed eventually. AgentCore is still pretty new, so it's possible this just hasn't been ironed out yet. If you've run into something similar, I'd love to hear about it.

The Outcome

After getting Tavily integrated, the agent started returning real job listings with real links. I set up EventBridge to run the searches on a schedule and configured SNS to email me the results.

The time savings were immediately obvious. Instead of spending 30+ minutes bouncing between career pages every few days, I'd get an email in my inbox with a summary of who's hiring and links directly to the postings. Even when nothing new showed up, it was nice to not wonder if I'd missed something.

A Note on Cost

If you decide to deploy this yourself, keep in mind that it will cost you money. Specifically:

Bedrock usage is the main cost driver here. Every time the agent runs, it's making calls to a foundation model, and those tokens add up. How much depends on how many companies you're monitoring and how often the scheduler runs.

Tavily has a free tier that's pretty generous for personal use, but if you're running searches frequently or monitoring a lot of companies, you could exceed it. Keep an eye on your usage.

I'd recommend starting with a low frequency schedule (maybe once a day or even once a week) and a small list of companies to get a feel for the costs before scaling up.

Everything else in the stack (AgentCore Runtime, EventBridge Scheduler, SNS) will likely fall within the AWS free tier for personal use. But "likely" isn't a guarantee, so check the pricing pages against the usage you expect before deploying.

What I Learned

I think the biggest takeaway from this project is that building something you actually need is still the best way to learn new tech. I could have followed a tutorial to build a generic chatbot, but because this was solving a real problem for me, I was way more motivated to push through the frustrating parts.

If you're curious about agents but not sure where to start, I'd suggest finding a small, annoying, repetitive task in your life and trying to automate it. It doesn't have to be fancy. The learning happens in the process of figuring out what works and what doesn't.

Give It a Try

The full source code is on GitHub. It includes instructions for running locally and deploying to AWS.

If you're currently job hunting and this looks useful, take it for a spin. If you find bugs or have ideas for improvements, open an issue or submit a PR. I'm curious to hear what other folks build with it.

Good luck out there 💪

A Strands Agent Template (For the Impatient)

Danielle Heberling — Fri, 23 Jan 2026 10:12:03 +0000

Photo by Tanja Tepavac on Unsplash

Ever since AWS announced Bedrock AgentCore Runtime back in July 2025, I've been experimenting with deploying agents to production. The hype around "agentic AI" is everywhere, but most examples I found were either too simple or way too complex for what I actually needed.

Here's what I wanted 👉 a way to go from zero to a deployed agent on AWS without spending three days on infrastructure setup. So I built a template that does exactly that.

What I Built

The strands-agent-template repo is a GitHub template that gives you everything you need to deploy a Strands agent to Amazon Bedrock AgentCore Runtime. Click "Use this template" and you're most of the way there.

What's included:

✅ Production-ready CDK infrastructure with IAM, logging, and tracing
✅ Local development setup so you can test before deploying
✅ GitHub Actions for CI/CD
✅ OpenTelemetry observability built in
✅ Testing, linting, and formatting configured

Why Python + TypeScript?

I went with Python for the agent code and TypeScript for infrastructure. This isn't because I love context switching (I don't), but because each framework is strongest in its native language. Strands has great Python support and documentation, while CDK is cleanest in TypeScript.

As these frameworks mature, I might consolidate to one language. But for now, this combo gives you access to the best libraries and examples for both.

Quick Start

If you just want to see it work after setting up the AWS CLI and configuring credentials:

# Test locally
cd agent && source .venv/bin/activate && python src/agentcore_app.py
# Test in another terminal
curl -X POST http://localhost:8080/invocations -H "Content-Type: application/json" -d '{"prompt": "What is 42 * 137?"}'
# Deploy to AWS
cd cdk && npm install && npm run build && npm run cdk:deploy

The template comes with a basic example agent. Replace it with your own logic and you're done. Under 10 minutes from clone to deployed.

What You Actually Get

Here's what stood out to me after using this in a few projects.

Local testing that doesn't lie. You can run your agent locally before deploying anything to AWS. This matters more than you'd think. I've wasted too much time debugging issues that only showed up after deployment.
Observability without the extra work. CloudWatch logs and OpenTelemetry tracing are configured out of the box. When something breaks (and it will), you can actually figure out why.
CI/CD that works. GitHub Actions run your tests on every push. Both Python and TypeScript tooling is configured. It catches the obvious stuff before you deploy.

If You're Just Getting Started

If this is your first time deploying an agent to AWS, here's what I'd focus on:

Start by running the example locally. Don't deploy anything yet. Get familiar with how Strands agents work.
Then make a tiny change to the agent logic. See how it behaves. Break something on purpose and watch what happens.
Only then deploy to AWS. Use the CloudWatch dashboard to see what your agent is actually doing.

The template includes detailed deployment docs in the DEPLOYMENT.md file. I tried to write them for someone who hasn't used CDK or AgentCore before.

What's Next

I'm actively using this template for a few projects, so it'll keep evolving based on what I learn. If you find gaps or have suggestions, open an issue or submit a PR. The CONTRIBUTING.md file has guidelines.

This is meant to be a starting point, not a finished product. Take it, modify it, make it yours.

You can find the template on GitHub.

Let me know if you use it. I'm curious what works and what doesn't for other folks trying to deploy agents on AWS.

Use Social Media Mindfully

Danielle Heberling — Sat, 17 Jan 2026 10:12:03 +0000

I quit Facebook in 2020 when a former coworker was spreading misinformation about what was happening in Portland, OR. He'd never been there and had no plans to visit. I was literally living in Portland at the time, telling him what I was seeing firsthand, but that didn't matter to him. That was it for me. I miss it sometimes, but mostly I don't.

Here's what I've noticed since then: the heyday of social media feels like it's behind us. In my opinion, Facebook peaked in 2008. Back then, it was about connecting with friends, sharing actually interesting updates about our lives. Minimal ads. It felt genuine.

Now? Wannabe influencers everywhere. More ads and brand accounts in your timeline than content from people you actually know. Bots running campaigns to get engagement through false things or distortions of reality. It's exhausting.

But here's the thing: I'm not saying abandon social media entirely. I'm saying use it differently.

How I Think About Social Media Now

I'm not scrolling feeds endlessly anymore. No traps of getting lost in reels or stories. I use Buffer to schedule posts, which keeps me from even looking at a timeline. I check in with intention when I need to, then I'm out. This one's harder than it sounds, but it makes a real difference in how much time you lose to these platforms.

With that said, social media still works for connections. DMs are good. Having actual conversations in comments is good. Longer discussions where you're genuinely exchanging ideas? Even better. This is where I think the platforms still have value if you're intentional about it.

I try to share things that might help someone else. Good articles I've read. Things I'm learning. Mistakes I've made. If it could save one person some time or frustration, it's worth sharing. The stuff you've learned the hard way, the patterns you're seeing in your day job...not to build a personal brand or chase engagement metrics, but because someone else is probably dealing with the same problems.

If you're job hunting, LinkedIn especially can help you connect with the right people. It's not your whole career strategy, but it's a useful tool when you need it.

Here's where I think we've lost the plot though: we've forgotten that coffee with friends to catch up beats any social media interaction. Travel somewhere to see people you care about. Those face-to-face conversations are what actually matter.

What This Actually Looks Like

For me, this means I spend my time learning, reading, and building things. When I do post, it's usually because I want to hear what other people think about something I'm working through. Or I've hit a problem that took me way too long to solve and I figure sharing it might save someone else the trouble.

I'm not trying to go viral. I'm not optimizing for engagement. I'm definitely not checking how many likes something got.

The goal is simple: be helpful. Connect with people who are thinking about similar problems. Have real conversations, even if they happen in threads or comment sections. That's what matters and what I'm focused on in 2026.

Spend your time learning, reading, and building things. Use social media when it serves a purpose. Skip it when it doesn't.

What's your approach? I'd be curious to hear how other people are thinking about this stuff.

How You Do It Matters

Danielle Heberling — Wed, 19 Nov 2025 15:12:03 +0000

Photo by Jack Anstey on Unsplash

Looking back on my career, the connections and opportunities I value most weren't the result of optimizing for speed. They grew from taking the long view.

How you do it matters just as much as what you do.

Take professional networking, for example. I've found that showing up consistently to community events, contributing even when I don't need anything, has led to the most meaningful connections and opportunities. That might look like speaking at a meetup, volunteering to help organize one, reviewing someone's resume, or answering career questions in Slack. Similarly, I'm drawn to organizations where solving real user problems is at the heart of what we build.

I once worked at a startup where the CEO's departing words really stuck with me. This man was far from perfect, but I enjoyed working with him professionally and did it twice, mostly because of our alignment on what matters.

Here's what he said upon his departure.

As you continue to grow without me, I'd ask you to keep one thing in mind: the way we do it matters. We haven't gotten here because we're the cleverest copywriters, the fastest programmers, or the slickest sales people. We don't work sixteen hour days, and we're not ruthless or cutthroat. We're kind. We care about each other. We bring out the best of each other's strengths, and we bridge each other's weaknesses. We create opportunities to experiment and take risks. We're humble. And we're better for it.

Of course, pragmatic tradeoffs are part of the job. I can move fast when needed, whether that's shipping a quick fix to unblock progress or meeting a tight deadline. The key is being intentional: planning to address the root cause rather than letting temporary solutions pile up. It's about being honest with yourself and your team about what you're doing and why.

In an age of viral videos for quick wins, I know it can be challenging to play the long game...but I encourage you to consider it.

This approach has led me to opportunities and relationships I wouldn't trade for anything. If you're wondering whether it's worth it to take a more intentional approach - I'd say give it a shot and see where it takes you.

Learning Outside Your Specialty | Why I Got a Kubernetes Cert

Danielle Heberling — Sun, 12 Oct 2025 10:12:03 +0000

Earlier this month I took the exam for the Kubernetes and Cloud Native Associate (KCNA) certification and passed.

After years of working in AWS Serverless, this raised some eyebrows. People asked: why?

Here's the honest answer: I've passed on applying to jobs because I didn't know Kubernetes (K8s). That needed to change.

The Reality of Job Postings

Look at DevOps and platform engineering roles today. Many of them want AWS experience, serverless knowledge, AND K8s expertise. Not all, but enough that it's a pattern.

Specializing deeply in serverless has been valuable for my career. But at some point, gaps in your knowledge start closing doors. When I'm scanning job postings and thinking "I could do this role, except for that one requirement," it's time to consider filling the gap.

This isn't about chasing every new technology. It's about recognizing when a skill has become common enough in your field that not having it limits options.

What "Best Tool for the Job" Actually Means

I still believe in "serverless first." But here's what I've learned: the best tool depends on context.

At the end of the day, users care that what you build is reliable, works, and solves their problem. Unless they're an engineer or tech geek, they don't care if you used K8s or serverless. They care about the outcome.

Tech decisions are made by people. People make choices based on their experiences and what they know. A lot of teams use K8s. Whether they "need" it is a value judgment that depends entirely on your perspective and experience. That's not what this post is about.

What I've realized: if I'm working with a team that's chosen K8s, fighting against that to use serverless creates friction. Understanding their world helps me work with them effectively, even if I'd make different choices.

This isn't about one technology being better than the other. It's about understanding enough of both to collaborate with different teams and make informed decisions when I do have a choice.

K8s Is Complex, But Learnable

People aren't wrong when they say K8s is complex. It is. There are a lot of moving parts.

But here's what studying for the KCNA taught me: if you take the time to understand the components and how they fit together, it's not magic. It's learnable. Breaking it down into pieces makes it manageable.

The complexity isn't a reason to avoid it. It's just a reason to approach it systematically.

My Advice: Depth First, Then Breadth

If someone asked me "should I learn K8s?" a year ago, I would have said: get really good at one thing first.

Master the fundamentals of whatever stack you're working in. Build that depth. Then expand your toolbelt.

Trying to learn everything at once means you'll be mediocre at many things instead of excellent at one. Depth first, breadth later.

Disclaimer: this is what works for me. Your situation might be different, and that's okay.

That's what this K8s certification represents for me. Not abandoning serverless or becoming a "K8s person." Just adding another tool to solve different problems.

What's Next

The plan is to complete the Kubernetes Resume Challenge and build some personal projects. The goal is practical knowledge, not just cert credentials.

Honestly, it's been fun to be a beginner again.

I'd love to hear from you: Have you learned outside your specialty? What pushed you to do it, and what did you learn in the process?

Most Valuable When Least Visible | The Security Paradox

Danielle Heberling — Wed, 18 Jun 2025 10:12:03 +0000

Photo by FlyD on Unsplash

The Problem

Tale as old as time: the friction between building out new features and the foundations necessary to deliver those features. By foundations, I'm referring to things such as but not limited to: security, testing, Continuous Integration/Continuous Delivery, observability, and Infrastructure as Code.

Recently I attended AWS re:Inforce, an AWS security focused conference. Speaking with fellow attendees it was very common for people ask some variant of "how can I get people in my organization to care about security?"

Disclaimer that I do not consider myself a security person™️, but I do champion security initiatives often. Security is everyone's responsibility. Sometimes in the real world people get excited about new features and forget about less visible things such as security.

Here are some of my thoughts in the form of unsolicited advice on how I attempt to navigate this.

1. Just Do It

If you work in an organization that gives you the freedom to allocate your time to projects beyond what is officially committed to in the sprint, just do it. Then once it's finished, you can share with your team what you did, how it works, and why it's important.

It is often faster to just do it rather than spend the same amount of time asking for permission. Especially at smaller early stage startups, delivering results speak louder than words.

2. View Yourself as an Enabler

Security is often (and mistakenly) seen as an obstacle to rapid feature development. I suggest reframing the conversation to highlight how security actually empowers the team to deliver features more quickly.

Many security measures may not seem urgent now, but by the time they are, it’s often too late. Focus on communicating the value of being proactive instead of reactive.

3. Learn Incentives

Incentives influence our perspectives and priorities. Invest time in understanding key stakeholders, especially decision-makers, to learn what drives them and what matters most to them.

This understanding will help you frame your arguments more effectively when proposing needs that don’t have the allure of a shiny new feature.

Bonus: Get Involved in the Community

As tech professionals, it’s easy to find ourselves working in silos within our organizations. But the more I connect with colleagues outside of my current company, the more I discover that the challenges I’m facing have often already been solved by someone else.

Why reinvent the wheel when we can learn from each other’s experiences - and share our own insights to help others grow? Engage in conversations, exchange ideas, and don’t hesitate to ask questions, even if they seem basic. Often, those “simple” questions lead to the most valuable discussions and solutions.

In person conferences aren't your thing? Shameless plug to check out the Believe in Serverless Community on Discord!

Closing

Ultimately, trust is the foundation of successful software. When it’s present, it’s invisible; when it’s missing, it becomes impossible to ignore. Once broken, trust is incredibly difficult to rebuild—especially for early-stage startups still working to establish their reputation. If users can’t trust your product, they won’t use it.

I’m not claiming to have all the answers when it comes to security, but I hope sharing these thoughts sparks reflection or helps someone else feel less alone in facing these challenges. Your efforts to prioritize security truly matter, even when they go unseen. Keep pushing forward and championing the importance of strong foundations.

I’d love to hear from you: What strategies or insights have helped you balance the tension between shipping new features and building secure, reliable products?

AWS Fargate on a Budget

Danielle Heberling — Tue, 18 Feb 2025 11:12:03 +0000

Photo by Alexander Grey on Unsplash

The Business Problem

At the day job, there's a new application that needs to be deployed in a manner that meets these requirements:

the code is server side and it needs to run longer than 15 minutes
this is an internal use app, it can tolerate occasional disruptions
non-engineering stakeholders should be able to access this application
the application will be used by fewer then 10 people and there will be periods of time where no one will be using it
it is faster to get approval if we use Amazon Web Services, because we currently use AWS heavily

A Solution

For non-disclosure agreement and security reasons this is a variation of the solution, not the exact solution.

It includes the following:

One Fargate Spot instance behind an Application Load Balancer
Two EventBridge Scheduler schedules that turn the Fargate Spot instances on at a specified time and off at a specified time

Disclaimer: this is not the only way to accomplish this.

Architecture Diagram and Example Code

Here's the high level architecture diagram of this solution.

Example code with instructions on how to deploy this into your AWS account, can be found in this GitHub repo.

Disclaimer: in this example, we are using a demo Docker image. If you are setting up an application that contains information that shouldn't be open to the public internet, ensure that authentication is in place. Authentication has been intentionally omitted, because it is not the focus of this article.

Let's zoom in a bit closer on the two major components of this solution:

Fargate Spot
EventBridge Scheduler

Fargate Spot

With Fargate there are two capacity provider types: Fargate and Fargate Spot.

Fargate is the standard which gives you on-demand access to containerized compute.

Fargate Spot is similar to Fargate, but cheaper (advertised as up to 70% discounted) and can be interrupted by AWS. The reason for this is because AWS operates at a massive scale and lots of times there are instances available that will run and cost AWS money regardless. In order to make money off of this extra capacity, AWS offers this extra capacity as Spot instances at a discounted rate. Because AWS might need this capacity back as demand rises, they reserve the right to give you a two minute warning before shutting down your instance to put it back into the regular Fargate on-demand pool.

The launch blog post has an excellent overview for an AWS official description.

Launching a Fargate Spot instance involves 1. launching it and 2. setting up the container to handle a graceful shutdown given a two minute warning from AWS.

Launch the Instance

The example code uses the ApplicationLoadBalancedFargateService CDK construct.

You can specify one or multiple capacity providers with the capacityProviderStrategies property with weights. The higher the weight, the more often that launch type will be utilized.

The example code sets things up to prefer Spot, but to use a regular Fargate launch type in case there is no Spot availability.

capacityProviderStrategies: [
          {
            capacityProvider: 'FARGATE_SPOT',
            weight: 50,
          },
          // Backup in case FARGATE_SPOT is not available
          {
            capacityProvider: 'FARGATE',
            weight: 1,
          },
        ],

Handle Graceful Shutdowns

In the example code, this is done in two places:

Adds a stopTimeout to the container on the Fargate Task for 120 seconds (needs to be <= 2 minutes)
Sets the deregistration delay on the Application Load Balancer at 30 seconds to give the instance time to separate from the Load Balancer before terminating (also needs to be <= 2 minutes)

EventBridge Scheduler

In the example, there are two EventBridge Scheduler schedules - "up" and "down".

This is accomplished by using an AWS CLI command to set the desiredCount on the ECS Service to either 1 (on/up) or 0 (off/down). Here's the official documentation for the command.

If this command were run on the command line via the AWS CLI it would look something like:

aws ecs update-service --cluster <cluster-name> --service <service-name> --desired-count <desired-count-int>

For this app, we're using EventBridge's Universal Targets feature. Universal Targets allow you to run most (not all - see the link to view unsupported commands) AWS CLI commands directly in an EventBridge Schedule without the need to add (and pay for) compute (such as Lambda).

In our example, we have one Schedule that sets the desiredCount to 1 (on/up) at 9am PT Mon-Fri and another that sets the desiredCount to 0 (off/down) at 5pm PT Mon-Fri.

These cron expressions can be adjusted...be sure to ask your stakeholder(s) when they plan to use the app to ensure it is available.

Summary

This was a walkthrough of a solution that allows us to run a Fargate instance on a budget. The solution utilizes Fargate Spot and EventBridge Scheduler to periodically shut down the Fargate Tasks during periods of no usage.

The example code repository can be found on GitHub.

What tips and tricks do you have for saving money with Fargate?

Ephemeral Jobs Longer than the Lambda Timeout

Danielle Heberling — Thu, 07 Nov 2024 15:12:03 +0000

Photo by Mathew Schwartz on Unsplash

The Problem

There's an ad-hoc job that runs in the background that doesn't have a client waiting for a synchronous response.

You could run this in AWS Lambda to save money. A benefit with an ephemeral job run is that AWS only charges for when the Lambda is running.

Problem is that this specific job runs longer than the Lambda timeout (at the time of writing this it is 15 minutes). We could write extra logic to make it work, but that adds unnecessary complexity.

You still want to run this ephemerally to save money and resources...so what can you do?

A Solution

One option is to use ECS run-task with a Fargate launch type.

AWS Resources Needed:

An ECS Cluster - a logical grouping of tasks or services (in my example we're using the default cluster that comes with new AWS accounts)
A Task Definition - a blueprint for your application which contains one or more containers and parameters to run
A Container - a lightweight, standalone, executable package of software that includes everything needed to run an application. This is commonly used with Docker.

Once those resources are deployed, the container can be triggered to run on demand with ECS run-task. Once the code is done running and the container exits, then the running ECS Task will disappear. The benefit is that you'll no longer be charged money for a running Fargate task.

There's a few ways to run an ECS task on demand:

AWS CLI
AWS SDK (could invoke from within Lambda etc)
In a Step Function state
EventBridge Scheduler
AWS Console

Here's an example of what this command looks like via the AWS CLI:

aws ecs run-task \
  --cluster default \
  --task-definition my-task \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet1,subnet2],securityGroups=[securityGroup1],assignPublicIp=ENABLED}"

Show Me the Code

There's a full code example of how to deploy these cloud resources and invoke the deployed task via ecs run-task with the AWS CLI.

Take a look at the README file for directions.

Bonus

There are a few more benefits to ECS run-task that I won't mention, but one notable one is the ability to override specific things in the task definition.

For example, if you want to override the CMD defined in the Dockerfile used to build the Container Image for the task, you could do this by adding the --overrides flag.

aws ecs run-task \
  --cluster default \
  --task-definition my-task \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet1,subnet2],securityGroups=[securityGroup1],assignPublicIp=ENABLED}" \
  --overrides '{"containerOverrides":[{"name":"my-container","command":["npm", "run", "migrate"]}]}'

I've seen this used for one off commands that need to run every now and then such as database migrations, though I'm certain there are other good use cases. Here's the full list of what can be overridden.

Special thanks to Chase Douglas for giving me this idea. 🙌🏻

Playdate Review

Danielle Heberling — Sat, 28 Sep 2024 05:12:03 +0000

What is a Playdate?

The Playdate is a small hand held game system with a crank, buttons (D-pad, a, b), and a small black and white screen (similar to e-paper).

With dimensions of 76 × 74 × 9 mm, the console can easily fit into most pockets for on the go game playing.

Panic is the company that created the Playdate. You might have heard of Panic from playing Untitled Goose Game or Firewatch. Or you might have used Mac apps such as Transmit or Coda (renamed to Nova).

The CEO and co-founder of Panic gave an in depth talk about the story behind the creation of the Playdate which is entertaining and informative if you would like to go deeper.

Nice!

These are some things that I enjoyed about the Playdate:

You get 24 free games (also called "season one") 2 games delievered each week until you have them all. It's delightful to see that notification light that there's new games to play at the beginning of owning this game system.
Games aren't limited to their catalog, you can also build your own games using their SDK or side load games from places such as itch.io.
The crank as an input device is surpisingly delightful and unique.
Most games have enhanced details. We're talking beautiful storytelling, visuals, and music.
This game system is minimally connected to the internet to allow downloading of games, but otherwise there aren't distractions that would exist on a smart phone.
The battery life has been great so far. Marketing copy for this product boasts 8 hours when playing games and 14 days while in standby clock mode. This has matched my experience. They also use a generic usb-c charger as well, so it's likely you already have the plug/cord for it (though it comes with one).

Oh.

These are some things that I think could be improved with the Playdate:

This game system does not have a backlight, so if you don't have good lighting it will be hard to see the screen.
I'm left handed and would love to have a "lefty" version with the crank on the left side of the game system.
Right now the only body color is yellow which is great, but I would like more color options especially when having multiple Playdates in my house.
There is not a way to shut it off. It will go in standby clock mode which is fine, but I would like the ability to fully shut the game system off without needing to let the battery die.
Right now for sound, it is wired headphones only. Looking through their website it appears that bluetooth is likely coming soon with a software update. To my knowledge the wifi chip they have in the game system should also double as bluetooth, so I don't think a hardware update is needed for this.

Where and how to buy

I'm seeing a few resellers, but the official place to buy and where I bought one was from the official playdate shop.

My opinion

I would highly recommend buying this game system. The Playdate is a lovely distraction from a world full of advertisements, pop ups, and notifications.

As a kid, I could sit for hours and play video games, but as an adult I find that I do not have much time to devote to playing video games. The games on this system are perfect for that. You pick up the game system, play for 15-30 min here and there plus the controls aren't complicated...so you're not spending time trying to remember how to play the game.

Also anything Panic creates, I know will be high quality and well thought out.

Side story

Panic holds a special place in my heart. Back when I recently relocated to Portland, OR I was looking to get out of my current company and saw that they had an open customer support role. I was currently working in customer support, but moonlighting by taking night/weekend classes at a local code school to switch into being a software engineer. I went into their office and did the full interview round (including an interview with their CEO, Cabel Sasser). The next day I received the nicest rejection email from Cabel stating that they really liked me, but think that I would be better suited to becoming a software engineer and recommended that I continue down that path.

Anyway I highly recommend the Playdate and anything that Panic creates, because I know it will be high quality in terms of the aesthetics and story telling.

Devs are Not the Target Market for AI

Danielle Heberling — Fri, 30 Aug 2024 12:12:03 +0000

Background

The year is 2024 in the tech industry. Layoffs and downsizing are everywhere. It is a very challenging job seeker market. There are lots of recent code school and computer science degree graduates seeking their first software engineer job. Amidst all of this noise, there is massive hype about "generative AI."

More on Artificial Intelligence (AI)

In my part of the world, software engineers are tired of hearing about AI and feel like it is mostly a solution looking for a problem. Everyday users I interact with also echo these sentiments.

To contast this, I'm seeing two groups of people that are very excited about AI 👉🏻 people selling things and business executives/investors at various companies.

The motivations of people trying to sell you something is pretty straighforward. They want to sell things to make money for their business.

The executives/investor's motivations are less clear. Here's my theory, many of them view employing humans as a liability both because dealing with humans can be messy but also because paying employees a salary is expensive and they think that AI can be a substitute.

A harsh truth is that many businesses exist to make money and to be profitable. If they no longer need to have the expense of paying employee salaries, this can help them move closer to that goal. Most companies do not exist for the purpose of providing jobs to people.¹

My Opinion on AI

I have concerns over AI ethics. Bias in the models, creator's work getting stolen for use in AI models without compensation, and environmental concerns of needing to run many servers to train these huge models. Many members of the United States government do not understand how technology works, so I don't trust that they can effectively regulate.

I think that these tools are quite good, but they're not sophisticated enough yet to fully replace human software engineering labor on a wider scale.

I have concerns over the ability for human or AI software engineers to be able to effectively troubleshoot and maintain unvetted AI generated code that gets pushed to production.

I am excited about the potential for AI to do boring and mundane tasks for us humans, so we can focus on imaginative creativity. An example: I'd like AI to do the dishes for me, so I can write my next novel, paint my next portrait, or compose my next song. I haven't seen many examples of this yet.

My Opinion Ultimately Doesn't Matter

What does matter if you want to be employed as a software engineer in this industry is the opinion of those who are in positions of power that make the budget and hiring decisions.

Unsolicited Advice

If you want to be hired and maintain a job in this industry as a software engineer, you need to figure out the motivations of people in positions of power and do the work to be able to effectively and genuinely sell yourself to them.

Some Ideas

Here's some themes I've seen lately with people in positions of power at various companies. Note that my bias leans toward small/medium size businesses since that is where the majority of my social network is employed. Your experience may be different.

They want to do more with less

I would advise being curious. When something isn't working as expected, be willing to adventure outside of your comfort zone to figure why, fix it, and learn new things.

On the web development front, more places desire to hire "fullstack" engineers. Just doing frontend is not enough for many companies.

They want to deliver solutions

Focus on shipping. It's not enough to only write code, you need to know how to get a full working application into production that satisfies all product requirements. You also need to know how to troubleshoot and diagnose bugs that happen to your users in production.

It's also important to be someone who can be trusted to follow through with what you say you will deliver. If you get stuck on something, it's your responsiblity to ask for help to get the assistance you need to follow through all the way.

They're excited about AI and some want to replace us devs with AI because it is cheaper than paying a salary

I would advise investing time in learning and using these new AI tools. Learn what they're good at and use them to accelerate your output. Learn what they're bad at and invest the time in learning how to do these things well by yourself. Learning those skills is a good way to emphasize your value to executives as to why they should hire you and pay you a salary instead of using AI.

Closing

Anyway, these are some thoughts I've had over the past few months. I'm just another person on the internet with opinions, giving unsolicited advice. What you do or do not do with it is up to you. I welcome polite engaged feedback and conversation on this topic.

¹ Note that this is not commentary from me saying that I like the status quo. This is a comment plainly stating the status quo.

I'm Worried About Generative AI

Danielle Heberling — Sat, 04 May 2024 22:12:03 +0000

I’m worried about generative AI. I say this as someone who uses AI to help with work tasks daily. I do believe there are some neat things that AI can do and there is high potential for it in the future as it improves, but I have concerns.

No this post isn’t about the common objections I’ve been seeing in the discourse. Things like bias in the models, hallucinations, and people’s jobs being replaced. Although they are very important things to monitor.

I’m concerned with AI monopolizing the focus of companies.

Take the big tech companies for example. There are reports everywhere about multiple layoff rounds but also news of higher rates of investment (both money and people resources) in AI than ever before. These large companies have huge ecosystems that many folks depend on. Does this mean since the main focus is AI that there will be less focus on their non-AI products and services? Does this mean that service reliability will degrade as time goes on because engineers who previously responded to outages were moved to teams working on AI or were laid off?

On the other end of the spectrum we have tech startups. I’m seeing many chatbots being built and their marketing tells me this is the solution to my problems (that I didn’t know I had). I’m excited that this opens up opportunity for new ideas and businesses to take shape, but when the core of a product is centered on what is essentially a black box that might be very wrong sometimes…this doesn’t feel like a good business model to me until the tech matures. My overall impression is that AI is a magic word that these companies can use to attract more customers who like shiny new things and also to attract venture capital funding.

I’ve also seen lots of companies throwing AI at things when it wasn’t needed. An example: a company who will remain nameless was demoing a new tool that uses AI to help users troubleshoot error messages that surface in their app. A developer watching the demo yelled out “or you could consider writing better error messages that users will understand instead of bringing AI into this.”

These are my opinions based off of observation of the world around me. I do not have data to present as evidence. This is all based off of vibes. I sure hope my vibes are wrong, that I’m overreacting to all of this, and that I should take off my tin foil hat.

Time will tell.

AI has promise and it does have the potential to be helpful for specific types of tasks, but let’s not forget that we should attempt to solve problems that already exist before bringing in tools that have the potential to cause even more problems.

What are your thoughts?