DEV Community: Victoria Drake

I Spent $78 Learning Why Bash Still Matters in the AI Age

Victoria Drake — Tue, 19 Aug 2025 12:00:00 +0000

Here's how a little laziness cost me $78.

While working on a personal project recently, I wanted Cline to process about a hundred files that were each in subdirectories of a project. I fired up Cline and picked Gemini 2.5 Pro (context window FTW) and asked it to recurse through the subdirectories, process the files, and put the results in a new file.

Cline got to work… slowly. I watched as the "API Request…" spinner appeared for each file read and each time it saved the results. About twenty minutes and $26 later, it finished.

Okay, I thought, that's not great, but not untenable. The cost of convenience, right? I opened up the results file to take a look and.. sigh. Not great work. It was obvious that some files had been skipped despite my very careful instructions to process each and every one.

So, like a glutton for punishment, I made a list of the files Cline had skipped and asked it to try again. Tired of babysitting, I raised the "Maximum Request Auto Approval" limit to more than I thought would be needed to finish processing the files that were left, and went to take a coffee break.

When I came back, Cline was done. The results? Still not great. Files had still been skipped, some files that were processed were missing results, and, oh, my task bill had risen to $78.

Okay, this was untenable. Reading all this data into context was costly and slow.

Then the coffee started to kick in, I guess, because it dawned on me: why in the world was I using expensive API calls to do something a Bash one-liner could do?

"Cline, write a Bash command that will recurse through the data/ directory and obtain the content of all the files and copy it into a single new file."

Which produced:

find data/ -type f -exec cat {} + > all_data.txt

This command:

find data/ - searches recursively in the data directory.
-type f - specifies that we're looking for files only (not directories, links, etc.).
-exec cat {} + - for all files found, execute the cat command. The {} is a placeholder for the filename, and the + is a crucial optimization that groups multiple filenames into a single cat command, avoiding the overhead of launching a new process for every single file.
> all_data.txt - redirects the standard output of the cat command (which is the concatenated content of all the files) into a new file named all_data.txt.

Then I asked Cline to read the resulting all_data.txt file, process it, and output the results.

It took about two minutes.

And it cost me $0.78.

What just happened?

My initial naive approach had accidentally created a perfect storm of computational inefficiency.

When Cline processed each file individually, it was making separate API calls for every single operation - reads, writes, the works. With about 100 files, that meant roughly 200+ API calls, each one spinning up its own network round-trip with all the latency that entails. Every time I saw that "API Request…" spinner, I was watching money float away into the ether.

But here's the kicker: large language models like Gemini charge based on token consumption.

It's not just the file content they're charging for; every single API call also included the entire conversation history, system prompts, and my instructions.

With a stateless API, that context has to be re-transmitted with every single request. If my average context was around 10,000 tokens and I made 200 calls, I burned through 2 million tokens (10,000 * 200) on overhead alone, before even counting the actual data.

Combining all the files with bash flipped this whole equation on its head. Instead of 200 API calls, I made exactly one. Instead of bearing the network latency for every file operation, combining the files locally on my machine meant the filesystem could actually optimize that work. What had taken almost an hour of network round-trips for Gemini to access all the data was reduced to a couple hundred milliseconds of local file operations.

The expensive lesson in algorithmic thinking

This whole debacle reminded me why understanding the cost model of your tools matters just as much as understanding their capabilities. API pricing is designed around per-request and per-token charges, which naturally punishes fine-grained operations. It's similar to how databases are optimized for bulk operations rather than processing individual rows - the overhead of each transaction quickly becomes the bottleneck.

My first approach had O(n) complexity for API calls, where n equals the number of files. The bash solution reduced that to O(1) by batching everything locally first. That's the difference between linear scaling and constant cost, and at $78, I felt every bit of that mathematical distinction.

There's also something to be said about data locality here. My original method couldn't take advantage of any local caching or filesystem optimizations. Every operation had to go over the network to an API server, get processed, and come back. The bash approach kept everything local until the very end, letting my machine's filesystem cache work its magic.

The real cost of convenience

I'd fallen into the trap of thinking that because I could use an AI tool for everything, I should use it for everything. But there's a difference between leveraging AI for tasks that require intelligence and using it as an expensive replacement for basic system utilities.

The irony is that I probably spent more mental energy managing and troubleshooting the AI approach than I would have just thinking through the problem for five minutes and reaching for the right tool from the start. Sometimes the most sophisticated solution is knowing when to employ a basic tool.

My little bit of laziness bought me a $78 lesson that boils down to this: always understand the economic model of your tools, especially when they're priced per operation. The most elegant and cost-effective solution isn't always the newest and most technically exciting one.

Create Better Code Documentation 10x Faster with AI

Victoria Drake — Tue, 12 Aug 2025 12:00:00 +0000

Documentation has always been one of those "we should do this" tasks that somehow never makes it to the top of the sprint. But what if creating comprehensive, useful documentation could be as straightforward as explaining your code to a colleague?

Conversational AI has changed the game entirely. Instead of starting with a blank page and trying to remember every detail a new team member might need, you can have AI help you think through the process systematically. The result isn't just better docs—it's documentation that actually serves your team's needs as you grow and evolve.

Here's how to use AI to build documentation that scales with your team and genuinely improves how you work together.

Documentation That Welcomes New Team Members

The best part about using AI for documentation is that it naturally thinks from an outsider's perspective. While you and your team already understand your system's quirks and design decisions, AI starts fresh every time—much like a new hire would.

Most conversational AI tools allow you to upload code files or paste code snippets. You can then use prompts that help surface the knowledge your team takes for granted:

Write documentation for a new software engineer joining our team. Assume they're experienced but know nothing about our specific domain, architecture decisions, or business logic. Include the "why" behind non-obvious technical choices and flag anything that might seem strange or unexpected to an outside developer.

This approach reveals the implicit knowledge that experienced team members forget to document—why certain patterns exist, what alternatives were considered, and where the potential gotchas are. It transforms documentation from a chore into a useful onboarding tool that actually reduces the time senior developers spend answering questions.

To create comprehensive documentation you can use immediately, provide the AI with additional context such as:

What the application does and who uses it
Key architectural decisions and their reasoning
Setup and deployment processes
Integration points with other systems
Common troubleshooting scenarios

Your role becomes reviewing and refining rather than writing from scratch—which is often the difference between documentation that gets done and documentation that gets skipped.

Operational Documentation That Actually Helps

One of the most valuable types of documentation is also the most overlooked: information organized for when things go wrong. During incidents, you need answers fast, not comprehensive explanations.

AI excels at creating focused, actionable documentation because you can specify exactly what situation you're optimizing for:

Create incident response documentation for this codebase. Focus on: 1) How to quickly identify what component is failing, 2) Common failure modes and their symptoms, 3) Step-by-step debugging workflows, 4) Who to contact for different types of issues. Write this as if the person reading it is stressed, tired, and needs answers in under 5 minutes.

This type of documentation serves a completely different purpose than your standard README or API docs. It's designed for when your most knowledgeable developers aren't available and someone needs to resolve an issue quickly.

The beauty of AI-generated operational docs is that they're naturally structured for scan-ability rather than linear reading—exactly what you need during high-pressure situations.

Capturing Institutional Knowledge

Here's where AI really shines: helping you identify and document the knowledge that exists only in people's heads. This institutional knowledge is often the difference between a change that takes 30 minutes and one that takes 3 hours of debugging.

You can surface these knowledge gaps by asking AI to analyze your code from a risk perspective:

Analyze this code and identify areas where domain knowledge or business context would be critical for modification. What would a developer need to know about our business, users, or regulatory requirements to safely change this code? What assumptions about data, timing, or external systems are embedded here?

For inline documentation, you can focus on the business logic and integration points that aren't obvious from the code itself:

Add inline documentation to this code file without changing any of the code. Focus on documenting business logic, data assumptions, and integration points that wouldn't be obvious to someone unfamiliar with our domain.

This process often improves the code itself—explaining your logic to AI sometimes reveals opportunities for clearer naming, better structure, or simplified approaches.

Making Documentation a Team Superpower

The real opportunity here isn't just better individual documentation—it's democratizing the ability to create good documentation across your entire team. Developers who previously avoided writing docs because they didn't know where to start now have a collaborative partner to help structure their thoughts.

Start with high-impact documentation: Focus on onboarding guides and operational runbooks first. These provide immediate value and create positive momentum around documentation practices.

Use AI to improve existing docs: You can ask AI to review and improve documentation you already have, suggesting missing information or better organization.

Make it iterative: Documentation doesn't need to be perfect on the first pass. Use AI to create initial drafts that you can refine based on team feedback and real usage patterns.

Leverage different formats: AI can help create everything from README files to inline comments to architectural decision records, adapting the style and depth based on the audience and purpose.

Practical Tips for Better Results

When working with AI to create documentation, providing context about the intended audience and use case dramatically improves the output. Explain not just what the code does, but who will be using the documentation and in what situations.

For complex codebases, you might get better results by working with smaller sections and then asking AI to help you organize everything into a coherent structure. Many AI tools can also provide downloadable files if you specify that in your prompt, which saves time on longer documents.

The goal isn't to replace human judgment in documentation—it's to remove the barriers that prevent good documentation from getting written in the first place. AI handles the initial structure and comprehensive coverage, while you focus on accuracy, team-specific context, and ensuring the documentation actually serves your workflows.

Good documentation transforms how teams work together. It reduces interruptions, accelerates onboarding, and creates resilience when key team members aren't available. With AI handling the heavy lifting of initial creation, maintaining comprehensive documentation becomes achievable rather than aspirational.

Your future team members (and your future self during the next production incident) will definitely appreciate the investment.

How to Choose a Great Tech Hire: Beyond Algorithm Tests and Whiteboard Coding

Victoria Drake — Tue, 05 Aug 2025 12:10:00 +0000

I've seen too many hiring processes that focus on the wrong things. Teams spend hours on algorithm puzzles and whiteboard exercises, then hire someone who can't write readable code or collaborate effectively with colleagues. Six months later, they're dealing with either a performance issue or an unexpected resignation from someone who never felt like they fit the team.

These candidates don't lack technical ability. The problem is that traditional hiring processes don't predict who will actually succeed and stay on your team. After years of hiring engineers and watching some thrive while others struggle, I've learned that the best predictors of long-term success are often the things most interviews completely miss.

Here's what I actually look for when hiring engineers, and why these signals matter more than most technical assessments.

Look for Builders, Not Just Coders

The question that matters most isn't "Can they solve algorithm problems?" It's "Can they build things that solve problems?" There's a fundamental difference between someone who can write code and someone who can deliver working software that serves a purpose.

When I review candidates, I'm looking for evidence that they've built complete projects from start to finish. Not just coding exercises or tutorial follow-alongs, but actual working software that solves real problems. This could be:

Command-line utilities
Web applications
Automation tools
Contributions to open source projects

The complexity matters less than the completeness.

What I'm really evaluating is their ability to navigate the full software development lifecycle. Can they scope a problem, make technical decisions, handle edge cases, write documentation, and ship something that actually works? These are the skills that translate directly to success on your team, regardless of whether they learned them in a computer science program or taught themselves on weekends.

The best candidates can walk you through their projects and explain not just how they built something, but why they made specific technical choices. They understand the trade-offs they made and can articulate what they learned from the experience. This kind of thinking is what distinguishes engineers who will contribute meaningfully to your team from those who will struggle to move beyond assigned tasks.

Evaluate Systems Thinking Over Syntax Knowledge

Most technical interviews focus on whether someone knows specific syntax or can solve isolated problems. But the engineers who succeed on teams are the ones who understand how their code fits into larger systems and affects other people's work.

I look for candidates who demonstrate awareness of follow-on effects. When they describe a project, do they consider:

Performance implications?
Maintainability concerns?
Impact on other developers or users?

Understanding concepts like mutability, thread safety, and code reusability shows technical competence as well as thinking systematically about software as something that exists in a larger context. Engineers who grasp these concepts naturally write code that's easier to debug, extend, and maintain.

During interviews, I ask candidates to explain technical trade-offs they've made in their projects. The specific technologies matter less than their ability to reason about complexity, performance, and maintainability. Engineers who think this way will continue learning and adapting as your company's tech stack evolves.

Assess Communication Skills Through Real Examples

Communication skills aren't just a "nice to have" for engineers—they're essential for team effectiveness. But most hiring processes assess communication through artificial interview scenarios rather than looking at how candidates actually communicate about technical topics.

I spend significant time reviewing candidates' written communication:

How do they explain their projects in README files?
How do they participate in open source discussions?
Can they write clear, helpful documentation?

These examples reveal how they'll communicate with your team when explaining technical decisions, documenting systems, or participating in code reviews.

Pay attention to how candidates describe complex technical concepts during interviews. Can they adjust their explanation based on their audience's technical background? Do they provide context and examples? Can they acknowledge when they don't know something without becoming defensive?

The engineers who succeed long-term are those who can collaborate effectively across different skill levels and backgrounds. They can explain technical concepts to non-technical stakeholders, provide helpful code review feedback, and contribute to architectural discussions. These collaborative skills are often better predictors of success than pure technical ability.

Identify Team Players Through Contribution Patterns

The best predictor of how someone will behave on your team is how they've behaved on other teams. Rather than asking hypothetical questions about teamwork, look at concrete examples of how candidates have collaborated with others.

Open source contributions provide excellent insight into someone's collaborative style:

How do they handle feedback on their code?
Do they contribute thoughtfully to discussions?
Can they work within existing conventions and standards?
Do they help other contributors or just focus on their own work?

For candidates without extensive open source history, look at how they talk about past team experiences. Do they credit others for successes? Can they describe situations where they helped colleagues or learned from feedback? How do they handle disagreement or conflict?

I'm particularly interested in candidates who show evidence of helping others grow. Engineers who mentor junior developers, contribute to team documentation, or improve development processes tend to have a positive impact that extends far beyond their individual contributions.

Evaluate Learning Ability Over Current Knowledge

Technology changes rapidly, which means the specific skills someone has today matter less than their ability to acquire new skills as needed. The engineers who thrive long-term are those who stay curious and adapt effectively to new challenges.

During interviews, I ask candidates about times they had to learn something completely new for a project:

How did they approach unfamiliar technologies?
What resources did they use?
How did they validate their understanding?

The process they describe reveals more about their potential than any specific technology they currently know.

I also look for evidence of intellectual humility. Can candidates acknowledge the limits of their knowledge? Do they ask thoughtful questions? Are they excited about learning from more experienced team members? Engineers who combine confidence in their abilities with openness to learning tend to grow quickly and integrate well with existing teams.

What This Means for Your Hiring Process

Identifying these qualities requires a different approach than traditional technical interviews. Instead of algorithm problems, focus on discussing real projects and technical decisions. Instead of whiteboard coding, review actual code they've written and ask them to explain their thinking.

Spend time on behavioral questions that reveal collaborative patterns and learning ability. Make time for informal conversation about what kind of work environment they thrive in and what they're excited to learn next.

Most importantly, involve your team in the hiring process. The people who will work directly with your new hire are often better at assessing team fit than individual interviewers making isolated decisions.

Remember that hiring is ultimately about predicting future success, not just evaluating current abilities. The candidates who can build complete projects, think systematically about technical decisions, communicate effectively, and continue learning will contribute more to your team's long-term success than those who simply perform well on coding tests.

Your perfect candidate isn't necessarily the most technically skilled or the most knowledgeable about your domain. It's the person who will grow with your team and contribute to the kind of collaborative, effective engineering culture that retains great people and delivers great software.

What qualities do you prioritize when hiring engineers? Share your experiences in the comments below.

Do One Thing: Mastering Prioritization for High-Performing Teams

Victoria Drake — Tue, 29 Jul 2025 13:03:00 +0000

In the engineering teams I lead, "priority" has no plural form. This drives some people slightly crazy, especially those who like to hedge their bets with phrases like "top priorities" or "critical priorities." But I've learned that the moment you allow multiple top priorities, you've essentially created zero priorities.

I discovered this the hard way while working with a team that was constantly context-switching between "urgent" projects. Everyone was busy, morale was decent, but we weren't actually shipping much of value. During one particularly frustrating week, I counted seventeen different tasks that had been labeled as "high priority" by various stakeholders. Our standups felt like disaster reports, and I realized we'd created a system where being busy had become more important than being effective.

The solution turned out to be surprisingly simple, though not easy to implement: put everything into a single, ordered list where only one thing can be most important at any given time.

The Radical Transparency of a Central List

Most teams I've encountered operate like a collection of individual to-do lists with some coordination meetings sprinkled on top. Engineering works on technical debt, product pushes for new features, leadership wants infrastructure improvements, and everyone optimizes their own piece of the puzzle. The result is a lot of activity that doesn't add up to meaningful progress.

A single, centralized, prioritized list changes the entire dynamic. Everyone can see what's actually being worked on, what's coming next, and most importantly, what's not getting done and why. This visibility creates natural conversations about trade-offs that simply don't happen when work is siloed.

I've watched teams discover they were working on competing solutions to the same problem, simply because no one had a complete view of active work. Others realized they were delaying important projects because someone assumed "someone else" was handling the dependency. When everything is visible and ordered, these coordination problems become obvious and fixable.

The transparency also creates a different kind of accountability. When priorities are public and explicit, it becomes much harder to justify working on pet projects or avoiding difficult tasks. The list becomes a shared source of truth that guides decisions rather than each person interpreting priorities through their own lens.

Autonomy Within Structure

One concern I hear frequently is that a single priority list will turn people into order-takers rather than creative problem-solvers. In practice, I've found exactly the opposite happens when you implement it correctly.

The key is encouraging people to choose the highest-priority task they can effectively tackle rather than assigning specific tasks to specific people. Someone might skip over the absolute top item because it requires domain knowledge they don't have, but they can pick up the second or third item that lets them contribute meaningfully while learning something new.

This approach leverages the fact that your team members understand their own capabilities and growth goals better than you do. A senior engineer might choose to mentor a junior developer on a complex task. A frontend specialist might want to tackle a backend task to broaden their skills. These decisions create better outcomes in the long term than top-down task assignment while still maintaining focus on organizational priorities.

The autonomy comes from trusting people to make good decisions about how to contribute most effectively, while the structure comes from ensuring those contributions align with actual business needs.

The Art of Making Yourself Redundant

If your team frequently asks you what they should work on next, you've accidentally created a bottleneck—and it's you. This is one of the most common scaling problems I see with engineering leaders who transition from individual contributor roles.

The goal is building a system where intelligent people can make good decisions without constant input from leadership. This requires making context painfully available—team goals, product strategy, architectural decisions, customer feedback, and anything else that influences prioritization should be accessible and current.

I've found that the difference between teams that scale smoothly and teams that hit velocity walls usually comes down to how well they've documented the reasoning behind decisions. When someone can understand not just what to build but why it matters and how it fits into the larger strategy, they can make smart trade-offs independently.

This redundancy becomes especially critical during high-pressure situations. When systems are down or deadlines are looming, you don't want your team waiting for permission to take action. Teams that have practiced autonomous decision-making within clear constraints can respond quickly and effectively without requiring heroic coordination efforts.

The Cultural Transformation

What surprises most leaders is how much this simple change affects team culture. When priorities are clear and transparent, several things happen that go far beyond improved task management.

First, political conversations about priority disappear. There's no point in lobbying for your favorite project when the criteria for prioritization are explicit and the current order is visible to everyone. Energy that was spent on organizational maneuvering gets redirected toward actual work.

Second, people start thinking about their contributions differently. Instead of optimizing for individual productivity, they begin considering how their work fits into team objectives. This naturally leads to better collaboration and knowledge sharing.

Third, the team develops a shared sense of progress and momentum. When everyone can see important work getting completed in priority order, it creates a satisfying rhythm that isolated individual achievements can't match.

Implementation Reality

The biggest challenge isn't creating the list—it's maintaining the discipline to use it consistently. Teams often start strong but gradually drift back to multiple priority tracks when pressure increases or when compelling new opportunities arise.

I've learned to treat priority discipline like any other technical practice that requires ongoing attention. Schedule regular review sessions to reorder the list, have explicit discussions about what we're choosing not to do, and consistently communicate why keeping a single-priority focus helps maintain development velocity.

The payoff: teams that ship more valuable work with less stress and confusion. When everyone understands what matters most and feels empowered to contribute effectively, both productivity and job satisfaction improve dramatically.

Most importantly, single-priority focus creates sustainable high performance rather than the boom-and-bust cycles that come from constantly shifting between competing urgent demands. Teams learn to work steadily toward important goals rather than reacting to whatever feels most pressing in the moment.

What's your experience with team prioritization? Have you found effective ways to maintain focus while preserving autonomy? Share your thoughts in the comments below.

The Descent Is Harder Than the Climb: Lessons in Leadership from Mt. Fuji

Victoria Drake — Wed, 23 Jul 2025 18:24:48 +0000

In 2017, I climbed Mt. Fuji in sneakers. This was not a deliberate choice to increase the challenge—it was the result of excellent research and poor judgment about what that research actually meant.

Everything I'd read suggested that Mt. Fuji was the "cakewalk of mountain climbing." Physically, the hardest portions amounted to scrambling over some big boulders. Most of the climb was no more taxing than hiking or climbing stairs. Japanese folks in their eighties made the journey for spiritual reasons. There were huts along the way for rest, food, and water. Based on this research, I concluded that sneakers would be perfectly adequate.

The ascent was everything I'd been promised. I experienced sights I'd never imagined—cities glowing through breaks in clouds from above, walking through paths of grey nothingness where the trail disappeared into cloud cover. Each station marker brought genuine pride and accomplishment. Even the pre-dawn summit queue with 5,000 other climbers, standing in freezing darkness for hours, felt manageable. We reached the summit before sunrise, and it remains one of the most beautiful moments I've experienced.

Then came the descent. That's where I learned that all the research in the world about reaching goals doesn't prepare you for what comes after you achieve them.

When Success Becomes the Set Up for Failure

The descent from Mt. Fuji is essentially a loosely-packed dirt and gravel road on a steep decline. With proper hiking boots and trekking poles, it's probably manageable. In flat-soled street shoes, I fell constantly, and fell hard—every three steps, for hours. I tried to take larger steps; it didn't help. I tried to take smaller steps; that didn't help, either. I tried cunningly to find a way to surf-slide my way down the mountainside and nearly ended up with a mouthful of dirt. As if literally rubbing salt into my wounds, without the gaiters I hadn't brought, sand found its way into my shoes. It was without a doubt the most stupefyingly discouraging experience of my life.

As I picked myself up repeatedly, covered in dirt with scratched elbows, seasoned hikers passed me with ease. Many of them could have been my grandparents, using proper equipment and technique to descend at a steady pace while I struggled and stopped to pour tiny rocks out of my sneakers. The contrast was humbling and instructive.

This experience taught me something crucial about leadership that I've applied countless times since: the skills and preparation that get you to success are often different from the skills required to maintain or scale that success. The descent is frequently harder than the climb, and most people don't prepare for it adequately.

The Post-Achievement Challenge

In business and team leadership, I've watched this pattern repeat consistently. The energy, skills, and resources required to achieve a goal are usually well-understood and planned for. But the challenges that come after success—maintaining market position, scaling team culture, or managing the operational complexity of growth—often catch leaders unprepared.

I've seen teams that executed brilliant product launches struggle with customer support and maintenance. Startups that successfully raised funding stumble when it comes to executing on their promises to investors. Engineering teams that built innovative solutions fail to create sustainable systems for maintaining and scaling those solutions.

The problem isn't lack of capability—it's that the descent requires different preparation and different skills than the ascent. What gets you to the summit (innovation, speed, breakthrough thinking) often isn't what gets you safely back to basecamp (consistency, processes, systematic execution).

Learning from Those Who've Made the Journey

Watching those experienced hikers pass me on Mt. Fuji was initially frustrating, but it became one of the most valuable parts of the experience. They had proper equipment, understood the terrain, and moved with confidence that came from experience. Most importantly, they had prepared specifically for the descent, not just the climb.

In leadership roles, I've learned to actively seek out people who've successfully navigated the "descent" phase of challenges I'm facing. Entrepreneurs who've managed hypergrowth. Product managers who've maintained market leadership over multiple years. Engineering leaders who've scaled teams from ten to fifty people, or CEOs who've scaled companies from fifty to five hundred.

These conversations can reveal patterns you may not have discovered on your own. Successful scaling requires different organizational structures than startup growth. Maintaining team culture during rapid hiring requires intentional systems that don't emerge naturally. Sustaining innovation while managing operational complexity demands new kinds of leadership skills.

People who've successfully managed the descent often have hard-won wisdom about preparation and technique that isn't captured in most "how to reach the summit" advice.

Building Skills Before You Need Them

The most effective leaders I know prepare for post-success challenges while they're still climbing toward their initial goals. They think systematically about what will be required to maintain and scale whatever they're building, not just achieve it.

This means:

Building operational capabilities alongside product capabilities
Developing team management skills in individual contributors
Creating sustainable processes while you're still in startup mode
Planning for the maintenance and evolution of systems as part of their initial implementation

It also means recognizing that the mindset and skills that drive breakthrough achievements—risk-taking, speed, creative problem-solving—need to be balanced with different capabilities like consistency, systematic thinking, and process optimization.

I've learned to explicitly ask: "What will success look like, and what challenges will that create?" This question reveals preparation gaps that aren't obvious when you're focused entirely on reaching your goals.

When You Find Yourself Unprepared

Despite best intentions, you'll sometimes find yourself in descent mode without proper preparation—leading a team through unexpected growth, managing a product that succeeded beyond projections, or scaling systems that weren't designed for current loads. The Mt. Fuji experience taught me how to navigate these situations effectively.

First, acknowledge the reality of your situation without wasting energy on regret about preparation gaps. You can't change what you didn't know or plan for previously, but you can adapt your approach based on current conditions. Take the time to solidify new goals in writing, then evaluate whether your efforts are serving them effectively.

Second, focus on learning from people who are managing similar challenges successfully. This isn't the time for pride or trying to figure everything out independently. The hikers who passed me weren't showing off—they had practical knowledge that could help. Conversations you have with others who came before you can save you from a lot of stumbles.

Third, lift your gaze. While the ascent phase requires day-to-day tactical thinking, the descent phase requires a strategic longer-term outlook. Implementing systems and culture that support continued success will require patience, persistence, and often a completely different pace than what got you to the summit. Expecting it to be as expedient as the climb leads to frustration and poor decision-making.

Finding Meaning in the Difficult Parts

Eventually, I reached the bottom of Mt. Fuji, exhausted and humbled but intact. At a tiny basecamp shop, I ate the most delicious bowl of ramen and the tastiest mountain-shaped sponge cake I'll likely ever have.

Even when you're unprepared and struggling, there's value in the journey itself. The descent taught me lessons about preparation, humility, and persistence that I've applied to all sorts of challenges for years since.

Preparing for Your Next Descent

There is a Japanese proverb: "A wise man will climb Mt Fuji once; a fool will climb Mt Fuji twice." I suspect this wisdom is based entirely on the difficulty of the descent. But in leadership, you don't get to choose how many times you'll face descent challenges—they're inevitable parts of any significant journey.

The key is recognizing that achieving your goals is often just the beginning of a different kind of challenge. Success creates new problems that require different skills, different preparation, and different mindsets than what got you there initially.

Whether you're building teams, scaling products, or managing organizational growth, prepare for the descent while you're planning the climb. Study what happens after success. Learn from people who've navigated similar transitions. Build operational capabilities alongside innovative ones.

Most importantly, remember that the descent is still part of the journey, not a failure of the ascent. The challenges that come with success are signs that you've accomplished something meaningful. Navigate them with patience, preparation, and the understanding that getting back to basecamp safely can be an even more important achievement than reaching the summit.

How to Future-Proof Your Software Engineering Career for the Age of AGI

Victoria Drake — Wed, 21 Aug 2024 13:30:24 +0000

In the viral essay The Decade Ahead, Leopold Aschenbrenner predicts that Artificial General Intelligence (AGI) will be a reality in only a few years.

If that happens, the skills and experiences that will prepare you for software engineering with AGI will likely focus on understanding, managing, and integrating highly autonomous systems.

Here are some key areas of work to focus on now to prepare you for the AGI era.

1. Mastering Machine Learning and Deep Learning

Engineers with expertise in these fields will be at the forefront. You’ll need to understand how to create and train models that can learn from a wide variety of data sources, adapt to new information, and generalize across different tasks. Focus on areas like reinforcement learning, unsupervised learning, and neural networks. Keep an eye out for new paradigms in AI that might emerge.

Job titles to watch : Machine Learning Engineer, AI Research Scientist.

2. Software Engineering with a Focus on AI Integration

Traditional software engineering roles will evolve to integrate AI components seamlessly. This includes developing frameworks where AGI can be applied or integrated into existing systems.

Job titles to watch : Full Stack Developer with AI specialization, AI Software Engineer.

3. Navigating Ethics and AI Governance

As AGI could pose significant ethical and governance challenges, roles focusing on the ethical implications, policy-making, and regulatory compliance will be crucial. This includes ensuring AGI systems operate within legal and ethical frameworks. Public as well as private sector experience will be valuable.

Job titles to watch : AI Ethics Analyst, Policy Advisor for AI, Compliance Officer for AI Systems.

4. Evolving Human Computer Interaction (HCI)

HCI will quickly transform into Human-AI Interaction Design. AGI systems will need to interact seamlessly with humans. Companies will need interfaces where humans can interact with AGI systems effectively, built by engineers who understand cognitive psychology and UX/UI design for AI systems.

Engineers skilled in designing intuitive interfaces and interactions between humans and intelligent systems will be highly successful in AGI integration.

Job titles to watch : Interaction Designer for AI, User Experience Researcher for AI Systems.

5. Enhancing Autonomous Systems and Robotics

If AGI leads to more autonomous robots, engineers who can design, build, and program robots with AGI capabilities will be in demand. This includes understanding not just robotics but how AGI can enhance robotic functionality.

Working on autonomous systems, whether in robotics, self-driving vehicles, or drones, can provide practical experience with highly independent systems. These skills will be transferrable to managing and optimizing AGI-based autonomous agents.

Job titles to watch : Robotics Engineer, Automation Specialist.

6. Pioneering Hardware Development for AGI

We’ll need engineers working on specialized hardware that can support AGI. Technologies like neuromorphic computing chips or quantum computing might be necessary for the computational power AGI would require.

Job titles to watch : Hardware Engineer for AI, Quantum Computing Engineer.

7. Securing the Future: Cybersecurity for AGI

AGI systems will introduce new security challenges. Engineers with expertise in cybersecurity will be in high demand to protect AGI systems from national security threats, ensure data privacy, and secure AI-driven decision-making processes against manipulation.

Job titles to watch : AI Security Specialist, Cybersecurity Analyst for AI.

8. Data Engineering: Fueling AGI with Information

Handling large-scale data systems will remain important. Engineers with expertise in big data, data pipelines, and real-time data processing will be essential in feeding AGI systems the information they need to operate.

Job titles to watch : Data Engineer, Big Data Architect.

9. Building Infrastructure for AGI

AGI will require robust and scalable infrastructure on a never-before-seen scale. Engineers with experience in cloud computing, distributed systems, and infrastructure as code (IaC) will be crucial in building the systems that support AGI.

Job titles to watch : Cloud Infrastructure Engineer, Systems Architect for AI.

10. Cross-Disciplinary Collaboration in the AGI Era

Working in roles that involve cross-disciplinary collaboration, such as roles in research or innovation labs, can provide engineers with the ability to think broadly and integrate knowledge from various fields. Combining skills from fields such as biology (for bioinformatics or synthetic biology with AGI) and psychology (for understanding human-AI interaction) will be vital in the AGI era.

Job titles to watch : Bioinformatics Engineer, Environmental Data Scientist.

11. Education and Training for an AGI-Ready Workforce

Developing new educational programs or training modules that teach engineers how to work with AGI, focusing on both technical skills and the philosophical or ethical considerations.

Job titles to watch : AI Curriculum Developer, Training Specialist for AI Technologies.

12. Shaping Regulations in an AGI-Driven World

Engineers working on regulatory technology (RegTech) will gain insight into compliance and governance, which will be critical as AGI evolves within legal frameworks. Understanding how to navigate and shape regulations will be vital.

Job titles to watch : Regulatory Engineer, Compliance Specialist for AI.

13. Research and Development (R&D) in AGI-related Areas

Finally, engineers who are involved in cutting-edge research in AGI, cognitive computing, or advanced AI labs will be directly contributing to and understanding the frontiers of AGI technology, giving them a head start in a world where AGI is a reality.

Job titles to watch : AGI Research Scientist, Cognitive Computing Engineer.

Future Job Titles

The transition to an AGI world would likely see a blend of these roles, where engineers might need to be polymaths, understanding not just one but multiple areas of technology and science. It’s important to grow your technical skills, but also practice adaptability and continuous learning.

Be on the lookout for roles that might not directly mention AGI but are foundational in AI, machine learning, and related technologies. As you choose your next role, think ahead to how you can tailor your focus and future-proof your work for the AGI era.

Post to your static website from your iPhone

Victoria Drake — Sun, 05 May 2024 00:00:00 +0000

I love websites. I love static sites in particular. But I know that sometimes it’s just not practical to write and post only from your computer. With my hands full raising a family, I do a lot more development in stops and starts from my phone these days than I thought I ever would.

So I brought together everything that’s great about Hugo plus everything that’s great about sharing your 3AM thoughts with the world from your phone, thanks to Collected Notes. I put it in a new Hugo site template with a fancy new theme I call Quint.

You can deploy the Quint site template with one button.

The Quint template can use the Collected Notes app as a CMS and also saves your posts to the site repository, for redundancy. It fetches new posts each time you build, and if you’re deploying via Netlify or GitHub Actions, you can use a webhook to deploy the site whenever you make a new post with Collected Notes.

To set up your own site:

Deploy the Quint template to Netlify with the button above, or clone the repo if you plan to use another deployment solution.
Sign up for Collected Notes if you haven’t already (there’s a free plan) and download the Collected Notes app on your iPhone.
Update the utils/fetch-posts.js file to use your Collected Notes site name.
Allow the GitHub Action to push changes back to your repository to save your posts. Under Settings > Actions > General > Workflow permissions, choose Read and write permissions.

Netlify will trigger a new build each time you push to your site repo, or, if you have a Collected Notes Premium subscription, you can set a Netlify Build Hook URL in your Collected Notes site settings to automatically redeploy the site when you make a post or update an existing post.

I hope this template helps out busy people like you! I’m using this solution myself, of course, to write the next chapter of my one-bag era – with my phone in one hand and a coffee in the other.

How to send long text input to ChatGPT using the OpenAI API

Victoria Drake — Tue, 26 Sep 2023 09:46:36 +0000

In a previous post, I showed how you can apply text preprocessing techniques to shorten your input length for ChatGPT. Today in the web interface (chat.openai.com), ChatGPT allows you to send a message with a maximum token length of 4,096.

There are bound to be situations in which this isn’t enough, such as when you want to read in a large amount of text from a file. Using the OpenAI API allows you to send many more tokens in a messages array, with the maximum number depending on your chosen model. This lets you provide large amounts of text to ChatGPT using chunking. Here’s how.

Chunking your input

The gpt-4 model currently has a maximum content length token limit of 8,192 tokens. (Here are the docs containing current limits for all the models.) Remember that you can first apply text preprocessing techniques to reduce your input size – in my previous post I achieved a 28% size reduction without losing meaning with just a little tokenization and pruning.

When this isn’t enough to fit your message within the maximum message token limit, you can take a general programmatic approach that sends your input in message chunks. The goal is to divide your text into sections that each fit within the model’s token limit. The general idea is to:

Tokenize and split text into chunks based on the model’s token limit. It’s better to keep message chunks slightly below the token limit since the token limit is shared between your message and ChatGPT’s response.
Maintain context between chunks, e.g. avoid splitting a sentence in the middle.

Each chunk is sent as a separate message in the conversation thread.

Handling responses

You send your chunks to ChatGPT using the OpenAI library’s ChatCompletion. ChatGPT returns individual responses for each message, so you may want to process these by:

Concatenating responses in the order you sent them to get a coherent answer.
Manage conversation flow by keeping track of which response refers to which chunk.
Formatting the response to suit your desired output, e.g. replacing \n with line breaks.

Putting it all together

Using the OpenAI API, you can send multiple messages to ChatGPT and ask it to wait for you to provide all of the data before answering your prompt. Being a language model, you can provide these instructions to ChatGPT in plain language. Here’s a suggested script:

Prompt: Summarize the following text for me

To provide the context for the above prompt, I will send you text in parts. When I am finished, I will tell you “ALL PARTS SENT”. Do not answer until you have received all the parts.

I created a Python module, chatgptmax, that puts all this together. It breaks up a large amount of text by a given maximum token length and sends it in chunks to ChatGPT.

You can install it with pip install chatgptmax, but here’s the juicy part:

import os
import openai
import tiktoken

# Set up your OpenAI API key
# Load your API key from an environment variable or secret management service
openai.api_key = os.getenv("OPENAI_API_KEY")

def send(
    prompt=None,
    text_data=None,
    chat_model="gpt-3.5-turbo",
    model_token_limit=8192,
    max_tokens=2500,
):
    """
    Send the prompt at the start of the conversation and then send chunks of text_data to ChatGPT via the OpenAI API.
    If the text_data is too long, it splits it into chunks and sends each chunk separately.

    Args:
    - prompt (str, optional): The prompt to guide the model's response.
    - text_data (str, optional): Additional text data to be included.
    - max_tokens (int, optional): Maximum tokens for each API call. Default is 2500.

    Returns:
    - list or str: A list of model's responses for each chunk or an error message.
    """

    # Check if the necessary arguments are provided
    if not prompt:
        return "Error: Prompt is missing. Please provide a prompt."
    if not text_data:
        return "Error: Text data is missing. Please provide some text data."

    # Initialize the tokenizer
    tokenizer = tiktoken.encoding_for_model(chat_model)

    # Encode the text_data into token integers
    token_integers = tokenizer.encode(text_data)

    # Split the token integers into chunks based on max_tokens
    chunk_size = max_tokens - len(tokenizer.encode(prompt))
    chunks = [
        token_integers[i : i + chunk_size]
        for i in range(0, len(token_integers), chunk_size)
    ]

    # Decode token chunks back to strings
    chunks = [tokenizer.decode(chunk) for chunk in chunks]

    responses = []
    messages = [
        {"role": "user", "content": prompt},
        {
            "role": "user",
            "content": "To provide the context for the above prompt, I will send you text in parts. When I am finished, I will tell you 'ALL PARTS SENT'. Do not answer until you have received all the parts.",
        },
    ]

    for chunk in chunks:
        messages.append({"role": "user", "content": chunk})

        # Check if total tokens exceed the model's limit and remove oldest chunks if necessary
        while (
            sum(len(tokenizer.encode(msg["content"])) for msg in messages)
            > model_token_limit
        ):
            messages.pop(1) # Remove the oldest chunk

        response = openai.ChatCompletion.create(model=chat_model, messages=messages)
        chatgpt_response = response.choices[0].message["content"].strip()
        responses.append(chatgpt_response)

    # Add the final "ALL PARTS SENT" message
    messages.append({"role": "user", "content": "ALL PARTS SENT"})
    response = openai.ChatCompletion.create(model=chat_model, messages=messages)
    final_response = response.choices[0].message["content"].strip()
    responses.append(final_response)

    return responses

Here’s an example of how you can use this module with text data read from a file. (chatgptmax also provides a convenience method for getting text from a file.)

# First, import the necessary modules and the function
import os

from chatgptmax import send

# Define a function to read the content of a file
def read_file_content(file_path):
    with open(file_path, 'r', encoding='utf-8') as file:
        return file.read()

# Use the function
if __name__ == " __main__":
    # Specify the path to your file
    file_path = "path_to_your_file.txt"

    # Read the content of the file
    file_content = read_file_content(file_path)

    # Define your prompt
    prompt_text = "Summarize the following text for me:"

    # Send the file content to ChatGPT
    responses = send(prompt=prompt_text, text_data=file_content)

    # Print the responses
    for response in responses:
        print(response)

Error handling

While the module is designed to handle most standard use cases, there are potential pitfalls to be aware of:

Incomplete sentences : If a chunk ends in the middle of a sentence, it might alter the meaning or context. To mitigate this, consider ensuring that chunks end at full stops or natural breaks in the text. You could do this by separating the text-chunking task into a separate function that:
1. Splits the text into sentences.
2. Iterates over the sentences and adds them to a chunk until the chunk reaches the maximum size.
3. Starts a new chunk when the current chunk reaches the maximum size or when adding another sentence would exceed the maximum size.
API connectivity issues : There’s always a possibility of timeouts or connectivity problems during API calls. If this is a significant issue for your application, you can include retry logic in your code. If an API call fails, the script could wait for a few seconds and then try again, ensuring that all chunks are processed.
Rate limits : Be mindful of OpenAI API’s rate limits. If you’re sending many chunks in rapid succession, you might hit these limits. Introducing a slight delay between calls or spreading out requests can help avoid this.

Optimization

As with any process, there’s always room for improvement. Here are a couple of ways you might optimize the module’s chunking and sending process further:

Parallelizing API calls : If OpenAI API’s rate limits and your infrastructure allow, you could send multiple chunks simultaneously. This parallel processing can speed up the overall time it takes to get responses for all chunks. Unless you have access to OpenAI’s 32k models or need to use small chunk sizes, however, parallelism gains are likely to be minimal.
Caching mechanisms : If you find yourself sending the same or similar chunks frequently, consider implementing a caching system. By storing ChatGPT’s responses for specific chunks, you can retrieve them instantly from the cache the next time, saving both time and API calls.

Now what

If you found your way here via search, you probably already have a use case in mind. Here are some other (startup) ideas:

You’re a researcher who wants to save time by getting short summaries of many lengthy articles.
You’re a legal professional who wants to analyze long contracts by extracting key points or clauses.
You’re a financial analyst who wants to pull a quick overview of trends from a long report.
You’re a writer who wants feedback on a new article or chapter… without having to actually show it to anyone yet.

Do you have a use case I didn’t list? Let me know about it! In the meantime, have fun sending lots of text to ChatGPT with chatgptmax.

Optimizing text for ChatGPT: NLP and text pre-processing techniques

Victoria Drake — Tue, 19 Sep 2023 16:06:36 +0000

In order for chatbots and voice assistants to be helpful, they need to be able to take in and understand our instructions in plain language using Natural Language Processing (NLP). ChatGPT relies on a blend of advanced algorithms and text preprocessing methods to make sense of our words. But just throwing a wall of text at it can be inefficient – you might be dumping in a lot of noise with that signal and hitting the text input limit.

Text preprocessing can help shorten and refine your input, ensuring that ChatGPT can grasp the essence without getting overwhelmed. In this article, we’ll explore these techniques, understand their importance, and see how they make your interactions with tools like ChatGPT more reliable and productive.

Text preprocessing

Text preprocessing prepares raw text data for analysis by NLP models. Generally, it distills everyday text (like full sentences) to make it more manageable or concise and meaningful. Techniques include:

Tokenization: splitting up text by sentences or paragraphs. For example, you could break down a lengthy legal document into individual clauses or sentences.
Extractive summarization: selecting key sentences from the text and discarding the rest. Instead of reading an entire 10-page document, extractive summarization could pinpoint the most crucial sentences and give you a concise overview without delving into the details.
Abstractive summarization: generating a concise representation of the text content, for example, turning a 10-page document into a brief paragraph that captures the document’s essence in new wording.
Pruning: removing redundant or less relevant parts. For example, in a verbose email thread, pruning can help remove all the greetings, sign-offs, and other repetitive elements, leaving only the core content for analysis.

While all these techniques can help reduce the size of raw text data, some of these techniques are easier to apply to general use cases than others. Let’s examine how text preprocessing can help us send a large amount of text to ChatGPT.

Tokenization and ChatGPT input limits

In the realm of Natural Language Processing (NLP), a token is the basic unit of text that a system reads. At its simplest, you can think of a token as a word, but depending on the language and the specific tokenization method used, a token can represent a word, part of a word, or even multiple words.

While in English we often equate tokens with words, in NLP, the concept is broader. A token can be as short as a single character or as long as a word. For example, with word tokenization, the sentence “Unicode characters such as emojis are not indivisible. ✂️” can be broken down into tokens like this: [“Unicode”, “characters”, “such”, “as”, “emojis”, “are”, “not”, “indivisible”, “.”, “✂️”]

In another form called Byte-Pair Encoding (BPE), the same sentence is tokenized as: [“Un”, “ic”, “ode”, " characters", " such", " as", " em, “oj”, “is”, " are", " not", " ind", “iv”, “isible”, “.”, " �", “�️”]. The emoji itself is split into tokens containing its underlying bytes.

Depending on the ChatGPT model chosen, your text input size is restricted by tokens. Here are the docs containing current limits. BPE is used by ChatGPT to determine token count, and we’ll discuss it more thoroughly later. First, we can programmatically apply some preprocessing techniques to reduce our text input size and use fewer tokens.

A general programmatic approach

For a general approach that can be applied programmatically, pruning is a suitable preprocessing technique. One form is stop word removal, or removing common words that might not add significant meaning in certain contexts. For example, consider the sentence:

“I always enjoy having pizza with my friends on weekends.”

Stop words are often words that don’t carry significant meaning on their own in a given context. In this sentence, words like “I”, “always”, “enjoy”, “having”, “with”, “my”, “on” are considered stop words.

After removing the stop words, the sentence becomes:

“pizza friends weekends.”

Now, the sentence is distilled to its key components, highlighting the main subject (pizza) and the associated context (friends and weekends). If you find yourself wishing you could convince people to do this in real life (cough_meetings_cough)… you aren’t alone.

Stop word removal is straightforward to apply programmatically: given a list of stop words, examine some text input to see if it contains any of the stop words on your list. If it does, remove them, then return the altered text.

def clean_stopwords(text: str) -> str:
    stopwords = ["a", "an", "and", "at", "but", "how", "in", "is", "on", "or", "the", "to", "what", "will"]
    tokens = text.split()
    clean_tokens = [t for t in tokens if not t in stopwords]
    return " ".join(clean_tokens)

To see how effective stop word removal can be, I took the entire text of my Tech Leader Docs newsletter (17,230 words consisting of 104,892 characters) and processed it using the above function. How effective was it? The resulting text contained 89,337 characters, which is about a 15% reduction in size.

Other pruning techniques can also be applied programmatically. Removing punctuation, numbers, HTML tags, URLs and email addresses, or non-alphabetical characters are all valid pruning techniques that can be straightforward to apply. Here is a function that does just that:

import re

def clean_text(text):
    # Remove URLs
    text = re.sub(r'http\S+', '', text)

    # Remove email addresses
    text = re.sub(r'\S+@\S+', '', text)

    # Remove everything that's not a letter (a-z, A-Z)
    text = re.sub(r'[^a-zA-Z\s]', '', text)

    # Remove whitespace, tabs, and new lines
    text = ''.join(text.split())

    return text

What measure of length reduction might we be able to get from this additional processing? Applying these techniques to the remaining characters of Tech Leader Docs results in just 75,217 characters; an overall reduction of about 28% from the original text.

More opinionated pruning, such as removing short words or specific words or phrases, can be tailored to a specific use case. These don’t lend themselves well to general functions, however.

Now that you have some text processing techniques in your toolkit, let’s look at how a reduction in characters translates to fewer tokens used when it comes to ChatGPT. To understand this, we’ll examine Byte-Pair Encoding.

Byte-Pair Encoding (BPE)

Byte-Pair Encoding (BPE) is a subword tokenization method. It was originally introduced for data compression but has since been adapted for tokenization in NLP tasks. It allows representing common words as tokens and splits more rare words into subword units. This enables a balance between character-level and word-level tokenization.

Let’s make that more concrete. Imagine you have a big box of LEGO bricks, and each brick represents a single letter or character. You’re tasked with building words using these LEGO bricks. At first, you might start by connecting individual bricks to form words. But over time, you notice that certain combinations of bricks (or characters) keep appearing together frequently, like “th” in “the” or “ing” in “running.”

BPE is like a smart LEGO-building buddy who suggests, “Hey, since ’th’ and ‘ing’ keep appearing together a lot, why don’t we glue them together and treat them as a single piece?” This way, the next time you want to build a word with “the” or “running,” you can use these glued-together pieces, making the process faster and more efficient.

Colloquially, the BPE algorithm looks like this:

Start with single characters.
Observe which pairs of characters frequently appear together.
Merge those frequent pairs together to treat them as one unit.
Repeat this process until you have a mix of single characters and frequently occurring character combinations.

BPE is a particularly powerful tokenization method, especially when dealing with diverse and extensive vocabularies. Here’s why:

Handling rare words: Traditional tokenization methods might stumble upon rare or out-of-vocabulary words. BPE, with its ability to break words down into frequent subword units, can represent these words without needing to have seen them before.
Efficiency: By representing frequent word parts as single tokens, BPE can compress text more effectively. This is especially useful for models like ChatGPT, where token limits apply.
Adaptability: BPE is language-agnostic. It doesn’t rely on predefined dictionaries or vocabularies. Instead, it learns from the data, making it adaptable to various languages and contexts.

In essence, BPE strikes a balance, offering the granularity of character-level tokenization and the context-awareness of word-level tokenization. This hybrid approach ensures that NLP models like ChatGPT can understand a wide range of texts while maintaining computational efficiency.

Sending lots of text to ChatGPT

At time of writing, a message to ChatGPT via its web interface has a maximum token length of 4,096 tokens. If we assume the prior mentioned percent reduction as an average, this means you could reduce text of up to 5,712 tokens down to the appropriate size with just text preprocessing.

What about when this isn’t enough? Beyond text preprocessing, larger input can be sent in chunks using the OpenAI API. In my next post, I’ll show you how to build a Python module that does exactly that.

Git branching for small teams

Victoria Drake — Mon, 23 May 2022 12:12:48 +0000

Here’s a practice I use personally and encourage within my open source projects and any small teams I run for work. I’ve seen major elements of it presented under a few different names: Short-Lived Feature Branch flow, GitHub flow (not to be confused with GitFlow), and Feature Branch Workflow are some. Having implemented features I like from all of these with different teams over the years, I’ll describe the resulting process that I’ve found works best for small teams of about 5-12 people.

A protected main branch

To support continuous delivery, no human should have direct push permissions on your master branch. If you develop on GitHub, the latest tag of this branch gets deployed when you create a release – which is hopefully very often, and very automated.

One issue, one branch, one PR

You’re already doing a great job of tracking future features and current bugs as issues (right?). To take a quick aside, an issue is a well-defined piece of work that can be merged to the main branch and deployed without breaking anything. It could be a new piece of functionality, a button component update, or a bug fix.

Author's illustration of issue branches and releases from master.

A short-lived branch-per-issue helps ensure that its resulting pull request doesn’t get too large, making it unwieldy and hard to review carefully. The definition of “short” varies depending on the team or project’s development velocity: for a small team producing a commercial app (like a startup), the time from issue branch creation to PR probably won’t exceed a week. For open source projects like the OWASP WSTG that depends on volunteers working around busy schedules, branches may live for a few weeks to a few months, depending on the contributor. Generally, strive to iterate in as little time as possible.

Here’s what this looks like practically. For an issue named (#28) Add user settings page, check out a new branch from master:

# Get all the latest work locally
git checkout master
git pull
# Start your new branch from master
git checkout -b 28/add-settings-page

Work on the issue, and periodically merge master to fix and avoid other conflicts:

# Commit to your issue branch
git commit ...
# Get the latest work on master
git checkout master
git pull
# Return to your issue branch and merge in master
git checkout 28/add-settings-page
git merge master

You may prefer to use rebasing instead of merging in master. This happens to be my personal preference as well, however, I’ve found that people generally seem to have a harder time wrapping their heads around how rebasing works than they do with merging. Interactive rebasing can easily introduce confusing errors, and rewriting history can be confusing to begin with. Since I’m all about reducing cognitive load in developers’ processes in general, I recommend using a merge strategy.

When the issue work is ready to PR, open the request against master. Automated tests run. Teammates review the work (using inline comments and suggestions if you’re on GitHub). Depending on the project, you may deploy a preview version as well.

Once everything checks out, the PR is merged, the issue is closed, and the branch is deleted.

Keep it clean

Some common pitfalls I’ve seen that can undermine this flow are:

Creating feature branches off of other feature/issue branches. This is a result of poor organization and prioritization. To avoid confusing conflicts and dependencies, always branch off the most up-to-date master.
Letting the issue branch live just a little longer. This results in scope creep and huge, confusing PRs that take a lot of time and mental effort to review. Keep branches tightly scoped to the one issue they’re meant to close.
Not deleting merged branches. There’s no reason to leave them about – all the work is in master. Not removing branches that are stale or have already been merged can cause confusion and make it more difficult than necessary to differentiate new ones.

If this sounds like a process you’d use, or if you have anything to add, let me know in the comments!

My paper to-do strategy

Victoria Drake — Mon, 25 Oct 2021 12:17:32 +0000

Coding up a to-do app may be the Hello, World of every framework, but when it comes to actually tracking tasks effectively (knock ‘em out not stack ‘em up) there’s no app that keeps things front of mind better than an open notebook on your desk.

Here’s my stupid-simple strategy for tracking and checking off my to-do list.

One page at a time

Plenty of methodologies recommend using sections or different pages of your book for monthly, weekly, and daily views; others advocate for creating sections for each category, such as “Home Tasks” and “Work Tasks” and other such time-wasters. All of this is unnecessary.

A to-do list works because it’s in your face and hard to miss. When you write things down on different pages, they become easy to miss. Don’t do that.

Use one page at a time. Write down one task under another. Don’t sort them, prioritize them (yet), or categorize anything. Just write them down on the current page, where you’re guaranteed to look when you lay eyes on your notebook next.

Intuitive notation

I use my notebook for two things: short notes (just a bit of information – nothing to do) and tasks (something to do). This translates to a notation system of three possible states:

It’s a note, indicated with a bullet point
It’s a new task, indicated with a checkbox
It’s a completed task, with the checkbox checked and the line struck out (because strike-throughs are satisfying)

I use a checkbox to distinguish tasks from notes because I’m an old-school HTML fan, but you do you.

You may like to add your own embellishments to this: I sometimes denote an urgent item with an asterisk. You might like to use a color pen or highlighter (avoid the bullet journal rabbit hole – another time-waster). Just keep it simple, repeatable, and intuitive.

When it’s time to turn the page

When life gets busy, you might fill up a page pretty quickly. If one or two tasks haven’t yet been crossed off, they’re liable to be forgotten. You can avoid this by carrying tasks over to the next page.

It’s straightforward: cross out the task on the page that’s filled up. Turn the page and write it down there again.

That’s silly, you might say, that’s a waste of energy! By the time I write it down all over again, I could’ve done half of it already.

…

I’ll wait.

…

The clever bit about carrying a task over is taking the opportunity to evaluate it. If the task is really a five-minute thing, more often than not, I go ahead and take care of it right there and then. If it’s a longer endeavor, the friction of writing it down again gives me the chance to answer the question of whether it’s something I feel strongly about doing (and hence whether it’s really important that I do it at all). It might not be, and that’s fine. I cross it out and don’t do it. If it is an important task, carrying it over means it remains front of mind until I can make the time to get it done.

Time well spent doing

I’ve explored a myriad of task list apps, pre-printed to-do lists and journals, and all kinds of digital notes for tracking work. I consistently keep returning to the feel of pen on paper and an open notebook on my desk. Why? Minimal cognitive load.

No time spent categorizing and labeling tasks in a complicated system. No time spent remembering how to open that app, where you stored that todo.txt file, or deciding whether to write something down under your weekly or daily plan. No tasks lost in an invisible backlog that grows over the years, becoming more and more infeasible.

Just pen and paper, one page at a time, and the satisfaction of getting things done.

Measuring productivity with GitHub issues

Victoria Drake — Mon, 30 Aug 2021 05:35:02 +0000

How long does it take for a bug to get squashed, or for a pull request to be merged? What kind of issues take the longest to close?

Most organizations want to improve productivity and output, but few technical teams seem to take a data-driven approach to discovering productivity bottlenecks. If you’re looking to improve development velocity, a couple key metrics could help your team get unblocked. Here’s how you can apply a smidge of data science to visualize how your repository is doing, and where improvements can be made.

Getting quality data

The first and most difficult part, as any data scientist would likely tell you, is ensuring the quality of your data. It’s especially important to consider consistency: are dates throughout the dataset presented in a consistent format? Have tags or labels been applied under consistent rules? Does the dataset contain repeated values, empty values, or unmatched types?

If your repository has previously changed up processes or standards, consider the timeframe of the data you collect. If labeling issues is done arbitrarily, those may not be a useful feature. While cleaning data is outside the scope of this article, I can, at least, help you painlessly collect it.

I wrote a straightforward Python utility that uses the GitHub API to pull data for any repository. You can use this on the command line and output the data to a file. It uses the list repository issues endpoint (docs), which, perhaps confusingly, includes both issues and pull requests (PRs) for the repository. I get my data like this:

$ python fetch.py -h
usage: fetch.py [-h] [--token TOKEN] repository months
$ python fetch.py OWASP/wstg 24 > data.json

Using the GitHub API means less worry about standardization, for example, all the dates are expressed as ISO 8601. Now that you have some data to process, it’s time to play with Pandas.

Plotting with Pandas

You can use a Jupyter Notebook to do some simple calculations and data visualization.

First, create the Notebook file:

touch stats.ipynb

Open the file in your favorite IDE, or in your browser by running jupyter notebook.

In the first code cell, import Pandas and load your data:

import pandas as pd

data = pd.read_json("data.json")
data

You can then run that cell to see a preview of the data you collected.

Pandas is a well-documented data analysis library. With a little imagination and a few keyword searches, you can begin to measure all kinds of repository metrics. For this walk-through, here’s how you can calculate and create a graph that shows the number of days an issue or PR remains open in your repository.

Create a new code cell and, for each item in your Series, subtract the date it was closed from the date it was created:

duration = pd.Series(data.closed_at - data.created_at)
duration.describe()

Series.describe() will give you some summary statistics that look something like these (from mypy on GitHub):

count 514
mean 5 days 08:04:17.239299610
std 14 days 12:04:22.979308668
min 0 days 00:00:09
25% 0 days 00:47:46.250000
50% 0 days 06:18:47
75% 2 days 20:22:49.250000
max 102 days 20:56:30

Series.plot() uses a specified plotting backend (matplotlib by default) to visualize your data. A histogram can be a helpful way to examine issue duration:

duration.apply(lambda x: x.days).plot(kind="hist")

This will plot a histogram that represents the frequency distribution of issues over days, which is one way you can tell how long most issues take to close. For example, mypy seems to handle the majority of issues and PRs within 10 days, with some outliers taking more than three months:

It would be interesting to visualize other repository data, such as its most frequent contributors, or most often used labels. Does a relationship exist between the author or reviewers of an issue and how quickly it is resolved? Does the presence of particular labels predict anything about the duration of the issue?

You aim for what you measure

Now that you have some data-driven superpowers, remember that it comes with great responsibility. Deciding what to measure is just as, if not more, important than measuring it.

Consider how to translate the numbers you gather into productivity improvements. For example, if your metric is closing issues and PRs faster, what actions can you take to encourage the right behavior in your teams? I’d suggest encouraging issues to be clearly defined, and pull requests to be small and have a well-contained scope, making them easier to understand and review.

To prepare to accurately take measurements for your repository, establish consistent standards for labels, tags, milestones, and other features you might want to examine. Remember that meaningful results are more easily gleaned from higher quality data.

Finally, have fun exercising your data science skills. Who knows what you can discover and improve upon next!