DEV Community: Lavkesh Dwivedi

Forty Percent of My Books Are Gifts

Lavkesh Dwivedi — Sat, 15 Aug 2026 13:41:40 +0000

Originally published on lavkesh.com

Verizon's DBIR puts the number of breaches attributed to stolen credentials at 44%, a figure that has remained largely unchanged over the past few years. This persistence suggests that despite increased awareness and improved security measures, the exploitation of stolen credentials remains a favored tactic among attackers. The DBIR's findings are based on a comprehensive analysis of breach data from around the world, providing a unique insight into how threats are changing.

The implications of this statistic are significant, as it highlights the critical importance of robust identity and access management practices. If nearly half of all breaches can be attributed to stolen credentials, it stands to reason that enhancing the security of these credentials should be a top priority for organizations. This could involve implementing more stringent authentication protocols, such as multi-factor authentication, as well as regularly reviewing and updating access permissions to minimize the potential damage in the event of a breach.

One of the primary challenges in combating the use of stolen credentials is the sheer volume of data that is available to potential attackers. With each new breach, millions of usernames and passwords are released into the wild, providing a treasure trove of information for those seeking to gain unauthorized access to sensitive systems. Furthermore, the increasing sophistication of password cracking tools and techniques means that even seemingly secure passwords can be compromised with relative ease.

In response to these threats, many organizations are turning to more advanced security solutions, such as behavioral biometrics and artificial intelligence-powered threat detection. These technologies have the potential to significantly enhance the security posture of an organization, but they are not a panacea. The most effective defense against the exploitation of stolen credentials will involve a multi-layered approach that incorporates a combination of people, processes, and technology.

The human factor is a critical component of this approach, as it is often the actions of individuals that provide the initial vulnerability that is exploited by attackers. Phishing campaigns, for example, rely on tricking users into divulging sensitive information, such as usernames and passwords. Educating users about the dangers of these types of attacks and providing them with the knowledge and skills necessary to identify and avoid them is essential for reducing the risk of a breach.

In addition to user education, organizations must also prioritize the implementation of strong technical controls. This could include the use of password managers to generate and store unique, complex passwords, as well as the deployment of intrusion detection and prevention systems to identify and block suspicious activity. By taking a comprehensive approach to security, organizations can reduce their risk of becoming the next victim of a breach.

The financial consequences of a breach can be severe, with the average cost of a data breach now exceeding $4 million. This figure is likely to continue to rise as the sophistication and frequency of attacks increase, making it even more critical for organizations to invest in their security infrastructure. The DBIR's findings serve as a stark reminder of the importance of prioritizing security and taking proactive steps to protect against the threats that are looming on the horizon.

As we move forward in 2026, it is clear that the types of threats will continue to change, with new and innovative attack vectors emerging all the time. In this environment, it is essential that organizations remain vigilant and adapt their security strategies to meet the changing needs of threats. By doing so, they can reduce their risk of becoming a statistic in next year's DBIR and protect their sensitive data from those who would seek to exploit it.

The 44% figure attributed to stolen credentials is a sobering reminder of the challenges that we face in the cybersecurity arena. However, by acknowledging these challenges and taking proactive steps to address them, we can work towards a more secure future for all. It is a future that will require the collaboration and cooperation of individuals, organizations, and governments around the world, but one that is essential for protecting the sensitive information that underpins our modern way of life.

A summary isn’t required; instead, the question is what the 2027 DBIR will report. Will the number of breaches attributed to stolen credentials continue to hold steady, or will we see a significant decrease due to the increased awareness and improved security measures?

Summer Nights in Orai

Lavkesh Dwivedi — Sat, 08 Aug 2026 13:46:36 +0000

Originally published on lavkesh.com

I grew up in Orai, a small town in central India, where the summer nights are warm and the streets are empty by 10 PM. The night air is thick with the smell of jasmine and the sound of crickets, a soundtrack that is both familiar and comforting. As I sit on the rooftop of my childhood home, I am reminded of the simple pleasures in life that often get lost in the hustle and bustle of city living.

In Dallas, where I currently live, the summer nights are hot and the streets are always bustling with activity. The city never sleeps, and the sounds of cars and sirens are a constant reminder of the fast-paced life that I have grown accustomed to. But reflecting on my childhood in Orai, I realize that there is a beauty to the slow pace of life in small towns that is hard to find in cities.

One of the things that I miss most about Orai is the sense of community that exists there. Neighbors know neighbors, and everyone looks out for each other. In Dallas, I have to make an effort to get to know my neighbors, and even then, it's not always easy to build meaningful relationships. But in Orai, community is woven into the fabric of everyday life, and it's something that I cherish deeply.

As I think about the differences between life in Orai and Dallas, I am reminded of the concept of 'pace of life.' It's a term that refers to the speed at which people live their lives, and it's something that can vary greatly from one place to another. In Orai, the pace of life is slow and deliberate, while in Dallas, it's fast and frenetic. And while both have their advantages and disadvantages, I find myself drawn to the slow pace of life in small towns like Orai.

But the pace of life is not just about the speed at which we live our lives; it's also about the way we experience time. In Orai, time is measured in terms of the sun and the seasons, while in Dallas, it's measured in terms of clocks and calendars. And while both ways of experiencing time have their own rhythms and rituals, I find myself preferring the more natural rhythms of small towns like Orai.

As I sit on the rooftop of my childhood home, watching the stars twinkle to life in the night sky, I am reminded of the beauty of the slow pace of life. It's a pace that allows us to appreciate the simple things in life, to connect with the people around us, and to experience time in a more natural and meaningful way. And while it's not always possible to live in a small town like Orai, I believe that we can all learn to appreciate the slow pace of life, no matter where we live.

In fact, I think that the slow pace of life is something that we can all benefit from, regardless of where we live. In a world that is increasingly fast-paced and frenetic, it's easy to get caught up in the hustle and bustle of city living. But by slowing down and appreciating the simple things in life, we can cultivate a sense of peace and contentment that is hard to find in the midst of chaos.

As I look out at the night sky, I am reminded of the importance of balance in our lives. We need the fast pace of cities to drive innovation and progress, but we also need the slow pace of small towns to cultivate community and connection. And while it's not always easy to find this balance, I believe that it's something that we can all strive for, no matter where we live.

In the end, the pace of life is a personal choice, and one that depends on our individual values and priorities. But as I sit on the rooftop of my childhood home, watching the stars twinkle to life in the night sky, I am reminded of the beauty of the slow pace of life, and the importance of appreciating the simple things in life.

The night air is filled with the sweet scent of jasmine, and the sound of crickets provides a soothing background noise. It's a moment of peace and tranquility, one that I will carry with me long after I leave Orai and return to the hustle and bustle of city living.

The Last Two Days of a Sprint Behave Like Options Decay

Lavkesh Dwivedi — Sun, 02 Aug 2026 14:43:45 +0000

Originally published on lavkesh.com

An at-the-money weekly option loses about 60% of its remaining time value in the final two trading days before expiry, according to Zerodha Varsity's options chapter. The decay accelerates instead of running flat, and the same curve shows up inside a two-week sprint. Most of what ships in the last two days was scheduled for the first eight, and it shows in the quality of the diff.

We tracked merged pull request size by day of the sprint for six sprints running. The median PR filed on day one or two ran about 140 lines. The median PR filed on day nine or ten ran under 40 lines, and nearly half of those skipped a second reviewer because the sprint review was already booked on the calendar.

That pattern is not an accident. It is the same mechanism that burns time value off an option. Work committed early still has room to be done right, and work committed late gets compressed until only the smallest, safest slice survives the deadline.

Standups do not catch this, because a standup measures whether a ticket moved columns, not whether the ticket that moved was the right size for the days left on the clock. A blocker announced on day nine gets the same thirty-second slot as a blocker announced on day two, even though one of them has no runway left to fix.

The fix that worked for us was not a longer sprint. It was a mid-sprint checkpoint on day six, where every ticket still in the backlog got re-estimated against the days actually remaining, not the days originally planned. Tickets that no longer fit got cut in that meeting, not discovered as a surprise on day ten.

Cutting a ticket on day six feels like admitting failure in the room. Shipping a rushed version of it on day ten feels like delivery, and the second one is worse for the codebase, but it gets rewarded anyway because nobody measures the review debt a compressed PR leaves behind.

Retrospectives that count velocity miss this entirely. A team can hit its story point target every sprint while quietly moving all the risk into the last forty-eight hours, the same way an option seller collects premium every week until the one week the underlying moves and the position that always worked stops working.

The mid-sprint checkpoint is now a fifteen-minute standing item on day six of every sprint we run. It has cut exactly one kind of failure: the ticket that ships on the last day because nobody looked at the calendar until it was too late to do anything but rush it.

Sprint Value Lost in the Home Stretch

Lavkesh Dwivedi — Sat, 01 Aug 2026 14:04:16 +0000

Originally published on lavkesh.com

An at-the-money weekly option loses roughly 60% of its time value in the final two trading days before expiry, according to Zerodha Varsity's 2025 options chapter. This phenomenon, known as theta decay, is not linear - it accelerates as the expiry date approaches. A similar pattern emerges in engineering sprints, where most teams shed a significant chunk of planned value in the final days before the deadline. I've observed this value decay in multiple agile teams, and it's time to examine the reasons behind it.

The agile methodology emphasizes flexibility and adaptability, but it can also create a culture of deadline-driven work. As the sprint deadline approaches, teams often find themselves in a state of high pressure, rushing to complete tasks and meet the planned value. However, this pressure can lead to a decline in engineering efficiency, as teams sacrifice quality and attention to detail in order to meet the deadline. The resulting value decay is a symptom of this approach, where the focus shifts from delivering high-quality work to simply meeting the deadline.

Data-driven retrospectives can help teams identify the root causes of value decay. By analyzing metrics such as cycle time, lead time, and throughput, teams can pinpoint areas where they can improve their workflow and reduce waste. For example, a team may discover that they are spending too much time on meetings and not enough time on actual coding. By streamlining their meeting schedule and focusing on core work, they can increase their productivity and reduce the likelihood of value decay.

Another strategy to mitigate value decay is to prioritize tasks based on their business value and complexity. By focusing on high-value tasks first and breaking them down into smaller, manageable chunks, teams can ensure that they are delivering the most important work even if they don't complete everything on their sprint backlog. This approach also helps to reduce the impact of deadline pressure, as teams can still deliver significant value even if they don't meet the original deadline.

Time value is a critical concept in options trading, but it's also relevant to engineering sprints. As the deadline approaches, the time value of each task decreases, making it more difficult to deliver the planned value. By recognizing this phenomenon and taking steps to mitigate it, teams can reduce the likelihood of value decay and deliver more consistent results. This requires a mindset shift, from focusing solely on meeting the deadline to prioritizing the delivery of high-quality work.

In my experience, teams that prioritize engineering efficiency and productivity tend to suffer less from value decay. These teams focus on delivering high-quality work, rather than just meeting the deadline, and they are more likely to adapt to changing circumstances and priorities. By emphasizing quality and attention to detail, teams can reduce the impact of deadline pressure and deliver more consistent results.

The concept of theta decay can also be applied to engineering sprints, where the time value of each task decreases as the deadline approaches. By recognizing this phenomenon, teams can take steps to mitigate its impact, such as prioritizing high-value tasks and focusing on core work. This requires a data-driven approach, where teams analyze their workflow and identify areas for improvement.

Ultimately, the key to avoiding value decay in engineering sprints is to recognize the predictable patterns that emerge as the deadline approaches. By prioritizing high-quality work, streamlining workflows, and focusing on core tasks, teams can reduce the likelihood of value decay and deliver more consistent results. This requires a mindset shift, from focusing solely on meeting the deadline to prioritizing the delivery of high-quality work.

As teams reflect on their sprint performance, they should ask themselves: what can we do to reduce the impact of deadline pressure and preserve the planned value? By examining their workflow, prioritizing high-value tasks, and focusing on core work, teams can mitigate the effects of value decay and deliver more consistent results. The answer lies in a combination of data-driven retrospectives, prioritization, and a focus on engineering efficiency.

The next time you're approaching the end of a sprint, take a step back and assess the planned value. Are you on track to deliver, or are you experiencing the familiar phenomenon of value decay? By recognizing the patterns and taking steps to mitigate them, you can reduce the impact of deadline pressure and deliver high-quality work that meets the needs of your stakeholders.

The Calendar Audit Every Engineering Manager Avoids

Lavkesh Dwivedi — Wed, 29 Jul 2026 19:32:37 +0000

Originally published on lavkesh.com

Forty hours in a week, and fourteen of them belonged to one recurring meeting. I counted, calendar block by calendar block, for the whole quarter, and the number held steady week after week. A Tuesday status sync with eleven people on the invite, most of whom sat through the full hour and said nothing.

That meeting existed for a reason once. Eighteen months earlier, a deployment slipped through three teams without anyone flagging a broken dependency until it hit production, and the postmortem's one durable output was a standing sync meant to catch exactly that kind of gap before it shipped.

It caught almost nothing after the first two months. The actual blockers surfaced in Slack threads and one-on-ones hours before the meeting ever happened, and the sync itself became a status report read aloud to people who had already read the same update in a different channel that morning.

Three other directors ran the same calendar count when I mentioned mine in a leadership sync of our own. Two of them found a nearly identical pattern, a recurring meeting born out of a real incident that had outlived the problem it was built to solve by well over a year.

None of us had noticed, because nobody schedules an hour to check whether last year's fix is still earning its slot on the calendar. A one-on-one that gets cancelled draws a complaint within the day. A standing meeting that stops mattering can run for quarters before anyone questions why it is still there.

I cut the invite list from eleven to four and replaced the live status round with a written update posted the night before. The call itself became fifteen minutes, reserved only for the one blocker that could not wait for async. Nobody on the smaller list has asked for the old format back.

One senior engineer did push back, and the pushback was fair. The old meeting had one working part. It forced two teams that rarely spoke outside of it to say something out loud in front of each other once a week, and a written update does not replicate that particular kind of friction.

So the fifteen minutes stayed. The written update was never meant to replace it; it sits alongside it, doing a smaller and more honest job than the hour used to.

Every manager on the team now blocks one hour a quarter to run the same count on their own calendar. The number is not always big. Some quarters it turns up nothing worth cutting. It remains the only exercise that catches a meeting nobody remembers agreeing to.

Spend Discipline in the Cloud

Lavkesh Dwivedi — Sat, 25 Jul 2026 14:07:32 +0000

Originally published on lavkesh.com

Monthly SIP inflows in India crossed 26,000 crore in March 2026, a fresh all-time high. This milestone is not just a testament to the growing popularity of systematic investment plans, but also a reflection of the power of automation and discipline in personal finance. However, the same principles that drive SIP success can also be applied to cloud cost management, where the lack of spend discipline can lead to bills that spiral out of control.

The key to successful SIP investing is regular contributions and a long-term perspective. Similarly, in cloud cost management, setting a budget alert is just the first step. It is essential to regularly review and adjust cloud resources to ensure they are aligned with changing business needs. This is where rightsizing comes in - the process of ensuring that cloud resources are optimized for current workloads, rather than being based on initial estimates or forecasts.

I recall a conversation with a colleague who was struggling to manage cloud costs for a Kubernetes cluster. The team had set up a budget alert at 20% over forecast, but nobody had revisited the numbers in months. As a result, the cloud bill had ballooned to an all-time high, with resources that were no longer needed or optimized. This experience highlights the importance of regular reviews and adjustments in cloud cost management.

The concept of rightsizing is not new, but it is often overlooked in the rush to deploy cloud resources. By regularly reviewing and adjusting cloud resources, teams can ensure that they are only paying for what they need, rather than being locked into unnecessary commitments. This approach requires discipline and a willingness to revisit assumptions, much like the discipline required to stick to a SIP investment plan.

In conclusion, the principles that drive SIP success can also be applied to cloud cost management. By setting budget alerts, regularly reviewing and adjusting cloud resources, and practicing rightsizing, teams can ensure that their cloud spend is aligned with their business needs. This approach requires a combination of automation, discipline, and regular reviews, but it can help to prevent cloud bills from spiraling out of control.

The next time you set up a budget alert for your cloud resources, remember that it is just the first step. Regular reviews and adjustments are essential to ensuring that your cloud spend is optimized and aligned with your business needs. By applying the principles of SIP investing to cloud cost management, you can avoid the pitfalls of unnecessary spend and ensure that your cloud resources are working for you, rather than against you.

As the cloud landscape continues to evolve, it is essential to stay vigilant and adapt to changing needs. By prioritizing spend discipline and rightsizing, teams can ensure that their cloud resources are optimized for performance and cost. This requires a mindset shift, from focusing solely on deployment and scalability, to prioritizing cost management and optimization.

The benefits of this approach are clear: reduced cloud spend, improved resource utilization, and increased agility. By applying the principles of SIP investing to cloud cost management, teams can achieve a better balance between deployment and optimization, and ensure that their cloud resources are working for them, rather than against them.

In the world of cloud cost management, discipline and regular reviews are essential. By prioritizing spend discipline and rightsizing, teams can ensure that their cloud resources are optimized for performance and cost. This approach requires a combination of automation, discipline, and regular reviews, but it can help to prevent cloud bills from spiraling out of control.

As teams continue to navigate the complexities of cloud cost management, it is essential to remember that budget alerts are just the first step. Regular reviews and adjustments are essential to ensuring that cloud spend is aligned with business needs. By applying the principles of SIP investing to cloud cost management, teams can achieve a better balance between deployment and optimization, and ensure that their cloud resources are working for them, rather than against them.

Cloud Cost Governance Needs SIP Discipline

Lavkesh Dwivedi — Mon, 20 Jul 2026 16:49:42 +0000

Originally published on lavkesh.com

Monthly SIP inflows in India crossed 26,000 crore in March 2026, a fresh all-time high. According to AMFI monthly data, this number is not just a testament to the growing financial discipline of Indians, but also a reminder of the importance of regularity in achieving long-term goals. I often wonder what if we applied the same discipline to cloud cost governance as we do to SIP contributions, considering SIP contributions have more than doubled in three years, with first-time investors now averaging 27 years old, down from 34 a decade ago.

The concept of SIP, or Systematic Investment Plan, is straightforward: invest a fixed amount of money at regular intervals, regardless of the market's performance. This approach helps investors ride out market fluctuations and avoid making emotional decisions based on short-term market volatility. But in contrast, cloud cost governance is often treated as an afterthought, with engineers focusing on deploying and managing applications without a clear understanding of the costs involved, which can lead to significant cost overruns, especially in complex environments like Kubernetes.

One of the primary challenges in applying SIP discipline to cloud cost governance is the lack of visibility into cloud spend. Unlike SIP contributions, which are typically fixed and regular, cloud costs can be variable and unpredictable, making it difficult to budget and plan for them. But this is where engineering leadership can play a crucial role in establishing a culture of cost governance by setting clear budgets and cost targets, and regularly reviewing and adjusting them.

The financial analogy between SIP and cloud cost governance can be taken further. Just as SIP investors benefit from the power of compounding, where small, regular investments can add up to significant returns over time, cloud engineers can benefit from the power of continuous cost optimization by regularly reviewing and optimizing cloud resources, which can help avoid cost overruns and ensure that cloud spend is aligned with business objectives. Requiring a mindset shift from treating cloud cost as an afterthought to making it a core part of the engineering workflow.

To apply SIP discipline to cloud cost governance, engineers need to establish clear budgets and cost targets, and regularly review and adjust them. This requires a deep understanding of cloud costs, including the costs of resources like compute, storage, and networking, and implementing cost governance tools and processes, such as cloud cost monitoring and optimization platforms, to track and optimize cloud spend, as well as establishing a culture of cost discipline, where cloud cost is treated as a key metric, alongside other engineering metrics like latency and throughput.

The benefits of applying SIP discipline to cloud cost governance are clear: by establishing a culture of cost discipline, engineers can avoid cost overruns, optimize cloud spend, and ensure that cloud resources are aligned with business objectives. Requiring a mindset shift from treating cloud cost as an afterthought to making it a core part of the engineering workflow. And as the Indian mutual fund industry continues to grow, with SIP contributions reaching new heights, it's time for cloud engineers to take a page out of the SIP book and apply the same discipline to cloud cost governance.

The discipline behind SIP contributions mirrors what most cloud budgets lack. By applying the principles of SIP to cloud cost governance, engineers can establish a culture of cost discipline, optimize cloud spend, and ensure that cloud resources are aligned with business objectives. Requiring a deep understanding of cloud costs, a mindset shift, and the implementation of cost governance tools and processes, which can help avoid cost overruns and ensure that cloud resources are optimized for maximum ROI.

Considering the state of cloud cost governance highlights how regularity and discipline support long-term goals. Whether it's SIP contributions or cloud cost governance, the principles are the same: establish clear targets, regularly review and adjust, and optimize for maximum ROI. And by applying these principles to cloud cost governance, engineers can ensure that cloud spend is aligned with business objectives and that costs are optimized for maximum ROI, which is essential for the growth and success of any organization.

The implications of this approach are significant. By treating cloud cost as a key metric, alongside other engineering metrics like latency and throughput, engineers can ensure that cloud resources are optimized for maximum ROI. Requiring a cultural shift from treating cloud cost as an afterthought to making it a core part of the engineering workflow. And as the cloud continues to grow in importance, it's time for engineers to take a proactive approach to cloud cost governance, using the principles of SIP as a guide, which can help them make informed decisions about cloud resource allocation and utilization.

Overall, applying SIP discipline to cloud cost governance is not just about optimizing cloud spend, but about establishing a culture of cost discipline that can benefit the entire organization. By taking a proactive approach to cloud cost governance, engineers can ensure that cloud resources are aligned with business objectives, and that costs are optimized for maximum ROI. Requiring a mindset shift, a deep understanding of cloud costs, and the implementation of cost governance tools and processes, which can help them achieve long-term success and growth in the cloud.

Cloud Bills Like SIPs

Lavkesh Dwivedi — Mon, 20 Jul 2026 16:13:46 +0000

Originally published on lavkesh.com

Monthly SIP inflows in India crossed 26,000 crore in March 2026, a fresh all-time high, according to AMFI monthly data, and this made me think of how engineering teams manage their cloud resources, because the pattern looks similar, with big inflows of cash and no clear exit strategy, just like investors dumping money into SIPs without a plan to withdraw it, and the bills just keep piling up

I've seen teams deploy Kubernetes without properly optimizing their resource allocation, and this leads to a huge cloud-cost, because they're not monitoring their usage closely, and the bills just keep growing, just like an SIP investment that's not regularly reviewed, and this is a problem that needs to be addressed, because cloud-cost is a significant part of any company's budget

The thing is, engineering teams are not just dumping money into cloud resources, they're also not planning for the future, and this is where the financial-analogy comes in, because just like a SIP investor needs to have a clear plan for their money, an engineering team needs to have a clear plan for their cloud spend, and that plan needs to include spend-optimization, and resource-management, and a clear exit strategy

I've worked with teams that have implemented cloud-cost monitoring tools, and the results are impressive, because they're able to track their usage in real-time, and make adjustments on the fly, and this leads to significant cost savings, and it's not just about the money, it's also about being more efficient, and using resources more effectively

But it's not just about the tools, it's also about the culture, and the way teams think about cloud resources, and engineering-leadership plays a big role in this, because they need to set the tone for the team, and make sure that everyone is on the same page, and that page needs to include a clear plan for cloud spend, and a commitment to spend-optimization

I've seen teams that have a clear plan for their cloud spend, and they're able to manage their resources effectively, and they're also able to innovate, and try new things, because they're not held back by a huge cloud-cost, and this is where the financial-analogy really comes in, because just like a SIP investor needs to have a clear plan for their money, an engineering team needs to have a clear plan for their cloud spend

The key is to find a balance between innovation, and cost savings, and this is where Kubernetes comes in, because it allows teams to deploy resources quickly, and scale up or down as needed, and this leads to a more efficient use of resources, and a lower cloud-cost, but it's not just about the technology, it's also about the people, and the way they think about cloud resources

Engineering teams need to think about cloud resources in the same way that they think about their own personal finances, and this means having a clear plan, and a commitment to spend-optimization, and resource-management, and it's not just about the money, it's also about being more efficient, and using resources more effectively

I think that if engineering teams can learn to treat cloud spend like a SIP, they'll be able to manage their resources more effectively, and innovate more quickly, and this will lead to a more efficient use of resources, and a lower cloud-cost, and it's not just about the technology, it's also about the people, and the way they think about cloud resources

The bottom line is that cloud-cost is a significant part of any company's budget, and engineering teams need to have a clear plan for managing their cloud resources, and this plan needs to include spend-optimization, and resource-management, and a clear exit strategy, and it's not just about the money, it's also about being more efficient, and using resources more effectively

Half of Cloudflare’s traffic is now automated bots

Lavkesh Dwivedi — Sat, 18 Jul 2026 14:01:33 +0000

Originally published on lavkesh.com

When Cloudflare released its 2025 traffic review, I saw the headline that more than half of every request they handle now comes from automated sources, and the growth curve for agentic AI traffic has been steeper than that of traditional crawlers for the past three quarters.

Agentic AI traffic isn’t the polite indexer that politely reads a sitemap; it’s a suite of autonomous agents that can fill forms, trigger transactions, and even generate content on the fly, meaning the requests they send carry intent and state.

That shift flips the economics of serving the web; rate-limit thresholds that once protected a handful of noisy scrapers now bite into legitimate user latency, and the price tag on bot-detection services has risen in step with the traffic volume.

If half the hits are machines, we have to ask whether those hits are actually contributing to revenue, or merely inflating our server bills while our dashboards flash green on request counts.

The answer isn’t to throw more models at the problem but to integrate AI into the core workflows that already generate profit, such as using an agent to pre-populate a checkout form for returning customers rather than letting it crawl endlessly.

Building that kind of system forces us to treat AI like any other critical service; we need versioned models, observability pipelines, and a deployment cadence that can survive a sudden surge of a million agent calls in a minute.

Companies that keep spending on flashy demo projects while neglecting the underlying engineering stack end up with a widening gap between AI spend and actual return, a gap that can be closed only by hiring engineers who understand both the model and the production environment.

At the same time the rise of agentic bots makes it harder to distinguish between legitimate and malicious traffic, so investing in AI-powered traffic classification that can flag anomalous behavior before it overwhelms the edge has become as essential as any load balancer.

Last month our team had to raise the Cloudflare rate limit from 10 k to 30 k requests per second after a new marketing AI assistant started polling product pages, a change that cost an extra $12 k in bandwidth but saved a potential $200 k loss from missed conversions.

More than Half of Web Traffic is AI

Lavkesh Dwivedi — Sun, 12 Jul 2026 00:56:19 +0000

Originally published on lavkesh.com

Cloudflare's 2025 review shows automated traffic, including bots, scrapers, and agents, now makes up over half of all web requests they see. This shift indicates a significant change in internet interaction, driven largely by the growth of agentic AI traffic. For the last three quarters, agentic AI traffic has grown faster than classic crawler traffic. These trends have far-reaching implications for how we approach AI adoption and its potential to deliver business value.

The surge in automated traffic raises questions about the effectiveness of current AI strategies. Despite significant investment in AI, returns are often unclear. Many organizations struggle to achieve the promised benefits of AI, and the gap between investment and actual business value is widening. The key issue is what's holding us back from realizing AI's benefits.

One reason for the gap lies in the way AI engineering is approached. AI projects are often siloed, with separate teams handling data science, engineering, and deployment. This fragmented approach can lead to a lack of cohesion and alignment, resulting in AI solutions that fail to deliver business value. To bridge this gap, a more integrated approach to AI engineering is needed; one that brings together cross-functional teams to design and deploy AI solutions meeting specific business needs.

Another challenge is measuring the ROI of AI initiatives. As AI becomes more pervasive, it's increasingly difficult to isolate its impact on business outcomes. Traditional metrics like click-through rates or conversion rates may not capture the complex effects of AI on business value. More nuanced metrics are needed to accurately measure the ROI of AI initiatives and provide actionable insights for improvement.

The growth of agentic AI traffic also highlights the need for sophisticated bot detection and rate limiting strategies. Advanced AI-powered bots can mimic human behavior, making it harder to distinguish between legitimate and automated traffic. This has significant implications for the economics of serving traffic; organizations must balance providing a good user experience with preventing abuse and optimizing resource utilization.

Addressing these challenges requires a holistic approach to AI adoption, considering the complex interplay between AI, business value, and technical feasibility. This demands a deep understanding of the business domain and the technical capabilities and limitations of AI. By bringing together business stakeholders, data scientists, and engineers, we can design and deploy AI solutions that deliver tangible business value and drive outcomes.

The recent surge in automated traffic underscores the importance of AI explainability and transparency. As AI becomes more pervasive, providing clear explanations of AI-driven decisions and actions is essential. This is not only about trust and accountability but also about driving business adoption and ROI. Transparent and interpretable AI models can build trust with stakeholders; they can ensure AI solutions align with business objectives.

Achieving AI's potential requires a pragmatic and business-focused approach to AI adoption. This involves experimenting, learning from failures, and continuously refining our approach to AI engineering and deployment. By doing so, we can bridge the gap between AI investment and business value; we can create a future where AI delivers impact.

AI is not a panacea but a tool to drive business value. The question remains: what's holding us back from achieving AI's potential? For leaders of AI initiatives, the key question is: what specific business problem are you trying to solve with AI, and how will you measure the ROI of your efforts?

Senior PM Pay Drops 18% in 2 Years

Lavkesh Dwivedi — Sun, 12 Jul 2026 00:40:29 +0000

Originally published on lavkesh.com

Senior PM total comp at Series C and later startups has dropped 18% in 24 months, the first sustained decline since 2015, according to Lenny's Newsletter PM benchmarks. This trend affects a crucial role for startups, with the drop more pronounced than the cooling engineering market, where senior IC roles are holding up better than management tracks.

Product management has always been highly valued in startups, with senior PMs' total compensation traditionally high, reflecting their importance in driving product development and growth. However, current market conditions, including the end of the 2021-22 over-hiring cycle, have led to a decline in compensation for senior PMs, with many startups reevaluating their compensation structures and executive pay.

The decline in senior PM pay is also a result of the changing nature of the role itself; as startups grow and mature, product management becomes more complex, requiring a broader range of skills and expertise. However, this increased complexity has not been matched by a corresponding increase in compensation, leading to a disconnect between the value senior PMs bring and the rewards they receive.

The implications of this trend are significant, as it may lead to a brain drain of top product management talent from startups to more established companies, where compensation is more stable and rewarding. This could negatively impact startups' ability to innovate and compete, as they will be losing the people responsible for driving product development and growth.

The shift in salary trends for senior PMs reflects the evolving tech industry, where lines between roles and functions are becoming increasingly blurred. Startups will need to adapt to these changes and find new ways to reward and retain top talent, including senior PMs, or risk falling behind in the competitive startup market.

The drop in senior PM pay is a complex issue, driven by market conditions, the changing role, and startups' evolving needs. The traditional model of compensating senior PMs is no longer sustainable; startups must find new ways to attract and retain top product management talent to remain competitive.

The current state of senior PM compensation presents a challenge for startups but also an opportunity to rethink their approach to rewards and talent management. By finding new ways to provide value to senior PMs and recognizing the importance of this role, startups can attract and retain top talent and better position themselves to compete and thrive.

The decline in senior PM pay is a trend that will be watched closely by startups and the tech industry, with significant implications for talent management and compensation. Startups will need to stay ahead of the curve and find new ways to attract and retain top talent, including senior PMs.

The future of senior PM compensation is uncertain, but one thing is clear: the traditional model is no longer sustainable. Startups will need to adapt to the changing market and employee needs, requiring a fundamental change in their approach to compensation and rewards, and a willingness to experiment and try new things.

Postgres Authentication in Production

Lavkesh Dwivedi — Sun, 12 Jul 2026 00:17:10 +0000

Originally published on lavkesh.com

Verizon's DBIR report that 44% of breaches are caused by stolen credentials got me thinking about Postgres's approach to authentication. I was struck by how different it was from other databases I'd worked with.

Postgres uses a separate authentication system, rather than relying on the operating system's authentication mechanisms. This flexibility is great, but it requires careful configuration.

On a project storing sensitive data, we used a combination of SSL certificates and password authentication to ensure only authorized users could access it. It was work-intensive, but effective.

Postgres's default password hashing algorithm is relatively simple, which isn't ideal. However, you can configure it to use more secure algorithms like bcrypt or PBKDF2 to protect user passwords.

Working with Postgres, I've come to appreciate its authentication handling. It's not always straightforward, but it's worth the effort, especially considering the 44% breach statistic.

Of course, there are trade-offs. The added complexity can make it harder to manage and maintain, especially for smaller teams. But overall, I think the benefits outweigh the costs.

Reading through the Postgres source code shows the attention to detail the developers brought to the authentication system. They've thought carefully about potential risks and tried to mitigate them.

Postgres uses 'roles' to manage access, defining a set of privileges a user has. You can assign multiple roles to a single user, making it a powerful way to manage access.

Even with precautions, breaches can happen. The 44% figure is a reminder that no matter how careful we are, there's always a chance something will go wrong.

Finding a balance between security and usability is key. Postgres shines here, offering flexibility to configure the authentication system while providing high security out of the box.