DEV Community: Zac_A_Clifton

Benchmarking Vercel AI Gateway against the native Anthropic SDK

Zac_A_Clifton — Fri, 13 Feb 2026 17:27:18 +0000

We're building SalesSage (Not fully announced yet) an AI-powered platform with the goal of making any person a salesperson.

One of our core features is real-time audio transcript analysis with AI systems.

That means making a lot of calls and sending a lot of context to Claude and other AIs.

This means that latency matters for us because we want to make sure we are responding in near realtime to what is being discussed.

Se we wanted to see if is routing our API calls through the Vercel AI Gateway slower than hitting Anthropic directly?

TL;DR

At small prompts (~10 tokens), the native Anthropic SDK is ~15-20% faster than the Vercel AI Gateway
At large context (120K tokens, 60% of the context window), the difference between native and gateway nearly vanishes
Gateway has occasional latency spikes that blow up tail latency — p99 TTFB spiked to 5.6s on one Sonnet call, though it's not statistically significant.
Tier 1 Anthropic rate limits (30K input tokens/min) make large context calls through the native SDK impractical without significant delays
The gateway handles rate limits for you, which is a real advantage

The setup

We wrote a benchmark suite in TypeScript that tests two providers:

Native Anthropic SDK (@anthropic-ai/sdk) — direct API calls to Anthropic
Vercel AI Gateway (gateway() from the ai package) — Anthropic routed through Vercel's proxy

Each provider was tested with Claude Sonnet 4 and Claude Opus 4, measuring:

TTFB (time to first token) via streaming
Total completion time via non-streaming

We ran two variants: a small prompt (~10 tokens, 5 iterations) and a large context prompt (120K tokens calibrated via Anthropic's countTokens API, 3 iterations).

We created a calibration step that used binary search with client.messages.countTokens() to build a prompt that lands within 500 tokens of our 120K target.

Small prompt results

With a tiny prompt, the native SDK wins consistently:

This was no real surprise here. The gateway adds a proxy hop, and at small payloads that hop is a measurable percentage of the total request time. Sonnet shows a ~200ms overhead, Opus is nearly identical.

Both models perform similarly at small context sizes where Sonnet and Opus TTFB are within 120ms of each other.

Large context results — this is where it gets interesting

We filled 60% of the 200K context window (~120K tokens of business meeting notes) and re-ran everything:

Wait — Opus through the gateway is basically the same speed as native? And total completion is actually faster through the gateway?

At large context sizes, the model's processing time dominates so heavily that the gateway proxy hop becomes noise. The ~200ms overhead that mattered at 10 tokens is irrelevant when the model is chewing through 120K tokens for 5+ seconds.

Opus gets hit harder than Sonnet

Context size doesn't affect all models equally:

Opus TTFB jumps 4x when you fill the context window, vs Sonnet's 2.5x. At small prompts they're nearly identical, but at 120K tokens Opus takes almost twice as long as Sonnet to produce the first token (4.8s vs 2.6s).

Since time-to-first-token matters for our product this told us that in a large context we need to exclusively consider streaming UI, real-time summaries.

The tail tells a different story

p50 looks clean. p99 does not.

That red bar on Gateway Sonnet? A 5.6s TTFB on a small prompt. That's a 4.5x multiplier over p50. The native SDK stays tight at 1.4x.

At large context sizes, the tail is less dramatic, but the pattern holds:

The native SDK stays tight and consistent. On the p99 it is 1.0-1.1x of p50 across the board. The gateway has wider variance and can spike to 6.7s on Opus.

We are not sure how to put this into perspective yet, and it's something we will be considering for our latency-sensitivity as we are unsure if we need predictable tail behavior.

The rate limit surprise

Here's what we didn't expect to be the biggest finding: Tier 1 Anthropic rate limits make large context calls through the native SDK impractical.

Our org is currently on Tier 1 since we are in closed alpha. This means we can only send 30K input tokens per minute for Sonnet and Opus. A single 120K-token request consumes 4 minutes of rate limit budget. Not 4 minutes of clock time — 4 minutes of token budget.

We initially tried 75-second delays between calls. Still got 429'd. Had to bump to 240-second (4-minute) delays, turning a benchmark into a 90-minute affair.

The Vercel AI Gateway? Processed every 120K token request without a single rate limit error. The gateway operates on its own rate limit tier, which for large context workloads is a real competitive advantage.

What we'd do differently

A few things we learned running these benchmarks:

@ai-sdk/anthropic@3.x is incompatible with ai@5.x at runtime. The SDK returns LanguageModelV3 models, but ai@5.x only supports V2. Type-casting compiles fine but the runtime check rejects it. We had to drop the Vercel AI SDK provider entirely.
Use countTokens for calibration. Don't estimate token counts from character length. The API gives you exact numbers — use it.
Budget ~$20 for a full benchmark run. Large context calls at 120K tokens add up fast across multiple providers, models, and iterations.

The bottom line

For short prompts, the native Anthropic SDK is ~15% faster with tighter tail latency. If you're making lots of small, fast calls, go direct.

For large context (which is our actual production use case at SalesSage — meeting prep briefs packed with transcripts, CRM data, and company research), the gateway is effectively the same speed, handles rate limits transparently, and gives you observability through the Vercel dashboard for free.

Conclusion:
Based on this data and other benefits of going with an AI Gateway as most of our calls will be large contexts.

Notifications For Your App: Should you build or buy?

Zac_A_Clifton — Fri, 16 Aug 2024 23:23:03 +0000

Disclosure: This article was commissioned by Novu.

Keeping users informed and engaged is critical for the success of any application or SAAS. In order to accomplish this, development and product teams need to either build their own notifications system, or buy a purpose-built solution. A capable, notification infrastructure consists of a backend system that manages the delivery of messages to users across various channels. These systems ensure that users receive timely and relevant updates, helping them stay connected and informed about important events, updates, and actions within the app or service.

Setting up a robust notification infrastructure involves several components, including message routing, user preference management, content personalization/customization, and delivery tracking. The goal is to provide a seamless and reliable notification experience that can scale with user growth and evolving needs. For solution-aware users, understanding the intricacies of notification infrastructure is crucial as it impacts user engagement, retention, and overall satisfaction. Whether you’re a startup building a new product or an enterprise looking to streamline your notification systems, the choice between building in-house or leveraging an third party service is a significant decision that can influence your project’s success.

The notification landscape

Organizations face the challenge of managing an overwhelming workload, including major architectural refactors, new cloud applications, and internal technology adoption. Amid these tasks, handling notifications has become particularly complex. Notifications have evolved from an overlooked feature to an essential component for user engagement across various applications. However, the rapid increase in notification requirements and compressed timelines often lead to fragmented, inconsistent systems developed by different teams without coordination, resulting in "notification chaos."

This chaos is marked by the proliferation of siloed notification components, complicating management and creating inefficiencies. Supporting multiple notification channels further adds to the complexity, as users demand control over their notification preferences. The disjointed systems can lead to inconsistent delivery, ignored user preferences, and increased maintenance costs.

Key Issues with Notification Chaos

Fragmentation: Disconnected notification systems developed independently by various teams.
Complexity: Multiple channels and user preference demands increase system complexity.
Operational challenges: Inconsistent delivery, user dissatisfaction, and high maintenance costs.
Strategic decision: Organizations must choose between building a custom solution or adopting an vendor like Novu, balancing customization needs with operational efficiencies.

Ultimately, the right choice will depend on the specific needs and resources of the organization, as well as its long-term strategic goals.

Organization 1:

A startup needs to add notifications to their app
Imagine you're running a tech startup and need a robust notification system for task updates, deadlines, and collaborative activities. You're facing notification chaos with multiple channels like email, SMS, push notifications, and in-app messages. You must decide whether to build this system in-house or leverage a third partyservice. Building in-house offers complete control and customization but diverts engineering resources from core product development. It's resource-intensive and time-consuming, which could slow down your product's growth.

On the other hand, using an third part service streamlines the process, offering multi-channel support, user preference management, and scalable delivery out of the box. This approach allows your team to focus on core features, accelerating your product's time-to-market and ensuring reliability and scalability. As a solution-aware startup, you must weigh the pros and cons: customization and control versus efficiency and focus on your core product. Choosing the right path will help you manage notification chaos while driving growth and innovation.

Organization 2: Enterprise notifications consolidation from disparate platforms

Consider a large enterprise with multiple applications, each using its own notification system, leading to notification chaos with inconsistent user experiences and redundant systems. The enterprise wants to centralize their notification infrastructure to improve consistency and enhance user experience across all platforms.

Developing an in-house unified notification platform offers maximum control and customization, integrating deeply with existing systems. However, it demands significant engineering resources and ongoing maintenance, making it a complex and resource-intensive endeavor.

Alternatively, adopting a centralized notification infrastructure provides an efficient and scalable solution. Such a service manages notification complexities, offering features like intelligent message routing and user preference management. This approach frees up engineering resources, allowing them to focus on core business objectives.

How to build a notifications system: core requirements

Building a notification system requires 7 core pieces to make it successful, maintainable, and compliant. These requirements encompass an inbox for centralized message management, user-configurable preferences, intelligent message routing, dynamic content management, team interactivity features, comprehensive analytics, and considerations for ongoing maintenance, scaling, and debugging. Each of these components is crucial for delivering timely, relevant, and personalized notifications while maintaining system efficiency and scalability. Leveraging the right infrastructure and third party dependencies to support these requirements is essential for meeting user expectations and ensuring the reliability of the notification system.

Core requirement 1: Inbox

The inbox provides a centralized place to view and manage user messages. The inbox can take different forms, such as a notification bell within an application or a dedicated notification box. Each type has its unique requirements and challenges:

Centralized message repository: Collects messages from various channels into one accessible stream for easy navigation and management.
Notification read/Unread status management: Allows users to distinguish and manage new and read notifications using visual cues.
User-friendly access and management: Enables intuitive navigation, marking, deleting, and archiving of notifications with simple controls and search functionality.

Supporting infrastructure and SAAS for user inbox requirements

Several infrastructure components and SAAS support these inbox features:

Message queues (e.g., RabbitMQ, Kafka, Nats): These ensure reliable delivery and processing of notifications, making sure messages are not lost and are delivered in the correct order. Databases (e.g., MongoDB, PostgreSQL): Used for storing notification data, including the read/unread state, in a structured and scalable manner.
Notification management platforms (e.g., Firebase Cloud Messaging, OneSignal): These SAAS provide tools for managing and delivering notifications across multiple channels.
Front-End frameworks (e.g., React, Vue.js, Angular, Svelte): Essential for building user-friendly interfaces that allow users to interact with their inbox efficiently.
Search and filtering tools (e.g., Elasticsearch ): These tools enhance the ability to search and filter notifications, especially in email inbox scenarios.

Choosing the right infrastructure and SAAS depends on the specific requirements of your notification system and the type of inbox you aim to implement. Whether it's a notification bell or an email box, having a robust and user-friendly inbox is key to maintaining an effective communication system and ensuring a positive user experience.

Core requirement 2: Subscriber preferences

Managing subscriber preferences is crucial yet often overlooked in notification systems. User-specified preferences allow for a tailored notification experience, enhancing satisfaction and engagement. Here are the key requirements for user-configurable notification settings and the supporting infrastructure.

User-configurable notification settings: Users should easily choose types, frequency, and channels for notifications through a user-friendly interface to manage their preferences.
Customizable delivery channels and schedules: Users can select preferred channels (email, SMS, push notifications) and schedule notifications for specific times, enhancing relevance and reducing ignored messages.
Unsubscribes (individual, category, and site-wide): Users should easily unsubscribe from individual, category, or all notifications, reducing fatigue and retaining engagement.
Language Integrations for multi-lingual support: Users can receive notifications in their preferred language, requiring robust translation capabilities for accurate and culturally appropriate messages.
Framework integrations for seamless embedding: Notification preferences should integrate smoothly with third party systems and frameworks like React, Angular, or Vue.js for a seamless user experience.

Supporting infrastructure and SAAS for user preference requirements

Several infrastructure components and SAAS can support these advanced notification preferences:

User preference management tools (e.g., Customer.io, UserEngage): These tools provide interfaces and backend systems for managing user preferences, ensuring users can easily customize their notification settings.
Messaging platforms (e.g., Twilio, SendGrid): These SAAS facilitate multi-channel delivery of notifications, including email, SMS, and push notifications, allowing for customizable delivery options.
Scheduling SAAS (e.g., Quartz Scheduler, DKron): Used to schedule notifications, ensuring they are delivered at user-specified times.
Internationalization libraries (e.g., i18next, Globalize): These libraries support multi-lingual capabilities, helping to translate and deliver notifications in various languages.
Integration platforms (e.g., Zapier, Integromat): These platforms help integrate notification systems with other applications and frameworks, ensuring smooth operation and user experience.

By leveraging these tools and SAAS, you can build a notification system that not only meets but exceeds user expectations, providing a highly personalized and efficient notification experience.

Core requirement 3: message routing

Effective message routing is essential for ensuring that notifications are timely, relevant, and delivered through the preferred channels. Advanced message routing capabilities can significantly enhance user satisfaction and engagement.

Intelligent routing based on user preferences and notification context: Tailors notification delivery by analyzing user settings, behavior, and context to ensure relevance and welcome. Ensuring timely and relevant notifications: Delivers notifications promptly (<1 min) by considering user activity and context, maximizing impact and avoiding message overload.
Fan-out capabilities for multiple recipients: Sends a single notification to multiple recipients efficiently, maintaining performance and delivery speed for collaborative applications. Timezone-aware delivery: Adjusts delivery times based on the recipient’s timezone to ensure notifications are received at optimal, non-disruptive times.
Handling online/offline status: Queues notifications for offline users and delivers them upon reconnection, ensuring no important updates are missed.

Supporting infrastructure for message routing requirements

Several infrastructure components and SAAS support these advanced message routing capabilities:

Message queues (e.g., RabbitMQ, Apache Kafka, NATS, Apache Pulsar): These ensure reliable and efficient message delivery and queuing, allowing for intelligent routing and fan-out operations.
Notification hubs (e.g., AWS SNS, Azure Notification Hubs): These provide scalable platforms for sending notifications across multiple channels and to large audiences.
Scheduling SAAS (e.g., Quartz Scheduler, DKron): These help in scheduling notifications to ensure they are delivered at optimal times based on timezone and user preferences.
Presence SAAS (e.g., Firebase Realtime Database, PubNub): These SAAS can track user online/offline status and manage the queuing and delivery of notifications accordingly.
User analytics platforms (e.g., Mixpanel, Segment): These provide insights into user behavior and preferences, enabling intelligent routing decisions based on context and user settings.

Core requirement 4: team interactivity

Ensuring smooth team interactivity is vital for the effective management of notifications. This involves using collaboration tools, setting up roles and permissions, and empowering non-technical users to make changes without needing developer assistance. Here’s a detailed look at these requirements.

Assessing engineering’s content management role: Determine if developers need to handle all content tasks or if they can delegate to improve resource management and productivity.
Non-technical staff managing notifications: Allow non-technical staff to handle routine notification tasks, enhancing efficiency and enabling developers to focus on technical issues.
Content management collaboration: Enables team collaboration on notification content with real-time updates and shared feedback using platforms like shared workspaces and document editors.
Team member roles and permissions: Defines specific roles and permissions to ensure appropriate access, prevent unauthorized changes, and enhance accountability within the team.
Non-technical users changing copy and images: Allows non-technical users to update text and images through user-friendly interfaces, reducing the need for developer involvement and speeding up notification deployment.

Supporting tools and infrastructure for team interactivity requirements

Several infrastructure components and SAAS support advanced team interactivity:

Several infrastructure components and SAAS support advanced team interactivity:
Collaboration platforms (e.g., Slack, Microsoft Teams, Discord, Google Chat): These platforms facilitate real-time communication and collaboration among team members, streamlining the process of managing notifications.
Content management systems (e.g., Contentful, WordPress): These systems often come with built-in roles and permissions, allowing for structured access control and enabling non-technical users to update content easily.
Document editors (e.g., Google Docs, Quip): These tools provide collaborative editing capabilities, enabling team members to work together on notification content and make real-time updates and comments.
WYSIWYG editors (e.g., TinyMCE, Froala): These editors allow non-technical users to make changes to notification content and images without needing to write code, making it easy to update and manage notifications.
Version control systems (e.g., Git, Bitbucket): While primarily used by developers, these systems can also support collaboration by tracking changes and ensuring that all updates are reviewed and approved before going live.

By leveraging these tools and SAAS, you can create a collaborative environment that enhances team interactivity, ensuring efficient content and message management while empowering non-technical users. This approach helps maintain a dynamic and responsive notification system that can quickly adapt to changing needs and user feedback.

Core requirement 5: analytics and metrics

Analytics and metrics are the backbone of an effective notification system. They provide insights into delivery success, user engagement, and system performance. Here's an in-depth look at these key aspects and the infrastructure that supports them.

Tracking delivery success rates and user engagement: Monitors notification delivery across channels and analyzes user interactions to refine strategies and enhance satisfaction.
Setting up delivery rates for various channels: Optimizes notification campaigns by understanding and adjusting delivery rates for different channels to ensure timely delivery.
Monitoring and handling integration downtimes: Uses monitoring tools and alerts to quickly address integration issues, minimizing disruptions in notification delivery.
Measuring user engagement and response rates: Analyzes metrics like open rates and click-through rates to determine notification effectiveness and improve content and delivery strategies.

Supporting infrastructure and tools for analytics requirements

Several infrastructure components and SAAS support advanced analytics and metrics for notification systems:

Analytics platforms (e.g., Google Analytics, Mixpanel): These platforms provide detailed insights into user behavior and engagement, helping you track and analyze key metrics.
Monitoring tools (e.g., Datadog, New Relic): These tools help monitor system performance and integration downtimes, ensuring timely detection and resolution of issues.
Email/SMS analytics (e.g., SendGrid, Twilio): These SAAS offer built-in analytics for tracking delivery success rates and user engagement across email and SMS channels.
Push notification SAAS (e.g., Firebase Cloud Messaging, OneSignal): These platforms provide detailed metrics on push notification delivery and engagement, helping you optimize your campaigns.
A/B testing tools (e.g., Optimizely, VWO): These tools allow you to test different notification strategies and measure their impact on user engagement and response rates.

By leveraging these tools and SAAS, you can build a comprehensive analytics and metrics framework that provides deep insights into the performance of your notification system. This enables you to make data-driven decisions, optimize your notification strategies, and ensure a high-quality user experience.

Core requirement 6: volume and scaling

Managing increasing notification loads: Design your system for peak performance to handle high volumes, scaling horizontally to accommodate growing demands.

Scaling strategies: Use horizontal scaling based on network requests and queue sizes to distribute load, improve redundancy, and maintain performance during peak times.
Custom metrics: Monitor metrics like CPU usage, memory, network latency, and queue lengths to make informed decisions about scaling and resource allocation for optimal performance.

Custom metrics challenges

Custom metrics require a thorough understanding of system architecture and the specific performance indicators that impact your notification infrastructure. These include:

Identifying relevant metrics: Determining which metrics are most indicative of system performance and resource needs.
Implementing monitoring tools: Integrating tools that can capture and report these metrics in real-time, such as Prometheus and Grafana or 3rd party monitoring system.
Configuring alerts and thresholds: Setting up alerts and thresholds to notify your team when certain metrics exceed predefined limits, indicating the need for scaling adjustments.
Testing and calibration: Continuously testing and calibrating the metrics to ensure they accurately reflect system performance and provide actionable insights.

Using custom metrics for dynamic scaling

Once custom metrics are in place, leveraging them for dynamic scaling involves:

Automated scaling policies: Developing automated policies that trigger scaling actions based on real-time metrics. For example, adding servers when CPU usage exceeds a certain percentage or reducing servers when queue sizes drop below a threshold.
Continuous monitoring and adjustment: Regularly monitoring the performance of the scaling policies and adjusting them as needed to respond to changing conditions and workloads.
Integrating with orchestration tools: Using orchestration tools like Kubernetes, nomad, or a hosted container solution to manage the scaling process seamlessly, ensuring that resources are allocated efficiently without manual intervention.

Challenges in using custom metrics

Using custom metrics for dynamic scaling presents several challenges:

Complexity: Managing the complexity of multiple metrics and ensuring they accurately reflect the system’s needs can be daunting.
Resource allocation: Balancing resource allocation to avoid over-provisioning (which leads to higher costs) and under-provisioning (which impacts performance) requires constant tuning.
Integration with existing systems: Ensuring that custom metrics and scaling policies integrate smoothly with existing infrastructure and workflows can be technically demanding.

Summary

Effectively setting up and leveraging custom metrics for dynamic scaling is critical for maintaining a robust, scalable, and efficient notification infrastructure. While the process is complex and requires significant effort, the benefits of improved performance, cost-efficiency, and resilience make it a worthwhile investment. By focusing on these strategies and overcoming the associated challenges, you can ensure your notification system can adapt to evolving demands and deliver a seamless user experience.

Core requirement 7: debugging and notification tracing

Using advanced tools and methods to troubleshoot message delivery issues allows you to diagnose and resolve problems promptly. Tracing message paths ensures accountability by tracking the journey of each message, confirming it reaches its destination. Analyzing queues helps identify bottlenecks and delivery failures, ensuring smooth message flow. Additionally, determining whether issues are caused by internal systems or third-party integrations helps isolate and address problems more efficiently.

Tools and methods for troubleshooting delivery issues: Use diagnostic tools to detect and resolve message delivery problems, ensuring consistent notifications.
Tracing message paths and ensuring accountability: Track each message to verify delivery and identify failure points, maintaining system integrity and improvement insights.
Analyzing queues to identify bottlenecks and failures: Monitor and analyze queue data to detect bottlenecks and delivery failures, enabling corrective actions for smooth message processing.
Determining problem source: Isolate issues to either internal systems or third-party integrations for efficient troubleshooting and resolution.

Summary

Advanced tools enable prompt problem diagnosis and resolution, while tracing message paths ensures accountability. Analyzing queues helps identify bottlenecks, and distinguishing between internal and third-party issues allows efficient troubleshooting. These practices ensure smooth and efficient notification operations.

Running a notifications platform: operational considerations

Ensuring a robust and reliable notification system requires careful attention to several operational aspects. These include ongoing maintenance, handling volume and scaling, debugging and tracing issues, and team considerations. Here’s a concise overview of these critical areas.

Ongoing maintenance/patching, updates, and security updates

Regular system updates and security patches are essential for protecting your notification infrastructure from vulnerabilities. By staying current with the latest patches, you can ensure smooth operation and prevent potential security breaches. This proactive approach minimizes downtime and maintains system integrity.

Handling evolving requirements and feature enhancements

Adapting your system to meet new user needs and incorporate advanced features is crucial for staying relevant. This involves regularly assessing user feedback and market trends to update your notification infrastructure. By doing so, you can enhance user experience and maintain a competitive edge.

Integration changes to adapt to new SAAS and platforms

As new SAAS and platforms emerge, ensuring compatibility is vital. Regularly updating your system to integrate with tools like Firebase Cloud Messaging (FCM) allows you to leverage new technologies and maintain seamless communication. This adaptability keeps your infrastructure flexible and future-proof.

Notification templates and content updates

Keeping notification templates and content up-to-date ensures that your messages remain relevant and engaging. Regular updates help you cater to changing user preferences and maintain high engagement rates. Fresh content and appealing templates can significantly enhance the effectiveness of your notifications.

Provider integrations

Maintaining and updating integrations with various service providers is crucial for seamless communication. This involves ensuring that your notification infrastructure can effectively interact with email, SMS, and push notification SAAS. Regular checks and updates keep these integrations functioning smoothly.

API and SDK maintenance and testing

Regular maintenance and testing of APIs and SDKs are necessary to ensure they work correctly with other system components. This includes updating to the latest versions and conducting thorough testing to identify and fix any issues. Effective API/SDK management ensures reliable and efficient system performance.

Credential management and security

Managing credentials securely is essential to prevent unauthorized access and protect data. Regularly updating and securely storing credentials reduces the risk of security breaches. Implementing strong authentication methods and monitoring access controls are key practices in credential management.

Summary

Regular system updates and security patches are essential for protecting your infrastructure from vulnerabilities, minimizing downtime, and maintaining system integrity. Adapting to evolving requirements and integrating with new SAAS keeps your system relevant and flexible, while continuous security upgrades protect against emerging threats.
Maintaining and updating notification templates ensures messages remain engaging and effective. Regular checks and updates of provider integrations, along with thorough maintenance and testing of APIs and SDKs, ensure seamless communication and reliable performance. Secure credential management prevents unauthorized access and protects data.
By focusing on these operational considerations, you can ensure your notification system remains robust, scalable, and efficient, delivering a seamless user experience and supporting the evolving needs of your business.

Resourcing

When deciding whether to build or buy a notification infrastructure, accurately estimating the time required is crucial. Here’s a breakdown of the different tiers of building notification infrastructure, focusing on the time commitment needed for each level, from simple integrations to complex systems, and including ongoing maintenance estimates.

Simple notification integration
- Scope: Basic notifications for essential updates.
- Resources: 1 engineer, 2 weeks.
- Complexity: Low; minimal customization and features.
- External packages: Typically 2-4 external packages (e.g., SMTP libraries, basic notification frameworks).
- Ongoing maintenance: Low. Expect quarterly updates requiring 1-2 days of engineering time for testing and integration.
Multi-channel notifications
- Scope: Support for multiple channels (e.g., email, SMS, push).
- Resources: 1-3 engineers, 1 quarter.
- Complexity: Moderate; requires integration with various communication APIs and preference management.
- External packages: Approximately 6-24 external packages (e.g., Twilio for SMS, Firebase for push notifications, email service providers, etc).
- Ongoing maintenance: Moderate. Monthly updates for at least one package, requiring around 1 week of engineering time per month for testing and integration.
Segmented notifications
- Scope: Segmentation of users for targeted notifications.
- Resources: 2-4 engineers, 2 quarters.
- Complexity: High; involves building infrastructure for user segmentation, managing different user groups, and ensuring scalability.
- External packages: Around 8-24 external packages (e.g., user segmentation libraries/systems, advanced notification SAAS).
- Ongoing maintenance: High. Bi-weekly updates for multiple packages, with an estimated 2-3 weeks of engineering time per month for testing and integration.
Internationalization
- Scope: Full support for multiple languages and regional preferences.
- Resources: 3-5 engineers, 6 quarters.
- Complexity: Very high; involves developing a robust system for language translations, managing different regional preferences, and ensuring compliance with international regulations.
- External packages: Around 10-24 external packages (e.g., translation SAAS, localization frameworks).
- Ongoing maintenance: Very high. Weekly updates for various packages, demanding about 4-6 weeks of engineering time per month for testing, integration, and continuous improvement.
Personalization
- Scope: Personalized notifications based on user behavior and preferences.
- Resources: 4-6 engineers, 1-2 Data Engineers, 2-4 Infrastructure Engineers at least 14 quarters.
- Complexity: Extremely high; requires sophisticated data processing, user profiling, and dynamic content generation.
- External Packages: Approximately 16-32 external packages (e.g., machine learning libraries, recommendation engines, infrastructure and data lake integrations).
- Ongoing Maintenance: Extremely high. Weekly updates and compliance checks, requiring 6-8 weeks of engineering time per month to maintain, test, and integrate updates while ensuring regulatory adherence.

Decision time

By understanding the time requirements for each tier of building a notification infrastructure, you can better evaluate whether to build or buy. Consider your organization's specific needs, available resources, and long-term goals to make a decision that aligns with your strategic priorities. Whether opting for a simple integration or a fully personalized and internationalized system, ensuring that you have the right resources and timeline in place is crucial for success.

By examining the different tiers of notification infrastructure—from simple integrations to complex, personalized, and internationalized systems—you can better understand the time and effort required for each level of implementation. This knowledge will help you make an informed decision that aligns with your strategic priorities, ensuring that your notification system is robust, scalable, and capable of meeting the evolving needs of your users.

Whether you choose to build or buy, the goal is to create a seamless and effective notification experience that enhances user engagement and satisfaction while supporting the growth and success of your organization.

We had a date bug that happened two times a year, and we didn't know, you might have it too 😱

Zac_A_Clifton — Mon, 11 Sep 2023 12:08:00 +0000

TL;DR

Novu's team encountered a significant bug affecting date calculations in their CI/CD pipelines, hindering all deployments.

The issue arose from the date-fns library's addMonths and subMonths functions.

We fixed this by using addDays and subDays functions instead.

Novu: Open-source notification infrastructure 🚀

Just a quick background about us. Novu is an open-source notification infrastructure. We basically help to manage all the product notifications. It can be In-App (the bell icon like you have in the Dev Community - Websockets), Emails, SMSs and so on.

The Mindset

When working in software development, we're always prepared for bugs to crop up.

Sometimes they're small, easy to identify, and quick to fix.

Other times, they're like this year's candidate for our 'Bug Of The Year'.

This was a bug so elusive and mysterious that it had us rummaging through our pipelines, questioning our code-base, and coming face-to-face with the intricacies of date manipulation.

Problems, Different Problems, and More Problems

Our CI/CD pipelines were failing. Specifically, two tests which were blocking ALL new deployments. It was time to put on our detective hats 🕵️.

We dove into our commit history using git bisect however it offered us no insight. Git bisect took us back to commits that where over 6 months in the past, long before any of our newest changes to the system that would have caused this. Was this bug created at the very beginning of Novu?

However, we did have a clue. Our failing unit tests showed us that we had incorrect date calculations.

Gathering the Clues 💡

Strangely, the difference was just one day.

const startDate = new Date("2023-08-31");
const oneMonthAhead = addMonths(startDate, 1);
const result = subMonths(oneMonthAhead, 1);
console.log(result);  // Expected: 31st of August, Reality: 30th of August

We also found that this does not happen on 31st July.

const startDate = new Date("2023-07-31");
const oneMonthAhead = addMonths(startDate, 1);
const result = subMonths(oneMonthAhead, 1);
console.log(result);  // Expected: 31st of July, Reality: 31th of July

But the bug shows up again January 31st.

const startDate = new Date("2023-01-31");
const oneMonthAhead = addMonths(startDate, 1);
const result = subMonths(oneMonthAhead, 1);
console.log(result);  // Expected: 31st of January, Reality: 28th of January

So this bug only happens when we add 1 month to a month that has more days then the next month and then subtract 1 month to go back to the month before.

This is a sneaky one

So here is what we know so far:

It would only show up on systems that does this specific sequence of logic.
The code would have to be ran on one of the few dates that are effected.
This effect is not documented anywhere on any of the libraries we use.

The worst thing is that this bug is also shows up HR tools, finance tools, salary tools, public government tools all rely on this package but unfortunately it is still better then us making the functions our-self's.

It has been said many times that date-times are among the trickiest aspects of programming, and our current predicament served as a hash reminder.

Why a simple actions can lead to bad things

After finding this out, we had a 'Eureka!' moment.
Our CTO, Dima Grossman, then had the idea to try it it on raycast. Interestingly enough it was happening in their product too.

We realized that the issue stemmed from being on the last day of the month, but what exactly was going awry?

The Culprit:

This popular utility library for date operations was at the heart of the problem.

Specifically, the addMonths and subMonths functions.

The addMonths function, when adding a month to the last day of any given month, would take you to the last day of the following month. Logical, right?

// source: https://github.com/date-fns/date-fns/blob/main/src/addMonths/index.ts
const daysInMonth = endOfDesiredMonth.getDate()
  if (dayOfMonth >= daysInMonth) {
    // If we're already at the end of the month, then this is the correct date
    // and we're done.
    return endOfDesiredMonth
  } else {
    // Otherwise, we now know that setting the original day-of-month value won't
    // cause an overflow, so set the desired day-of-month. Note that we can't
    // just set the date of `endOfDesiredMonth` because that object may have had
    // its time changed in the unusual case where where a DST transition was on
    // the last day of the month and its local time was in the hour skipped or
    // repeated next to a DST transition.  So we use `date` instead which is
    // guaranteed to still have the original time.
    _date.setFullYear(
      endOfDesiredMonth.getFullYear(),
      endOfDesiredMonth.getMonth(),
      dayOfMonth
    )
    return _date
  }

But the subMonths function, rather than having its own dedicated logic, simply reused addMonths with a negative number. D.R.Y principles in action, but with an unintended consequence.

// source: https://github.com/date-fns/date-fns/blob/main/src/subMonths/index.ts
export default function subMonths<DateType extends Date>(
  date: DateType | number,
  amount: number
): DateType {
  return addMonths(date, -amount)
}

Here is what exactly caused our issue

Let's put it this way:

For 28th February, add one month and then subtract one month, and you get 28th February. No problems there.
But, for 31st August, add one month and then subtract one month, and you land on... 30th August. That's one day lost in date limbo!

The core of the issue was the way addMonths determined the end of the desired month.

For days that were not at the end of the month, the logic was sound.

However, for the last day of a month, the function defaulted to the end of the next month instead of adding the correct amount of days.

The Simple Fix

To ensure a consistent approach to date manipulation, we shifted from using addMonths and subMonths to addDays and subDays.

This provided a more granular and precise way to handle date calculations, and importantly, allowed us to sidestep the addMonths pitfall.

Lessons Learnt

This bug served as a strong lesson in a few key areas:

Assumptions are Risky: Never assume that widely-used libraries are infallible. Even the most popular ones have their quirks.
Tests are Gold: If not for our rigorous testing suite, this bug might have remained hidden, only to wreak havoc at the most inopportune moment.
Dates are Tricky: They've always been, and will continue to be, a challenging aspect of software development. Always handle with care.

While this bug threw a wrench in our pipes, it also reinforced the importance of comprehensive tests and the need to continually question and challenge our assumptions.

Death of this Bug

In a world of code where dates and times form such a crucial part of our applications, bugs like these provide not just a hiccup, but a learning opportunity. The next time you find a weird issue in your application, dig deep. Who knows, you might just uncover the next 'Bug Of The Year'.

You can find the PRs and Issues here:

📜 Novu's Communication Manifest: Lighting the Path to our Future 💡

Zac_A_Clifton — Mon, 28 Aug 2023 12:13:31 +0000

TL;DR

As part of our second offsite, the team at Novu created a set of principles to help drive our interactions with our communities, the companies we support, and internally with our self.

Principles

Assume Good Intentions
Equality of Opinions
Mindfulness
Support Each Other
Transparency
Publicize as Much as Possible
Adaptability
Plan Ahead
Consistent Communication
Make an Effort

What we were doing was not scaling

At Novu, we like to think of our mission as more than just business - it's about creating a powerful 💪 , shared culture that promotes effective communication. We are committed to building a robust, asynchronous communication channels with talented individuals around the globe.

However, as we where going about our daily async communication we would have to constantly have to talk between each other on how we can do better.

While talking between each other was getting us closer to what we wanted, we need to have a target 🎯 to guide us.

This is where our communication manifest 📜 comes in. This manifest holds our principles for communication with us and the people we support.

These principles steer our decision-making procedures, supporting our mission to adapt and thrive amidst the rapidly evolving digital landscape.

Now we want to share this Manifest with you.

📜 Novu's Manifest 📜

1. 🧡 Always Assume Good Intentions

No matter the form everyone has something we can learn from.

We firmly uphold the principle of seeing the best in people and that everyone we interact with can offer us valuable insights, enhancing our collective wisdom.

2. 🟰 ALL Ideas and Opinions are Equal, Regardless of Hierarchy or Source

Every voice matters at Novu.

We promote a culture where everyone's opinions are held as equal, irrespective of their role or level within the company.

3. 🧠 Be Mindful of How You Communicate

We understand that communication is a powerful tool.

We strive to be mindful of our words, tone, and actions to ensure we respect and value each other's feelings, perspectives, and experiences.

4. 🏛️ Support Each Other to Become Better Communicators

We share a collective passion for shaping a better world.

Our dedication is reflected in our relentless pursuit to enhance our communication prowess and cultivate a culture rooted in learning and growth.

5. 🪟 Be Transparent and Explicit with Expectations

Transparency is key to our operations.

We express our expectations clearly and explicitly, making sure that everyone has a clear understanding of their roles, responsibilities, and the goals we are striving to achieve together.

6. 💬 Publicize Decisions and Consider All Affected Individuals

We are committed to open communication.

We publicize our decisions and make sure that all affected individuals are considered and informed before finalizing any major decisions.

7. 🌪️ Remain Adaptable to Others' Priorities and Needs

We understand that in a fast-paced environment, adaptability is critical.

We articulate our priorities and needs while staying flexible to accommodate the changing needs and priorities of our colleagues, customers, and community.

8. 📝 Assume Unavailability and Plan Accordingly

We operate in a global environment, and as such, unavailability is a given.

We plan and execute our tasks with this in mind to ensure that our operations run smoothly and efficiently, regardless of time zones and personal schedules.

9. 📡 Communicate All Necessary Details for Planned Interactions

We value clarity and precision in our interactions.

We ensure that all necessary details for planned interactions are communicated effectively, making our meetings productive and valuable for everyone involved.

10. 🙏 Make an Effort to Promptly Respond to Others

We are committed to going above and beyond.

We make an effort to respond quickly to others, acknowledging their contributions, and keeping our collaborative efforts moving forward as we understand that prompt responses are crucial for effective collaboration.

The way forward

These ten principles are the bedrock of our operations at Novu and the people we support.

They guide us in our quest for excellence, and they serve as a beacon to light our path as we continue to innovate and grow.

Novu

Novu a leading open-source notification infrastructure for developers. Check out how we embody these principles and more on our public handbook at: https://handbook.novu.co/

Thank you for reading!