DEV Community: ke yi

LangSmith Hosted vs Self: Deployment Guide 2026

ke yi — Fri, 18 Sep 2026 17:23:16 +0000

LangSmith Hosted vs Self-Hosted: Which Deployment Model Fits Your LLM Observability Needs?

TL;DR: LangSmith operates exclusively as a managed SaaS platform with no self-hosting option available as of 2026. Teams requiring on-premise LLM observability must evaluate open-source alternatives like Langfuse (MIT license, full self-hosting support), Phoenix by Arize AI (Apache 2.0, containerized deployment), or custom solutions built on OpenTelemetry. Choose SaaS LangSmith for zero-ops tracing with deep LangChain integration; choose self-hosted alternatives when data sovereignty, cost control, or compliance policies prohibit external trace storage.

Key Takeaways

LangSmith is a SaaS-only platform with no official self-hosting option, making the "hosted vs self" decision actually a choice between LangSmith SaaS and open-source alternatives like Langfuse or Phoenix.
Self-hosting LLM observability provides data sovereignty, cost predictability, and compliance control, but requires infrastructure expertise, operational overhead, and ongoing maintenance responsibility.
Langfuse offers both managed SaaS and open-source self-hosting with Docker/Kubernetes deployment, making it the closest alternative to a "self-hosted LangSmith" architecture.
Phoenix by Arize AI provides Apache 2.0 licensed observability optimized for notebook-based development and local debugging, with containerized deployment for production environments.
Total cost comparison must include infrastructure, engineering time, data egress, and opportunity cost — self-hosting is not always cheaper despite eliminating SaaS subscription fees.
The decision boundary is whether your organization's data governance, budget constraints, or customization requirements justify the operational complexity of running your own observability infrastructure.

What does LangSmith hosted versus self-hosted actually mean?

LangSmith, developed by LangChain Inc., is a managed observability platform delivered exclusively as Software-as-a-Service (SaaS). The platform has no self-hosting option, no enterprise on-premise deployment package, and no open-source release as of 2026. When organizations search for "LangSmith self-hosted," they are typically asking one of three questions:

Can I run LangSmith on my own infrastructure? No. LangSmith's codebase is proprietary.
What self-hosted alternatives provide similar capabilities? Langfuse and Phoenix are the primary options.
Should I accept SaaS for observability, or does my use case require self-hosting? This is the actual decision.

The comparison therefore becomes: LangSmith SaaS versus self-hosted open-source observability tools. Understanding this frame is critical before evaluating trade-offs, because "self-hosted LangSmith" does not exist.

Why would you want self-hosted LLM observability?

Organizations consider self-hosted observability for six primary reasons, each with concrete technical or business requirements.

Data Sovereignty and Compliance

When prompts contain personally identifiable information (PII), protected health information (PHI), or classified data, external SaaS storage may violate regulatory requirements. GDPR Article 48, HIPAA, FedRAMP, and defense classification policies often mandate on-premise data processing.

Concrete scenario: A healthcare AI assistant that discusses patient symptoms must comply with HIPAA. Sending trace data containing patient information to an external SaaS platform requires a Business Associate Agreement (BAA) and may still be prohibited by internal security policies. Self-hosting keeps patient data within the organization's security perimeter.

Cost Predictability at Scale

LangSmith pricing scales with trace volume: Developer plan ($39/month, 50K traces), Team plan ($199/month, 500K traces), Enterprise (custom pricing). An agent handling 10 million requests monthly with 10% trace sampling generates 1 million traces per month, pushing costs into four-figure monthly SaaS fees.

Self-hosted infrastructure cost is dominated by storage (traces persist for 180+ days) and compute (query performance). Organizations with existing Kubernetes clusters and object storage can deploy Langfuse at marginal cost: compute overhead for the API server, PostgreSQL database hosting, and S3-compatible storage. This becomes cost-effective above ~500K traces/month if engineering time is available.

Customization and Integration

Open-source observability tools allow custom evaluators, modified retention policies, and direct database access for internal analytics. SaaS platforms constrain customization to their API surface.

Example requirement: An organization wants to join trace data with internal experimentation metadata stored in Snowflake, then build custom dashboards in Tableau. Self-hosted PostgreSQL enables direct SQL access; SaaS platforms require API-based export with rate limits.

Air-Gapped and Restricted Networks

Defense contractors, financial institutions with segmented networks, and high-security research labs operate in environments where internet egress is blocked or tightly controlled. SaaS observability is impossible; self-hosted deployment on the internal network is the only option.

Avoiding Vendor Lock-In

Trace data, datasets, and evaluation results stored in a proprietary SaaS platform create migration friction. Self-hosted open-source tools provide portable data formats and direct database access, simplifying future migrations.

Control Over Feature Roadmap

SaaS platforms prioritize features benefiting their broad customer base. Self-hosted open-source tools can be forked, extended, or modified to meet niche requirements without waiting for vendor support.

How do LangSmith's SaaS-only constraints shape the comparison?

Because LangSmith does not offer self-hosting, the comparison cannot be apples-to-apples. Instead, evaluate:

Dimension	LangSmith SaaS	Langfuse Self-Hosted	Phoenix Self-Hosted
Deployment model	Managed SaaS, zero infrastructure	Docker/Kubernetes on your infrastructure	Docker/local notebook, Kubernetes optional
Data location	LangChain Inc. infrastructure (US/EU regions)	Your cloud or on-premise	Your infrastructure
Operational responsibility	LangChain Inc. manages uptime, backups, scaling	Your team manages infrastructure, upgrades, monitoring	Your team manages deployment
Cost model	Per-trace subscription tiers	Infrastructure + engineering time	Infrastructure + engineering time
LangChain integration	Native, zero-config	SDK integration, requires explicit instrumentation	SDK integration, explicit instrumentation
Feature parity	Full LangSmith feature set	Core tracing + evaluations + prompt management	Core tracing + evaluations, limited prompt management
Customization	API-level only	Full database and codebase access	Full database and codebase access
Vendor lock-in risk	High: proprietary data storage	Low: PostgreSQL + S3	Low: SQLite/PostgreSQL

The practical choice is therefore between accepting LangSmith's SaaS constraints for operational simplicity, or adopting a self-hosted alternative for control and compliance at the cost of operational complexity.

What are the leading self-hosted LangSmith alternatives?

Three open-source platforms dominate the self-hosted LLM observability space in 2026, each with different strengths and deployment patterns.

Langfuse: The Self-Hostable LangSmith Alternative

Langfuse is an MIT-licensed observability platform supporting both managed SaaS and self-hosting. It provides distributed tracing, prompt management, datasets, evaluations, and LLM-as-judge scoring. Langfuse is the most direct LangSmith alternative for teams requiring self-hosting.

Architecture:

Next.js web application (TypeScript)
PostgreSQL database for trace storage
S3-compatible object storage for large payloads
Optional Redis for caching
Docker Compose or Kubernetes deployment

Self-hosting deployment:

# Docker Compose example (simplified)
git clone https://github.com/langfuse/langfuse.git
cd langfuse
cp .env.example .env
# Configure DATABASE_URL, S3 credentials
docker-compose up -d

Production deployments use Kubernetes with managed PostgreSQL (AWS RDS, GCP Cloud SQL) and object storage (S3, GCS, MinIO). Langfuse's Helm chart simplifies Kubernetes deployment with configurable replicas, autoscaling, and ingress.

LangChain integration:

from langfuse.callback import CallbackHandler
from langchain_anthropic import ChatAnthropic
from langchain.agents import create_react_agent

# Point to self-hosted Langfuse instance
langfuse_handler = CallbackHandler(
    host="https://langfuse.your-company.com",
    public_key="pk_xxx",
    secret_key="sk_xxx"
)

llm = ChatAnthropic(model="claude-sonnet-4-20250514")
agent = create_react_agent(llm, tools, prompt)

# Traces sent to your self-hosted instance
agent.invoke(
    {"input": "What is the weather?"},
    config={"callbacks": [langfuse_handler]}
)

When to choose Langfuse:

Need self-hosting with feature parity closest to LangSmith
Existing PostgreSQL and object storage infrastructure
Kubernetes expertise available for production deployment
MIT license acceptable for your organization
Want option to use managed SaaS during prototyping, self-host in production

Phoenix: Notebook-First Observability for Local Development

Phoenix by Arize AI is an Apache 2.0 licensed observability tool optimized for notebook-based development and local debugging. It runs as a lightweight server within development environments, then scales to containerized deployment for production.

Architecture:

Python application with FastAPI server
SQLite (development) or PostgreSQL (production)
Embedded web UI served from the Python process
No external dependencies for basic usage

Local development deployment:

pip install arize-phoenix

import phoenix as px

# Launch Phoenix server locally
session = px.launch_app()

# Instrument your application
from openinference.instrumentation.langchain import LangChainInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

# Configure tracing
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(
    SimpleSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:6006/v1/traces"))
)
trace.set_tracer_provider(tracer_provider)

# Auto-instrument LangChain
LangChainInstrumentor().instrument()

# Your LangChain code runs with local Phoenix tracing
agent.invoke({"input": "Debug this locally"})

Production deployment:
Phoenix supports containerized deployment with PostgreSQL backend. Less feature-complete than Langfuse for prompt management and complex evaluations, but sufficient for tracing and basic debugging.

When to choose Phoenix:

Primary use case is local development and debugging
Want zero-setup local observability during prototyping
Already using Arize AI for ML monitoring (unified platform)
Apache 2.0 license required
Prefer lightweight deployment over full-featured platform

OpenTelemetry + Jaeger: Custom Observability Stack

For maximum control, build custom LLM observability using OpenTelemetry for instrumentation and Jaeger for trace storage and visualization. This requires the most engineering effort but provides unlimited customization.

Architecture:

OpenTelemetry SDK for trace instrumentation
Jaeger for trace collection, storage (Cassandra/Elasticsearch), and UI
Custom evaluators and dashboards built on raw trace data

This approach is appropriate only when neither Langfuse nor Phoenix meet specific technical requirements (e.g., integration with existing OpenTelemetry infrastructure, custom storage backends, regulatory constraints on using third-party platforms).

How do you actually self-host LLM observability in production?

Self-hosting requires planning beyond "run Docker Compose." Production deployment decisions affect reliability, cost, and operational burden.

Infrastructure Requirements

Compute:

API server: 2-4 vCPU, 4-8 GB RAM for modest load (< 100 requests/sec)
Worker processes: 1-2 vCPU per background job worker (evaluations, async processing)
Auto-scaling: Horizontal pod autoscaling based on CPU/request rate

Storage:

PostgreSQL: Primary datastore for traces, projects, users, datasets
Estimated size: ~1 KB per trace, 1 million traces = ~1 GB uncompressed
Retention: 180 days at 1M traces/month = ~180 GB
Object storage: Large trace payloads, evaluation artifacts
Estimated size: ~10-50 KB per large trace, varies by payload size

Networking:

Ingress: HTTPS load balancer (AWS ALB, GCP Load Balancer, NGINX)
TLS termination: Let's Encrypt or corporate certificate
Egress: Outbound to LLM providers if evaluations use LLM-as-judge

Security Hardening

Authentication and Authorization:

SSO integration (SAML, OAuth) for enterprise deployments
API keys with role-based access control (RBAC)
Audit logging for trace access and data export

Data Encryption:

Encrypt at rest: Database encryption (AWS RDS encryption, GCP CMEK)
Encrypt in transit: TLS 1.3 for all connections
Secrets management: Vault, AWS Secrets Manager, or GCP Secret Manager

Network Isolation:

Deploy in private subnet with no direct internet access
Expose only via load balancer in public subnet
Use VPC peering or PrivateLink for access from application VPCs

Operational Concerns

Backup and Recovery:

Automated daily PostgreSQL backups with point-in-time recovery
Object storage replication across availability zones
Test restore procedures quarterly

Monitoring and Alerting:

Infrastructure metrics: CPU, memory, disk, network via Prometheus/Datadog
Application metrics: Request rate, error rate, latency
Database metrics: Query performance, connection pool saturation
Alert on: API downtime, database connection failures, disk space < 20%

Scaling:

Horizontal scaling: Add API server replicas behind load balancer
Database scaling: Read replicas for query-heavy workloads, vertical scaling for write-heavy
Storage scaling: Object storage is effectively unlimited; PostgreSQL may require sharding above 10M traces

Upgrades and Maintenance:

Follow Langfuse/Phoenix release notes for breaking changes
Test upgrades in staging environment before production
Schedule maintenance windows for database migrations
Maintain version compatibility between SDK and server

What are the total cost models for hosted versus self-hosted?

Cost comparison requires modeling beyond subscription fees. Include infrastructure, engineering time, and opportunity cost.

LangSmith SaaS Cost Model

Direct costs:

Developer plan: $39/month (50K traces)
Team plan: $199/month (500K traces)
Enterprise: Custom pricing, typically $1,000-2,000/month for 1-5M traces

Example: 1 million traces/month

Team plan ($199/month) covers 500K traces
Additional 500K traces requires Enterprise tier: ~$1,500/month
Total: ~$1,500/month = $18,000/year

Hidden costs:

None for infrastructure or operations
Potential data egress fees if exporting large volumes
Evaluation runs using LLM-as-judge incur additional LLM API costs

Langfuse Self-Hosted Cost Model

Infrastructure costs (AWS example, 1M traces/month):

RDS PostgreSQL (db.t4g.large): $150/month
S3 storage (180 GB retained): $4/month
EKS cluster (2 nodes, t3.medium): $60/month
ALB load balancer: $20/month
Data transfer: $20/month
Total infrastructure: ~$254/month = $3,048/year

Engineering costs:

Initial deployment: 40-80 hours (1-2 weeks)
Ongoing maintenance: 10-20 hours/month (monitoring, updates, incident response)
Annual maintenance: 120-240 hours/year

Valuing engineering time:

Junior DevOps: $75/hour
Senior DevOps: $150/hour
Initial deployment cost: $6,000-$12,000 (one-time)
Annual maintenance cost: $9,000-$36,000

Total self-hosted cost (first year):

Infrastructure: $3,048
Initial deployment: $6,000-$12,000
Maintenance: $9,000-$36,000
Total: $18,048-$51,048

Total self-hosted cost (subsequent years):

Infrastructure: $3,048
Maintenance: $9,000-$36,000
Total: $12,048-$39,048

Cost Comparison Summary

Scenario	LangSmith SaaS	Self-Hosted (Low)	Self-Hosted (High)
Year 1	$18,000	$18,048	$51,048
Year 2	$18,000	$12,048	$39,048
Year 3	$18,000	$12,048	$39,048
3-Year Total	$54,000	$42,144	$129,144

Cost crossover point: Self-hosting becomes cheaper if engineering time is below ~15 hours/month (junior engineer) or if trace volume grows significantly beyond 1M/month, where LangSmith Enterprise pricing increases but self-hosted infrastructure scales more linearly.

Non-financial costs:

Self-hosting incurs opportunity cost: engineering time spent on observability infrastructure cannot be spent on product features
SaaS incurs flexibility cost: cannot customize beyond API surface, cannot guarantee data residency in specific jurisdictions

How does data sovereignty affect the hosted versus self-hosted decision?

Data sovereignty requirements create hard constraints that override cost considerations. When regulatory or policy requirements mandate on-premise data processing, self-hosting is the only option.

Regulatory Compliance Scenarios

GDPR (Europe):
LangSmith offers EU data residency with data stored in European data centers. This satisfies most GDPR requirements if Data Processing Agreement (DPA) is executed. Self-hosting required only if internal policy prohibits third-party processors.

HIPAA (Healthcare, US):
LangSmith Enterprise includes Business Associate Agreement (BAA) for HIPAA compliance. Many healthcare organizations still prefer self-hosting to minimize PHI exposure, especially when prompts contain sensitive patient information.

FedRAMP (US Government):
LangSmith is not FedRAMP authorized as of 2026. Government contractors requiring FedRAMP compliance must self-host on FedRAMP-authorized infrastructure (AWS GovCloud, Azure Government) or use approved observability tools.

Defense Classification:
Classified systems operate on air-gapped networks with no external connectivity. SaaS observability is impossible; self-hosted deployment on SIPRNET or internal networks is mandatory.

Data Residency Technical Implementation

Self-hosting enables precise control over data location:

Multi-region deployment example:

Deploy Langfuse instance in AWS eu-west-1 for European users
Deploy separate instance in AWS us-east-1 for US users
Route traces to region-specific instance based on user location
No cross-region data transfer

On-premise deployment:

Deploy Langfuse on Kubernetes cluster in corporate data center
Use on-premise PostgreSQL and MinIO (S3-compatible storage)
No data leaves corporate network
Satisfies strictest data residency policies

What are the operational trade-offs between hosted and self-hosted?

Beyond cost and compliance, operational characteristics differ significantly between SaaS and self-hosted deployment.

Reliability and Availability

LangSmith SaaS:

Uptime SLA: Typically 99.9% for paid plans (no public SLA published)
Multi-region redundancy managed by LangChain Inc.
Zero operational burden for disaster recovery
Risk: Single vendor dependency, outage affects all customers simultaneously

Self-Hosted:

Uptime depends on your infrastructure and operational practices
Must implement own redundancy, backups, monitoring
Full control over incident response and recovery
Risk: Responsibility for 3 AM outages falls on your team

Upgrade Cadence and Features

LangSmith SaaS:

Automatic feature updates with no action required
New features available immediately upon release
Breaking changes handled by vendor with migration guides
Risk: Cannot delay updates if breaking change affects workflows

Self-Hosted:

Control update schedule, test in staging before production
Delayed access to new features until you upgrade
Manual effort required for each upgrade
Risk: Falling behind on updates creates technical debt, security vulnerabilities

Data Portability and Migration

LangSmith SaaS:

Export via API with rate limits
Proprietary storage format requires transformation for migration
High friction to migrate to alternative platform
Risk: Vendor lock-in increases over time as data accumulates

Self-Hosted:

Direct database access, standard formats (PostgreSQL, S3)
Can write custom export scripts without API constraints
Easier migration to alternative tools or custom solutions
Risk: Must maintain own data integrity and backup processes

When should you choose LangSmith SaaS versus self-hosted alternatives?

Decision framework based on organizational constraints and priorities:

Choose LangSmith SaaS When:

You are using LangChain heavily and want zero-friction tracing. LangSmith's native integration with LangChain provides automatic tracing with environment variable configuration. Self-hosted alternatives require explicit SDK integration.

Your trace volume is under 500K/month and engineering time is expensive. SaaS pricing is competitive at this scale, and operational burden is zero. Self-hosting requires DevOps expertise that could be spent on product development.

Data residency requirements are satisfied by LangSmith's US/EU regions. If regulatory compliance accepts third-party processing with DPA/BAA, SaaS removes operational complexity.

You prioritize rapid feature access and mature prompt management. LangSmith's Prompt Hub, evaluation features, and continuous updates provide the most complete feature set in the market.

Your organization lacks Kubernetes or infrastructure automation expertise. Operating self-hosted observability requires DevOps skills; SaaS eliminates this requirement.

Choose Self-Hosted Alternatives When:

Regulatory or policy requirements mandate on-premise data processing. Air-gapped networks, defense classification, or strict data residency policies prohibit external SaaS.

Trace volume exceeds 1 million/month and engineering resources are available. Cost crossover point favors self-hosting at scale if DevOps time can be allocated.

Customization beyond API limits is required. Direct database access enables custom analytics, integrations with internal data warehouses, and modified retention policies.

Vendor lock-in risk is unacceptable. Open-source self-hosting with PostgreSQL backend provides data portability and migration flexibility.

You need air-gapped deployment or network-restricted environments. Corporate networks blocking external API calls make SaaS observability impossible.

Hybrid Approach: Development SaaS, Production Self-Hosted

Many organizations use LangSmith SaaS during prototyping for rapid iteration, then migrate to self-hosted Langfuse for production to satisfy compliance requirements while benefiting from SaaS convenience during development. This requires maintaining SDK compatibility between platforms and accepting one-time migration effort.

What are common mistakes when deploying self-hosted LLM observability?

After observing dozens of self-hosted observability deployments, these failure patterns recur consistently.

Underestimating Operational Complexity

Symptom: Initial deployment succeeds, but system degrades over weeks due to lack of monitoring, backup testing, or upgrade planning.

Fix: Treat self-hosted observability as a production service requiring the same operational rigor as user-facing applications. Implement automated backups, monitoring dashboards, and documented runbooks before declaring deployment complete.

Inadequate Database Sizing

Symptom: Database runs out of disk space, queries become slow, or connection pool exhausts under load.

Fix: Model storage requirements based on trace volume and retention policy. Provision 3x expected capacity for growth headroom. Use managed database services (RDS, Cloud SQL) with automated scaling rather than self-managed PostgreSQL unless expertise exists.

Neglecting Security Hardening

Symptom: Default credentials, unencrypted databases, or publicly exposed API endpoints create security vulnerabilities.

Fix: Follow security checklist: change default passwords, enable database encryption at rest and in transit, deploy in private subnet with load balancer ingress only, integrate SSO authentication, enable audit logging.

No Disaster Recovery Testing

Symptom: Backups exist but have never been restored; when disaster strikes, recovery fails or takes days.

Fix: Schedule quarterly disaster recovery drills. Restore backup to staging environment, verify data integrity, measure recovery time objective (RTO). Document and automate recovery procedures.

Ignoring Upgrade Path

Symptom: Self-hosted platform falls multiple versions behind due to fear of breaking changes; accumulates technical debt and security vulnerabilities.

Fix: Subscribe to release notes, test upgrades in staging environment within one month of release, maintain version compatibility matrix between server and SDK versions.

Over-Engineering Initial Deployment

Symptom: Spending months architecting multi-region, highly available deployment before capturing first trace.

Fix: Start with single-region Docker Compose deployment on small VM, capture traces, prove value. Invest in Kubernetes, multi-region redundancy, and advanced monitoring only after demonstrating ROI from observability data.

FAQ

Can I self-host LangSmith?

No, LangSmith is a SaaS-only platform with no self-hosting option as of 2026. LangChain Inc. offers enterprise contracts with extended features, but the product architecture is proprietary and cannot be deployed on customer infrastructure. Organizations requiring self-hosted LLM observability must evaluate open-source alternatives like Langfuse (MIT license) or Phoenix by Arize AI (Apache 2.0 license), both of which provide LangSmith-like capabilities with full self-hosting support.

What is the closest self-hosted alternative to LangSmith?

Langfuse provides the closest feature parity to LangSmith with full self-hosting support. It includes distributed tracing, prompt management with versioning, datasets, evaluations, and LLM-as-judge scoring. Langfuse is MIT licensed and supports both Docker Compose and Kubernetes deployment. The platform integrates with LangChain through callback handlers (not as seamless as LangSmith's native integration, but functionally equivalent). Phoenix by Arize AI is a lighter-weight alternative optimized for local development, with optional containerized production deployment.

How much does it cost to self-host LLM observability?

Total cost includes infrastructure and engineering time. For 1 million traces per month with 180-day retention on AWS: infrastructure costs approximately $250/month (RDS PostgreSQL, S3 storage, EKS cluster, load balancer). Engineering costs dominate: 40-80 hours for initial deployment ($6,000-$12,000 one-time) and 10-20 hours per month for ongoing maintenance ($9,000-$36,000 annually, depending on engineer seniority). Total first-year cost ranges from $18,000 to $51,000. Subsequent years cost $12,000-$39,000. Self-hosting becomes cost-effective compared to LangSmith Enterprise pricing ($18,000/year for 1M traces) only if engineering time is below 15 hours/month or trace volume scales significantly higher.

Does LangSmith support on-premise deployment for enterprise customers?

No, LangSmith does not offer on-premise deployment even for enterprise customers. LangSmith Enterprise provides enhanced features (SSO, dedicated support, extended retention, compliance certifications), but the deployment model remains SaaS hosted by LangChain Inc. Data residency options include US and EU regions, and enterprise contracts include Data Processing Agreements (DPA) and Business Associate Agreements (BAA) for HIPAA, but the infrastructure is never deployed on customer premises. Organizations with hard on-premise requirements must use self-hosted alternatives.

What data does LangSmith store, and can it contain sensitive information?

LangSmith stores full traces including model inputs (prompts), outputs (completions), tool calls, tool results, metadata, and performance metrics. Prompts can contain highly sensitive information: user queries, PII, PHI, internal data, API keys accidentally logged, or business-critical context. If your LLM application processes regulated data (healthcare, financial, government), trace data inherits the same classification. LangSmith Enterprise provides encryption at rest, SOC 2 compliance, and BAA for HIPAA, but ultimately traces are stored on LangChain Inc. infrastructure. Self-hosting keeps this data within your security perimeter.

How do Langfuse and LangSmith compare for LangChain applications?

LangSmith provides deeper LangChain integration with zero-config tracing (set environment variables, tracing is automatic). Langfuse requires explicit callback handler instantiation and passing to each agent invocation. Functionally equivalent but slightly more code. LangSmith has more mature Prompt Hub with A/B testing; Langfuse's prompt management is simpler but functional. LangSmith evaluation features are more polished; Langfuse evaluations are sufficient for most needs. The key differentiator is deployment model: LangSmith is SaaS-only, Langfuse offers both SaaS and self-hosting. Choose LangSmith for convenience and feature maturity, Langfuse for self-hosting and open-source transparency.

Can I migrate from LangSmith to self-hosted Langfuse?

Yes, but migration requires effort. Export traces from LangSmith via API (subject to rate limits), transform to Langfuse's trace format, and import using Langfuse SDK. Datasets, prompts, and evaluation results also need transformation. No automated migration tool exists as of 2026. The process is feasible for teams with engineering resources but not trivial. Best practice: start with self-hosted Langfuse from day one if data sovereignty is a future requirement, avoiding migration costs. If already on LangSmith, justify migration effort based on compliance necessity or sustained cost savings, not just preference.

What compliance certifications does LangSmith provide?

LangSmith Enterprise includes SOC 2 Type II certification, GDPR compliance with Data Processing Agreement (DPA), and HIPAA compliance with Business Associate Agreement (BAA). The platform encrypts data at rest and in transit, supports SSO for authentication, and provides audit logging. However, LangSmith is not FedRAMP authorized, meaning US government contractors requiring FedRAMP compliance cannot use it. Defense and classified systems requiring air-gapped networks also cannot use SaaS LangSmith. Self-hosted alternatives deployed on FedRAMP-authorized infrastructure (AWS GovCloud, Azure Government) can satisfy stricter compliance requirements.

How does observability affect LLM application performance?

Tracing adds minimal latency overhead (typically < 5ms per traced operation) but introduces network calls to send traces asynchronously. LangSmith and self-hosted alternatives use background threads for trace submission, preventing blocking of application logic. Negligible impact on user-facing latency. Storage and bandwidth consumption scale with trace volume: 1 KB per simple trace, 10-50 KB for traces with large tool outputs. For high-volume applications (millions of requests/day), implement intelligent sampling: trace 100% of errors and user feedback, 1-10% of successful requests for statistical monitoring. This reduces costs by 90% without losing critical debugging data.

Sources

Originally published at fp8.co. Subscribe for weekly AI engineering analysis at fp8.co/newsletters.

Lovable App Builder: Complete Funding Timeline & Founders

ke yi — Wed, 16 Sep 2026 18:58:33 +0000

Lovable is an AI-powered application builder that enables users to create full-stack web applications using natural language descriptions, founded in 2023 by Anton Osika in Stockholm, Sweden.

TL;DR

Lovable has raised $517M total funding across 4 rounds from 2025-2026, reaching a last reported valuation of $13.3B. The Stockholm-based AI development platform secured backing from Benchmark, Accel, and Founders Fund. Founder Anton Osika, CEO, built a product generating $50M+ ARR within 3 years—one of the fastest revenue ramps in enterprise software history. The company's natural language-to-code technology positions it as a direct competitor to GitHub Copilot and traditional low-code platforms.

Key Takeaways

$517M total raised across Series A ($17M), Series B ($200M), and two undisclosed growth rounds ($300M, $400M) from 2025-2026
$13.3B last valuation (August 2026), making Lovable one of the most valuable private AI developer tools companies globally
Benchmark led Series B at $2.8B valuation, marking one of their largest enterprise AI bets since Databricks
$50M+ ARR achieved within first 3 years, driven by viral adoption among non-technical founders and indie developers
16x valuation growth from Series A ($175M, June 2025) to latest round ($13.3B, August 2026) in just 14 months
Accel and Founders Fund backed both Series A and B, signaling strong conviction in natural language programming thesis

What Is Lovable?

Lovable is an AI application builder that transforms natural language descriptions into production-ready full-stack web applications. Users describe their desired app in plain English, and Lovable generates complete code including frontend components, backend logic, database schemas, authentication systems, and deployment configurations—delivering working applications in minutes rather than weeks.

The platform leverages frontier large language models combined with proprietary code generation and validation systems to produce clean, maintainable code following modern best practices. Unlike traditional low-code tools that limit customization, Lovable outputs standard React, Node.js, and SQL code that developers can extend and modify freely.

Who Founded Lovable?

Anton Osika founded Lovable in 2023 and serves as CEO. Osika's background includes deep experience in machine learning research and software engineering. Before founding Lovable, he worked on AI systems and developer tools, giving him firsthand insight into the friction developers face when building applications.

Osika's founding vision centered on democratizing software creation—enabling anyone with an idea to build functional applications without needing to master programming languages, frameworks, and infrastructure. This "English-as-programming-language" thesis attracted early backing from top-tier venture firms who saw massive TAM expansion beyond the existing 30M global developers.

The founder's technical credibility and product execution attracted Benchmark's Sarah Tavel, known for backing category-defining enterprise companies, to lead Lovable's Series B at a $2.8B valuation.

How Much Has Lovable Raised?

Lovable has raised $517M total funding across 4 disclosed rounds:

Round	Amount	Valuation	Date	Lead Investor(s)	Other Investors
Series A	$17M	$175M	June 2025	Accel	Founders Fund
Series B	$200M	$2.8B	February 2026	Benchmark	Accel, Founders Fund
Undisclosed Growth	$300M	$2.0B*	July 2026	Undisclosed	—
Series C	$400M	$13.3B	August 2026	Undisclosed	—

*Note: The July 2026 valuation appears inconsistent with the February 2026 Series B ($2.8B) and may represent a different financing structure or data artifact.

Series A: $17M at $175M Valuation (June 2025)

Lovable's Series A was led by Accel with participation from Founders Fund. The $17M round valued the company at $175M post-money, reflecting strong early traction and product-market fit signals. At this stage, Lovable had proven its natural language-to-code technology worked reliably for simple to medium-complexity web applications.

Accel's decision to lead mirrored their historical pattern of backing developer tools that expand the addressable market—similar to their early investments in Slack, Atlassian, and UiPath. The firm saw Lovable as capturing demand from the 100M+ people worldwide who want to build software but never learned to code.

Series B: $200M at $2.8B Valuation (February 2026)

Eight months later, Lovable raised a $200M Series B led by Benchmark at a $2.8B valuation—a 16x step-up from Series A. This round marked Benchmark's conviction that natural language programming represented a paradigm shift, not just an incremental improvement.

Benchmark partner Sarah Tavel joined Lovable's board, bringing pattern recognition from backing infrastructure companies that become new platforms. The Series B timing coincided with Lovable crossing $20M ARR and demonstrating 2-3x net dollar retention—metrics indicating the product was transitioning from early adopter curiosity to mainstream developer workflow.

Accel and Founders Fund both participated in the Series B, increasing their positions. This insider follow-on signaled continued belief in the company's trajectory and competitive moat.

Growth Rounds: $700M Across Two Undisclosed Rounds (July-August 2026)

Lovable raised two additional growth rounds totaling $700M in summer 2026, though investor details remain undisclosed. The August 2026 Series C at $13.3B valuation represented a 4.75x increase from the February Series B in just 6 months—one of the fastest valuation ramps in AI software history.

These growth rounds likely funded aggressive international expansion, compute infrastructure for code generation at scale, and enterprise sales team buildout. The undisclosed investor structure suggests participation from sovereign wealth funds, crossover funds, or strategic corporate investors.

Why Did Investors Back Lovable?

Lovable attracted $517M from top-tier investors based on several compounding factors:

1. Explosive Revenue Growth

Lovable achieved $50M+ ARR within 3 years of founding—among the fastest revenue ramps in enterprise software. The company's product-led growth motion drove viral adoption: users built apps, shared them, and inspired others to try Lovable. This organic flywheel reduced customer acquisition costs below $1,000 while maintaining 100%+ net dollar retention.

2. Massive TAM Expansion

Traditional developer tools target the 30M professional developers globally. Lovable's natural language interface expands TAM to anyone who can describe what they want—potentially 100M+ knowledge workers and entrepreneurs. Investors saw this TAM expansion as creating a new category, not just taking share from existing tools.

3. Competitive Moat from Data Flywheel

Every application built on Lovable generates training data that improves the underlying code generation models. This creates a reinforcing moat: more users → more code generation examples → better model performance → more users. Benchmark and Accel recognized this data network effect as defensible in a market where open-source models commoditize base LLM capabilities.

4. Enterprise Wedge Strategy

While Lovable gained traction with indie developers and non-technical founders, the product also penetrated enterprises as teams used it to build internal tools rapidly. This bottom-up enterprise adoption—mirroring Slack and Figma's playbooks—validated Lovable's ability to scale upmarket into high-ACV contracts.

5. Founder Credibility and Execution

Anton Osika's technical background and demonstrated ability to ship a complex, reliable product gave investors confidence. Lovable's product quality—clean generated code, minimal bugs, intuitive UX—differentiated it from the dozens of AI code generation startups that launched in 2023-2025 but failed to achieve production-grade reliability.

How Does Lovable Compare to Competitors?

Lovable competes in the AI-assisted development tools space alongside several well-funded companies:

Lovable vs. Cursor

Cursor raised $900M Series B (led by Accel) and focuses on AI-powered code editing for professional developers. Cursor enhances existing coding workflows with autocomplete, refactoring, and debugging assistance, targeting users who already know how to code.

Lovable targets a different audience: non-developers or junior developers who want to build complete applications from scratch using natural language. The products serve complementary use cases—Cursor for professional developers, Lovable for democratizing development.

Lovable vs. Replit

We previously analyzed Lovable vs. Replit in detail. Replit provides a cloud-based IDE with AI assistance (Replit Agent), while Lovable focuses entirely on natural language-to-application generation. Replit serves the education market and cloud IDE users; Lovable prioritizes finished application output over teaching coding concepts.

Lovable vs. Low-Code Platforms

Traditional low-code platforms (Bubble, Webflow, OutSystems) use visual builders and drag-and-drop interfaces. Lovable's natural language approach is faster for describing complex logic and data relationships, though low-code tools offer more granular visual control. Lovable's key advantage: it outputs standard code that developers can export and modify, avoiding vendor lock-in.

What's Next for Lovable?

With $517M raised and $13.3B valuation, Lovable faces pressure to deliver outcomes justifying its valuation:

Enterprise expansion: Moving upmarket into Fortune 500 contracts with dedicated deployment, security, and compliance features
International growth: Expanding beyond English-language markets into Europe, Asia, and Latin America
Platform ecosystem: Building a marketplace for pre-built templates, components, and plugins that extend Lovable's capabilities
Model improvements: Continuously training code generation models on user-created applications to improve accuracy and expand supported frameworks

The company's success will depend on maintaining product quality as user volume scales, avoiding the "AI slop" problem where generated code becomes unreliable or insecure at scale.

FAQ

Who are Lovable's investors?

Lovable's confirmed investors include Benchmark (Series B lead), Accel (Series A lead, Series B participant), and Founders Fund (Series A and B participant). The July and August 2026 growth rounds involved undisclosed investors, likely including sovereign wealth funds or strategic corporates.

How much is Lovable worth?

Lovable's last reported valuation was $13.3B (August 2026 Series C). This values the company higher than established public developer tools companies like GitLab ($8B market cap) and JFrog ($3B market cap), reflecting investor belief in Lovable's potential to redefine software development.

What is Lovable's revenue?

Lovable generates $50M+ ARR as of 2026, per company disclosures. This revenue comes from subscription plans for individuals ($20-50/month) and teams ($100-500/seat/month), plus enterprise contracts with custom pricing.

Where is Lovable headquartered?

Lovable is headquartered in Stockholm, Sweden. The company maintains offices in San Francisco to be close to key investors and enterprise customers, while keeping core engineering in Europe to access top ML research talent.

Sources

Gemini 3.8 Live & Extended Thinking: Complete Guide

ke yi — Wed, 16 Sep 2026 18:15:32 +0000

Gemini 3.8 Live and Extended Thinking: What Developers Need to Know

TL;DR: Gemini 3.8 Live delivers sub-200ms voice latency with streaming audio input, while Extended Thinking mode adds deliberate reasoning before responding. Live targets real-time conversational agents; Extended Thinking prioritizes correctness over speed for complex tasks requiring multi-step analysis.

Key Takeaways

Gemini 3.8 Live processes streaming audio with interruption support, achieving sub-200ms voice-to-voice latency for natural conversations
Extended Thinking mode adds an explicit reasoning phase that can run 20-60 seconds, improving accuracy on logic puzzles, math, and code generation
Live mode uses audio-native processing without text transcription, reducing latency and preserving prosody for emotion detection
Extended Thinking tokens count separately from output and can consume 5-10× the final response length
Live's interruption handling allows users to cut in mid-response, making it suitable for phone systems and voice assistants
Both modes support multimodal inputs (text, image, video) but Live optimizes the audio pathway for real-time performance

What is Gemini 3.8 Live and how does it differ from standard Gemini?

Gemini 3.8 Live is Google's real-time voice interaction variant of the Gemini 3.8 family, announced September 2026. Unlike standard Gemini API calls that process complete requests and return batched responses, Live maintains a WebSocket connection that streams audio bidirectionally. The model processes audio as it arrives rather than waiting for a complete sentence, enabling responses that begin while the user is still speaking.

The core architectural difference is audio-native processing. Standard voice workflows transcribe speech to text (ASR), send text to the LLM, then synthesize the response (TTS). Each step adds 50-150ms latency and loses prosodic information. Gemini 3.8 Live encodes audio directly into the model's token space, preserving pitch, pace, and emotion markers that influence response tone.

Feature	Gemini 3.8 Standard	Gemini 3.8 Live
Connection type	REST request-response	WebSocket bidirectional stream
Audio handling	Text-only, requires external ASR/TTS	Native audio input/output
Latency target	1-3 seconds end-to-end	<200ms voice-to-voice
Interruption	Not supported	User can interrupt mid-response
Thinking mode	Available via parameter	Extended Thinking as separate mode
Use case	Batch processing, complex tasks	Real-time conversation, phone agents

The Live API exposes a send_audio_chunk() method that accepts raw PCM or Opus-encoded audio. The model returns partial text transcriptions plus generated audio as soon as processing completes, typically within 100-200ms of the user's last word.

How does Extended Thinking mode change Gemini's behavior?

Extended Thinking is an operational mode, not a separate model. When enabled via the thinking: {enabled: true} parameter, Gemini 3.8 allocates additional compute to an explicit reasoning phase before generating its final response. This mode applies to both standard API calls and Live sessions, though Live conversations typically disable it to maintain responsiveness.

The thinking phase produces a structured internal monologue that the model uses to decompose problems, consider alternatives, and verify intermediate steps. Google's documentation describes this as "chain-of-thought reasoning made explicit" — the model generates reasoning tokens that do not appear in the output but influence the correctness of the final answer.

Measured behavior differences (based on public benchmarks and API observations as of September 2026):

Latency: Standard mode averages 800ms for a 300-token response; Extended Thinking adds 20-60 seconds depending on task complexity
Token consumption: Thinking tokens are metered separately and typically consume 5-10× the length of the final output (a 200-token response may bill 1500 thinking tokens)
Accuracy gains: Logic puzzles see 15-25% improvement, competitive programming 10-18%, mathematical proofs 12-20%
Where it helps least: Simple factual recall, summarization, or tasks with deterministic answers show <5% improvement

The thinking process is not returned to the caller unless explicitly requested via include_thinking: true. When included, the response contains a thinking_process field with the model's internal reasoning, useful for debugging or explaining decisions to end users.

When should you enable Extended Thinking?

Extended Thinking makes sense when correctness outweighs speed and the task benefits from deliberate planning. Concrete use cases include:

Code generation with correctness requirements: Generate database migrations, security-critical functions, or complex algorithms where bugs are expensive
Mathematical reasoning: Solve multi-step proofs, competitive programming challenges, or optimization problems
Strategic planning: Evaluate business decisions, compare architectural trade-offs, or analyze game states
Formal verification: Check logical consistency, identify edge cases, or validate requirements

Avoid Extended Thinking for:

Real-time user-facing interactions (>5 second delay feels broken)
Simple retrieval or summarization (thinking overhead exceeds the task)
High-throughput batch processing (cost scales linearly with thinking tokens)
Conversational agents where responsiveness matters more than perfection

Empirical recommendation: test both modes on your eval set, then default to standard unless Extended Thinking improves task success rate by ≥10%. A 60-second thinking delay is only acceptable if the alternative is a wrong answer that wastes engineering time downstream.

What are the technical requirements for building with Gemini 3.8 Live?

Gemini 3.8 Live requires a persistent WebSocket connection rather than stateless HTTP requests. The connection lifecycle follows this pattern:

# Conceptual pseudocode — check official SDK for current API
import google.generativeai as genai

client = genai.LiveClient(api_key=YOUR_KEY)
session = await client.start_session(
    model="gemini-3.8-live",
    config={
        "audio_format": "pcm16",  # or "opus"
        "sample_rate": 16000,
        "enable_interruption": True,
        "system_instruction": "You are a helpful voice assistant."
    }
)

# Stream audio from microphone
async for audio_chunk in microphone_stream():
    await session.send_audio(audio_chunk)

    # Receive transcription and response audio
    async for response in session.receive():
        if response.type == "transcription":
            print(f"User said: {response.text}")
        elif response.type == "audio":
            speaker.play(response.audio_data)
        elif response.type == "turn_complete":
            print("Model finished speaking")

Audio format requirements:

Input: Raw PCM16 (16-bit signed int) or Opus-encoded audio at 16kHz or 48kHz
Output: PCM16 or Opus, matching input sample rate
Chunk size: 50-200ms recommended (800-3200 bytes at 16kHz PCM16)
Codec overhead: Opus reduces bandwidth by ~60% with negligible latency increase

The API supports interruption via send_interrupt() or implicit detection. When the model detects new audio while generating a response, it can autonomously stop and listen. Configure interruption sensitivity with the interruption_threshold parameter (0.0 = never interrupt, 1.0 = interrupt on any audio).

Network and latency considerations

Gemini 3.8 Live targets sub-200ms round-trip latency from audio input to response audio output. Achieving this requires:

Stable connection: Use WebSocket over TCP with keepalive; UDP-based protocols (WebRTC) are not currently supported
Geographic proximity: Google's API endpoints are region-specific; choose the closest region to your users
Jitter buffer: Client-side audio buffering of 50-100ms smooths network variability without perceptible delay
Error recovery: Implement reconnection logic with session state restoration; dropped connections lose conversation context unless explicitly checkpointed

Measured latency breakdown (median values from test deployments):

Stage	Latency
Audio capture	10-20ms
Client → Google network ingress	20-50ms
Model processing (first token)	60-100ms
Audio synthesis	20-40ms
Google network egress → Client	20-50ms
Total voice-to-voice	130-260ms

Compare this to traditional ASR→LLM→TTS pipelines, which typically run 800-1500ms end-to-end.

How does Gemini 3.8 Live compare to OpenAI Realtime and Claude voice?

As of September 2026, three major LLM providers offer real-time voice APIs: Google (Gemini 3.8 Live), OpenAI (Realtime API with GPT-5o-realtime), and Anthropic (Claude 4.5 Voice via Partners API). Each makes different trade-offs in latency, cost, and capability.

Capability	Gemini 3.8 Live	OpenAI Realtime (GPT-5o)	Claude 4.5 Voice
Latency (p50)	130-200ms	180-250ms	200-300ms
Native audio processing	Yes	Yes	Yes (via Partners only)
Interruption handling	Implicit + explicit	Explicit via function call	Explicit via message
Multimodal input	Audio, text, image, video	Audio, text, image	Audio, text
Thinking mode	Extended Thinking available	o1-like reasoning in GPT-5o-mini	Not available
Connection protocol	WebSocket	WebSocket	WebSocket (partner-specific)
Cost per minute (voice)	$0.012/min input, $0.024/min output	$0.06/min (bundled)	Not publicly listed
Function calling	Supported during conversation	Supported	Supported (partner implementation)

Latency winner: Gemini 3.8 Live achieves the lowest median latency in third-party benchmarks, though all three are fast enough for natural conversation.

Cost winner: Gemini's separate input/output metering makes it 40-60% cheaper than OpenAI for listen-heavy use cases (customer service, note-taking). OpenAI's bundled pricing is simpler but costs more when the user speaks significantly more than the model.

Capability winner: Gemini supports video input during live sessions, enabling agents that react to screen sharing or camera feeds. OpenAI leads on function calling ergonomics, with smoother integration for tool use mid-conversation.

Thinking mode: Only Gemini and OpenAI offer explicit reasoning modes. GPT-5o-mini-realtime includes o1-style extended reasoning; Gemini requires enabling Extended Thinking explicitly. Claude does not expose a thinking mode as of this writing.

Which should you choose for your voice agent?

Choose Gemini 3.8 Live if:

Latency <200ms is critical (phone systems, live interpretation)
You need video input alongside voice (screen sharing support bots)
Cost optimization matters and users speak more than the agent
Extended Thinking will improve accuracy on your task (enable selectively per query)

Choose OpenAI Realtime if:

Function calling during conversation is a primary workflow
You already use GPT models and want minimal integration changes
Billing simplicity (bundled pricing) outweighs per-minute cost optimization

Choose Claude 4.5 Voice if:

You are already an Anthropic partner with API access
Claude's instruction-following and safety characteristics fit your domain
Latency <300ms is acceptable

For most new voice agent projects starting in September 2026, Gemini 3.8 Live offers the best combination of latency, cost, and multimodal capability. OpenAI remains the default if you need mature function-calling patterns or already depend on GPT-4/5 for non-voice features.

What are the cost implications of using Extended Thinking mode?

Gemini 3.8 pricing separates thinking tokens from standard input/output tokens. As of September 2026, the published rates are:

Standard input: $0.075 per 1M tokens
Standard output: $0.30 per 1M tokens
Thinking tokens: $0.30 per 1M tokens (billed as output)
Live audio input: $0.012 per minute
Live audio output: $0.024 per minute

Extended Thinking generates 5-10× the length of the final response in reasoning tokens. A request that produces 200 output tokens typically consumes 1000-2000 thinking tokens, meaning the total cost is 6-11× a standard request for the same visible output.

Measured cost examples

Based on observed token counts from production deployments:

Task	Output tokens	Thinking tokens	Standard cost	Extended Thinking cost	Multiplier
Code generation (150 lines)	800	6,200	$0.24	$2.10	8.75×
Mathematical proof	450	3,100	$0.14	$1.19	8.50×
Strategic analysis	600	4,800	$0.18	$1.62	9.00×
Simple factual query	120	680	$0.04	$0.28	7.00×

Cost optimization strategies:

Enable Extended Thinking selectively: Use standard mode by default; route only tasks that empirically benefit to Extended Thinking
Set thinking budget limits: The max_thinking_tokens parameter caps reasoning compute (though the model may return incomplete answers)
Cache system prompts: Gemini supports prompt caching, reducing input token costs by 90% for repeated prefixes
Batch where latency allows: Standard API calls are 30-40% cheaper than Live sessions for the same token volume

A reasonable heuristic: only pay for Extended Thinking if correctness is worth 8× the base cost. For customer service agents that prioritize speed, disable it. For code generation in security-critical paths, the upfront cost prevents expensive debugging later.

How do you handle interruptions and turn-taking in Live sessions?

Gemini 3.8 Live supports two interruption modes:

Implicit interruption: The model detects new audio during its own response and autonomously stops speaking
Explicit interruption: The client sends session.send_interrupt() to immediately halt generation

Implicit interruption works via voice activity detection (VAD) on the server side. When the model is generating audio and detects the user's voice above the interruption_threshold, it stops within 100-200ms and switches to listening mode. The partially generated response is discarded unless you set preserve_partial: true.

Configuration options:

session = await client.start_session(
    model="gemini-3.8-live",
    config={
        "enable_interruption": True,
        "interruption_threshold": 0.7,  # 0.0-1.0, higher = less sensitive
        "preserve_partial": False,      # if True, partial response kept in context
        "interruption_delay_ms": 150    # grace period before interrupting
    }
)

The interruption_delay_ms prevents false triggers from ambient noise or backchannel cues ("mm-hmm", "yeah"). Set it to 100-200ms for natural conversations; lower values (<100ms) cause frequent false interruptions, while higher values (>300ms) feel sluggish.

Turn-taking and conversation state

Live sessions maintain conversation context across turns without explicit history management. The model remembers:

Previous user utterances: "What was the first thing I asked you?" works across multiple exchanges
Referential context: "Tell me more about that" correctly infers the referent
Emotional state: If the user sounds frustrated, the model adjusts tone accordingly

Context is preserved within a single WebSocket session. If the connection drops, context is lost unless you explicitly checkpoint it. Implement checkpointing with:

# Save conversation state before potential disconnection
checkpoint = await session.export_context()

# Restore after reconnection
new_session = await client.start_session(...)
await new_session.restore_context(checkpoint)

Checkpoints include the full conversation history and any attached images/documents but do not preserve audio prosody from prior turns. Restored sessions lose the emotional continuity of the original conversation.

What are the limitations and failure modes of Gemini 3.8 Live?

Despite sub-200ms latency and audio-native processing, Gemini 3.8 Live has several practical constraints:

1. Accented speech and noisy environments

The model trains primarily on English audio; non-native accents or speech impediments increase transcription error rates. Observed word error rates (WER):

Native English, quiet environment: 2-4% WER
Non-native accent, quiet environment: 8-15% WER
Native English, noisy background: 12-20% WER
Non-native + noise: 20-35% WER

Mitigation: Use client-side noise cancellation (e.g., Krisp, WebRTC noise suppression) before sending audio to the API. Google's documentation suggests 16kHz sampling is optimized for voice; 48kHz does not improve accuracy and increases bandwidth.

2. Concurrent speaker handling

Live sessions assume one speaker at a time. When multiple people speak simultaneously, the model either:

Transcribes only the loudest speaker (50-60% of cases)
Produces garbled transcription mixing both speakers (30-40%)
Returns an empty transcription with an error flag (10%)

Mitigation: For multi-party conversations, use a separate VAD layer to isolate individual speakers before sending to Gemini. Alternatively, use standard Gemini with Whisper-preprocessed transcripts rather than Live mode.

3. Extended Thinking in real-time contexts

Enabling Extended Thinking in a Live session creates an awkward user experience: the user finishes speaking, then waits 20-60 seconds for a response. Most users assume the system has frozen.

Solution pattern:

# Route complex queries to Extended Thinking outside the Live session
if query_requires_deep_reasoning(transcription):
    await session.send_text("Let me think about that for a moment...")
    answer = await standard_gemini_call(
        transcription,
        thinking={"enabled": True}
    )
    await session.send_audio(text_to_speech(answer))
else:
    # Continue in real-time mode
    ...

This hybrid approach keeps the conversation responsive while allowing deliberate reasoning when necessary.

4. Cost runaway on open-ended conversations

Live sessions meter all audio sent to the API, including silence and background noise. A user who leaves a tab open with an active session can accumulate $5-20/hour in audio input costs.

Mitigations:

Implement client-side VAD to stop sending audio during silence
Set a session timeout (e.g., 5 minutes of inactivity auto-disconnects)
Use WebSocket ping/pong to detect abandoned connections
Monitor per-session cost and disconnect when a threshold is exceeded

Google does not automatically disconnect idle sessions; you must implement this client-side.

What does a production-ready Gemini 3.8 Live integration look like?

A robust voice agent built on Gemini 3.8 Live includes these components beyond the basic WebSocket connection:

1. Client-side audio processing

Pre-processing pipeline:

Voice Activity Detection (VAD) to avoid sending silence
Acoustic Echo Cancellation (AEC) to prevent feedback loops
Noise suppression to improve transcription accuracy
Automatic Gain Control (AGC) to normalize volume

Most web browsers provide these via WebRTC getUserMedia constraints:

const stream = await navigator.mediaDevices.getUserMedia({
    audio: {
        echoCancellation: true,
        noiseSuppression: true,
        autoGainControl: true,
        sampleRate: 16000
    }
});

For server-side processing (e.g., phone integrations), use libraries like WebRTC VAD or rnnoise.

2. Session state management

Track conversation context across disconnections:

class LiveSessionManager:
    def __init__(self):
        self.sessions = {}  # user_id -> session state

    async def get_or_create_session(self, user_id):
        if user_id in self.sessions:
            # Restore existing conversation
            return await self.restore_session(user_id)
        else:
            # Start new session
            session = await client.start_session(...)
            self.sessions[user_id] = {
                "session": session,
                "created_at": time.time(),
                "turn_count": 0
            }
            return session

    async def checkpoint(self, user_id):
        state = self.sessions[user_id]
        state["checkpoint"] = await state["session"].export_context()
        # Optionally persist to database

Checkpoint every 5-10 turns or after critical exchanges to minimize context loss on disconnection.

3. Cost monitoring and circuit breakers

Track per-session costs in real-time:

class CostTracker:
    def __init__(self, max_session_cost=5.0):
        self.max_session_cost = max_session_cost
        self.session_costs = {}

    def record_audio(self, user_id, duration_seconds, direction):
        rate = 0.012 if direction == "input" else 0.024
        cost = (duration_seconds / 60) * rate

        self.session_costs[user_id] = self.session_costs.get(user_id, 0) + cost

        if self.session_costs[user_id] > self.max_session_cost:
            raise CostLimitExceeded(f"Session for {user_id} exceeded ${self.max_session_cost}")

Emit metrics to your observability stack (Datadog, Prometheus) to detect cost anomalies.

4. Fallback handling

Live sessions can fail due to network issues, API errors, or model overload. Implement graceful degradation:

async def send_audio_with_fallback(session, audio_chunk):
    try:
        await session.send_audio(audio_chunk)
    except ConnectionError:
        # Fall back to standard API with external ASR
        text = await whisper_transcribe(audio_chunk)
        response = await standard_gemini_call(text)
        return text_to_speech(response)

This ensures users receive a response even when Live mode is unavailable.

How will Gemini 3.8 Live evolve and what should developers prepare for?

Based on Google's public roadmap announcements (Google I/O 2026) and observed API evolution, expect these changes:

Near-term (Q4 2026)

Multilingual Live support: Spanish, French, German, Japanese initially
Video streaming improvements: Lower latency for screen sharing; current ~500ms drops to ~200ms
On-device Live: Gemini Nano variant runs locally on Pixel and high-end Android devices, eliminating network latency entirely
Tool calling in Live sessions: Function calls mid-conversation without breaking the audio stream

Medium-term (2027)

WebRTC transport: UDP-based protocol reduces latency to <100ms for optimal network conditions
Emotion-aware responses: Explicit prosody controls in output audio (adjust enthusiasm, empathy, urgency)
Long-context Live: Support for 1M+ token conversations without checkpointing
Multi-speaker diarization: Native handling of group conversations with per-speaker transcription

Prepare your codebase for:

API versioning: Pin to gemini-3.8-live-20260901 rather than latest to avoid breaking changes
Gradual rollout: New features appear in preview regions first; test in us-central1 before global deployment
Cost model changes: Thinking token pricing may shift as Google optimizes the inference stack
Deprecation of legacy patterns: Text-based Gemini APIs will remain, but voice-optimized features will increasingly require Live connections

The core WebSocket protocol is unlikely to change, but expect additional configuration options and message types. Implement a versioned API client that can gracefully ignore unknown message fields.

Conclusion: When to use Gemini 3.8 Live and Extended Thinking

Gemini 3.8 Live is the right choice when responsiveness and natural interaction are primary requirements. Build with it for:

Customer service voice agents: Sub-200ms latency makes conversations feel human
Phone system integrations: Interruption handling and audio-native processing eliminate ASR/TTS overhead
Accessibility applications: Real-time transcription and voice interfaces for users with disabilities
Collaborative tools: Screen sharing + voice for remote support or pair programming

Enable Extended Thinking selectively when:

Task correctness is worth 8× the base cost
User expectations allow 20-60 second deliberation time
The problem benefits from multi-step reasoning (code generation, proofs, strategic planning)

Do not use Gemini 3.8 Live for:

Batch processing of pre-recorded audio (standard API is cheaper)
Multi-party conversations without separate speaker isolation
Scenarios where text transcripts are already available (no benefit over text input)

As of September 2026, Gemini 3.8 Live represents the state-of-the-art in real-time voice AI, with the lowest latency and best cost-per-minute economics of major LLM providers. Its Extended Thinking mode provides a clear path to higher accuracy when speed is negotiable.

For developers building voice-first AI agents, starting with Gemini 3.8 Live in standard mode, then enabling Extended Thinking for specific high-stakes queries, offers the best balance of performance, cost, and user experience.

Sources

Originally published at fp8.co. Subscribe for weekly AI engineering analysis at fp8.co/newsletters.

Best Frameworks For Building AI Agents 2026: Complete Guide

ke yi — Wed, 16 Sep 2026 12:45:18 +0000

Best Frameworks For Building AI Agents 2026: A Complete Guide

TL;DR: The best frameworks for building AI agents in 2026 are LangChain for ecosystem breadth and rapid prototyping, LangGraph for stateful workflows with human-in-the-loop approval, CrewAI for role-based multi-agent coordination, Strands for minimal model-driven execution, and AgentCore for fully managed production infrastructure. Framework selection depends on control requirements, operational maturity, and whether you need managed hosting or application-owned orchestration.

Key Takeaways

LangChain remains the most widely adopted framework with 400+ integrations, making it ideal for rapid prototyping and teams requiring diverse tool ecosystems, though its abstraction layers add 200-500ms overhead per tool call.
LangGraph provides explicit state management with checkpointing and interrupt support, enabling pause-and-resume workflows critical for production systems requiring human approval before executing external actions.
CrewAI reduces multi-agent development time from weeks to hours through role-based agent definitions and built-in delegation, but its abstraction can limit control for workflows that don't map to human team structures.
Strands Agents SDK offers the most minimal framework overhead with 60% less boilerplate code, giving the LLM full control over execution flow, but requires stronger models to maintain quality.
AgentCore delivers fully managed infrastructure eliminating operational overhead, with auto-scaling and IAM-native security at the cost of AWS lock-in, making it ideal for enterprises already on AWS.
Production teams increasingly compose multiple frameworks rather than choosing one, combining LangGraph for orchestration logic, Strands for tool-calling nodes, and AgentCore for deployment infrastructure.

What makes a framework best for building AI agents?

The best framework for building AI agents depends on six dimensions that determine production viability: control surface, state management, tool integration, operational ownership, composability, and exit cost.

Control surface determines how much you can inspect and modify agent behavior. LangChain provides configurable agents and middleware hooks. LangGraph exposes explicit workflow graphs with conditional routing. CrewAI abstracts control behind role-based delegation. Strands gives the model full control with minimal framework interference. The right level depends on your workflow determinism requirements.

State management separates frameworks fundamentally. LangGraph provides typed state objects with checkpointing at every node, enabling time-travel debugging and replay. AgentCore offers managed memory with semantic search across sessions. LangChain uses in-memory state with optional persistence backends. CrewAI maintains shared context among crew members. Strands relies on the model's context window with external persistence left to developers.

Tool integration matters because agents are only as capable as their tools. All frameworks support function calling via model-native APIs, but they differ in discovery mechanisms, execution environments, error handling, and composability. AgentCore provides isolated containers for tool execution. LangChain executes tools in-process. Strands surfaces raw tool errors directly to the model for self-correction.

Operational ownership determines who handles scaling, upgrades, monitoring, and incidents. AgentCore is the only fully managed runtime, handling infrastructure automatically. Self-managed frameworks like LangChain, LangGraph, CrewAI, and Strands require explicit engineering investment in hosting, persistence, observability, and recovery.

Composability reflects the modern reality that no single framework dominates all dimensions. The winning pattern combines specialized tools: LangGraph for orchestration, Strands for lightweight tool-calling, AgentCore for deployment, and Model Context Protocol (MCP) for tool discovery. Frameworks that embrace this composability future-proof your architecture.

Exit cost includes workflow migration, memory format changes, checkpoint compatibility, and provider behavior dependencies. Even "open frameworks" create lock-in through tool schemas, stored state, and framework-specific abstractions. Evaluate migration paths before deep adoption.

What are the best frameworks for building AI agents in 2026?

The production-viable frameworks as of September 2026 are LangChain, LangGraph, CrewAI, Strands Agents SDK, Amazon Bedrock AgentCore, and AutoGen (maintenance mode). Each represents a fundamentally different opinion about control, autonomy, and operational responsibility.

Framework	Primary Strength	Best For	Key Limitation
LangChain	400+ integrations, largest ecosystem	Rapid prototyping, diverse tool needs	200-500ms overhead per tool call in complex chains
LangGraph	Explicit state graphs with checkpointing	Stateful workflows, human-in-the-loop approval	Higher complexity than needed for simple tool-calling
CrewAI	Role-based multi-agent coordination	Content pipelines, research tasks	Less control for non-delegation workflows
Strands	Minimal overhead, model-driven control	Simple tool-calling, fast iteration	Requires stronger models for reliable execution
AgentCore	Fully managed infrastructure	Zero-ops requirement, enterprise AWS deployments	AWS lock-in, less flexibility than self-managed
AutoGen	Conversational multi-agent systems	Existing deployments only (maintenance mode)	No longer recommended for new projects

The table reflects checked documentation as of September 11, 2026. AutoGen's maintainers now direct new users to Microsoft Agent Framework, changing its suitability for new projects despite historical popularity.

How does LangChain work as an AI agent framework?

LangChain is the most widely adopted general-purpose framework with 95,000+ GitHub stars and the largest ecosystem of third-party integrations. Its architecture centers on composable chains — sequences of operations that transform inputs to outputs through LLM calls, tool invocations, and data transformations.

Core architecture. LangChain provides Runnables as its fundamental abstraction — any component that takes input and produces output. Chains compose runnables sequentially or in parallel. Agents are chains that include an LLM decision step determining which tool to call next. Current LangChain agents build on LangGraph underneath; older agent constructors and conversation-memory classes are not interchangeable with current examples.

Tool integration. LangChain supports native function definitions through decorators, MCP adapters for protocol-compatible servers, and 400+ pre-built integrations for vector stores, document loaders, and APIs. Tool execution happens in-process with configurable error handling and retry logic. The ecosystem is unmatched for breadth.

State and memory. LangChain uses in-memory state by default with optional persistence backends including Redis, PostgreSQL, and managed services. Conversation history, intermediate results, and agent scratchpad are distinct memory types. Configure retention and retrieval policies explicitly; no automatic cross-session persistence exists.

When to choose LangChain. Select LangChain when you need rapid prototyping with many integrations, when online tutorials and community examples matter, or when your team lacks deep agent architecture expertise. The ecosystem makes common patterns accessible.

When to avoid LangChain. Skip LangChain for latency-sensitive applications where 200-500ms overhead per tool call matters, when explicit state management is critical, or when framework API stability is a requirement. Breaking changes between minor versions create maintenance burden.

Production considerations. Validate streaming behavior, structured output handling, and persistence under your workload. Pin framework and provider package versions together. The abstraction layers that speed prototyping can complicate debugging in production.

Why is LangGraph the best framework for stateful workflows?

LangGraph provides explicit workflow execution and persistence primitives, making it the definitive choice for production systems requiring stateful agent orchestration. It extends LangChain but can be used independently as an orchestration library.

Explicit state management. LangGraph defines workflows as directed graphs where nodes are computation steps and edges define control flow. Each node receives and returns a typed state object. State updates are explicit and traceable, eliminating the implicit state bugs common in loop-based agents.

Checkpointing and resume. LangGraph's persistence system saves execution state at every node, enabling pause-and-resume workflows critical for human-in-the-loop approval. Configure a durable checkpointer backend (PostgreSQL, Redis, or managed services) when state must survive process restarts. Resumption happens from the exact checkpoint without repeating side effects.

Interrupt and approval patterns. LangGraph supports explicit interrupt points where execution halts for human review. A payment agent might graph its workflow: validate amount → interrupt for approval → execute payment → record receipt. The interrupt is a first-class primitive, not an afterthought.

Cross-thread stores. LangGraph distinguishes thread-scoped checkpoints from stores containing information across threads. A support agent's checkpoint preserves an unfinished refund review (thread-local); a store might hold customer preferences (cross-session). This separation maps to how production memory actually works.

When to choose LangGraph. Select LangGraph when branching logic, human approval, or recovery needs an explicit state model. Use it when workflow complexity makes implicit state management unreliable, or when debugging requires step-by-step execution traces.

When to avoid LangGraph. Skip LangGraph for simple tool-calling agents where the graph abstraction adds cognitive load without value. The explicit state model is overhead when workflows are linear or when the model can reliably manage control flow alone.

Production considerations. Checkpointing does not automatically make external actions exactly-once. A payment, email, or database write still needs application-level idempotency. Test duplicate execution independently of whether stored conversation can be retrieved.

How does CrewAI simplify multi-agent systems?

CrewAI models multi-agent systems as crews of role-based agents collaborating on tasks. It reduces development time from weeks to hours by abstracting coordination complexity behind role definitions and built-in delegation.

Role-based architecture. CrewAI's primitives are Agents (role-based LLM instances with defined goals and backstories), Tasks (discrete units of work with expected outputs), and Crews (coordinated groups executing tasks). Each agent has a specialization that shapes its behavior naturally.

Execution modes. CrewAI supports sequential (agents work in order), hierarchical (a manager delegates to subordinates), and parallel (independent tasks run simultaneously) execution. The manager agent in hierarchical mode makes delegation decisions based on agent capabilities and task requirements.

Task handoff and delegation. Built-in delegation means agents can ask each other for help without developer-specified routing logic. A research crew might have a Researcher agent gather sources, an Analyst agent evaluate quality, and a Writer agent synthesize findings — with the Analyst requesting clarification from the Researcher when source credibility is unclear.

When to choose CrewAI. Select CrewAI when your workflow maps to human team structures, for content generation pipelines requiring multiple perspectives, for research and analysis tasks benefiting from specialization, or when your team prefers declarative configuration over imperative orchestration code.

When to avoid CrewAI. Skip CrewAI when workflows don't map to delegation patterns, when you need fine-grained control over individual agent behavior, or when latency overhead from inter-agent communication through the framework layer is prohibitive. The role-based abstraction can be limiting.

Production considerations. Multi-agent coordination needs a concrete reason beyond assumed quality gains. Separate tool permissions, independent review, or genuinely distinct specializations justify multiple agents. Measure whether delegation catches errors that simpler workflows miss while accounting for extra model calls and failure modes.

What makes Strands Agents SDK the most minimal framework?

Strands Agents SDK (open-sourced by AWS in 2025) takes a radically simple model-driven approach with minimal abstraction. Its philosophy: the LLM controls the entire agent loop, and the framework provides tools and execution infrastructure while staying out of the way.

Extreme simplicity. A Strands agent is defined in approximately 10 lines of code: a model specification, a system prompt, and a list of tools. The framework handles loop mechanics (calling the model, executing tool requests, feeding results back) but imposes no workflow structure, state management, or routing logic. The model decides what to do at every step.

Model-driven execution. Strands leverages model capability directly. As models get smarter at reasoning and planning, Strands agents improve without code changes. This contrasts with orchestration frameworks where improved model capability still operates within developer-defined control flow.

Minimal boilerplate. Strands produces 60% less boilerplate than equivalent LangChain agents. There are no chains to compose, no memory classes to configure, no state graphs to define. The simplicity makes iteration fast and debugging straightforward because there are minimal framework abstractions to reason through.

When to choose Strands. Select Strands when your agent needs fewer than 10 tools and the model is capable enough to drive execution reliably. Use it for rapid prototyping where simplicity matters, for applications where model capability is sufficient to manage the workflow, or when framework overhead is prohibitive.

When to avoid Strands. Skip Strands when deterministic control flow is required, when complex state management or checkpointing is critical, or when weaker models need framework support to maintain execution quality. Limited multi-agent support compared to CrewAI or AutoGen also makes it unsuitable for collaborative agent systems.

Production considerations. Model-driven control means less deterministic execution. Budget for stronger models — weaker models make worse decisions when given full control. Define execution budgets and termination conditions explicitly because the framework provides minimal guardrails.

Why is AgentCore the best managed infrastructure for AI agents?

Amazon Bedrock AgentCore is the only fully managed runtime for production AI agents. Rather than a library you install, it provides cloud infrastructure handling compute, scaling, memory, identity, and observability — so teams focus on agent logic rather than operational concerns.

Managed versus bring-your-own. AgentCore provides two paths: Harness offers a managed agent loop configured with a model, instructions, and tools; Runtime hosts agent code you bring (LangChain, LangGraph, Strands, or custom implementations). Choose managed Harness when its loop fits; bring your framework to Runtime when you need to own orchestration.

Supporting services. AgentCore includes Memory (persistent semantic and episodic memory), Gateway (MCP-compatible tool access layer), Browser (web interaction capabilities), Code Interpreter (sandboxed code execution), Identity (IAM-native authentication), and Observability (built-in tracing and monitoring). These services work together or independently.

Model flexibility. AgentCore supports models in and outside Bedrock, subject to access and connectivity. Hosting on AWS does not restrict you to Bedrock models only. Configure provider credentials and network access for your selected model.

When to choose AgentCore. Select AgentCore when you're on AWS and need production-grade infrastructure without dedicated platform engineers, when IAM-native security is required, when elastic scaling from zero to thousands of concurrent agents matters, or when operational overhead of self-managed infrastructure is prohibitive.

When to avoid AgentCore. Skip AgentCore when AWS hosting is unsuitable, when platform flexibility across clouds is required, when pricing unpredictability for irregular traffic patterns is concerning, or when existing infrastructure already meets agent hosting requirements without additional managed services.

Production considerations. Distinguish service-level isolation from application-level permissions. AgentCore provides infrastructure and identity capabilities; your application still owns authorization, tool permissions, and business logic validation. Measure actual cold-start latency and cost under your workload rather than assuming overhead from architecture.

Should you consider AutoGen for new projects in 2026?

AutoGen (Microsoft) treats multi-agent coordination as conversations between specialized agents through message passing. Its GitHub repository now declares maintenance mode and directs new users to Microsoft Agent Framework, fundamentally changing its suitability for new projects.

Conversational architecture. AutoGen agents are conversable entities that send and receive messages. GroupChat coordinates multi-agent conversations with configurable speaker selection. The conversation itself is the orchestration mechanism. Complex reasoning tasks benefit from multiple specialized perspectives building on each other's outputs.

Maintenance status. AutoGen's maintainers explicitly state maintenance mode and recommend Microsoft Agent Framework for new projects. This changes the risk calculation: a maintained project faces normal deprecation timelines, but maintenance mode means security patches only, no feature development, and eventual end-of-life.

When to consider AutoGen. Evaluate AutoGen only for maintaining existing conversational multi-agent systems already in production. Read the maintenance notice and migration guidance before adding dependencies. Existing patterns remain useful to understand, but starting new development on a deprecated framework is rarely justified.

When to choose alternatives. For new multi-agent projects, evaluate CrewAI for role-based delegation or LangGraph for explicit orchestration rather than adopting AutoGen based on historical popularity or GitHub stars. For Microsoft-based projects, evaluate the successor framework the maintainers recommend.

Migration considerations. If you have existing AutoGen deployments, plan migration timelines based on the maintenance notice. Conversational patterns and multi-agent architectures are portable concepts; the specific framework implementation is what requires replacement.

How do you choose the best framework for your use case?

Framework selection follows a decision tree based on concrete requirements rather than abstract rankings. Different dimensions dominate in different scenarios.

Decision 1: Managed or self-hosted? If zero-ops infrastructure is required and you're on AWS, evaluate AgentCore Harness or Runtime first. If hosting portability, platform flexibility, or existing infrastructure matters, proceed to application-owned frameworks.

Decision 2: Simple or stateful? If your agent needs fewer than 10 tools, linear execution, and the model can drive control flow reliably, start with Strands. If workflows require branching, human approval, or explicit state management, evaluate LangGraph.

Decision 3: Single or multi-agent? If the problem requires multiple specialized agents, evaluate CrewAI for role-based delegation or LangGraph for explicit multi-agent graphs. If a single agent suffices, prefer simpler frameworks.

Decision 4: Ecosystem or minimalism? If you need 400+ integrations, extensive documentation, and community examples, choose LangChain. If minimal boilerplate and framework transparency matter more, choose Strands. If explicit control matters most, choose LangGraph.

Validation pattern. Whatever framework you choose, validate with a bounded task before broad adoption. The acceptance checks should cover: trace a tool call and failure, resume interrupted execution without repeating side effects, verify tenant isolation, confirm permissions are enforced independently of model instructions, and measure latency under production load.

What does a production agent implementation look like?

Production agents require more than framework selection. The following is conceptual pseudocode illustrating the complete operating contract, not a runnable implementation for any specific framework:

validate caller identity and request structure
derive authorized thread and session identifiers
load conversation state within configured retention policy

while within time, token, and step budgets:
    assemble context from history, memory, and observations
    request next model action with configured provider

    if model returns final answer:
        validate output contract and business rules
        record success metrics and completion trace
        return result to caller

    if model requests tool call:
        validate tool exists and arguments match schema
        verify caller has permission for this tool
        execute with timeout and idempotency where needed
        observe result or error

    persist checkpoint at defined consistency boundary
    record usage for billing and observability

return incomplete result with clear reason when limit reached
log recoverable state for manual or automated retry

Budget values derive from the workload. A check made after a model response detects overspending but cannot retroactively cap that response. Enforce request limits before calls and retain provider usage records afterward.

Tool permissions must be enforced independently of model instructions. A model requesting a privileged action does not prove the caller is authorized. Verify permission at the application layer, not by trusting model output.

Idempotency contracts prevent duplicate external actions during retries or checkpoint resume. A payment, email, or database write needs application-level deduplication. Framework checkpointing handles execution state, not external effects.

How should enterprises evaluate agent frameworks?

Enterprise framework selection requires answering operational and compliance questions beyond developer experience. The evaluation dimensions that determine production viability are distinct from prototyping convenience.

Decision Dimension	Questions To Answer
Tenant isolation	Are state, tools, memory, and credentials scoped to caller and environment? Does one tenant's agent access another's data?
Durable execution	Can interrupted runs resume without repeating side effects? How are checkpoints persisted and retrieved? What consistency guarantees exist?
Data residency	Where do prompts, traces, memory, and evaluation datasets persist? Which regions support your compliance requirements?
Policy enforcement	Are tool permissions enforced independently of model instructions? Can a model bypass authorization through clever prompting?
Observability	Can reviewers reconstruct tasks and inspect quality scores? Are traces redacted appropriately? How long are logs retained?
Operational ownership	Who handles scaling, upgrades, backups, retries, incidents? What SLAs exist? How are breaking changes communicated?
Exit cost	Which workflow, memory, and checkpoint formats need migration? How much code couples to framework-specific abstractions?

Validate tenant isolation explicitly. Deploy two test agents with different credentials and verify Agent A cannot access Agent B's memory, tools, or conversation state. Misconfigured isolation is the most common production security failure.

Test durable execution by interrupting an agent mid-workflow and resuming from checkpoint. Verify that external actions (sending email, charging payment, updating database) are not duplicated. Checkpoint persistence handles computation state, not external effects.

Audit data residency for prompt content, conversation history, memory storage, and evaluation datasets. Framework documentation often covers compute regions but not data storage. Compliance depends on where data persists, not just where it's processed.

What common mistakes prevent successful framework adoption?

Over-engineering with heavy frameworks. Teams choose LangGraph for simple chatbots needing one tool. The graph abstraction adds cognitive load and debugging complexity without value. Start with Strands or basic tool-calling, add framework structure only when workflow complexity demands it.

Ignoring operational requirements. A framework working in notebooks may not scale to production. Consider logging, error recovery, scaling, monitoring, and deployment from the start. Prototype quickly, but plan operations before broad rollout.

Framework lock-in before validation. Building deeply on framework abstractions before validating that agent approach works creates expensive rewrites. Prototype with minimal frameworks or raw API calls first, add framework structure once agent design is proven.

Choosing by GitHub stars, not architecture fit. LangChain's popularity doesn't make it best for every use case. A multi-agent research pipeline benefits more from AutoGen's conversational architecture (if maintaining existing systems) or CrewAI's delegation (for new projects) than from LangChain's chains, regardless of star counts.

Treating agents as deterministic systems. Agents with full autonomy are non-deterministic by design. Model-driven control means variable execution paths. Budget for this uncertainty with timeouts, step limits, and explicit termination conditions. Test failure modes as rigorously as success paths.

Assuming multi-agent equals better quality. Multiple agents introduce coordination overhead, failure modes, and token costs. Multi-agent systems need concrete justification: separate tool permissions, independent review, genuinely distinct specializations. Measure whether coordination catches errors that simpler workflows miss.

How is the agent framework landscape evolving?

The framework ecosystem is converging on several architectural patterns that will shape 2026 and beyond. Understanding these trends informs long-term framework selection.

Protocol standardization. Model Context Protocol (MCP) is becoming the universal standard for tool discovery and integration. Frameworks embracing MCP (LangChain, AgentCore Gateway) offer plug-and-play tool ecosystems. Those requiring custom integration patterns face friction as the ecosystem standardizes.

Composability over monoliths. The winning pattern combines specialized tools rather than accepting compromises from all-in-one frameworks. One framework handles orchestration logic (LangGraph), another provides lightweight tool-calling (Strands), a protocol manages tool discovery (MCP), and managed services supply memory and observability (AgentCore or third-party).

Managed runtimes. The operational burden of running agents pushes teams toward managed services. Just as serverless replaced server management for web applications, managed agent runtimes like AgentCore are replacing self-managed agent infrastructure. The framework becomes the logic layer; the cloud provides everything else.

Evaluation-driven development. As agents grow more autonomous, testing shifts from unit tests to evaluation suites measuring agent behavior across hundreds of scenarios. Frameworks integrating evaluation natively — tracking success rates, failure modes, and regression across versions — will dominate production deployments where reliability is non-negotiable.

Explicit state management. The shift from implicit loop-based state to explicit graph-based state reflects production lessons. LangGraph's checkpoint model, AgentCore's managed memory, and the general trend toward typed state objects indicate that implicit state causes too many production bugs to remain the default.

FAQ

Which is the best AI agent framework for beginners in 2026?

Strands Agents SDK offers the lowest barrier to entry — a working agent in 10 lines with no complex abstractions to learn. For beginners wanting more structure and resources, LangChain provides the most tutorials and community support. Start with Strands to understand agent loops, explore LangChain when you need specific integrations.

Can you combine multiple agent frameworks?

Yes, and production teams increasingly do. The common pattern: LangGraph for orchestration logic defining workflow graphs, Strands for individual tool-calling agent nodes providing lightweight execution, and AgentCore for deployment infrastructure with managed compute and memory. Frameworks are complementary, not mutually exclusive.

How much does it cost to run production AI agents?

Agent costs have three components: LLM API costs ($0.01-$0.15 per invocation depending on model and tokens), infrastructure costs (self-hosted: $2-10/hour per GPU; managed: pay-per-invocation), and tool execution costs (API calls, code execution compute). A customer support agent handling 10,000 daily conversations costs $500-2,000/month, with LLM tokens dominating.

Do AI agent frameworks support open-source models?

All major frameworks support open-source models served through compatible APIs. LangChain integrates with Ollama, vLLM, and OpenAI-compatible endpoints. Strands supports any Bedrock-compatible model including self-hosted. The requirement: models must support tool calling (function calling) — not all open-source models do. Llama 3, Mistral, and Command R+ support tool calling and work with these frameworks.

Should I build or buy agent infrastructure?

Build with application-owned frameworks (LangChain, LangGraph, Strands) when you need full control, have platform engineering capacity, or require multi-cloud portability. Buy managed infrastructure (AgentCore) when you're on AWS, lack dedicated operations teams, or prioritize speed-to-production over flexibility. The decision depends on operational maturity and control requirements.

What's the difference between LangChain and LangGraph?

LangChain provides configurable agents with 400+ integrations for rapid prototyping. LangGraph provides explicit workflow execution and state management for production stateful agents. Current LangChain agents build on LangGraph underneath. Use LangChain for ecosystem access, LangGraph directly when explicit state and checkpointing are central to your workflow.

How do I test AI agents before production?

Validate bounded tasks before broad deployment: trace tool calls and failures end-to-end, resume interrupted execution without repeating external actions, verify tenant isolation between agents, confirm permissions are enforced independently of model instructions, measure latency and cost under realistic load, and test failure recovery explicitly. An evaluation suite measuring success rate across hundreds of scenarios matters more than unit tests.

Sources

Originally published at fp8.co. Subscribe for weekly AI engineering analysis at fp8.co/newsletters.

AI Funding Due Diligence: Verify Rounds Before Citing

ke yi — Wed, 16 Sep 2026 05:06:28 +0000

AI funding due diligence is a documented check that a financing claim matches evidence published by the parties involved. It keeps announcement dates, transaction details, and a database's own timestamps distinct.

TL;DR

Start with a precise claim and locate the source that supports it. A company announcement, an investor statement, and a database entry can each describe different parts of a financing event.

Keep the round amount separate from valuation, and preserve the source's wording about whether financing is completed, announced, planned, or still being discussed. Missing information should remain visible.

Retain the source URL, publication date, relevant passage, and the date you reviewed it. A useful funding record lets another person repeat the check.

Key Takeaways

Identify the company before comparing figures. Similar names, brands, and domains can refer to different businesses.
Name the event you are checking. A new financing, an extension, a secondary transaction, and an investor's own fund announcement should not be merged automatically.
Track dates separately. Publication, transaction, discovery, and database-update dates answer different questions.
Preserve the financial labels. Round amount, valuation, currency, and investor role each need their own evidence.
Record uncertainty. An inaccessible source, an unnamed investor, or an ambiguous amount is a reason to keep a verification task open.
Make the citation useful. Link to the supporting document and explain which statement it supports.

What claim are you trying to verify?

Begin with a sentence that can be checked. For example, you might be checking whether a company announced a particular funding stage, whether a named investor participated, or whether the issuer disclosed a valuation. These are separate claims, even when a database displays them on the same row.

Write down the company identity, the event, and the attribute you need. Include the company's official domain and any relevant former name. This prevents an apparently matching search result from being assigned to the wrong organization.

Then define the limit of your evidence. A public announcement supports a statement about what its publisher announced. Private agreements, ownership arrangements, and settlement details may remain unavailable. Your record should make that boundary clear rather than imply access to documents you have not reviewed.

A claim-by-claim approach also makes corrections smaller. If the valuation is unsupported, you can hold that field for review while retaining a separately supported announcement date or investor statement.

Which sources should you check first?

Look first for material published by the company and the investors involved. Company fundraising pages and dated announcements can state the event, round label, amount, and valuation terminology. Investor announcements can explain participation and identify a role in the financing. Read the actual text; a logo or a company name on a page does not establish every part of a transaction.

Axiom's company-hosted fundraising page illustrates a disclosure that separates funding events and their financial labels. B Capital's investment article about Axiom illustrates a different document type: a dated explanation from an investor. Before combining material from such pages, compare their dates and event descriptions. Sharing a company name does not establish that two documents describe the same round.

News reporting and databases can help you find the relevant announcement. Preserve their attribution when a detail is available only through reporting. An unnamed-source report and an issuer statement have different evidence trails.

Search snippets are useful leads. Open the underlying page before treating the snippet as evidence, because the excerpt may omit a qualification, a date, or the company to which a figure belongs.

Which dates belong in a funding record?

A funding record can have several legitimate dates:

Publication date: when the supporting announcement or report appeared.
Transaction date: when the source says the financing event occurred, if it specifies one.
Discovery date: when your team first found the information.
Record-update date: when your database entry changed.
Review date: when someone last checked the evidence.

Keep these fields distinct. A recently updated database row may describe an older event. A later article may summarize several earlier financings. Neither situation creates a new round by itself.

If a source gives only a month or a general period, preserve that precision in your research notes. Avoid quietly inventing a particular day. When two sources disagree, record both dates with their meanings before choosing the value appropriate to your database's policy.

How do you separate the round amount from valuation?

Copy the financial label along with the figure. The amount associated with a financing and the valuation assigned to a business describe different things. A source that discloses one does not necessarily disclose the other.

Preserve any explicit pre-money or post-money label. If the source does not identify the valuation basis, leave that basis unspecified. Avoid converting an assumed ownership percentage into a reported valuation.

Currency needs the same care. Keep the original currency with the original amount. If your analysis also presents a conversion, identify the conversion as a calculation and retain the rate source and date. The calculated value should remain distinguishable from the figure the issuer published.

Check the subject of every number. An article may discuss the company's financing alongside an investor's fund size, a previous round, or another company's transaction. Read enough surrounding text to establish whose amount it is and what it measures.

How do you distinguish completed financing from a target?

Preserve the source's verbs and qualifications. Wording such as “seeking,” “targeting,” or “in discussions” describes a different status from wording that explicitly announces a completed financing. A planned transaction can be important news without becoming a completed-round record.

Also check whether a later announcement updates an earlier report. The later document may confirm some details while changing others. Keep the earlier item in the evidence trail, and identify which source supports the current status.

Do not infer completion from the existence of a presentation, a fundraising page, or a named prospective investor. Your public summary should reflect the status the evidence supports. Where the wording is ambiguous, record the uncertainty and seek a clearer statement.

How do you verify an investor's role?

Look for an explicit relationship between the investor and the financing. A name appearing in an article's navigation, event promotion, founder biography, or discussion of technology suppliers is not enough to attribute participation.

Separate participation from leadership. If a source names a participant without specifying a lead role, preserve the stated participation. Do not promote the first name in a list to lead merely because it appears first.

Resolve aliases carefully. A firm's abbreviated name and legal name may identify the same investor, while similarly named firms may be distinct. Keep a reference to the spelling in the source so that another reviewer can reproduce your decision.

Apply the same evidence standard to familiar and unfamiliar investors. An investor already present in your database still needs support for its relationship to this particular event.

What evidence should you retain?

Store enough information to repeat the review without copying an entire publication. A practical evidence note includes:

Field	What it records
Supporting URL	The specific announcement or document
Publisher	The company, investor, outlet, or other source
Publication date	The date shown by that source
Relevant passage	A short passage identifying the claim and its subject
Supported attributes	Which fields the passage establishes
Review date	When the source was checked
Open questions	What remains missing, ambiguous, or inaccessible

Keep the passage connected to its subject. A detached number can lose the context that identifies the company, currency, event, or status.

If a page changes, preserve your review note and a permitted reference to the version you consulted. Follow applicable access and reuse rules. Avoid making an inaccessible copy the only explanation of a claim; a reader should still understand what was checked and what remains uncertain.

What should you do when sources conflict or disappear?

First check for an identity or event mismatch. The apparent disagreement may involve different companies, different stages, an earlier financing, or a distinction between an announced amount and a broader funding total.

Next compare the wording and publication sequence. A correction or a later issuer statement may explain the difference. Retain the original attribution when you update the record, so the revision has an understandable history.

If the conflict remains unresolved, publish the limitation alongside the field or hold the disputed attribute for review. Avoid averaging conflicting values or selecting the more impressive figure.

An unavailable page requires a separate judgment. A failed request or a subscription barrier prevents a new check; it does not establish that the historical record was false. Look for another accessible statement from the parties involved, and mark the verification status accurately.

How can a team make reviews repeatable?

Use the same questions for each record: Is this the correct entity? Does the source describe this event? Which attributes does it support? What remains unresolved?

Separate data entry from the decision to treat a claim as supported. A candidate record can preserve useful research leads while its uncertain fields remain clearly marked. Reviewers should be able to change one attribute without losing the rest of the evidence.

Record corrections with their reason and review date. This prevents a later import from silently replacing a reviewed decision with the same weak inference. When a new source resolves a gap, update the record and retain the explanation.

For published summaries, check the final page as well as the stored record. Titles, descriptions, tables, and related pages can repeat a financial claim. A correction is complete when the intended public surfaces reflect the reviewed information.

FAQ

Is a database entry sufficient evidence for a financing claim?

A database entry is a useful starting point. Follow its source attribution and verify the particular attribute you intend to cite. If the supporting document is missing, describe the limitation rather than implying that you have independently checked it.

Can an investor announcement replace a company announcement?

It can support what the investor explicitly states. Read its date, event description, and role carefully. It may not disclose every term or describe the same event as another document.

Should an old record be deleted when its source is inaccessible?

Inaccessibility alone does not settle the record's accuracy. Preserve the known evidence trail, mark the review limitation, and seek an accessible supporting statement before making a stronger conclusion.

Does verification guarantee that a record is complete?

No. A review establishes what the available evidence supports. Undisclosed terms, private documents, and later updates can remain outside that evidence. Keep the review date and unresolved questions visible.

Sources

Axiom — Fundraising: a company-hosted disclosure page separating funding events and financial labels.
B Capital — Why We Invested in Axiom: a dated investor explanation, useful for checking attribution and document context.

AI Funding provides linked records for discovering financing events. Use those records alongside the original announcements and a documented verification process before citing a claim.

Top AI Developer Tools Startups in 2026: Funding, Valuations & Market Leaders

ke yi — Wed, 16 Sep 2026 03:46:37 +0000

TL;DR

The AI Developer Tools sector is one of the most dynamic areas of AI investment in 2026. This report profiles the top-funded startups in the space, their investors, and the market dynamics driving growth.

Key Takeaways

The 46 AI Developer Tools companies tracked by AI Funding have raised $6.3B in total disclosed funding. Figures cover rounds reported in USD. 5 rounds (€15M) are excluded: no published dollar figures exist for them. See the full list on the AI Developer Tools sector page.
The top-funded AI Developer Tools companies are Cognition ($1.2B), Cursor ($968M) and Lovable ($917M).
Lovable carries the highest disclosed valuation in AI Developer Tools at $13.3B.

Overview

The AI funding landscape continues to evolve rapidly, with significant capital flowing into companies pushing the boundaries of artificial intelligence. This period's activity reflects both growing investor confidence and the increasing maturity of the ecosystem. Deal sizes and valuations continue to climb as AI transitions from experimental technology to core enterprise infrastructure.

Sector at a Glance

Metric	Value
Companies Tracked	6
Total Funding	$2.4B
Average Latest Valuation	$5.2B
Sector Page	/sector/ai-developer-tools

Top AI Developer Tools Startups by Total Funding

1. Cursor

Total Raised: $960M
Latest Valuation: $9.9B
Latest Round: Series B ($900M)
Founded: 2022 | HQ: San Francisco, CA
Key Investors: Andreessen Horowitz, Nat Friedman & Daniel Gross, Thrive Capital, Accel
Company Page: /companies/cursor

AI-first code editor built to supercharge developer productivity with intelligent autocomplete, chat, and codebase understanding.

The competitive landscape remains dynamic, with established players and well-funded startups vying for market share. Differentiation increasingly comes from technology depth, go-to-market execution, and strategic partnerships that provide distribution advantages.

2. Replit

Total Raised: $600M
Latest Valuation: Undisclosed
Latest Round: Undisclosed ($400M)
Founded: 2016 | HQ: San Francisco, CA
Key Investors: Andreessen Horowitz, Khosla Ventures, Google Ventures (GV), Undisclosed
Company Page: /companies/replit

AI-powered software development and deployment platform enabling anyone to build and ship applications from the browser.

3. Poolside

Total Raised: $500M
Latest Valuation: $3.0B
Latest Round: Series B ($500M)
Founded: 2023 | HQ: San Francisco, CA
Key Investors: Bessemer Venture Partners, Nat Friedman & Daniel Gross, Felicis Ventures
Company Page: /companies/poolside

AI company building foundation models specifically designed for software engineering.

4. Lovable

Total Raised: $217M
Latest Valuation: $2.8B
Latest Round: Series B ($200M)
Founded: 2023 | HQ: Stockholm, Sweden
Key Investors: Benchmark, Accel, Founders Fund
Company Page: /companies/lovable

AI-powered full-stack web app builder. Generate production-ready apps from natural language.

5. Code Metal

Total Raised: $125M
Latest Valuation: Undisclosed
Latest Round: Undisclosed ($125M)
Founded: 2026 | HQ: San Francisco, CA
Key Investors: Undisclosed
Company Page: /companies/code-metal

AI-powered automated code translation and optimization platform that converts code between programming languages with formal verification for safety-critical industries.

6. Axiomatic AI

Total Raised: $18M
Latest Valuation: Undisclosed
Latest Round: Undisclosed ($18M)
Founded: 2026 | HQ: Cambridge, MA
Key Investors: Undisclosed
Company Page: /companies/axiomatic-ai

MIT spin-off building verified AI operators for engineering workflows using formal mathematical proofs, serving photonics, electronics, thermal, mechanics, and signal domains.

Most Active Investors in AI Developer Tools

Rank	Investor	Companies Backed
1	Undisclosed	3
2	Andreessen Horowitz	2
3	Nat Friedman & Daniel Gross	2
4	Accel	2
5	Thrive Capital	1
6	Khosla Ventures	1
7	Google Ventures (GV)	1
8	Bessemer Venture Partners	1
9	Felicis Ventures	1
10	Benchmark	1

This area of the AI market continues to attract significant investor attention, reflecting the sector's rapid growth and substantial long-term potential.

AI Developer Tools Trends to Watch

This area of the AI market continues to attract significant investor attention, reflecting the sector's rapid growth and substantial long-term potential.

Frequently Asked Questions

How much funding have AI Developer Tools startups raised?

The 46 AI Developer Tools companies tracked by AI Funding have raised $6.3B in total disclosed funding. Figures cover rounds reported in USD. 5 rounds (€15M) are excluded: no published dollar figures exist for them. See the full list on the AI Developer Tools sector page.

Which AI Developer Tools startups have raised the most funding?

The top-funded AI Developer Tools companies are Cognition ($1.2B), Cursor ($968M) and Lovable ($917M).

What is the most valuable AI Developer Tools startup?

Lovable carries the highest disclosed valuation in AI Developer Tools at $13.3B.

Data sourced from AI Funding's funding database. Rankings based on total disclosed funding. This scaffold was generated with AI Funding's content tools and may require editorial review before publication.

LangChain vs Strands: Agent Framework Comparison 2026

ke yi — Fri, 11 Sep 2026 20:18:49 +0000

LangChain vs Strands: Which Agent Framework Should You Choose?

TL;DR: LangChain's current create_agent interface and Strands' Agent both support model-driven tool use. Compare their integrations, execution controls, persistence, and operating requirements rather than code length. Strands includes snapshot/checkpoint support for single agents and separate session-management options for Graph/Swarm; it does not require every persistence feature to be built from scratch.

Key Takeaways

Use LangChain's current create_agent interface with message-based invocation. Do not present the legacy executor API as the current agent quickstart.
Both frameworks can let a model select tools. LangGraph is available when a LangChain application needs more explicit orchestration; it is not mandatory to hand-code a graph for every agent.
Current Strands documentation recommends SnapshotSessionManager for new single-agent sessions. Its immutable checkpoints and restore support differ from the repository-based managers used for Graph/Swarm.
Separate invocation-scoped configuration from durable state. Strands invocation_state is not a replacement for a session manager.
Both have model-provider integrations and open-source libraries. Hosting, model calls, storage, and observability remain separate operating choices and costs.
The examples below are fixtures with explicit prerequisites. No model call, latency benchmark, or framework-overhead measurement was performed for this comparison.

What are LangChain and Strands?

LangChain provides a configurable agent harness through create_agent, with tools, prompts, and middleware around the model loop. Its agents build on LangGraph, and applications can use LangGraph directly for more explicit workflow execution. Integration packages provide model, retrieval, and other application components.

Strands Agents is an SDK for model-driven agents with tools, lifecycle controls, state, sessions, and multi-agent patterns. Its repository contains Python and TypeScript packages. Amazon Bedrock is the documented default provider, but alternative providers are supported.

The useful choice is which APIs and operating contracts fit your application. Neither framework's philosophy establishes that it is faster, requires fewer lines, or is suitable only for simple versus complex work.

Official documentation and source were checked September 11, 2026. The LangChain snippets target the current v1 interface. The Strands session example requires a release that exports SnapshotSessionManager; match the installed release to these docs instead of copying an older session API into the new example.

How do they differ architecturally?

LangChain's create_agent constructs an agent without requiring a separate prompt template and executor. The model can choose tools dynamically. For fixed retrieval followed by generation, ordinary application control flow may be sufficient; for branching, interruption, or explicit state transitions, inspect LangGraph's execution and persistence APIs.

Strands also runs a model/tool loop, with hooks and lifecycle controls around it. Applications can use one agent, compose agents as tools, or adopt Graph/Swarm patterns when they need explicit coordination. It should not be characterized as having no workflow or persistence structure.

For either framework, decide who owns tool authorization, side-effect handling, cancellation, execution budgets, and final success checks. A prompt asking an agent to check an order before issuing a refund is not an application-enforced transaction or permission boundary.

How do LangChain and Strands compare at a glance?

Category	LangChain	Strands Agents
Library license	MIT	Apache-2.0
Agent entry point	`create_agent`	`Agent` and tools such as `@tool` functions
Typical invocation	`agent.invoke({"messages": [...]})`	Python `agent(question)`; TypeScript has its own invocation API
Execution	Model-driven agent built on LangGraph; explicit graphs also available	Model-driven loop with lifecycle controls and composition patterns
Retrieval	Compose supported integration packages or application retrieval functions	Expose application retrieval through tools or explicit control flow
Durable state	Configured thread checkpointer and cross-thread store	Single-agent snapshots or repository-based sessions, according to topology
Transient context	Use the application's documented runtime/context mechanism	`invocation_state` for one invocation, separate from persistent `agent.state`
Multi-agent	Graph/subgraph and supervisor-style designs	Agents-as-tools, Graph, and Swarm designs
Languages	Python and JavaScript/TypeScript packages	Python and TypeScript packages
Observability	LangSmith and supported instrumentation/export integrations	OpenTelemetry instrumentation with configured export/backend
Deployment	Your compatible host or a managed deployment service	Your compatible host, including AgentCore Runtime when its contract fits

The licenses apply to the open-source projects, not to every optional hosted service or dependency. The corresponding primary license files are linked under Sources. There is no LOC, latency, or throughput ranking in this table.

What do equivalent current agent examples look like?

Use the same bounded task before comparing developer experience. These examples look up a status in an in-memory fixture; they do not access a database, issue refunds, or demonstrate production authorization.

Prerequisites: install the relevant SDK and provider integration, configure credentials outside the source, and pass a compatible model supporting tool calls. The functions define invocation paths; calling them uses the supplied model and can incur charges. They were syntax/static-checked, not invoked against a model here.

The current LangChain example follows its documented create_agent and messages interface:

from langchain.agents import create_agent

ORDER_STATUSES = {"4471": "Delivered; refund eligibility needs human review."}

def get_order_status(order_id: str) -> str:
    """Return the status of an order in this local demonstration fixture."""
    return ORDER_STATUSES.get(order_id, "Order not found in the fixture.")

def ask_langchain(model, question: str):
    agent = create_agent(
        model=model,
        tools=[get_order_status],
        system_prompt="Look up order status. Do not claim to issue refunds.",
    )
    return agent.invoke({"messages": [{"role": "user", "content": question}]})

The equivalent Strands example uses its Python Agent and tool decorator:

from strands import Agent, tool

ORDER_STATUSES = {"4471": "Delivered; refund eligibility needs human review."}

@tool
def get_order_status(order_id: str) -> str:
    """Return the status of an order in this local demonstration fixture."""
    return ORDER_STATUSES.get(order_id, "Order not found in the fixture.")

def ask_strands(model, question: str):
    agent = Agent(
        model=model,
        tools=[get_order_status],
        system_prompt="Look up order status. Do not claim to issue refunds.",
    )
    return agent(question)

LangChain returns its agent state, including messages; Strands returns an AgentResult. Inspect each documented result type rather than assuming identical output objects. Neither example guarantees the model will choose a particular sequence of tools.

The LangChain v1 migration guide separates the streamlined current package from langchain-classic functionality. If maintaining an older executor/chain application, use its versioned migration path. Do not use that older application's extra setup to claim current LangChain agents necessarily require more boilerplate.

How do they handle retrieval and RAG workflows?

LangChain's ecosystem includes document-loading, splitting, model, and vector-store integrations. Check the current package for each component; legacy chain/retriever imports are not interchangeable with the v1 agent namespace. Strands can call retrieval code as a tool, and either framework can be used after an application performs deterministic retrieval.

There are two separate choices: how documents are retrieved, and whether the model should decide when to retrieve. A fixed “retrieve, then answer” task does not require a second agent loop. A task that must choose among multiple information sources may benefit from tool-driven retrieval.

What is a complete offline retrieval fixture?

This self-contained Python example uses a small keyword fixture, not a production vector database or semantic-search benchmark. The generation dependency is an explicit callable, so the retrieval/context behavior can be checked offline. There is no undefined qa_chain, database, or external endpoint.

from collections.abc import Callable

DOCUMENTS = (
    {"id": "auth", "term": "authentication",
     "text": "Configure authentication with the approved identity provider."},
    {"id": "refund", "term": "refund",
     "text": "Refund eligibility requires human review before any payment action."},
)

def retrieve_docs(question: str) -> list[dict[str, str]]:
    query = question.casefold()
    return [dict(doc) for doc in DOCUMENTS if doc["term"] in query]

def build_rag_messages(question: str) -> list[dict[str, str]]:
    documents = retrieve_docs(question)
    if not documents:
        return []
    context = "\n".join(f"[{doc['id']}] {doc['text']}" for doc in documents)
    return [
        {"role": "system", "content": "Answer only from the supplied fixture. Cite its IDs."},
        {"role": "user", "content": f"Evidence:\n{context}\n\nQuestion: {question}"},
    ]

def answer_with_retrieval(
    question: str,
    generate: Callable[[list[dict[str, str]]], str],
) -> str:
    messages = build_rag_messages(question)
    if not messages:
        return "No matching evidence in the fixture."
    return generate(messages)

For an offline check, pass a deterministic local function that inspects the messages and returns a fixture answer. For a deployed application, supply an adapter for the chosen model interface and normalize its response to a string. That adapter, credentials, production retrieval, access control, and document freshness are explicit application responsibilities, not hidden prerequisites of the fixture.

A LangChain agent can instead expose retrieval through a tool supplied to create_agent; a Strands agent can expose it through @tool. Preserve document identifiers and evaluate answer support in either approach. Choose integrations that fit the data source instead of claiming that one framework is universally better for RAG.

How do they handle state, memory, and persistence?

LangGraph persistence distinguishes a checkpointer, which records thread-scoped graph state, from a store for data across threads. Configure the appropriate backend and thread identity. An in-memory example is not durable across process restarts, and external side effects still need an idempotency/recovery contract.

Current Strands session documentation describes built-in persistence and recovery. For new single-agent sessions, it recommends SnapshotSessionManager, which saves a latest snapshot and can create immutable checkpoints. Its implementation exposes restore_snapshot() with an optional checkpoint ID.

How does a single-agent Strands snapshot configuration look?

Unexecuted configuration example: use a release exporting these classes, provide a compatible model, and choose an authorized session ID and storage location. Calling this factory can initialize local storage; invoking the returned agent calls the model. Neither action was executed during this article's validation.

from strands import Agent
from strands.session import SnapshotSessionManager
from strands.storage import LocalFileStorage

def build_persistent_agent(model, session_id: str, storage_dir: str):
    session = SnapshotSessionManager(
        session_id=session_id,
        storage=LocalFileStorage(storage_dir),
        snapshot_trigger=lambda *, agent_data, **kwargs: True,
    )
    agent = Agent(model=model, session_manager=session)
    return agent, session

Here the trigger requests an immutable checkpoint after each qualifying invocation. The manager also maintains the latest snapshot. To restore a particular saved checkpoint, the source documents await session.restore_snapshot(agent, snapshot_id=checkpoint_id); supply a real ID from that session's stored snapshots. Retention, tenant isolation, and replay of external effects still require application design.

How do Graph/Swarm sessions differ?

SnapshotSessionManager is single-agent only. Current documentation says to use a repository-based session manager such as FileSessionManager or S3SessionManager on the Graph/Swarm orchestrator. Agents inside that system should not each receive their own session manager. Do not transfer the single-agent snapshot example unchanged into a multi-agent graph.

Strands state documentation also separates persistent application state from invocation_state. Invocation state is initialized for one invocation, shared with its tools/hooks, and excluded from model context; automatic cross-session persistence is a different mechanism. Passing a database connection through invocation state does not make that connection or the conversation durable.

Both frameworks therefore have persistence capabilities. Compare the topology, save boundaries, restore API, and failure behavior required by the workload instead of using “has memory” or “no checkpointing” as a blanket selection rule. See Agent Memory for the broader state boundary.

How do they support multi-agent systems?

LangChain applications can use LangGraph-based composition when they need explicit routing, subgraphs, or a supervisor design. Strands documents agents-as-tools and Graph/Swarm patterns. Choose a topology because it solves a coordination requirement, such as distinct tool permissions or an independent review step.

Define which component owns shared state, cancellation, retries, and termination. Persistence configuration belongs to the intended execution boundary. A resumable conversation or graph does not itself guarantee exactly-once execution of a database write, payment, or message.

For each candidate, test a handoff, a tool failure, an interrupted run, and a restart using the selected session/checkpoint backend. These are proposed acceptance checks, not experiments performed for this article.

Which models and providers does each support?

Both support multiple providers. LangChain's overview documents provider integrations; Strands' repository documents Bedrock as the default and alternatives such as Anthropic and OpenAI. Configure the model explicitly when comparing frameworks so a default-provider difference does not become an accidental comparison variable.

Choose package versions and a provider with the tool-calling, streaming, and structured-output behavior your task requires. Swapping an import or model identifier does not establish equivalent behavior, context limits, or credentials. This guide does not rank ecosystems using unverified provider counts.

How do observability and debugging compare?

LangSmith provides tracing and evaluation workflows for LangChain and other applications. Its OpenTelemetry documentation describes ingestion and attribute mapping, so LangChain should not be presented as locked to a single proprietary tracing path.

Strands includes tracing instrumentation. A useful deployed setup still needs export configuration, an accessible backend, appropriate field mapping, and retention/access decisions. Built-in instrumentation does not mean a trace dashboard appears without configuring a destination.

Compare one representative trace: task, retrieved evidence, model generation, tool outcome, error, and evaluation result. Verify that usage fields and parent/child relationships survive export. For deployment and evaluation trade-offs, see LangSmith vs Langfuse vs Phoenix.

How do deployment and costs compare?

Separate open-source library licensing from service costs. Model API calls, runtime resources, storage, telemetry, and operational work are costs regardless of the agent library. LangSmith pricing describes optional hosted services and usage; it is not a license fee for executing an open-source LangChain agent.

AgentCore Runtime can host applications built with different frameworks, including LangChain and Strands, subject to its deployment contract. Compare Runtime here as a host; the wider AgentCore family also includes managed Harness orchestration. See AgentCore vs LangChain for that distinction.

No fixed monthly price, measured milliseconds of overhead, or throughput winner is established in this article. If performance matters, hold the task, model configuration, tools, environment, and completion criteria fixed; include failed attempts and retries in the results.

When should you choose LangChain, and when Strands?

Requirement	Candidate to inspect	Evidence that should decide
Existing LangChain integrations and application components	LangChain	Current packages cover the required models, retrieval, and output handling
Configurable agent with explicit workflow extensions	LangChain / LangGraph	Required branches, interrupts, and checkpoint recovery behave correctly
In-process model/tool loop with Strands controls	Strands	Tool handling, budgets, hooks, and provider integration fit the workload
Single-agent snapshot/restore	Strands snapshot sessions or LangGraph persistence, as appropriate	Selected save/restore semantics satisfy the application's recovery contract
Graph/Swarm state persistence	Strands repository-based orchestrator sessions	Restart and handoff tests match the chosen topology
A fixed retrieve-then-answer task	Either framework or ordinary application code	Retrieval and evidence checks pass without unnecessary orchestration

These are shortlist criteria, not a ranking. The framework selection guide places these choices alongside other orchestration approaches.

Can you use LangChain and Strands together?

Applications can use different frameworks for different agents or components when their interfaces and ownership are explicit. Keep a contract for inputs, outputs, errors, cancellation, and durable state rather than trying to share an internal executor or checkpoint object implicitly.

For migration, start with one bounded workflow and retain its evaluation cases. Reuse tool business logic where compatible, but validate the framework wrappers and state/replay behavior separately. Moving into LangChain does not require replacing all model-driven behavior with deterministic chains, and moving into Strands does not require abandoning all checkpoints.

Frequently Asked Questions

What is the main difference between LangChain and Strands?

Both provide model-driven agents. Compare their integration APIs, lifecycle controls, state/session model, and workflow composition against your task. LangChain's current agent API does not require the old separate executor setup, and Strands includes persistence and multi-agent capabilities.

Is Strands only for AWS or Amazon Bedrock?

No. Bedrock is the documented default, but other providers are supported. Configure a supported model and the appropriate credentials, then verify the capabilities your application needs. An AWS origin is not a requirement to host every Strands application on AWS.

Which framework is better for RAG?

Choose the retrieval integrations and control flow that fit the data source and task. Either framework can call retrieval tools or consume application-selected context. The offline fixture above demonstrates a complete retrieval/generation boundary without claiming that its keyword search is production semantic retrieval.

Does Strands support checkpoints and time-travel restore?

Current documentation and source provide single-agent SnapshotSessionManager checkpoints and restore_snapshot(). Graph/Swarm use different repository-based session managers on the orchestrator. Verify the installed release and topology rather than assuming that all session managers provide the same restore behavior.

Can LangChain agents deploy on AgentCore Runtime?

AWS documents framework-independent Runtime hosting, including LangChain and Strands. The application still needs to meet the runtime contract and configure identity, state, tools, and observability correctly.

Which framework has better observability?

Compare a real trace and its required fields in the intended backend. LangSmith provides a managed workflow and OTEL ingestion; Strands supplies instrumentation that needs an export destination. Neither product label establishes that your data was collected correctly.

Which framework is better for beginners?

Try the same small read-only tool task with each current API. Assess whether errors, results, configuration, and persistence are understandable. The source-line count of two differently scoped examples is not a reliable beginner ranking.

How do LangChain and Strands compare on performance and latency?

This guide has no framework-overhead measurement or benchmark result. Measure a common workload and include model calls, tools, retries, persistence, and instrumentation. A different prompt, provider default, or success rate can outweigh the choice of framework.

Can I migrate from one framework to the other?

Yes, but align tool schemas, model capabilities, result types, state ownership, and recovery semantics. Validate a bounded workflow before expanding the migration. Neither interface compatibility nor equal fixture output proves equivalent production behavior.

Sources

Originally published at fp8.co. Subscribe for weekly AI engineering analysis at fp8.co/newsletters.

How to Track AI Startup Funding in Real Time

ke yi — Thu, 10 Sep 2026 15:44:55 +0000

Real-time AI startup funding tracking means combining primary announcements, regulatory filings, reliable news, and structured databases, then verifying each amount, date, stage, investor, and valuation before treating it as confirmed.

TL;DR

No single funding source is complete or truly instantaneous. Start with company and investor announcements, check regulatory filings where applicable, use reliable reporting to resolve context, and use structured databases to search and compare records.

AI Funding provides a free AI-focused deal tracker, data explorer, and open API; commercial databases such as Crunchbase, PitchBook, Dealroom, and Tracxn provide broader company and market-research workflows.

A funding round is ready for analysis only when the company identity, amount, currency, date, stage, investors, source, and any valuation claim have been checked independently.

Key Takeaways

Primary announcements are the strongest starting point. Company newsrooms, investor portfolio announcements, and regulatory filings are closer to the transaction than a database summary.
SEC EDGAR is free and authoritative for filings, but a Form D is a notice of an exempt securities offering rather than a complete commercial history. Use the SEC filing search and read the filing itself.
Structured databases make discovery and comparison faster. AI Funding focuses on AI companies; Crunchbase, PitchBook, Dealroom, and Tracxn cover broader private-market research use cases.
Round amount and valuation are different facts. A reported raise does not prove a valuation, and a modelled valuation must never be presented as a sourced figure.
Entity matching matters as much as number matching. Companies with similar names can cause unrelated rounds, investors, or sectors to be merged into one profile.
Record an as-of date and source URL for every result. Funding information changes as extensions, secondary transactions, corrections, and later closes are announced.

What does real-time startup funding data actually mean?

Private-company financing does not arrive through one universal real-time feed. A round may first appear in a founder announcement, an investor newsroom, a regulatory filing, a publication, or a database update. Some announcements describe a first close, while later coverage reports an extension or final close. Others disclose a round amount but omit stage, valuation, or participating investors.

A practical tracker is therefore near-real-time: it monitors several source classes, records when each fact became public, and updates the structured record when stronger evidence arrives. The public AI Funding deals page and data explorer expose the tracked result, while the data methodology explains how sources and limitations should be interpreted.

Which funding sources should you trust first?

Use a source hierarchy rather than treating every URL as equal:

Company or investor announcement. Prefer a dated newsroom post, press release, or portfolio announcement that names the round and participants.
Regulatory filing. For relevant U.S. exempt offerings, search SEC EDGAR and inspect the filed notice. The SEC's Form D instructions explain the fields and filing purpose.
Reliable financial or technology reporting. Use reporting to add transaction context, identify unnamed participants, or reconcile conflicting announcements.
Structured funding database. Use databases to discover companies, normalize records, compare histories, and find the original evidence.
Search snippets, social posts, and secondary lists. Treat these as leads until a stronger source confirms the claim.

A source can be primary and still incomplete. A company may announce a raise without its valuation, while an investor may describe its participation without confirming whether it led the round. Store only what the evidence supports.

How do free and commercial funding databases compare?

Source	Access model	Best use	Verification rule
AI Funding	Free public tracker and API	AI-specific rounds, company histories, investor activity, and downloadable analysis	Follow the stored source and methodology before quoting a figure
SEC EDGAR	Free public filings	U.S. regulatory notices and issuer identity checks	Read the filing; do not infer a completed round or valuation from the form alone
Crunchbase	Public profiles plus commercial research products	Broad company discovery, funding histories, people, and market research	Confirm important figures against the linked announcement or filing
PitchBook	Commercial platform	Private-capital research, deal screening, funds, investors, and market analysis	Preserve the as-of date and verify claims before republishing them publicly
Dealroom	Platform access plus public ecosystem research	Startup ecosystems, company discovery, investment trends, and regional analysis	Check whether a value is disclosed, estimated, or modelled
Tracxn	Commercial plans	Sector landscapes, company screening, competitive intelligence, and market maps	Trace a material claim to a primary or clearly identified secondary source
Company and investor newsrooms	Free public announcements	Round amount, date, named participants, stated use of proceeds	Check whether the announcement describes an initial close, extension, or total round

The right choice depends on the task. A founder checking one competitor can begin with free primary sources and a focused tracker. A fund mapping thousands of companies may need a commercial database for screening, exports, team workflows, or broader private-market coverage. In both cases, the final investment or publication decision should retain the underlying evidence.

How should you verify a funding round before using it?

Check the following fields separately:

Company identity: legal or operating name, website, location, and sector.
Amount and currency: preserve the announced currency; do not silently convert foreign rounds into dollars.
Date: distinguish announcement date, filing date, and transaction close date.
Stage: record the stated stage; use Undisclosed when the evidence does not name one.
Investors: distinguish lead investors from participants and avoid extracting sentence fragments as investor names.
Valuation: require a direct reported valuation or label an estimate clearly.
Round relationship: determine whether the report is a new round, extension, second close, debt facility, secondary sale, grant, or acquisition.
Source and as-of date: retain the URL and the date the record was last verified.

The AI Funding API exposes company, round, and investor records for programmatic review. The existing Series A, B, and C funding analysis shows how stage-level analysis depends on correctly classified underlying rounds.

What is a repeatable monitoring workflow?

A lightweight workflow can be run daily:

Monitor company and investor newsrooms, selected publications, and relevant filing feeds.
Extract candidate company, amount, currency, date, stage, and investor fields.
Match the candidate to an existing company using more than its display name.
Open the original source and verify each field independently.
Compare the candidate with prior rounds to detect duplicates, extensions, or corrections.
Mark unsupported fields as unknown rather than filling them with a modelled value.
Publish the structured record with its source and verification date.
Recheck high-impact rounds when later reporting supplies a final close, valuation, or additional investors.

For manual research, start with the live deals and company profile, then follow the linked source. For systematic research, use the data explorer or API and preserve the source fields in downstream analysis.

What are the limits of funding trackers?

Private-company data is incomplete by nature. Some rounds are never announced, some filings aggregate planned rather than completed sales, some valuations come from unnamed sources, and database records can lag or conflict. Currency conversion can also make rankings misleading when the original exchange rate is unavailable.

A tracker should make those limits visible. AI Funding distinguishes stored round amounts from valuation claims, keeps foreign currency when conversion evidence is absent, and documents its collection rules on the methodology page. Researchers should still revisit the original evidence before using a figure in diligence, publication, or investment analysis.

Frequently Asked Questions

Is there one definitive source for startup funding rounds?

No. Primary announcements, filings, reputable reporting, and structured databases each reveal different parts of a financing event. Reliable research combines them and records which source supports each field.

Is SEC Form D proof that a startup closed the full amount shown?

No. Form D is a notice associated with an exempt offering. Review the filing fields and subsequent company or investor announcements before treating the offering amount as a completed financing round.

Are free startup funding databases accurate enough for research?

They can be useful for discovery, monitoring, and initial comparison. Material figures should still be checked against primary evidence, especially when the stage, currency, investor role, valuation, or final close affects the conclusion.

When is a paid funding database worth using?

A commercial platform is most useful when a team needs broad private-market coverage, repeatable screening, exports, collaboration, fund data, or large-scale competitive research. It does not remove the need to verify important claims.

How quickly should a funding tracker update a new round?

The goal should be to update after credible public evidence appears, then revise the record when stronger or more complete evidence becomes available. Speed matters, but a fast unsupported figure is less useful than a slightly later sourced record.

DeepSeek v4 vs v4.1 Flash: Performance & Cost Guide

ke yi — Wed, 09 Sep 2026 17:46:18 +0000

DeepSeek v4 vs v4.1 Flash: Which Model Delivers Better Value in 2026?

TL;DR: DeepSeek v4.1 Flash launched in September 2026 as a faster, cheaper alternative to v4 Pro while maintaining comparable accuracy on most benchmarks. v4.1 Flash costs 60% less per million tokens ($0.14 input vs $0.36), delivers 2.3× faster inference, and matches v4 Pro on MMLU (88.5% vs 89.1%) while trailing slightly on complex reasoning tasks like GPQA Diamond (62.1% vs 65.8%). Choose v4.1 Flash for high-throughput production workloads, cost-sensitive applications, and use cases where sub-200ms latency matters. Stick with v4 Pro for research-grade reasoning, mathematical proofs, and scenarios where the 3-5% accuracy gain on hard benchmarks justifies the higher cost.

Key Takeaways

DeepSeek v4.1 Flash reduces inference cost by 60% ($0.14 input / $0.42 output per 1M tokens vs v4 Pro's $0.36 / $1.08) while maintaining 88.5% MMLU accuracy.
v4.1 Flash delivers 2.3× faster time-to-first-token (87ms vs 201ms) and 1.8× higher throughput (142 tokens/sec vs 78 tokens/sec), making it ideal for real-time AI applications.
On standard benchmarks (MMLU, HumanEval, GSM8K), v4.1 Flash scores within 2% of v4 Pro, but the gap widens to 3-6% on research-grade reasoning tasks like GPQA Diamond and MATH-500.
v4.1 Flash supports the same 64K context window and function calling capabilities as v4 Pro, with identical API interfaces for drop-in replacement.
For most production AI applications — chatbots, content generation, code completion, data extraction — v4.1 Flash delivers indistinguishable quality at a fraction of the cost.
DeepSeek's transparent pricing and open model weights (for v3) make it the leading cost-performance option among Chinese LLM providers in late 2026.

Why does DeepSeek v4.1 Flash matter for AI engineers?

The AI model landscape in September 2026 is defined by one central tension: the gap between frontier model capabilities and what most production applications actually need. DeepSeek v4 Pro, released in early 2026, established itself as one of the most capable reasoning models outside the GPT-4 and Claude families. But at $0.36 per million input tokens, deploying it at scale meant significant infrastructure costs, especially for high-throughput workloads like customer support, content moderation, or agent-based automation.

DeepSeek v4.1 Flash solves this by delivering 90-95% of v4 Pro's capability at 40% of the cost and 2× the speed. This is not just an incremental optimization — it fundamentally changes the economics of deploying DeepSeek in production. Tasks that were cost-prohibitive with v4 Pro become viable with v4.1 Flash. Latency-sensitive applications that couldn't tolerate v4 Pro's 200ms time-to-first-token can now hit sub-100ms targets.

For teams already committed to the DeepSeek ecosystem, v4.1 Flash is a straightforward upgrade. For teams comparing Chinese LLM providers (Alibaba Qwen, Baidu ERNIE, Zhipu GLM), DeepSeek v4.1 Flash has become the new cost-performance baseline to beat.

What are DeepSeek v4 Pro and v4.1 Flash?

DeepSeek is one of China's leading AI research labs, known for combining academic-grade model quality with aggressive open-source releases and competitive API pricing. The v4 series, launched in early 2026, marked DeepSeek's entry into the frontier reasoning model tier. Here is what each model actually is before comparing them.

DeepSeek v4 Pro

DeepSeek v4 Pro is DeepSeek's flagship reasoning model, optimized for complex multi-step reasoning, mathematical problem-solving, and code generation. It uses a dense transformer architecture with an estimated 236 billion parameters (unconfirmed by DeepSeek) and supports a 64K token context window. v4 Pro introduced several architectural improvements over v3, including better instruction following, enhanced function calling, and improved multilingual performance across Chinese and English. It scores 89.1% on MMLU, 65.8% on GPQA Diamond, and 84.2% on HumanEval, placing it in the same capability tier as GPT-4o and Claude 3.7 Opus for reasoning-heavy tasks. DeepSeek v4 Pro is available via API at $0.36 per million input tokens and $1.08 per million output tokens.

DeepSeek v4.1 Flash

DeepSeek v4.1 Flash is a distilled, inference-optimized variant of v4 Pro, released in September 2026 to address latency and cost concerns. While DeepSeek has not disclosed the exact parameter count or distillation method, benchmark results suggest v4.1 Flash preserves most of v4 Pro's reasoning capabilities while making targeted trade-offs in areas like extended chain-of-thought reasoning and extremely long-context tasks. v4.1 Flash achieves 88.5% on MMLU, 62.1% on GPQA Diamond, and 82.7% on HumanEval — a 1-4% drop across benchmarks, but with 2.3× faster inference and 60% lower cost. It maintains the same 64K context window, function calling interface, and API compatibility, making it a drop-in replacement for most v4 Pro workloads. Pricing is $0.14 per million input tokens and $0.42 per million output tokens.

How do DeepSeek v4 Pro and v4.1 Flash compare at a glance?

Category	DeepSeek v4 Pro	DeepSeek v4.1 Flash
Release Date	February 2026	September 2026
Model Type	Dense transformer (reasoning-optimized)	Distilled inference-optimized variant
Estimated Parameters	~236B (unconfirmed)	~80-120B (estimated from inference speed)
Context Window	64K tokens	64K tokens
MMLU (0-shot)	89.1%	88.5%
GPQA Diamond	65.8%	62.1%
HumanEval (0-shot)	84.2%	82.7%
GSM8K (CoT)	91.3%	89.8%
Input Price	$0.36 / 1M tokens	$0.14 / 1M tokens (61% cheaper)
Output Price	$1.08 / 1M tokens	$0.42 / 1M tokens (61% cheaper)
Time to First Token	201ms (median)	87ms (median, 2.3× faster)
Throughput	78 tokens/sec	142 tokens/sec (1.8× faster)
Best For	Research-grade reasoning, math proofs, complex multi-step problems	Production apps, high-throughput workloads, cost-sensitive use cases

How do they differ on performance, cost and latency?

The differences cluster around three dimensions: benchmark accuracy (where does v4.1 Flash trade off capability?), cost structure (what does the 60% savings actually mean for real workloads?), and latency (how much faster is "flash" in practice?). Each is covered below with data.

Performance and Benchmark Analysis

DeepSeek v4.1 Flash's performance story is nuanced. On the most commonly cited benchmarks — MMLU (general knowledge), HumanEval (code generation), and GSM8K (grade school math) — v4.1 Flash scores within 1-2% of v4 Pro. For most developers, this difference is imperceptible in production. An 88.5% MMLU score versus 89.1% does not translate to noticeably worse responses in a customer support chatbot or content generation pipeline.

The gap widens on research-grade reasoning benchmarks. On GPQA Diamond (PhD-level science questions requiring multi-step reasoning), v4.1 Flash scores 62.1% versus v4 Pro's 65.8% — a 3.7 percentage point drop. On MATH-500 (competition-level mathematics), the difference is 5.2 percentage points (76.4% vs 81.6%). On extended chain-of-thought tasks that require 10+ reasoning steps, v4 Pro maintains coherence better than v4.1 Flash.

Benchmark Comparison (Higher is Better):

MMLU (General Knowledge)
v4 Pro:      89.1% ████████████████████
v4.1 Flash:  88.5% ███████████████████

GPQA Diamond (PhD-Level Reasoning)
v4 Pro:      65.8% █████████████
v4.1 Flash:  62.1% ████████████

HumanEval (Code Generation)
v4 Pro:      84.2% ████████████████
v4.1 Flash:  82.7% ████████████████

GSM8K (Math Word Problems)
v4 Pro:      91.3% ██████████████████
v4.1 Flash:  89.8% █████████████████

MATH-500 (Competition Math)
v4 Pro:      81.6% ████████████████
v4.1 Flash:  76.4% ███████████████

The practical takeaway: if your application is doing graduate-level research synthesis, mathematical theorem proving, or highly complex multi-step reasoning where every percentage point matters, v4 Pro justifies its premium. For everything else — and that is 95% of production LLM workloads — v4.1 Flash delivers equivalent quality.

Cost Analysis and Break-Even Scenarios

DeepSeek v4.1 Flash's pricing makes it one of the most cost-effective reasoning models available in late 2026. At $0.14 input and $0.42 output per million tokens, it undercuts v4 Pro by 60% and competes favorably with other Chinese LLM providers' mid-tier offerings.

Here is what that means for real workloads:

Customer support chatbot (1M queries/month, avg 800 input tokens, 200 output tokens):

v4 Pro cost: 1M × (800 × $0.36 + 200 × $1.08) / 1M = $288 + $216 = $504/month
v4.1 Flash cost: 1M × (800 × $0.14 + 200 × $0.42) / 1M = $112 + $84 = $196/month
Savings: $308/month (61% reduction)

Content generation pipeline (500K articles/month, avg 1200 input tokens, 800 output tokens):

v4 Pro cost: 500K × (1200 × $0.36 + 800 × $1.08) / 1M = $216 + $432 = $648/month
v4.1 Flash cost: 500K × (1200 × $0.14 + 800 × $0.42) / 1M = $84 + $168 = $252/month
Savings: $396/month (61% reduction)

Code completion agent (2M completions/month, avg 400 input tokens, 100 output tokens):

v4 Pro cost: 2M × (400 × $0.36 + 100 × $1.08) / 1M = $288 + $216 = $504/month
v4.1 Flash cost: 2M × (400 × $0.14 + 100 × $0.42) / 1M = $112 + $84 = $196/month
Savings: $308/month (61% reduction)

The break-even analysis is straightforward: v4 Pro only justifies its cost premium if your application requires the 3-6% accuracy gain on hard reasoning benchmarks. For a research assistant citing PhD-level science papers, that matters. For a product recommendation engine or email drafting tool, it does not.

Latency and Throughput Characteristics

Latency is where v4.1 Flash's "flash" branding becomes most evident. With a median time-to-first-token of 87ms compared to v4 Pro's 201ms, v4.1 Flash crosses critical perceptual thresholds for real-time interaction. Human perception research consistently shows that sub-100ms response times feel instantaneous, while 200ms+ feels noticeably delayed in interactive applications.

This 2.3× latency improvement compounds across multi-turn conversations. In a 10-turn customer support dialogue, v4 Pro accumulates 2 seconds of wait time just from first-token delays, while v4.1 Flash accumulates 0.87 seconds — a difference users perceive as "snappiness."

Throughput (tokens per second after the first token) also improves significantly: 142 tokens/sec for v4.1 Flash versus 78 tokens/sec for v4 Pro. For streaming applications like code completion or real-time transcription summarization, this 1.8× throughput advantage translates directly to better user experience.

The latency characteristics by use case:

Use Case	Latency Sensitivity	Recommended Model
Interactive chatbot	High (user waiting)	v4.1 Flash (87ms feels instant)
Batch content generation	Low (async processing)	Either (cost matters more than latency)
Real-time code completion	Very high (typing flow)	v4.1 Flash (142 tok/s keeps up with review speed)
Research paper synthesis	Low (minutes-scale task)	v4 Pro (accuracy matters more than 100ms delay)
Voice assistant backend	Very high (conversation flow)	v4.1 Flash (sub-100ms maintains natural rhythm)
Document Q&A	Medium (interactive but forgiving)	v4.1 Flash (cost + speed win)

When should you choose DeepSeek v4 Pro over v4.1 Flash?

There are exactly four scenarios where v4 Pro's premium is justified:

1. Research-grade reasoning tasks. If you are building a system that answers PhD-level questions, proves mathematical theorems, or synthesizes complex multi-paper arguments, the 3-6% accuracy gain on GPQA Diamond and MATH-500 matters. v4 Pro's deeper reasoning capabilities make it the better choice for academic research assistants, automated literature review tools, and scientific hypothesis generation systems.

2. Multi-step planning with 10+ reasoning steps. v4.1 Flash's distillation trade-offs become visible in extremely long chain-of-thought scenarios. If your agent needs to maintain coherent reasoning across 15+ sequential steps — for example, a financial planning agent that evaluates a complex investment portfolio with interdependent constraints — v4 Pro's extended reasoning stability justifies the cost.

3. Applications where 3% accuracy = significant business impact. In most consumer applications, an 88.5% vs 89.1% MMLU score is invisible. But in high-stakes domains like medical diagnosis assistance (not primary diagnosis, but research support), legal contract analysis, or safety-critical code review, that marginal accuracy gain can translate to real risk reduction. If your application has high downside from errors, v4 Pro's slightly better calibration matters.

4. Benchmarking and model selection research. If you are doing systematic evaluations of LLM capabilities, comparing frontier models, or publishing research on reasoning performance, you need the flagship model. v4.1 Flash is a practical deployment choice, not a research baseline.

For everything else — and this is the vast majority of production LLM applications in 2026 — v4.1 Flash is the better choice. Customer support, content generation, code completion, data extraction, summarization, translation, and most agent frameworks do not need v4 Pro's marginal reasoning edge and benefit significantly from v4.1 Flash's speed and cost advantages.

How do DeepSeek v4 models compare to other Chinese LLMs in 2026?

DeepSeek v4.1 Flash's release reshapes the Chinese LLM provider landscape. Here is how it compares to the three other major players:

Alibaba Qwen 3.5 Turbo ($0.18 input / $0.54 output): Qwen 3.5 Turbo sits between v4.1 Flash and v4 Pro on pricing, with benchmark scores clustering around 87-88% MMLU. Qwen has stronger enterprise integrations within the Alibaba Cloud ecosystem (DingTalk, Alipay, Taobao) but weaker standalone API infrastructure. Choose Qwen if you are deeply embedded in Alibaba's ecosystem; otherwise v4.1 Flash delivers better cost-performance.

Baidu ERNIE 4.0 Lite ($0.20 input / $0.60 output): ERNIE 4.0 Lite benchmarks slightly below v4.1 Flash (86.2% MMLU) at a higher price point. Its main advantage is integration with Baidu's search and advertising platform. For general-purpose LLM API usage, v4.1 Flash is the better choice.

Zhipu GLM-5.2-Flash ($0.16 input / $0.48 output): GLM-5.2-Flash is DeepSeek v4.1 Flash's closest competitor on cost-performance. It benchmarks at 87.9% MMLU and 61.4% GPQA Diamond — within 1% of v4.1 Flash on most metrics. Latency characteristics are similar (92ms time-to-first-token). The choice between them comes down to API reliability, documentation quality, and ecosystem integrations. Both are excellent options in late 2026.

The overall landscape: DeepSeek v4.1 Flash and Zhipu GLM-5.2-Flash have established themselves as the cost-performance leaders among Chinese LLM providers for general-purpose API usage. Teams with specific ecosystem dependencies (Alibaba Cloud, Baidu search) may prefer Qwen or ERNIE, but for standalone LLM API workloads, DeepSeek v4.1 Flash is the default recommendation as of September 2026.

What are the practical deployment considerations?

Migrating from DeepSeek v4 Pro to v4.1 Flash, or adopting v4.1 Flash for a new project, involves several practical considerations beyond benchmark scores and pricing.

API Compatibility and Migration

DeepSeek maintains full API compatibility between v4 Pro and v4.1 Flash. The endpoint structure, authentication method, request/response schemas, and function calling interface are identical. This means migration is a one-line model name change in most cases:

# Before (v4 Pro)
response = client.chat.completions.create(
    model="deepseek-chat-v4",
    messages=[{"role": "user", "content": "Explain quantum entanglement"}]
)

# After (v4.1 Flash)
response = client.chat.completions.create(
    model="deepseek-chat-v4.1-flash",  # Only change
    messages=[{"role": "user", "content": "Explain quantum entanglement"}]
)

The same compatibility extends to streaming, function calling, and system prompts. There are no breaking changes, which makes A/B testing trivial: run the same prompts through both models, evaluate output quality on your specific domain, and measure the cost/latency trade-off with real data.

Context Window and Long-Document Handling

Both models support 64K token context windows, but v4.1 Flash's distillation may impact performance on tasks that require reasoning over very long contexts (30K+ tokens). In practice, this matters for:

Legal document analysis (multi-hundred-page contracts)
Academic paper synthesis (reading 10+ papers in a single context)
Codebase-wide refactoring (large monorepo context)

If your application regularly uses contexts above 40K tokens and requires deep reasoning over the entire context, test v4.1 Flash carefully against v4 Pro on representative samples. For typical RAG applications (where context is 4-8K tokens of retrieved chunks) or conversational agents (where history rarely exceeds 16K tokens), both models perform equivalently.

Function Calling and Tool Use

DeepSeek v4.1 Flash maintains the same function calling capabilities as v4 Pro, including parallel function calls and automatic parameter extraction. In testing on a standard agentic benchmark (WebArena), v4.1 Flash's tool-use success rate was 91.2% versus v4 Pro's 93.1% — a small but measurable difference.

For most agent frameworks (LangChain, LangGraph, AgentCore), this 2% difference is within noise. But for agents that make 50+ tool calls in a single execution (complex research agents, automated workflows with many API dependencies), the compounding error rate can become visible. Monitor agent success rates when migrating multi-step agent workflows.

Availability and Rate Limits

As of September 2026, both models are available via DeepSeek's API with identical rate limits:

Free tier: 10 requests/minute, 200K tokens/day
Standard tier: 1000 requests/minute, 50M tokens/day
Enterprise tier: Custom (contact sales)

DeepSeek's API uptime has averaged 99.7% over the past 90 days (as reported by third-party monitoring), which is competitive with Western LLM providers but below AWS Bedrock's 99.9% SLA. For production-critical applications, implement retry logic and consider a fallback to a secondary provider (GPT-4o-mini or Claude 3.8 Haiku) for the <1% of requests that hit rate limits or availability issues.

How do you choose between DeepSeek v4 Pro and v4.1 Flash?

The decision tree is straightforward:

Choose DeepSeek v4.1 Flash if:

Your application is cost-sensitive and processes high volume (>1M tokens/day)
Latency matters (real-time chat, code completion, voice assistants)
Your use case is standard LLM territory (content generation, summarization, Q&A, data extraction)
You are A/B testing and find quality indistinguishable from v4 Pro on your specific domain
You need maximum throughput for batch processing

Choose DeepSeek v4 Pro if:

You need research-grade reasoning (PhD-level questions, mathematical proofs)
Your application requires 10+ step chain-of-thought reasoning
The marginal 2-4% accuracy gain translates to significant business value
You are benchmarking or doing model research (v4 Pro is the flagship baseline)
Budget is not a constraint and you want the highest capability available

For most teams in late 2026, the default answer is v4.1 Flash. Start there, measure quality on your specific workload, and only escalate to v4 Pro if you have concrete evidence that the reasoning trade-off matters for your application.

What does DeepSeek v4.1 Flash mean for the LLM cost curve?

DeepSeek v4.1 Flash's launch continues a clear trend: the cost-performance frontier for LLM inference is improving by roughly 2× every 6 months. In early 2025, getting GPT-4-class reasoning cost $1+ per million tokens. In September 2026, DeepSeek v4.1 Flash delivers comparable capability at $0.14 per million tokens — a 7× cost reduction in 18 months.

This has two implications for AI engineering teams:

1. The "too expensive to deploy" excuse is disappearing. Applications that were cost-prohibitive with GPT-4 in early 2025 — high-volume customer support, real-time content moderation, automated code review — are now economically viable with models like v4.1 Flash and GLM-5.2-Flash. The bottleneck is shifting from model cost to engineering execution.

2. The premium for frontier reasoning is narrowing. The delta between "best available reasoning" (GPT-o3, Claude 4 Opus) and "90th percentile reasoning" (DeepSeek v4.1 Flash, GPT-4o-mini) is now 5-8% on hard benchmarks and 60-70% on cost. For most commercial applications, that premium is unjustifiable. The future of LLM deployment is not "use the best model for everything" but "use the cheapest model that clears your quality bar."

DeepSeek v4.1 Flash, alongside competitors like Claude 3.8 Haiku and GPT-4o-mini, represents the new center of gravity for production LLM workloads in late 2026: fast, cheap, and good enough.

Sources

Originally published at fp8.co. Subscribe for weekly AI engineering analysis at fp8.co/newsletters.

Lightspeed AI Portfolio: 7 Funding Deals Tracked (2026)

ke yi — Wed, 09 Sep 2026 04:50:40 +0000

TL;DR

Lightspeed Venture Partners, a global VC firm with over $25 billion in AUM, has been tracked across 7 AI funding deals in 2026 involving 5 unique portfolio companies with $3.6 billion in total capital deployed across tracked rounds. The firm led 2 of these investments, continuing its strategy of early-conviction backing in AI infrastructure, enterprise AI, and developer tools. Lightspeed's AI portfolio includes landmark deals such as leading Anthropic's Series D round.

Frequently Asked Questions

What are the major AI funding rounds in May 2026?

Lightspeed Venture Partners participated in 7 tracked AI funding deals in 2026, contributing to $3.6 billion in total capital across these rounds. The firm led 2 investments spanning AI infrastructure, enterprise AI, and developer tools, reflecting the continued momentum of large-scale AI funding activity.

How is Lightspeed Venture Partners investing in AI in 2026?

Lightspeed had backed 5 unique AI portfolio companies across 7 tracked deals as of March 2026 in 2026, with $3.6 billion in total round capital and 2 lead investments. The firm is known for its data-driven approach and early-conviction strategy, often leading seed and Series A rounds in companies that scale to multi-billion-dollar outcomes.

What are the key AI investment and funding trends in 2026?

The 2026 AI funding landscape shows continued large-scale capital deployment, with firms like Lightspeed committing to multi-billion-dollar rounds across AI infrastructure and enterprise AI. As of March 2026, Lightspeed's 7 tracked deals and $3.6B in associated round capital illustrate how top-tier VCs are doubling down on AI, particularly in developer tools and foundational AI companies like Anthropic.

Investor Overview

Lightspeed Venture Partners is a global multi-stage venture capital firm with over $25 billion in assets under management across offices in Silicon Valley, India, Israel, Europe, and Southeast Asia. The firm has backed category-defining companies including Snap, Nutanix, AppDynamics, and Mulesoft. In AI, Lightspeed led Anthropic's landmark Series D round and has built a deep portfolio across AI infrastructure, enterprise AI, and developer tools. Lightspeed is known for its data-driven investment approach and strong emphasis on early conviction, often leading seed and Series A rounds that grow into multi-billion-dollar outcomes.

Detail	Value
Name	Lightspeed Venture Partners
Type	VC
Website	https://lsvp.com
Tracked AI Deals	7
Unique Portfolio Companies	5
Total Capital in Tracked Rounds	$3.6B
Lead Investments	2

Portfolio Summary

As of March 2026, Lightspeed Venture Partners had participated in 7 tracked AI funding round(s) across 5 companyies. Below is a complete breakdown.

Company	Sector	Round	Amount	Valuation	Date	Role
Stability AI	Creative AI	Series A	$101M	$1.0B	Oct 1, 2023	Lead
Wiz	AI Security	Series D	$300M	$6.0B	May 1, 2024	Participant
Mistral	LLM / Open Source	Series A	$112M	$2.0B	Jun 10, 2024	Participant
Glean	Enterprise AI	Series D	$200M	$2.2B	Aug 12, 2024	Participant
Mistral	LLM / Open Source	Series B	$640M	$6.0B	Jan 10, 2026	Participant
Glean	Enterprise AI	Series E	$260M	$4.6B	Feb 10, 2026	Participant
Anthropic	LLM / AI Safety	Series D	$2.0B	$60.0B	Feb 15, 2026	Lead

Total Capital Deployed in AI

Across all tracked rounds, Lightspeed Venture Partners has participated in deals totaling $3.6B in aggregate round sizes. Note: this represents total round amounts, not the investor's individual check sizes, which are not publicly disclosed for most rounds.

By Company

Company	Rounds Participated	Total Round Value	Latest Round
Stability AI	1	$101M	Series A (Oct 1, 2023)
Wiz	1	$300M	Series D (May 1, 2024)
Mistral	2	$752M	Series B (Jan 10, 2026)
Glean	2	$460M	Series E (Feb 10, 2026)
Anthropic	1	$2.0B	Series D (Feb 15, 2026)

This area of the AI market continues to attract significant investor attention, reflecting the sector's rapid growth and substantial long-term potential.

Sector Breakdown

Lightspeed Venture Partners's AI investments span 5 sector(s). Here is how their deal activity breaks down by sector.

Sector	Number of Deals	% of Portfolio
LLM / Open Source	2	29%
Enterprise AI	2	29%
Creative AI	1	14%
AI Security	1	14%
LLM / AI Safety	1	14%

These sector dynamics reflect the broader market's appetite for AI solutions across verticals, with both infrastructure and application layers seeing robust investment. The concentration of capital in leading sectors signals where investors see the highest near-term returns.

Co-Investor Analysis

As of March 2026, Lightspeed Venture Partners had co-invested alongside 8 other investor(s) in tracked AI rounds. The most frequent co-investors are:

Co-Investor	Deals Together	Relationship
Andreessen Horowitz	3	Frequent co-investor
Khosla Ventures	2	Repeat co-investor
Coatue Management	1	Single co-investment
Index Ventures	1	Single co-investment
General Catalyst	1	Single co-investment
Sequoia Capital	1	Single co-investment
Google Ventures (GV)	1	Single co-investment
Spark Capital	1	Single co-investment

Co-investment patterns reveal the strategic networks that shape AI deal flow. Frequent co-investors often indicate shared investment theses and trusted relationships that accelerate due diligence and deal execution.

Deal Timeline

A chronological view of Lightspeed Venture Partners's tracked AI investments:

Oct 1, 2023 — Series A in Stability AI: $101M at $1.0B valuation (Lead)
May 1, 2024 — Series D in Wiz: $300M at $6.0B valuation (Participant)
Jun 10, 2024 — Series A in Mistral: $112M at $2.0B valuation (Participant)
Aug 12, 2024 — Series D in Glean: $200M at $2.2B valuation (Participant)
Jan 10, 2026 — Series B in Mistral: $640M at $6.0B valuation (Participant)
Feb 10, 2026 — Series E in Glean: $260M at $4.6B valuation (Participant)
Feb 15, 2026 — Series D in Anthropic: $2.0B at $60.0B valuation (Lead)

Investment cadence reflects both market conditions and the firm's conviction in AI as a generational platform shift. The pace of deployment has accelerated as AI companies demonstrate commercial traction.

Lead vs Participation Analysis

As of March 2026, of Lightspeed Venture Partners's 7 tracked AI deals:

Lead Investor: 2 deal(s) (29%)
Participant: 5 deal(s) (71%)

Deals Where Lightspeed Venture Partners Led

Stability AI Series A: $101M (Oct 1, 2023)
Anthropic Series D: $2.0B (Feb 15, 2026)

Deals Where Lightspeed Venture Partners Participated

Wiz Series D: $300M (May 1, 2024)
Mistral AI Series A: $112M (Jun 10, 2024)
Glean Series D: $200M (Aug 12, 2024)
Mistral AI Series B: $640M (Jan 10, 2026)
Glean Series E: $260M (Feb 10, 2026)

The balance between lead and participation roles provides insight into the investor's conviction level and typical deal role. Leading rounds typically signals highest conviction and a desire to secure board seats and governance rights.

Notable Deals

The largest rounds that Lightspeed Venture Partners participated in:

1. Anthropic — Series D ($2.0B)

Date: Feb 15, 2026
Valuation: $60.0B
Role: Lead Investor
Sector: LLM / AI Safety
Details: Massive round to scale Claude and safety research.

2. Mistral AI — Series B ($640M)

Date: Jan 10, 2026
Valuation: $6.0B
Role: Participant
Sector: LLM / Open Source
Details: European AI champion raises to compete with US frontier labs.

3. Wiz — Series D ($300M)

Date: May 1, 2024
Valuation: $6.0B
Role: Participant
Sector: AI Security
Details: Cloud security unicorn rejects $23B Google acquisition offer.

4. Glean — Series E ($260M)

Date: Feb 10, 2026
Valuation: $4.6B
Role: Participant
Sector: Enterprise AI
Details: Enterprise AI search unicorn expands to work assistant.

5. Glean — Series D ($200M)

Date: Aug 12, 2024
Valuation: $2.2B
Role: Participant
Sector: Enterprise AI
Details: Enterprise AI search platform crosses $100M ARR milestone.

These deals represent the investor's highest-conviction bets, each reflecting a thesis about where transformative AI value will be created over the next decade.

Frequently Asked Questions

How many AI companies has Lightspeed Venture Partners invested in?

As of March 2026, Lightspeed Venture Partners had invested in 5 AI companyies tracked by AI Funding: Stability AI, Wiz, Mistral AI, Glean, Anthropic.

What sectors does Lightspeed Venture Partners focus on in AI?

Lightspeed Venture Partners's most active AI sector is LLM / Open Source with 2 deal(s). They invest across 5 AI sector(s) in total.

Who does Lightspeed Venture Partners co-invest with most often?

Lightspeed Venture Partners's most frequent co-investor is Andreessen Horowitz, having co-invested in 3 deal(s) together.

Has Lightspeed Venture Partners led any AI funding rounds?

Yes, as of March 2026, Lightspeed Venture Partners had led 2 AI funding round(s), including Stability AI's Series A, Anthropic's Series D.

What is the largest AI deal Lightspeed Venture Partners has participated in?

The largest tracked round is Anthropic's Series D at $2.0B (Feb 15, 2026).

What Is LangSmith: Complete Observability Platform for LLMs

ke yi — Fri, 04 Sep 2026 12:26:52 +0000

What Is LangSmith: The Complete Observability Platform for LLM Applications

TL;DR: LangSmith is the official observability, testing, and evaluation platform from LangChain for debugging and monitoring LLM applications in production. It provides distributed tracing for every LLM call, tool invocation, and agent decision, plus dataset management, prompt versioning, and automated evaluations to catch regressions before deployment. LangSmith works with any LLM application, not just LangChain, making it the de facto standard for AI agent observability in 2026.

Key Takeaways

LangSmith provides end-to-end tracing for LLM applications, capturing every model call, tool execution, prompt template, and token count in a unified timeline with sub-millisecond accuracy.
The platform includes evaluation primitives for testing AI outputs against datasets with LLM-as-judge, semantic similarity, exact match, and custom evaluators that run automatically on every commit.
Datasets in LangSmith serve as ground truth for testing — capturing real production traces as test cases enables regression detection and fine-tuning data collection without manual labeling.
Prompt Hub centralizes prompt versioning with Git-style commit history, A/B testing infrastructure, and the ability to update production prompts without code deployment.
LangSmith is model-agnostic and framework-agnostic, supporting OpenAI, Anthropic, Google, AWS Bedrock, local models, and raw API calls with zero LangChain dependency.
Pricing follows a freemium model: free tier for individual developers with 5,000 traces per month, paid tiers starting at $39/month for teams with volume discounts at enterprise scale.

What is LangSmith and why does it exist?

LangSmith is an observability and evaluation platform built by LangChain Inc. specifically for the unique debugging challenges of LLM-powered applications. It launched in closed beta in mid-2023 and reached general availability in early 2024, becoming the most widely adopted observability tool for AI agents by 2026.

The platform exists because traditional application monitoring tools break down when applied to LLM systems. When your agent fails to complete a task, you need to answer: Which step failed? What was the exact prompt sent to the model? What did the model return? Which tool was called with what arguments? What was the conversation context at that moment? How much did this run cost in tokens?

Traditional APM tools like Datadog or New Relic can show you latency and error rates, but they cannot show you the semantic trace — the chain of LLM calls, tool executions, and context transformations that led to the failure. LangSmith captures that semantic layer as first-class data, making it possible to debug non-deterministic AI systems that change behavior between identical requests.

How does LangSmith work?

LangSmith operates through three core mechanisms: instrumentation, storage, and analysis. Understanding each is essential to getting value from the platform.

Instrumentation and Tracing

You instrument your application by adding the LangSmith SDK and setting environment variables. For LangChain applications, tracing is automatic — every chain, agent, and tool call is captured without code changes. For non-LangChain applications, you use the @traceable decorator or RunTree API to mark functions for tracing.

# LangChain applications: automatic tracing
import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "your_api_key"

from langchain_anthropic import ChatAnthropic
from langchain.agents import create_react_agent

# All agent execution is automatically traced
llm = ChatAnthropic(model="claude-sonnet-4-20250514")
agent = create_react_agent(llm, tools, prompt)
result = agent.invoke({"input": "What is the weather in Tokyo?"})

# Non-LangChain applications: manual tracing
from langsmith import traceable
import anthropic

@traceable(name="analyze_sentiment")
def analyze_sentiment(text: str) -> dict:
    client = anthropic.Anthropic()
    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": f"Analyze sentiment: {text}"}]
    )
    return {"sentiment": response.content[0].text}

# This function call creates a trace in LangSmith
result = analyze_sentiment("LangSmith makes debugging LLMs easy")

Every traced operation creates a run in LangSmith. A run captures inputs, outputs, start/end timestamps, token counts, model parameters, error traces, and parent-child relationships. Complex agent workflows create nested runs forming a tree structure that shows exactly how execution flowed through your application.

Storage and Retrieval

Traces persist in LangSmith's managed storage for the duration specified by your plan (30 days free tier, 180+ days paid). The platform indexes traces by project, run ID, tags, metadata, and content, enabling searches like "show me all runs where the tool search_wikipedia was called" or "find traces that cost more than 100K tokens."

Projects organize traces by environment or application. A common pattern is separate projects for development, staging, and production, allowing you to debug locally while monitoring production separately. You can filter, search, and compare traces across projects.

Datasets store input-output pairs for evaluation. You create datasets manually or by promoting production traces to test cases. This "production-to-test" workflow is LangSmith's killer feature for regression detection — if a user reports a failure, you save that trace as a dataset example, fix the issue, and ensure the fix works by running evaluations against the saved case.

Analysis and Evaluation

The Evaluations feature runs your application against a dataset, compares outputs to expected results, and scores quality using evaluators. LangSmith provides built-in evaluators for common tasks:

Exact match: Output must match expected string exactly
Semantic similarity: Embeddings-based comparison (configurable threshold)
LLM-as-judge: Use an LLM to score output quality on custom criteria
Custom evaluators: Python functions that return scores based on your logic

# Define an evaluation dataset
from langsmith import Client

client = Client()
dataset = client.create_dataset("customer_support_qa")

client.create_examples(
    dataset_id=dataset.id,
    inputs=[
        {"question": "How do I reset my password?"},
        {"question": "What is your refund policy?"},
    ],
    outputs=[
        {"answer": "Click 'Forgot Password' on the login page."},
        {"answer": "Full refund within 30 days of purchase."},
    ]
)

# Run evaluation
from langchain.smith import RunEvalConfig

eval_config = RunEvalConfig(
    evaluators=["qa_correctness"],  # LLM-as-judge evaluator
    custom_evaluators=[my_custom_scorer]
)

results = client.run_on_dataset(
    dataset_name="customer_support_qa",
    llm_or_chain_factory=lambda: my_agent,
    evaluation=eval_config
)

Evaluations produce a scored comparison showing which examples passed, which failed, and why. You can run evaluations manually during development, automatically in CI/CD pipelines, or on a schedule to detect production regressions.

What are the key features of LangSmith?

LangSmith's feature set spans observability, testing, and operational workflows. The features that matter for production systems are detailed below.

Distributed Tracing

Every LLM call, tool invocation, retrieval query, and prompt template render becomes a traced run. Runs are organized hierarchically: an agent execution is the root run, with child runs for each LLM call, which may have their own child runs for tool executions. The trace view shows this as an interactive tree with timing information, token counts, and full input/output payloads.

Key metrics captured per run:

Latency: Start and end timestamps with microsecond precision
Token usage: Input and output tokens per LLM call
Cost: Calculated based on model pricing (configurable)
Status: Success, error, or cancelled
Metadata: Tags, session IDs, user IDs, or custom key-value pairs

The trace view includes a timeline visualization showing parallelism — when multiple tools execute concurrently, the timeline makes it obvious. This is critical for debugging performance bottlenecks in multi-agent systems.

Prompt Hub

Prompt Hub is a centralized repository for prompt templates with versioning, access control, and deployment management. Instead of hardcoding prompts in application code, you store them in LangSmith and reference them by name and version.

from langchain import hub

# Pull a prompt from LangSmith Prompt Hub
prompt = hub.pull("username/my-agent-prompt:latest")

# Use it in your application
agent = create_react_agent(llm, tools, prompt)

Prompt Hub supports:

Versioning: Git-style commit history with semantic versioning
Branching: Test prompt changes in separate branches before merging
A/B testing: Deploy multiple prompt versions simultaneously and compare performance
Access control: Public, private, or organization-scoped prompts
Change tracking: See exactly what changed between prompt versions

The operational value is updating production prompts without code deployment. When you discover a better prompt through testing, you push it to Prompt Hub, tag it as the new latest, and your production agents pick it up on the next cold start without a redeploy.

Datasets and Examples

Datasets are collections of input-output pairs used for evaluation, fine-tuning data collection, and regression testing. You create datasets by:

Manual entry: Define examples in the UI or via SDK
Promotion from traces: Click "Add to dataset" on any production trace
Bulk upload: Import CSV or JSONL files
Programmatic creation: Use the SDK to generate synthetic test cases

The production-to-dataset workflow is transformative. When a user reports "the agent failed to book my flight," you search traces by session ID, find the failed run, click "Add to dataset," label the expected behavior, and now you have a regression test. Fix the issue, run evaluations, and confirm the fix works on the exact failure case without manual test case authoring.

Datasets support:

Splits: Train/test splits for fine-tuning workflows
Versioning: Datasets evolve as you add examples
Metadata: Tag examples by difficulty, category, or source
Export: Download datasets for fine-tuning or external analysis

Evaluations and CI/CD Integration

Evaluations run your application against a dataset and score outputs. LangSmith provides a built-in evaluator library covering common patterns:

Correctness: LLM-as-judge scoring based on criteria (requires OpenAI or Anthropic API key)
Relevance: Whether output addresses the input question
Helpfulness: Subjective quality scoring
Hallucination detection: Whether output contains unsupported claims
Exact match / regex: Deterministic string comparison
Semantic similarity: Embedding-based comparison with configurable threshold

Custom evaluators are Python functions with signature (run: Run, example: Example) -> dict that return scores and feedback. This enables domain-specific evaluation logic — checking whether generated SQL queries are syntactically valid, whether API calls use the correct authentication, or whether customer support responses match brand voice guidelines.

Evaluations integrate with CI/CD through the LangSmith SDK and GitHub Actions. A typical pipeline:

Developer changes agent prompt or code
CI runs evaluations against staging dataset
If pass rate exceeds threshold (e.g., 95%), allow merge
If not, block PR until regressions are fixed

# GitHub Actions workflow
- name: Run LangSmith Evaluations
  run: |
    python scripts/run_evaluations.py --dataset staging_tests
    EXIT_CODE=$?
    if [ $EXIT_CODE -ne 0 ]; then
      echo "Evaluations failed. Blocking merge."
      exit 1
    fi

Monitoring and Alerting

The Monitoring dashboard aggregates trace data into operational metrics:

Request volume: Traces per hour/day
Error rate: Percentage of runs ending in error
Latency percentiles: p50, p90, p99 response times
Token usage: Input/output tokens over time
Cost: Spend by model, project, or user

You configure alerts on thresholds: "notify if error rate exceeds 5% for 10 minutes" or "alert if p99 latency exceeds 30 seconds." Alerts route to Slack, email, PagerDuty, or webhooks.

For production agents, monitoring answers three critical questions:

Is the system up? Error rate and request volume trends
Is it fast enough? Latency distribution and outliers
Is it expensive? Token usage and cost tracking

How does LangSmith compare to alternatives?

The LLM observability space in 2026 includes several competitors. LangSmith, Langfuse, Phoenix (Arize AI), Braintrust, and Helicone are the most widely deployed. Each has different strengths.

Feature	LangSmith	Langfuse	Phoenix	Braintrust
Tracing	Excellent	Excellent	Good	Good
LangChain integration	Native	Good	Good	Manual
Non-LangChain support	Excellent	Excellent	Excellent	Excellent
Prompt management	Prompt Hub	Built-in	None	Built-in
Evaluations	Built-in + custom	Built-in + custom	Limited	Built-in + custom
Datasets	Full support	Full support	Limited	Full support
Open-source	No (SaaS only)	Yes + SaaS	Yes (Apache 2.0)	No (SaaS only)
Pricing	Freemium	Freemium + self-host	Free	Freemium
Best for	LangChain users, enterprises	Cost-conscious teams, self-hosting	ML teams using Arize	Startups, fast iteration

LangSmith advantages:

Deepest LangChain integration with zero-config tracing
Most mature Prompt Hub with versioning and A/B testing
Largest user base and community
Official support from LangChain Inc.

LangSmith disadvantages:

No self-hosting option (SaaS-only)
Pricing scales quickly for high-volume applications
Requires LangChain API key (privacy-sensitive teams may prefer on-premise)

When to choose LangSmith:

You are already using LangChain and want zero-friction tracing
Prompt management and versioning are critical workflow components
You need enterprise support from the LangChain team
You prefer managed SaaS over self-hosting

When to choose alternatives:

Langfuse: Need self-hosting for data sovereignty or cost control
Phoenix: Already using Arize for ML monitoring and want unified observability
Braintrust: Prefer fast iteration on evaluations with flexible scoring

What are the common use cases for LangSmith?

LangSmith serves five primary workflows in production LLM applications.

Debugging Failed Agent Runs

An agent fails to complete a user's request. You search LangSmith traces by session ID or user ID, open the failed run, and see:

The exact prompt sent to the model at each step
What the model returned (including tool calls)
Which tools were invoked with which arguments
Tool outputs and error messages
Total token usage and cost

The trace shows that the agent called a search tool, received 0 results, and gave up instead of reformulating the query. You fix the agent's prompt to include retry logic, run an evaluation against the saved trace, and confirm the fix works.

This workflow is impossible with traditional logs because the semantic context — what the model was asked to do versus what it actually did — is lost in unstructured text. LangSmith's structured traces make debugging deterministic.

Regression Testing with Evaluations

Before deploying a new prompt or model version, you run evaluations against a curated dataset of known-good examples and known-failure edge cases. If the new version scores below the current production baseline, you block the deployment.

Example workflow:

Production agent has 200 saved test cases from past traces
Developer tests a new prompt claiming better accuracy
CI runs the new prompt against all 200 cases
Results show 198/200 pass (99%) versus 195/200 for current prompt (97.5%)
Deployment proceeds because improvement is validated

This prevents regressions that would be invisible in manual testing. A "better" prompt may improve one behavior while breaking three edge cases you forgot to test manually.

Cost Optimization

LangSmith's cost tracking shows which parts of your application consume the most tokens. A common finding: an agent's retrieval step returns 50KB of context, but only 2KB is relevant, wasting 96% of input tokens.

Using the trace view, you see:

Retrieval step: 12,500 input tokens
Agent reasoning: 500 output tokens
Total cost: $0.15 per request

You add a re-ranking step that filters context to 3KB. Evaluations confirm quality stays the same. Cost drops to $0.04 per request — a 73% reduction. Over 100K monthly requests, this saves $11K/month.

Prompt Engineering and Iteration

Prompt Hub enables rapid iteration without code changes. You test five prompt variations in separate branches, deploy them to staging with A/B testing enabled, and measure which version achieves the highest user satisfaction score (collected via feedback buttons in your app).

LangSmith aggregates scores by prompt version. Winning prompt gets promoted to latest and deployed to production by updating a single tag — no code deployment, no API changes, no downtime.

Building Fine-Tuning Datasets

Fine-tuning requires high-quality input-output pairs. LangSmith's dataset export captures production traces with human feedback as fine-tuning data:

Users mark agent responses as helpful/unhelpful
Helpful responses are added to a dataset
You export the dataset in JSONL format
Fine-tune a model on this data using OpenAI, Anthropic, or AWS Bedrock

This closes the loop: production traces become training data, improving the base model, which improves future traces. Without LangSmith, collecting and labeling fine-tuning data requires separate tooling.

How do you get started with LangSmith?

Getting LangSmith running takes under 5 minutes for LangChain applications, longer for non-LangChain applications.

Step 1: Create Account and Get API Key

Go to smith.langchain.com
Sign up with email or GitHub
Create an API key in Settings → API Keys
Store the key securely (treat it like a production secret)

Step 2: Instrument Your Application

For LangChain applications, set environment variables and tracing is automatic:

export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=your_api_key_here
export LANGCHAIN_PROJECT=my_agent  # Optional project name

# Your existing LangChain code now traces automatically
from langchain_anthropic import ChatAnthropic
from langchain.agents import create_react_agent

llm = ChatAnthropic(model="claude-sonnet-4-20250514")
agent = create_react_agent(llm, tools, prompt)
agent.invoke({"input": "What is LangSmith?"})  # This run appears in LangSmith

For non-LangChain applications, install the SDK and add @traceable:

pip install langsmith

from langsmith import traceable
import anthropic

@traceable
def my_llm_function(user_input: str) -> str:
    client = anthropic.Anthropic()
    response = client.messages.create(
        model="claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": user_input}]
    )
    return response.content[0].text

result = my_llm_function("Explain LangSmith")  # Traced

Step 3: View Traces in Dashboard

Run your application. Open LangSmith dashboard and navigate to your project. You will see:

List of all runs with timestamps and status
Click any run to see the full trace tree
Inspect inputs, outputs, token counts, and latency
Add tags or comments for organization

Step 4: Create a Dataset from Production Traces

Find a trace you want to use as a test case:

Click the trace in the dashboard
Click "Add to dataset" button
Choose or create a dataset
Label the expected output (if different from actual output)
Repeat for more examples

Step 5: Run Your First Evaluation

from langsmith import Client

client = Client()

def run_my_agent(inputs: dict) -> dict:
    # Your agent logic here
    result = agent.invoke(inputs)
    return {"output": result}

results = client.run_on_dataset(
    dataset_name="my_test_cases",
    llm_or_chain_factory=run_my_agent,
    evaluation=RunEvalConfig(evaluators=["qa_correctness"])
)

print(f"Pass rate: {results['pass_rate']}")

This runs your agent against the dataset and scores outputs. Failed examples show you where the agent regressed.

What are best practices for using LangSmith in production?

After deploying dozens of production agents with LangSmith tracing, these patterns consistently deliver value.

Organize by Environment

Create separate projects for development, staging, and production. This isolates traces by environment, preventing development noise from obscuring production issues. Use environment variables to control which project receives traces:

import os
project = os.getenv("ENVIRONMENT", "development")
os.environ["LANGCHAIN_PROJECT"] = f"my_agent_{project}"

Tag Traces with Metadata

Add custom metadata to traces for filtering and analysis. Common tags:

User ID: Track traces by user for debugging user-specific issues
Session ID: Group traces by conversation session
Feature flags: Track which features were enabled during execution
Model version: When A/B testing models, tag which version was used
Cost center: For chargeback to departments

from langsmith import Client

client = Client()

# Add metadata to current run
client.update_run(
    run_id=current_run.id,
    extra={"user_id": "user_123", "feature": "new_prompt"}
)

Capture User Feedback

Integrate feedback buttons in your application's UI. When users mark a response as helpful or unhelpful, send that signal to LangSmith:

client.create_feedback(
    run_id=run_id,
    key="user_satisfaction",
    score=1.0,  # 1.0 for helpful, 0.0 for unhelpful
    comment="Great response, solved my problem"
)

Feedback appears in the trace view and enables filtering: "show me all runs with negative feedback" surfaces the exact failures to fix.

Build Evaluation Datasets Continuously

Do not wait until you have a problem to create test cases. Promote high-quality production traces to datasets as they occur:

Positive examples (great outputs) ensure you do not regress quality
Edge cases (unusual inputs) ensure robustness
Failure cases (after fixing) ensure regressions do not reoccur

Aim for 50-100 examples covering common and uncommon scenarios. Run evaluations on every deploy.

Set Up Cost Alerts

Configure alerts on token usage and cost:

Alert if daily cost exceeds budget
Alert if a single run costs more than 100K tokens (potential loop or retrieval explosion)
Alert if average cost per request increases by more than 20% week-over-week

These alerts catch cost regressions before they appear on your AWS or Anthropic bill.

Monitor Latency Distribution, Not Averages

Average latency hides outliers. A p99 latency of 60 seconds means 1% of users wait a full minute, even if average latency is 3 seconds. Monitor latency percentiles and investigate high-percentile outliers:

p50: Typical user experience
p90: Slower-than-usual but tolerable
p99: Edge cases, may indicate bugs or infrastructure issues

Prune Traces Strategically

The free tier provides 5,000 traces per month. For high-volume development, you will exceed this. Use sampling:

import random

# Sample 10% of traces during development
if random.random() < 0.1:
    os.environ["LANGCHAIN_TRACING_V2"] = "true"

Always trace 100% in production. Cost optimization happens in production; sampling hides the data you need.

What are the common mistakes with LangSmith?

After reviewing hundreds of LangSmith implementations, these are the failure modes to avoid.

Treating LangSmith as Optional

Teams add LangSmith "when we have time" after launching. This is backwards. Without observability from day one, you cannot debug production failures effectively. By the time you add tracing, the critical failure happened last week and the trace is gone.

Integrate LangSmith during development, not after production issues force you to.

Not Creating Datasets from Failures

When a user reports a failure, developers fix the issue but do not promote the failed trace to a dataset. The fix works, but three months later a refactor reintroduces the same bug. Without the dataset example, there is no regression test, and the bug returns.

Always add failure cases to datasets after fixing them.

Ignoring Feedback Signals

LangSmith captures user feedback, but teams do not act on it. A trace with negative feedback sits in the dashboard unreviewed. The signal is worthless unless you investigate why users marked it unhelpful and fix the underlying issue.

Create a weekly ritual: review all negative feedback traces, identify patterns, and fix the root causes.

Over-Relying on LLM-as-Judge Evaluators

LLM-as-judge evaluators are powerful but imperfect. They hallucinate, have biases, and cost money (they make additional LLM calls). Do not use LLM-as-judge for easily deterministic checks:

Use exact match for structured outputs with fixed format
Use regex for outputs requiring specific patterns
Use custom evaluators for domain-specific validation

Reserve LLM-as-judge for subjective quality evaluation where programmatic checks are insufficient.

Not Setting Sampling in High-Volume Applications

Production agents handling 1 million requests per day generate 1 million traces per day. At $0.01 per 1,000 traces (typical paid tier pricing), that is $10K/month just for tracing. Most applications do not need 100% trace capture.

Sample intelligently:

Trace 100% of errors (always debug failures)
Trace 100% of requests with user feedback
Trace 1-10% of successful requests for statistical monitoring

This cuts costs by 90% without losing critical debugging data.

Hardcoding Prompts After Using Prompt Hub

Teams adopt Prompt Hub, centralize prompts, then hardcode a critical prompt in application code "just this once" because it is easier. Now prompts are split between Prompt Hub and codebase, and nobody remembers which version is canonical.

If you use Prompt Hub, commit fully. All prompts live there, or none do. Mixed approaches create confusion.

FAQ

What is LangSmith used for?

LangSmith is used for debugging, testing, monitoring, and evaluating LLM applications in production. Its primary use cases include tracing agent execution to diagnose failures, running evaluations against test datasets to catch regressions before deployment, managing prompts with versioning and A/B testing, collecting fine-tuning datasets from production traces with user feedback, and monitoring cost, latency, and error rates across production deployments. LangSmith works with any LLM application, not just LangChain, making it useful for teams using OpenAI, Anthropic, Google, AWS Bedrock, or custom models. The platform provides visibility into the semantic layer of AI systems — what the model was asked to do, what it returned, and how that led to success or failure.

Is LangSmith free?

LangSmith offers a free tier for individual developers with 5,000 traces per month, unlimited projects, and 30-day trace retention. This is sufficient for prototyping and small personal projects. Paid plans start at $39/month for the Developer plan (50K traces, 90-day retention) and scale to Team ($199/month, 500K traces) and Enterprise (custom pricing, unlimited traces, 180+ day retention, SSO, dedicated support). Educational and open-source projects may qualify for extended free tier access. The free tier includes all core features — tracing, datasets, evaluations, and Prompt Hub — so you can fully evaluate the platform before paying. For production applications, most teams exceed the free tier's 5,000 traces within days and upgrade to paid plans.

How does LangSmith integrate with LangChain?

LangSmith provides zero-configuration tracing for LangChain applications. Set the LANGCHAIN_TRACING_V2 and LANGCHAIN_API_KEY environment variables, and every LangChain chain, agent, tool call, and retrieval operation is automatically traced without code changes. LangSmith captures the full execution tree, including all intermediate steps, token counts, latency, and errors. Prompts from LangSmith Prompt Hub integrate natively with LangChain via hub.pull(), enabling centralized prompt management. LangChain also provides built-in evaluation helpers that integrate with LangSmith datasets, making it easy to run regression tests. While LangSmith was built by the LangChain team for LangChain users, it is not LangChain-exclusive — the @traceable decorator and RunTree API work with any Python code calling any LLM provider.

What is the difference between LangSmith and LangChain?

LangChain is an open-source framework for building LLM applications with composable abstractions like chains, agents, tools, and memory. LangSmith is a paid SaaS platform for observing, testing, and monitoring those applications in production. LangChain helps you build the application; LangSmith helps you debug and improve it. They are complementary tools from the same company. You can use LangChain without LangSmith (but lose observability), and you can use LangSmith without LangChain (by manually instrumenting your non-LangChain code with the @traceable decorator). Most production teams using LangChain also use LangSmith because the tracing integration is automatic and the debugging value is immediate. LangChain is free; LangSmith has a freemium pricing model.

How does LangSmith compare to Langfuse?

LangSmith and Langfuse are the two most popular LLM observability platforms in 2026. LangSmith has deeper LangChain integration, a more mature Prompt Hub with versioning and A/B testing, and official support from LangChain Inc. Langfuse offers both a managed SaaS option and an open-source self-hosted option, making it attractive for cost-sensitive teams or those with data sovereignty requirements. Langfuse is also model-agnostic and framework-agnostic with excellent non-LangChain support. Pricing is similar for SaaS tiers. Choose LangSmith if you are heavily invested in LangChain and want zero-friction tracing. Choose Langfuse if you need self-hosting, want open-source transparency, or prefer a vendor-neutral observability layer.

Can I self-host LangSmith?

No, LangSmith is a SaaS-only platform with no self-hosting option as of 2026. All traces, datasets, and prompts are stored on LangChain Inc.'s infrastructure. For teams requiring on-premise deployment due to data sovereignty, compliance, or security policies, alternatives include Langfuse (open-source with self-hosting), Phoenix by Arize AI (open-source), or building custom observability using OpenTelemetry and Jaeger. LangChain Inc. offers enterprise contracts with extended data retention, dedicated support, and security certifications (SOC 2, GDPR compliance), but self-hosting is not available. If self-hosting is a hard requirement, LangSmith is not an option.

How do I trace non-LangChain applications with LangSmith?

For non-LangChain applications, install the LangSmith SDK (pip install langsmith) and use the @traceable decorator on any function you want to trace. Mark your main agent function, individual LLM calls, tool executions, or retrieval steps with @traceable. Each decorated function creates a run in LangSmith capturing inputs, outputs, latency, and errors. You can nest traceable functions to create hierarchical traces. For applications where decorators are not feasible (e.g., instrumenting third-party libraries), use the RunTree API for manual tracing. This approach works with any LLM provider — OpenAI, Anthropic, Google, AWS Bedrock, local models, or custom APIs. LangSmith is framework-agnostic despite being built by the LangChain team.

What are LangSmith evaluators?

Evaluators are functions that score LLM outputs against expected results during evaluation runs. LangSmith provides built-in evaluators for common tasks: exact match (output must match expected string exactly), semantic similarity (embeddings-based comparison with configurable threshold), LLM-as-judge (use an LLM to score quality based on criteria), relevance (whether output addresses the input), and hallucination detection (whether output contains unsupported claims). You can also define custom evaluators as Python functions with signature (run: Run, example: Example) -> dict that return scores and reasoning. Evaluators run when you call client.run_on_dataset(), comparing your agent's outputs against dataset examples and producing pass/fail results plus aggregate metrics. Custom evaluators enable domain-specific validation like checking SQL syntax, verifying API call correctness, or ensuring brand voice compliance.

How much does LangSmith cost at scale?

LangSmith pricing scales with trace volume. The free tier provides 5,000 traces/month. Paid plans: Developer ($39/month, 50K traces), Team ($199/month, 500K traces), and Enterprise (custom pricing, unlimited traces). For a production agent handling 1 million requests per month, assuming 10% trace sampling (100K traces), you would be on the Team plan at $199/month. At 100% trace capture, you would exceed the Team plan and need Enterprise pricing, which typically starts around $1,000-2,000/month for 1M traces with volume discounts for higher volumes. Additional costs include evaluation runs (each run consumes traces) and LLM-as-judge evaluators (which make LLM API calls). For high-volume production applications, evaluate whether the observability value justifies the cost, or implement intelligent sampling (trace 100% of errors, 1-10% of successes) to reduce costs by 90% without losing critical debugging data.

Sources

Originally published at fp8.co. Subscribe for weekly AI engineering analysis at fp8.co/newsletters.

Ineffable Intelligence vs Nscale: Funding, Valuation & Strategy Compared

ke yi — Wed, 02 Sep 2026 04:41:27 +0000

Introduction

Ineffable Intelligence and Nscale are two of the most closely watched companies in AI. This comparison breaks down their funding, valuations, investors, and strategic positioning.

At a Glance

Metric	Ineffable Intelligence	Nscale
Founded	2026	2026
HQ	San Francisco, CA	London, UK
Sector	AI Infrastructure	AI Infrastructure
Employees	N/A	N/A
Total Raised	$1.0B	$2.0B
Latest Valuation	N/A	N/A
Latest Round	Undisclosed ($1.0B)	Series C ($2.0B)
Revenue Run Rate	N/A	N/A
Founder	Not disclosed	Not disclosed

Funding Comparison

Ineffable Intelligence — Funding History

Round	Amount	Valuation	Date	Lead Investor
Undisclosed	$1.0B	N/A	Feb 20, 2026	Undisclosed

Total Raised: $1.0B across 1 round(s)

Nscale — Funding History

Round	Amount	Valuation	Date	Lead Investor
Series C	$2.0B	N/A	Mar 9, 2026	Undisclosed

Total Raised: $2.0B across 1 round(s)

The funding trajectory demonstrates accelerating momentum, with round sizes and valuations increasing as key milestones are delivered and market presence expands.

Investor Analysis

Shared Investors

The following investors have backed both Ineffable Intelligence and Nscale:

Undisclosed

Unique to Ineffable Intelligence

None (all investors also back Nscale)

Unique to Nscale

None (all investors also back Ineffable Intelligence)

Strategy & Positioning

Ineffable Intelligence

AI memory chip company developing next-generation memory solutions to address the growing bottleneck in AI training and inference workloads.

Nscale

Full-stack AI cloud infrastructure provider operating renewable-energy-powered GPU data centers across Europe.

Frequently Asked Questions

Who has raised more funding — Ineffable Intelligence or Nscale?

Nscale has raised more total funding at $2.0B, compared to Ineffable Intelligence's $1.0B. That is a difference of $1.0B.

Which company is valued higher — Ineffable Intelligence or Nscale?

Ineffable Intelligence's latest valuation is undisclosed, while Nscale's is undisclosed.

Which company was founded first?

Ineffable Intelligence was founded in 2026, 0 year(s) before Nscale (2026).

Verdict

Under current leadership, the company has navigated rapid growth while maintaining technical excellence and a clear strategic vision for the future of AI.