The Hottest Trend: Vector Databases and AI
The capabilities of AI, particularly LLMs, seem to have no visible ceiling. Applications based on these technologies are thriving across various industries. One significant intersection between databases and AI lies in vector storage and search. Over the past year, nearly every mainstream database has introduced or enhanced related features. Forrester has also led the charge by releasing the independent Vector Database Wave, evaluating the vector processing capabilities of various databases.
Forrester's Vector Database Wave
In Q3 of this year, Forrester published its Vector Database Wave, featuring companies such as Zilliz, DataStax, Microsoft, Amazon, Oracle, and Pinecone.
Now, almost all database or cloud vendors are investing in "vector" technologies. These AI-driven innovations are evolving rapidly, and competition in vector storage and applications is far from over.
Vector Features Introduced by Database and Cloud Providers
In 2024, database providers raced to launch their support for vector-related features. Notable examples include:
- pg_vector released version 0.8, widely adopted across numerous projects.
- Azure introduced its vector database extension DiskANN, integrated into several of its database products.
- MariaDB began full support for vector features starting with version 11.6.
- OceanBase 4.3.3 added support for vector data storage and indexing.
- MySQL 9.0 introduced vector storage support.
- TiDB launched Vector Search (Beta), integrating vector capabilities seamlessly into its ecosystem.
- Additionally, nearly all cloud providers supporting PostgreSQL now include the pg_vector plugin.
Other AI-Database Integrations
Beyond vector capabilities, other AI-database integrations, such as Text2SQL and database optimization, are also gaining traction. Examples include:
- AWS Redshift officially supports natural language-generated SQL queries through Amazon Q, available in the Redshift Query Editor.
- Copilot for Azure extended its support to Azure SQL and Azure Database for MySQL.
Graph Databases
Graph databases have become a critical part of the large language model (LLM) ecosystem, experiencing rapid growth over the past year:
- Neo4j announced that its ARR (Annual Recurring Revenue) surpassed $200 million, doubling over the past three years. Its leadership in the graph technology market has played a pivotal role in enhancing the accuracy, transparency, and interpretability of generative AI results.
- This year, the ISO issued the new ISO/IEC GQL (Graph Query Language) standard, the "SQL" of graph databases, addressing gaps in existing standards for processing graph data. NebulaGraph became the first distributed graph database to support this standard.
Open Source and Commercial Developments
At the start of this year, the widely-used caching database Redis changed its open-source license. Starting from version 7.4, Redis adopted the RSALv2+SSPLv1 license, moving away from the BSD license.
Shortly thereafter, the Valkey project was initiated under the Linux Foundation, with backing from companies like Amazon, Google, and Oracle. Later in the year, AWS officially supported Valkey on its cloud platform.
Meanwhile, CockroachDB imposed significant restrictions on its open-source license. For small businesses (annual revenue under $10 million), the software remains free, but enterprises exceeding this revenue threshold must now pay for its use.
PostgreSQL's Steady Ascent
As in previous years, PostgreSQL continues its slow and steady rise, while MySQL seems to have plateaued.
In the DB-Engines rankings, MySQL still holds a significant lead. However, PostgreSQL's momentum is evident, with innovations like pg_vector gaining widespread use, whereas MySQL’s recent support for vectors in version 9.0 remains limited.
In the JetBrains State of Developer Ecosystem Report 2024, similar trends emerge:
- MySQL adoption fell from 60% in 2019 to 52% in 2024.
- PostgreSQL adoption grew from 30% in 2019 to 45% in 2024.
Funding Highlights in the Database Domain
The database sector saw numerous notable funding events in 2024:
- Tembo, a PostgreSQL hosting startup, raised $14 million in Series A funding, offering both open-source products and Tembo Cloud for managed PostgreSQL services.
- OpenAI acquired real-time analytics database developer Rockset for an estimated $500–$1,000 million to enhance real-time data search and analysis for large models.
- ClickHouse acquired PostgreSQL data synchronization startup PeerDB, whose seed round in 2023 raised $3.6 million.
- Open-source time-series database GreptimeDB secured multi-million-dollar funding, emphasizing cost-effective real-time data handling.
- K1 acquired MariaDB, cementing its position as a leading small business software investor.
- Supabase raised $80 million in Series C funding, centered around PostgreSQL, offering a range of developer tools including vector databases.
- ApertureDB raised $8 million to advance multi-modal data flow services for AI.
Gartner’s Magic Quadrant for Databases
Gartner recently released the 2024 Magic Quadrant for Databases. The leaders remain Google, AWS, Azure, and Oracle, with MongoDB, DataBricks, and Snowflake making progress toward the upper-right quadrant.
Key Announcements from Cloud Providers
- At AWS re:Invent, AWS launched Aurora DSQL, featuring cross-region, strongly consistent, multi-region read-write capabilities with 99.999% availability (multi-region deployment) and PostgreSQL compatibility. DynamoDB also introduced similar cross-region consistency capabilities.
- At Oracle CloudWorld, Larry Ellison emphasized a multi-cloud strategy, unveiling Oracle@Google, Oracle@Azure, and Oracle@AWS solutions.
- GCP focused on enhancing AlloyDB and its foundational products, such as introducing Enterprise Plus and Enterprise versions of Cloud SQL for SQL Server.
- Microsoft Azure prioritized Cosmos DB and SQL Database, introducing DiskANN vector indexing for its vector databases.
We're Leapcell, a better platform to deploy your projects, including those with cloud databases.
Leapcell is the Next-Gen Serverless Platform for Web Hosting, Async Tasks, and Redis:
- Multi-Language Support
- Develop with JavaScript, Python, Go, or Rust.
- Deploy unlimited projects for free
- pay only for usage — no requests, no charges.
- Unbeatable Cost Efficiency
- Pay-as-you-go with no idle charges.
- Example: $25 supports 6.94M requests at a 60ms average response time.
- Streamlined Developer Experience
- Intuitive UI for effortless setup.
- Fully automated CI/CD pipelines and GitOps integration.
- Real-time metrics and logging for actionable insights.
- Effortless Scalability and High Performance
- Auto-scaling to handle high concurrency with ease.
- Zero operational overhead — just focus on building.
Explore more in the Documentation!
Follow us on X: @LeapcellHQ
Top comments (0)