All Data and AI Weekly #212: 20 Oct 2025

#llm #mcp #snowflake #agenticai

All Data and AI Weekly

( AI, Data, NiFi, Iceberg, Polaris, Streamlit, Flink, Kafka, Python, Java, SQL, MCP, LLM, RAG, Cortex AI, AISQL, Search, Unstructured Data )

#212: 20 Oct 2025

https://bsky.app/profile/paasdev.bsky.social

NiFi + AI + AI Data Cloud + Iceberg.

https://www.reddit.com/r/DataEngineeringForAI/hot/

Monthly NYC and Youtube Events

https://lu.ma/PINSAI

Code and Open Source Projects

https://github.com/tspannhw/TrafficAI/tree/main/Agents

https://github.com/tspannhw/conferences

events
https://pydata.org/global2025/schedule
[Discount Code] On November 20th Online and November 21st in Warsaw (PGE Narodowy, Warsaw), we invite you to the next edition of Data Science Summit —one of the largest cybersecurity conferences in Poland. The Organizers and Program Council Members have prepared an exciting and diverse Agenda, featuring 20+ thematic tracks, over 250 presentations, networking, an afterparty, and much more…

Hope to see you there!

More information and registration at: https://main.dssconf.pl/

My New Projects

Ghosts: A new project worth checking out.
Everything Snowflake One-Pager: A concise guide and summary for all things Snowflake.

📚 Articles & Blog Posts

Documenting Snowflake Objects with Cortex AI: Learn how to empower your documentation process using AI.
Data Engineering for the Lakehouse with Open Formats: An overview of data engineering strategies in a modern lakehouse architecture.
From Discovery to Delivery with Cortex Agents: Exploring the end-to-end intelligence capabilities of Snowflake Cortex Agents.
Document Processing with Snowflake Cortex AI+SQL: A practical guide to processing documents using AI functions directly in Snowflake.
Understanding Snowflake Cortex AI Services Cost: A breakdown of the costs associated with using Cortex AI.
Optimizing Snowflake Cortex Analyst Performance: Tips and tricks for getting the best performance out of Cortex Analyst.
Migrating Oracle PL/SQL Out Parameters: A technical guide for migrating from Oracle to Snowflake.
The Next-Gen Snowpipe Streaming Architecture: An engineering deep-dive into the new Snowpipe streaming architecture.
Understanding Snowflake Data Lineage: An article from Integrate.io on the importance of data lineage in Snowflake.
OpenLineage and Snowflake Integration: Learn how OpenLineage integrates with Snowflake for better data lineage tracking.

📄 Documentation

Snowflake Cortex AI+SQL: Official documentation for using AI and ML functions in Snowflake.
Snowflake Cortex Agents: Official documentation for the new Cortex Agents feature.
Snowflake SEARCH Function: Documentation for the SEARCH function for text analysis.
Neo4j & Snowflake Graph Analytics: Documentation for using Neo4j for graph analytics on Snowflake data.
Envoy Proxy: Official website for the Envoy Proxy.

💡 Examples & Tutorials

Make Your Lakehouse AI-Ready: A developer solution guide for preparing your lakehouse for AI applications.
Data Lake with Iceberg and AWS Glue: A hands-on quickstart for building a data lake with open formats.
Snowflake Document Classification & Extraction: A GitHub repository with an example of document classification and extraction.
OpenLineage AccessHistory Setup: A Snowflake Labs repo to help set up OpenLineage.

🗓️ Upcoming Events

Webinar: Architect for AI with Open Formats (Oct 15, 2025): Join this product demo to learn about architecting for AI.
Hands-On Lab: Declarative Data Pipelines (Nov 13, 2025): A virtual lab on building pipelines with Dynamic Tables.

💻 Code Repositories

AutoSwagger: Automatically generate Swagger/OpenAPI documentation from your code.
Erdos: An open-source platform for building high-performance, real-time data stream processing applications.
nanochat: A small, educational repository for understanding the basics of chat models.
dbt-colibri: A dbt package for data contracts and monitoring.