Timothy Spann

Posted on Apr 13

All Data and AI Weekly #237-13April2026

#apachenifi #cortexcode #llm #mcp

All Data and AI Weekly #237-13April2026

( AI, Data, Agentic AI, Cortex Code, NiFi, Iceberg, Polaris, Streamlit, Python, Java, SQL, MCP, LLM, RAG, Cortex AI, AISQL, Search, Unstructured Data )

Tim's Corner: NiFi 2.9, Document Intelligence, and Summit Countdown

Welcome to issue 237! This week delivers a massive wave of platform announcements. Apache NiFi 2.9.0 dropped with initial Connector support and GCS Iceberg integration. Snowflake shipped AI_COMPLETE Document Intelligence to GA - reason over PDFs, Word docs, charts and tables natively. Budgets for AI features are now GA, giving teams granular cost control over Cortex Code, Agents, and Intelligence. The AI_SERVICES billing breakout brings per-service visibility for Cortex Agents, Cortex Code CLI, Cortex Code Snowsight, and Snowflake Intelligence. Iceberg write support for Unity Catalog on Azure is GA, Snowpipe Streaming error logging is GA, and medical/health data classifiers landed for HIPAA-aligned sensitive data detection. On the community front, there are new guides on embedding Cortex Agents in Tableau, connecting Cortex Code to Slack, and the Iceberg v3 interoperability story. I'm speaking at Snowflake Summit 2026 in June on NiFi and Agents - come say hello! Let's dig in.

Tim's Content

Snowflake Summit 2026 - Tim is Speaking!

I'll be speaking at Snowflake Summit 2026 on NiFi and Agents. Check the session catalog:

New SNACK-AI Repositories

Repository	Description
SNACKAI-CoCo-PolyMarket	Prediction market analytics built with Cortex Code and SNACK-AI
SNACKAI-CoCo-VPSCalculator	VPS cost calculator built with Cortex Code

Blog: Prediction Markets at Warp Speed - What the SNACKAI-CoCo-PolyMarket Repository Reveals

Upcoming Events: Catch Tim Live!

April 9: The Agentic Enterprise: From Strategy to ROI - Virtual
April 13-17: D4BV - Virtual APAC
April 14: D4BNA - Virtual US
April 15: Nodes AI Conference - Virtual
April 22: Snowflake Connect: Building Transformation Pipelines for AI-Ready Data - Virtual
April 29: Tysons Corner Hands-On Lab: Automate Data Workflows with Cortex Code and Evolv - Tysons Corner, VA
April 30: Snowflake Accelerate - Virtual
May 20: Budapest Data and AI - Budapest, Hungary - Tim Virtual speaking
June 2-5: Snowflake Summit 2026 - San Francisco, CA - Join me, I will be speaking on NiFi and Agents.

Snowflake Dev Day (In-Person): https://www.snowflake.com/en/dev-day/in-person/

All Snowflake Events: https://www.snowflake.com/about/events/

Snowflake Summit 2026 - San Francisco, June 2-5

The biggest event of the year is coming! Snowflake Summit 2026 returns to San Francisco June 2-5. Expect keynotes, hands-on labs, deep technical sessions, partner showcases, and announcements that will shape the future of the data and AI ecosystem. Whether you're a data engineer, data scientist, architect, or business leader - this is the event to be at.

Register now: https://www.snowflake.com/summit/

Featured: Apache NiFi 2.9.0 Released

Apache NiFi 2.9.0 is here with over 150 issues resolved! This is a significant release that adds initial support for Connectors - a new NiFi concept - plus Google Cloud Storage support for Iceberg and a refactored ConsumeKinesis processor. A key part of the SNACK-AI stack continues to evolve.

Key Highlights:

Initial support for Connectors
Google Cloud Storage support for Iceberg
Refactored ConsumeKinesis to avoid KCL dependency
Bug fix for ConsumeKafka duplicate message processing during rebalance

Release Notes: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version2.9.0

Cortex AI & Agents

AI_COMPLETE Document Intelligence (GA - Apr 2)

AI_COMPLETE document intelligence is now Generally Available. The multimodal AI_COMPLETE function supports document inputs including PDFs and Word files stored in stages. Answer questions about charts and diagrams, compare information across documents, generate audience-tailored summaries, and extract structured insights from reports, contracts, and technical documentation.

Cortex Code CLI Meets Slack

Connect Cortex Code CLI to Slack for AI-powered data workflows directly from your team's communication hub.

Build Custom Subagents in Cortex Code CLI (Part 3)

Third installment in the series on building custom subagents - covering advanced delegation patterns and multi-agent coordination.

Subagent for Cortex Code (Snowflake Labs)

Official Snowflake Labs repository with reference implementations for building subagents in Cortex Code.

GitHub: https://github.com/Snowflake-Labs/subagent-cortex-code

Tableau + Cortex Agent Integration

Embed a Snowflake Cortex Agent directly into Tableau dashboards, giving business users natural language data exploration inside their existing BI tool.

Resources:

Snowflake Intelligence Artifacts

Artifacts for Snowflake Intelligence allow agents to produce, store, and reference structured outputs like reports, charts, and data summaries.

Documentation: https://docs.snowflake.com/en/user-guide/snowflake-cortex/snowflake-intelligence/artifacts

Snowflake Platform Updates

Budgets for AI Features (GA - Apr 10)

Use custom budgets to track and control credit consumption for AI features - AI Functions, Cortex Code, Cortex Agents, and Snowflake Intelligence - broken down by team or cost center. Tag users to a business unit and the budget tracks only their credits.

AI_SERVICES Billing Breakout (Apr 6)

More granular billing service types for AI Credits. Cortex Agents, Cortex Code CLI, Cortex Code Snowsight, and Snowflake Intelligence are now broken out of AI_SERVICES as separate SERVICE_TYPE values in metering views.

Medical and Health Data Classifiers (GA - Apr 3)

Native semantic classifiers for medical and health-related data are now GA. Covers ICD codes, lab terminology, medical conditions, procedures, and medication names aligned with HIPAA requirements.

Performance Explorer Granular Access Control (Apr 2)

Performance Explorer now applies granular access control aligned with your privileges on warehouses, databases, and Snowflake database roles. More users can access Performance Explorer while seeing only what their roles permit.

Snowpipe Streaming Error Logging (GA - Apr 8)

Error logging for Snowpipe Streaming is GA. Failed rows are captured in a dedicated error table with full original payloads and detailed error metadata instead of being silently dropped.

Resources:

DML Error Logging

Configure DML error logging for data load operations to capture failed rows for investigation.

Documentation: https://docs.snowflake.com/en/user-guide/data-load-overview#dml-error-logging

Cost Anomaly: Hourly Consumption by Service Type (Mar 12)

Investigate cost anomalies using hourly consumption broken down by service type - identify which services contribute to consumption spikes during each hour.

Resources:

Multi-Location Resilience for Data Pipelines (GA - Mar 12)

Safeguard data pipelines against region-wide outages. Snowpipe and COPY INTO resume processing in secondary locations with exactly-once semantics and zero-engineering overhead.

Manage External Volumes in Snowsight (GA - Mar 31)

Create, grant, verify, and drop external volumes for Iceberg tables directly from the Snowsight UI.

Git Integration Setup

Updated documentation for setting up Git repositories as Snowflake integrations for version-controlled workflows.

Documentation: https://docs.snowflake.com/en/developer-guide/git/git-setting-up

Network Egress IP Addresses

Reference documentation for network egress IP addresses used by Snowflake services.

Documentation: https://docs.snowflake.com/en/user-guide/egress-ip/network-egress

Startup Challenge Semi-Finalists 2026

Snowflake announces the 2026 Startup Challenge semi-finalists - innovative companies building on the Snowflake platform.

Apache Iceberg Updates

Iceberg Write Support for Unity Catalog on Azure (GA - Apr 6)

Write support for externally managed Iceberg tables on Databricks Unity Catalog is now GA for Azure storage, using Azure Data Lake Storage Gen2 external volumes.

The Battle for Compute: Unlocked by Iceberg v3 and Snowflake Interoperability

Engineering blog on how Iceberg v3 and Snowflake's interoperability capabilities are reshaping the compute landscape - enabling true engine choice on shared data.

Companion Code: https://github.com/sfc-gh-naveenalanthomas/iceberg-v3-snowflake-interoperability

Iceberg v3 Specification Support

Documentation for Snowflake's support of the Iceberg v3 table specification.

Documentation: https://docs.snowflake.com/en/user-guide/tables-iceberg-v3-specification-support

External Query Engine with Snowflake Horizon

Access Iceberg tables managed by Snowflake Horizon using external query engines (Spark, Trino, etc.).

Documentation: https://docs.snowflake.com/en/user-guide/tables-iceberg-access-using-external-query-engine-snowflake-horizon

DuckLake Demo

Demo project showcasing DuckLake - DuckDB's lakehouse format - alongside Iceberg patterns.

GitHub: https://github.com/jeff-skoldberg-gmds/ducklake-demo

Semantic Views & Intelligence

Exporting Semantic Views to Tableau TDS Files (Preview - Mar 5)

Export a semantic view to a Tableau Data Source (TDS) file using SYSTEM$EXPORT_TDS_FROM_SEMANTIC_VIEW, bridging Snowflake semantic models directly into Tableau workflows.

Snowflake Postgres

Postgres Logs to Observe

Guide for shipping Snowflake Postgres logs to observability platforms for monitoring and alerting.

Guide: https://www.snowflake.com/en/developers/guides/snowflake-postgres-logs-to-observe/

From the Community: Medium & Blogs

Article	Description
Cortex Code CLI Meets Slack	Connect Cortex Code to Slack for AI-powered data workflows in your team's comms
The Battle for Compute: Iceberg v3 + Snowflake	How Iceberg v3 reshapes engine choice on shared data
Prediction Markets at Warp Speed	Tim Spann on the SNACKAI-CoCo-PolyMarket architecture
Build Custom Subagents in Cortex Code (Part 3)	Advanced subagent delegation and multi-agent coordination
Giving Your Tableau Dashboard a Brain	Embed a Cortex Agent in Tableau in 4 steps
Taking Snowflake Cortex Code CLI for a Spin	Mechanical Rock's hands-on review of Cortex Code CLI

GitHub: Snowflake Labs & Community

Repository	Description
subagent-cortex-code	Reference implementations for building subagents in Cortex Code
tableau-cortex-agent-demo	Demo for embedding Cortex Agents in Tableau dashboards
iceberg-v3-snowflake-interoperability	Companion code for Iceberg v3 interoperability blog
snowflake-account-usage-dashboard	Pre-built dashboard for Snowflake account usage analytics
ducklake-demo	DuckLake demo project alongside Iceberg patterns
opendataloader-pdf	Open data loader for PDF document ingestion

MCP & Agent Tooling

Tool	Description
Agentic AI APIs	Collection of API patterns for building agentic AI applications
Graphify	Turn structured data into knowledge graphs for agent consumption
Floci	Flow-based orchestration for AI agent pipelines
Open Semantic Interchange (OSI)	Open standard for semantic data interchange between AI systems
OSI Discussions	Community discussion forum for the OSI standard

AI Research & Models

TerminalBench 2.0

TerminalBench 2.0 leaderboard for benchmarking AI coding agents in terminal environments.

Leaderboard: https://www.tbench.ai/leaderboard/terminal-bench/2.0

Reverse SynthID

Research tool for analyzing and understanding Google's SynthID watermarking in AI-generated content.

GitHub: https://github.com/aloshdenny/reverse-SynthID

Hyper-Extract

Hyper-Extract - hypergraph-based feature extraction for structured and relational data.

GitHub: https://github.com/yifanfeng97/Hyper-Extract

RecapAI

AI-powered summarization and recap tool for meetings, documents, and long-form content.

GitHub: https://github.com/RecapAI/Recap

Amazon S3 Files

AWS announces Amazon S3 Files - a new way to work with S3 objects as a mounted file system.

Developer Tools

Browsers, Crawlers & Automation

Tool	Description
Scrapling	Intelligent web scraping library with anti-detection capabilities
Camofox Browser	Privacy-focused browser with fingerprint camouflage
MarkItDown	Microsoft's tool for converting documents to clean markdown
JSON Render	Visual JSON rendering and exploration tool

Terminal, Shell & System

Tool	Description
rpg	Role-playing game for PostgreSQL - gamified database learning
Linux on Android	Run a full Linux environment on Android devices
VibeTracer	Application tracing and debugging visualization tool
AWS Doctor	Diagnostic tool for AWS infrastructure health checks
Happy Engineering	Resources and practices for engineering team happiness

ROI of Generative & Agentic AI

Snowflake published extensive research demonstrating the radical ROI organizations are seeing from generative and agentic AI implementations.

Resources:

Data Sharing & Open Table Formats

Snowflake's vision for data sharing with open table formats enables seamless data exchange while maintaining interoperability across engines and platforms.

SNACK-AI: The 2026 Pattern

As we look forward in 2026, SNACK-AI is the architectural pattern for powering AI applications at any scale - from mobile phones and robots to massive enterprise clusters.

SNACK-AI integrates: