The Unofficial Snowflake Monthly Release Notes: March 2024

Monthly Snowflake Unofficial Release Notes #New features #Previews #Clients #Behavior Changes

Welcome to the fantastic Unofficial Release Notes for Snowflake! You’ll find all the latest features, drivers, and more in one convenient place. As an unofficial source, I am excited to share my insights and thoughts. Let’s dive in! You can also find all of Snowflake’s releases here.

This month, we provide coverage up to release 8.12.

I would appreciate your suggestions on continuing to combine these monthly release notes. Feel free to comment below or chat with me on LinkedIn.

Note: There is a NEW Deprecated Features for March with Bundle 2024_03. See the link for details.

What’s New in Snowflake

New Features

• Preview: Data Quality Monitoring and data metric functions (DMF), monitor the data quality metrics such as completeness, accuracy, uniqueness, and validity. You can use the DMF in a query to test the data quality in your pipeline or associate the DMF to desired tables to continuously monitor•General Availability: Snowflake Data Clean Rooms in three AWS regions, allowing multiple parties to collaborate in a secure environment. These collaborators can combine and analyze data without worrying about the privacy concerns of sharing raw data
• Preview: Custom Classification, provides the CUSTOM_CLASSIFIER class in the SNOWFLAKE.DATA_PRIVACY schema. Define your semantic category, specify the privacy category, and specify regular expressions to match column value patterns while optionally matching the column name
• Preview: Snowflake Cortex LLM functions (see Machine Learning Updates)
• General Availability: Geospatial Functions: ST_GEOMFROMGEOHASH and ST_GEOMPOINTFROMGEOHASH, both related to geohashes
• Preview: Snowflake Cortex ML functions: Classification function (see Machine Learning Updates)
• Preview: Snowflake CLI 2.x, an open-source command-line tool explicitly designed for developer-centric workloads in addition to SQL operations. You can create, manage, update, and view apps running on Snowflake across workloads such as Streamlit in Snowflake, the Snowflake Native App Framework, Snowpark Container Services, and Snowpark

SQL Updates

• General Availability: SELECT supports trailing commas
• General Availability: Organization Usage: Improved views for billing reconciliation, improved to make it easier to reconcile Snowflake usage with monthly billing statements

Client, Drivers, Libraries and Connectors Updates

• New features: .NET Driver 3.1.0 (asynchronous queries), Go Snowflake Driver 1.9.0 (Arrow 15, WithHigherPrecision in Arrow, date and time converter from the Snowflake format to the Golang format, replaces UTF-8 characters in Arrow), Snowflake CLI 2.1.1 (Initial public release), Snowflake CLI 2.1.2 (pip as a Snowflake CLI dependency, Optimized the snow connection test), Snowflake Connector for Kafka 2.2.1 (offset verification logic to make sure there is no missing or duplicate data, client provider overridden map for Snowpipe Streaming comma limited, few libraries updated), Snowflake API for Python 0.7.0 (pip install snowflake[ml] to install the Snowpark ML library as an extra package dependency, Snowflake.core package updates: Task predecessors now return their fully qualified name, __str__() and __repr_html__() methods to DAGRun to make it notebook compatible, doc improvements), Snowpark Library for Python 1.14.0 (support for creating vectorized UDTFs with the process method, dataframe functions locate and timestamp, ASOF JOIN type, support for the following local testing APIs, telemetry to local testing, Improved the error message of DataFrameReader to raise FileNotFound error ), Snowpark ML 1.3.1 (snowflake.ml.fileset.sfcfs.SFFileSystem can now be used in UDFs and stored procedures), SnowSQL 1.2.32 (keyring dependency to 23.1.0 to address a security vulnerability), Snowflake Connector for Google Analytics Raw Data 0.16.1 (optimizations made to scheduling ingestions and ingesting, mixed-case and lowercase secret names support), Snowflake Connector for Google Analytics Aggregate Data 0.18.0 (Generated IDs for all CONNECTOR_EXECUTION_LOG entries), Snowflake Connector for Google Analytics Aggregate Data 0.19.0 (Added alerting handling. Procedures DISABLE_ALERTS and CONFIGURE_ALERTS were added), Snowflake Connector for Google Analytics Aggregate Data 0.21.0 (CONNECTOR_EXECUTION_LOG table is deprecated and will be removed in the future, more details to the CONNECTOR_ERRORS view, including the error reason), Snowflake Connector for Google Analytics Aggregate Data 0.21.1 (request for the EXECUTE MANAGED TASK privilege to the connector’s manifest), Snowflake Connector for Google Analytics Aggregate Data 0.22.0 (GET_TROUBLESHOOTING_DATA procedure), Snowflake Connector for ServiceNow 4.21.0 (new procedure EXPORT_CONNECTOR_STATE() to export the connector’s state and configuration to be available to import in the GA version of the application)
• Bug fixes: .NET Driver 3.1.0 (exceptions Okta authenticator, validating very short (1–2 character) account names, WAREHOUSE property from a connection string with quoted content), Go Snowflake Driver 1.9.0 (Amazon S3 accelerated configuration, dividing big numbers in Arrow, racing issue during logging initialization, few others bugs), Snowflake CLI 2.1.2 (creating virtual environments), Snowflake Connector for Kafka 2.2.1 (Cleaned up streaming ingest threads when SinkTask stop () is called), Snowflake API for Python 0.7.0 (Fixed code generator and updated OpenAPI-spec driven models, Pydantic compatibility, Task.error_integration property, Task.config property when the REST property was missing), Snowpark Library for Python 1.14.0 (SnowflakePlanBuilder where save_as_table does not correctly filter columns, statement parameters might have no effect when resolving imports and packages, lots of bugs in local testing), Snowflake Connector for Google Analytics Raw Data 0.16.1 (ingestions should no longer fail due to issues with refreshing the access token, validation to prevent enabling the same Google Analytics property ingestion from multiple Google Cloud Platform projects, validation for adding multiple properties at once is now more resilient against issues with BigQuery connectivity), Snowflake Connector for Google Analytics Aggregate Data 0.18.0 (Fixed initial load start date calculation), Snowflake Connector for Google Analytics Aggregate Data 0.21.0 (CONNECTOR_CONFIGURATION, CONNECTOR_STATS, and CONNECTOR_ERRORS views are now visible to the VIEWER application role), Snowflake Connector for ServiceNow 4.21.0 (explicit timeout on the task creating flattened views), Snowflake Connector for ServiceNow 4.21.1 (Fix MERGE_INTO_SINK_TABLE task failing on a specific connector state and an empty event log table)

Machine Learning Updates

• Preview: Snowflake Cortex LLM Functions, instant access to a suite of features powered by state-of-the-art large language models (LLMs), with the following functions: COMPLETE, EXTRACT_ANSWER, SENTIMENT, SUMMARIZE and TRANSLATE, great to see a proper SaaS version of an LLM integrated into Snowflake although early tries still look a little rough around the edges
• Preview: Snowflake Cortex ML Classification Function, a ML-powered function that sorts data into different classes using patterns detected in the training data

Hybrid Tables Updates

• Availability of hybrid tables in the following additional Amazon Web Services (AWS) regions: US East (Ohio), US East (N. Virginia), EU (Frankfurt) and Asia Pacific (Tokyo), Canada (Central), South America (Sao Paulo), Europe (London), EU (Paris), EU (Stockholm) and all the Asia regions

Data Lake Updates

• General Availability: Primary key information added to Iceberg table metadata, writes information for primary key columns to Iceberg table metadata using the Apache Iceberg identifier-field-ids property.

Applications Updates

• Preview: Support for Streamlit 1.26.0, select the Streamlit library version in Snowsight or pin the version in the app’s environment.yml file, support for 1.23 to 1.26 versions, new Streamlit features support: st.chat_message, st.chat_input, st.status, st.toggle, st.data_editor and st.column_config
• Preview: Limit functionality of your Snowflake Native App, limit functionality during the trial period; this includes functionality tied to secure view, secure UDF, or a Streamlit using a system function, SYSTEM$IS_LISTING_TRIAL.

Data Pipelines/Data Loading/Unloading Updates

• General Availability: Performance improvements for loading JSON files, improvement results in lower ingestion latency of up to 25% without changing any of the load queries
• General Availability: Improvements to the SNOWPIPE_STREAMING_CLIENT_HISTORY view, a bug fix that now shows all events since Feb 1st, 2024
• General Availability: Replication: Stages, pipes, storage integrations, load history, and Snowpipe Streaming, gives you the ability to replicate this objects for failover use-cases
• General Availability: Schema detection and evolution for Kafka connector with Snowpipe Streaming, this provide you with ability to flatten RECORD_CONTENT into COLUMN’s directly

Data Governance Updates

• Preview: Custom Classification, provides the CUSTOM_CLASSIFIER class in the SNOWFLAKE.DATA_PRIVACY schema. Define your semantic category, specify the privacy category, and specify regular expressions to match column value patterns while optionally matching the column name
• General Availability: Memoizable functions with constant arguments,
• General Availability: Share data protected by a role-based policy, use the IS_DATABASE_ROLE_IN_SESSION function in the conditions of a masking policy or row access policy
• General Availability: Access History: Stored procedure ancestor queries, track the chain of queries that call a stored procedure by using the parent_query_idand root_query_id columns
• General Availability: Shared tag references, allow users in the data sharing consumer account to view shared tags and the tag references on shared objects when the tags and shared objects are in the same database
• General Availability: Access History: Track objects modified by a DDL operation, you can also track how tag and policy assignments change and track the table and column lifecycle

Conclusion

It seems that Snowflake focused this month on stability, with many releases around making things Generally Available while pushing new features, particularly in Machine Learning and Governance.

I am busy trying this, like Snowflake Cortex ML functions; thank you, Snowflake team, for doing the labs so quickly and the new Command Line CLI.

I am super happy to see Data Quality Monitoring being released in Preview to everyone. Although it is only on Enterprise accounts or above which is a little sad I guess. It is still great to see not just fancy features but also care about improving your client’s core data practices.

I am Augusto Rosa, VP of Engineering for Infostrux Solutions. Snowflake Data Super Hero and SME. Thanks for reading my blog post. You can follow me on LinkedIn. Subscribe to Infostrux Medium Blogs https://medium.com/infostrux-solutions for the most interesting Data Engineering and Snowflake news.

Sources:

• https://community.snowflake.com/s/announcements
• https://docs.snowflake.com/en/release-notes/

---