augusto kiniama rosa

Posted on Jan 1 • Originally published at blog.infostrux.com on Jul 1, 2024

The Unofficial Snowflake Monthly Release Notes: June 2024

#datasuperhero #newreleases #snowflake #data

Monthly Snowflake Unofficial Release Notes #New features #Previews #Clients #Behavior Changes

Welcome to the fantastic Unofficial Release Notes for Snowflake! You’ll find all the latest features, drivers, and more in one convenient place.

As an unofficial source, I am excited to share my insights and thoughts. Let’s dive in! You can also find all of Snowflake’s releases here.

This month, we provide coverage up to release 8.24 (General Availability — GA). I hope to extend this eventually to private previews notices as well.

I would appreciate your suggestions on continuing to combine these monthly release notes. Feel free to comment below or chat with me on LinkedIn.

Behavior change bundles 2024_03 active by Default, 2024_04 enabled by default but able to be disabled and 2024_05 is available to be enabled.

What’s New in Snowflake

New Features

Entity-Level Privacy (GA): Aggregation policies with entity-level privacy ensure groups contain a minimum number of entities to enhance data privacy.
Cost Management Overview Page (GA): New Organization Overview page in Snowsight for insights into Snowflake cost management.
Snowflake Notebooks (Preview): development interface in Snowsight that offers an interactive, cell-based programming environment for Python and SQL. #summit
Snowpark pandas API (Preview): run your pandas code directly on your data in Snowflake. #summit
Iceberg tables (GA): single copy of data with interoperability across a variety of compute engines. #summit
Sharing data in non-secure views (Preview): sharing data in non-secure views
Account Usage: TABLE_DML_HISTORY (Preview): : determine the magnitude and effects of the DML operations performed on a table
Account Usage: TABLE_PRUNING_HISTORY views (Preview): determine pruning efficiency for all tables.
TO_QUERY table function (GA): returns a result set based on SQL text and an optional set of arguments that are passed to the SQL text if it is parameterized. The function compiles the SQL text as the definition of a subquery in the FROM clause. When writing an application or a stored procedure, you can call this function to construct a SQL statement.

Snowsight Updates

Specify Appearance (Preview): Dark mode Option to specify color schemes, including a dark mode to reduce eye strain in low-light conditions #summit
Tasks Improvements (GA): Enhanced Snowsight tasks for better task management.
Universal Search (GA): find even more objects than before, quickly and securely. Searching from the Search tab finds tables, functions, databases, data products available to you in the Snowflake Marketplace, relevant Snowflake Documentation topics, and related articles in the Snowflake Community Knowledge Base. #summit

Snowflake Applications

Snowflake Native SDK for Connectors (Preview): library with templates and quickstarts in Java that you can use to quickly build your own Snowflake Native App based Connectors to easily ingest data from an external data source into Snowflake. #summit
Snowflake Native Apps with Snowpark Container Services (Preview): run any containerized service supported by Snowpark Container Services within a Snowflake Native App. #summit
Updates to logging and tracing for a Snowflake Native App (Preview): enhancements to the logging and tracing.
New events generated during app installation and upgrade (Preview): addition of trace events that are generated when a Snowflake Native App is installed or upgraded.

Snowpark Updates

Snowpark Python local testing framework (GA): an emulator that lets you test your Python code locally when working with the Snowpark Python library. #summit

Streamlit Updates

Custom UI in Streamlit in Snowflake (Preview): Custom UI enables customization of the look, feel, and front-end behavior of Streamlit in Snowflake apps. Supports Custom HTML and CSS using unsafe_allow_html=True in st.markdown and Iframed HTML, CSS, and JavaScript using st.components.v1.html.

Geospatial Updates

geospatial H3 functions (GA): H3 functions are now generally available.
New geospatial functions (Preview): H3_TRY_COVERAGE, H3_TRY_COVERAGE_STRINGS, H3_TRY_POLYGON_TO_CELLS and H3_TRY_POLYGON_TO_CELLS_STRINGS: functions for GEOGRAPHY objects

Data governance updates

Data quality: add new system DMFs (Preview): SNOWFLAKE.CORE.AVG, SNOWFLAKE.CORE.BLANK_COUNT, SNOWFLAKE.CORE.BLANK_PERCENT, SNOWFLAKE.CORE.MAX, SNOWFLAKE.CORE.MIN, SNOWFLAKE.CORE.NULL_PERCENT, SNOWFLAKE.CORE.STDDEV

SQL Updates

Window functions: extended support for RANGE BETWEEN (Preview): extended range-based functionality makes it easier to run moving aggregations when expected or unexpected missing records cause gaps to occur in time-series data sets for COUNT, SUM, MIN, MAX, AVG

Hybrid Tables

Time Travel for hybrid tables (Preview): enabled support for time travel on hybrid tables

Machine Learning Updates (Cortex and ML)

Snowflake Cortex Fine-Tuning (Preview): adapt pre-trained models to more specialized tasks.
Anomaly Detection ML Function (GA): improves its algorithm, and results in improved performance
EMBED_TEXT_1024 function (Preview): vector embedding function that outputs 1024 dimension vectors, which enables important applications that require semantic vector search and retrieval.
TRY_COMPLETE and COUNT_TOKENS LLM helped functions (Preview): purpose-built and managed functions that help to reduce cases of query failures when the number of input tokens exceed a model limit.
DocumentAI — new version of the Arctic-TILT model (Preview): improvements: extraction of lists of values, checkbox identification and query paraphrasing recognition to improve recognizing queries built as sentences, such as Give me the date of the agreement.
DocumentAI-increase in number of documents that you can process with !PREDICT in one query. Process a maximum of 1000 documents in one query, instead of 20.

Data Clean Rooms Updates

Additional supported regions(GA): (AWS South America (Sao Paulo), AWS Asia Pacific (Tokyo), Azure UAE North (Dubai), AWS Asia Pacific (Osaka), AWS Asia Pacific (Seoul), AWS Asia Pacific (Jakarta), Azure Japan East (Tokyo).
Granular access management for Snowflake data (GA): account administrators and object owners can grant access to Snowflake data at the schema and object level.
Choosing a warehouse when running an analysis (GA): clean room users can select which warehouse they want to use when running an analysis with select templates.
Support for multiple custom templates in web app (GA): run different types of custom analyses in a single clean room.
Provider-run analyses (GA): directly extract insights from data shared by the consumer within the clean room.
Consumer-defined templates (GA): Consumers can now request that a provider allow them to add their own template to a clean room.
Granular access controls for tables & templates (GA): control which consumers can access a specific table or template in the clean room.
Activating results across regions(GA): Consumers can now activate results back to the provider even if the provider is in a different cloud/region. Consumers must enable cross-cloud auto-fulfillment on their account.
SQL Template enhancement (GA): provider allows another party to query data via the SQL template, the provider can now specify custom aggregation threshold values for each entity column they want to protect.

Data Pipelines/Data Loading/Unloading Updates

ALTER DYNAMIC TABLE command: Support for adding search optimization and setting additional properties (GA): command can now set the following, search optimization for your dynamic tables, and set the following properties for your dynamic tables: Retention time, Comments, Default collation, Tags and Row access and masking policies

Security Updates

Trust Center(GA): fully available with support
Authentication policies: New multi-factor authentication parameters (GA): MFA_AUTHENTICATION_METHODS and MFA_ENROLLMENT

Open-Source Updates

terraform-snowflake-provider 0.92.0 (API Authentication security integration to SDK, External OAuth security integration to SDK, network rules, SCIM and SAML2 security integrations to SDK, Snowflake OAuth security integration to SDK, Database resource v1 readiness, Database SDK upgrade)
Streamlit 1.36 (Allow passing on_change_callback for CustomComponents, Use raw number for number column overlay and copy, Introduce st.navigation and st.Page, Make st.write call st.json to display Streamlit secrets object, Streamlit Charts: Customizable Axis Labels, vertical alignment parameter to st.columns, icon parameter to st.expander, Use default widget height for non-stacked checkbox & toggle widgets, Horizontal st.bar_chart)
Modin 0.31.0 (UserWarning no longer displayed when Series/DataFrames are small, reload_modin feature, Automatic publication of Modin wheel to PyPI, MinRowPartitionSize and MinColumnPartitionSize, NumPy 2.0 support)
Modin 0.29.1 (Automatic publication of Modin wheel to PyPI)

Client, Drivers, Libraries and Connectors Updates

New features:

Snowflake Connector for Google Analytics Raw Data 1.3.0 (reload property feature)
Snowflake Connector for ServiceNow V2 5.5.0 (default way to obtain the schema of a table when starting its ingestion)
ODBC Driver 3.3.2 (General stability improvements)
PHP PDO Driver for Snowflake 3.0.0 (Improved data integration capabilities)
Snowflake CLI 2.5.0 (Enhanced command-line functionality)
Snowflake Connector for Python 3.11.0 (Enhanced Python integration)
Snowflake API for Python 0.10.0 (API enhancements for Python)
Snowpark Library for Python 1.19.0 (Additional functionalities for Snowpark)
Snowpark ML 1.5.3 (Machine learning enhancements.
SnowSQL 1.3.1 (Updated SQL functionalities)

Bug fixes:

Snowpark ML 1.5.2 (Minor bug fixes)
Snowflake API for Python 0.9.0 (Minor bug fixes)
Snowflake CLI 2.4.1 (Various minor fixes)
Snowflake Connector for Google Analytics Aggregate Data 1.3.0, 1.3.1, 1.4.0 (small improvements and bug fixes)
Snowflake Connector for ServiceNow V2 5.4.0 (bug fixes)
Snowflake Connector for ServiceNow V2 5.5.0 (health check bug)

Conclusion

This was the summit month and it has been a busy but maybe not as busy as usual when compared to the last two months.

The enhancements introduced in June 2024 have expanded Snowflake’s capabilities in data privacy, cost management, Snowpark, Streamlit, geospatial functions, data clean rooms, data pipelines, security, and open-source updates. Notable improvements include the general availability of Entity-Level Privacy, Iceberg tables, and Trust Center, as well as previews of Snowflake Notebooks, Snowpark pandas API, and Document AI enhancements. These updates bolster Snowflake’s position as a robust platform for data management, analysis, and application development. Stay tuned for more upcoming innovations and improvements!

I hope you continue to enjoy these articles. Thanks for reading my blog post.

I am Augusto Rosa, VP of Engineering for Infostrux Solutions. Snowflake Data Super Hero and Snowflake SME. You can follow me on LinkedIn.

Subscribe to Infostrux Medium Blogs https://medium.com/infostrux-solutions for the most interesting Data Engineering and Snowflake news.

Sources:

How I Cut 22.3 Seconds Off an API Call with Sentry 👀

Struggling with slow API calls? Dan Mindru walks through how he used Sentry's new Trace View feature to shave off 22.3 seconds from an API call.

Get a practical walkthrough of how to identify bottlenecks, split tasks into multiple parallel tasks, identify slow AI model calls, and more.

DEV Community