<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Nagaraju Bondala</title>
    <description>The latest articles on DEV Community by Nagaraju Bondala (@nagaraju_bondala).</description>
    <link>https://dev.to/nagaraju_bondala</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4259290%2Fe444f221-8e1c-4310-bd0a-d77d7b7bef0c.png</url>
      <title>DEV Community: Nagaraju Bondala</title>
      <link>https://dev.to/nagaraju_bondala</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/nagaraju_bondala"/>
    <language>en</language>
    <item>
      <title>How to Forecast in Databricks SQL with ai_forecast</title>
      <dc:creator>Nagaraju Bondala</dc:creator>
      <pubDate>Sun, 22 Nov 2026 20:38:02 +0000</pubDate>
      <link>https://dev.to/nagaraju_bondala/how-to-forecast-in-databricks-sql-with-aiforecast-4m9i</link>
      <guid>https://dev.to/nagaraju_bondala/how-to-forecast-in-databricks-sql-with-aiforecast-4m9i</guid>
      <description>&lt;p&gt;Forecasting usually means exporting data, training a model and writing the results back, so many teams skip it. The &lt;code&gt;ai_forecast&lt;/code&gt; function in Databricks SQL does the forecast in one query, and version 2 adds holiday support, covariates and non-negative forecasts on top of a time series foundation model. This guide builds a daily sales forecast step by step: you prepare the input, run a basic forecast, switch to version 2, add holidays, add a covariate and constrain the output to non-negative values. In testing, the biggest gain did not come from the new model, but from telling it about planned promotions. The complete code for this guide is on GitHub: &lt;a href="https://github.com/ven-nbondala-tf/databricks-release-lab/tree/main/databricks/ai-forecast" rel="noopener noreferrer"&gt;ai-forecast&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/-7MijS-_BKQ" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;br&gt;
&lt;em&gt;Forecast error for five setups, from version 1 to planned promotions, in 30 seconds.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  What is ai_forecast in Databricks SQL
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;ai_forecast&lt;/code&gt; is a table-valued function that reads a table of history and returns forecasted values with an upper and a lower bound. You call it in the FROM clause, so the result is an ordinary table you can filter, join or save.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Version 2 is generally available in Databricks SQL, announced on November 18, 2026. Version 1 is in Public Preview.&lt;/li&gt;
&lt;li&gt;Version 2 uses a time series foundation model. Version 1 uses a prophet-like piecewise linear and seasonality model.&lt;/li&gt;
&lt;li&gt;It forecasts one or more value columns, optionally per group, and supports holidays, covariates and non-negative forecasts in version 2.&lt;/li&gt;
&lt;li&gt;It runs on the compute you submit it from, with no separate inference cost.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The diagram shows what goes in and what comes out.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6y71tfkfty1ebg15rvth.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6y71tfkfty1ebg15rvth.png" alt="A flow diagram. History rows, optional covariate rows and optional arguments feed ai_forecast with version 2. It returns forecast rows only, with a forecast, an upper and a lower column for each value column." width="799" height="364"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The inputs and outputs of ai_forecast version 2.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  When to use ai_forecast
&lt;/h2&gt;

&lt;p&gt;Use &lt;code&gt;ai_forecast&lt;/code&gt; when your data is already in a table, you want a forecast from SQL, and a general-purpose model is accurate enough: sales, traffic, demand or revenue per day or another regular interval, for one series or many groups. It fits well inside a scheduled query, a dashboard dataset or a pipeline step.&lt;/p&gt;

&lt;p&gt;Use a custom model when you need control over the model itself, such as custom features, your own training and evaluation loop, or a model registered and served separately. Forecasting in Databricks SQL with &lt;code&gt;ai_forecast&lt;/code&gt; is a fast starting point, and a custom model is the next step when its accuracy is not enough.&lt;/p&gt;
&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;A Pro or Serverless SQL warehouse. AI Functions are not available on Classic SQL warehouses.&lt;/li&gt;
&lt;li&gt;Read access to the table with your history, and permission to create tables if you want to save the forecast.&lt;/li&gt;
&lt;li&gt;Version 2 ran on a serverless SQL warehouse with no preview setting turned on in the test workspace. The function page says that for some workspaces, including those with the compliance security profile, a workspace admin must turn on "AI Forecast V2" on the Previews page. If version 2 is not available, ask your admin to check that setting.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  Step 1: Prepare the input
&lt;/h2&gt;

&lt;p&gt;The function needs a time column of type DATE or TIMESTAMP, and a value column that can be cast to DOUBLE. Create a table with one row per day and region. The setup script in the repository builds two years of daily sales for two regions, a promo flag and a second series that trends close to zero.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;SCHEMA&lt;/span&gt; &lt;span class="k"&gt;IF&lt;/span&gt; &lt;span class="k"&gt;NOT&lt;/span&gt; &lt;span class="k"&gt;EXISTS&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="k"&gt;REPLACE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;region&lt;/span&gt; &lt;span class="no"&gt;STRING&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;sales&lt;/span&gt; &lt;span class="kt"&gt;DOUBLE&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;promo&lt;/span&gt; &lt;span class="kt"&gt;BOOLEAN&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The examples use the data before 2026-11-16 as history, and compare the forecast with the 60 days that follow. The generated data runs to mid-January 2027, so the last 60 days can act as "actuals" to compare against. With real data, you hold out the most recent 60 days instead, and for the final forecast you pass all the history you have.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: Run a basic forecast
&lt;/h2&gt;

&lt;p&gt;A forecast needs the table, the end of the horizon, and the names of the time and value columns. Add &lt;code&gt;group_col&lt;/code&gt; to forecast each region independently.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="no"&gt;AI_FORECAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;TABLE&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sales&lt;/span&gt;
        &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt;
        &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;horizon&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2027-01-15'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;time_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'ds'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;value_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'sales'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;group_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'region'&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The result has the columns &lt;code&gt;ds&lt;/code&gt;, &lt;code&gt;region&lt;/code&gt;, &lt;code&gt;sales_forecast&lt;/code&gt;, &lt;code&gt;sales_upper&lt;/code&gt; and &lt;code&gt;sales_lower&lt;/code&gt;. The horizon is right-exclusive, so this query returns the 60 days from 2026-11-16 to 2027-01-14. The output contains forecasted rows only, not the history. This query uses version 1, because version 1 is the default when you do not pass &lt;code&gt;version&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: Switch to version 2
&lt;/h2&gt;

&lt;p&gt;Version 2 is recommended for most use cases, and you select it with the &lt;code&gt;version&lt;/code&gt; argument. Add it to the same query.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="no"&gt;AI_FORECAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;TABLE&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sales&lt;/span&gt;
        &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt;
        &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;horizon&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2027-01-15'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;time_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'ds'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;value_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'sales'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;group_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'region'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2'&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Version 1 is on a deprecation path, and the default will change to version 2 in an upcoming release. Pass &lt;code&gt;version&lt;/code&gt; explicitly so your results do not change when it does.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: Add holidays
&lt;/h2&gt;

&lt;p&gt;Holiday effects are modeled when you pass a region code, such as &lt;code&gt;'US'&lt;/code&gt;. Use it when your series dips or peaks around public holidays.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="no"&gt;AI_FORECAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;TABLE&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sales&lt;/span&gt;
        &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt;
        &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;horizon&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2027-01-15'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;time_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'ds'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;value_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'sales'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;group_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'region'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;holiday_region&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'US'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2'&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The chart compares the forecasts with the actual values for the East region. Sales dip on Thanksgiving, Christmas and New Year in the test data. The forecast with holidays predicts part of each dip, and the forecasts without holidays predict none.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiplf8af1g5g0x25n4t46.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiplf8af1g5g0x25n4t46.png" alt="A line chart. The actual sales line dips sharply on Thanksgiving, Christmas and New Year. The version 1 and version 2 forecasts stay flat through those days. The version 2 forecast with US holidays dips on all three days, by less than the actual values. A shaded band shows the version 2 prediction interval." width="800" height="446"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Actual sales and forecasts for the 60 held-out days in the East region.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 5: Add a covariate
&lt;/h2&gt;

&lt;p&gt;A covariate is a column that explains the value, such as a promotion flag. To use one as a future covariate, add rows that cover the forecast horizon with the covariate filled in and the value column set to NULL.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="no"&gt;AI_FORECAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;TABLE&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sales&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;promo&lt;/span&gt;
    &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt;
    &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;
    &lt;span class="k"&gt;UNION&lt;/span&gt; &lt;span class="k"&gt;ALL&lt;/span&gt;
    &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;CAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;NULL&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="kt"&gt;DOUBLE&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;sales&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;promo&lt;/span&gt;
    &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt;
    &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;
  &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;horizon&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2027-01-15'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;time_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'ds'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;value_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'sales'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;group_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'region'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;covariate_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'promo'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2'&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you fill the covariate only on the history rows, &lt;code&gt;ai_forecast&lt;/code&gt; treats it as a past-only covariate. That is not an error, but the forecast gains little, because the model cannot see the promotions that are planned.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 6: Constrain the forecast to non-negative values
&lt;/h2&gt;

&lt;p&gt;Set &lt;code&gt;positive_only&lt;/code&gt; for series that cannot go below zero, such as returns, units or counts. Without it, the lower bound of the interval can fall below zero even when the forecast itself does not.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="no"&gt;AI_FORECAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;TABLE&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;returns&lt;/span&gt;
        &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;returns_daily&lt;/span&gt;
        &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;horizon&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2027-01-15'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;time_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'ds'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;value_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'returns'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;positive_only&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;TRUE&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2'&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The chart shows a daily returns series that trends close to zero. In the test, none of the point forecasts were negative either way. Without &lt;code&gt;positive_only&lt;/code&gt;, all 60 lower bounds were below zero, and with it the lower bounds stop at zero.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftihrbuxs7t97gdxfg2uc.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftihrbuxs7t97gdxfg2uc.png" alt="Two line charts of daily returns with a forecast line and a shaded interval. In the left chart the lower edge of the interval goes below zero. In the right chart the lower edge stops at zero." width="799" height="411"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The same near-zero series forecast without and with positive_only.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Example: forecast sales by region with planned promotions
&lt;/h2&gt;

&lt;p&gt;A retailer plans promotions for the next 60 days and wants the sales forecast per region to reflect them. The input is the history plus the planned horizon rows with &lt;code&gt;promo&lt;/code&gt; set and &lt;code&gt;sales&lt;/code&gt; NULL, which is the query in Step 5. You save the result so a dashboard can read it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="k"&gt;REPLACE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;fc_promo&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="no"&gt;AI_FORECAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;TABLE&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sales&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;promo&lt;/span&gt;
    &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt;
    &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;
    &lt;span class="k"&gt;UNION&lt;/span&gt; &lt;span class="k"&gt;ALL&lt;/span&gt;
    &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;CAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;NULL&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="kt"&gt;DOUBLE&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;promo&lt;/span&gt;
    &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt;
    &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-16'&lt;/span&gt;
  &lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;horizon&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2027-01-15'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;time_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'ds'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;value_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'sales'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;group_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'region'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;covariate_col&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'promo'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="s1"&gt;'2'&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To measure accuracy, join the forecast to the actual values for the same days and average the absolute difference.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
       &lt;span class="nb"&gt;ROUND&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;AVG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;ABS&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_forecast&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;mae&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;fc_promo&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;
&lt;span class="k"&gt;JOIN&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forecast_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales_daily&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;
  &lt;span class="k"&gt;ON&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ds&lt;/span&gt; &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;region&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;region&lt;/span&gt;
&lt;span class="k"&gt;GROUP&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;region&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The query returns the mean absolute error (MAE) for each region. Running it on each forecast gives the table below for the 60 held-out days. The main finding is the covariate result: a future covariate roughly halved the error, while a past-only covariate gave no real gain. The model can only use what it knows about the forecast window, so planned promotions help and promotions it never sees ahead of time do not. Version 2 with defaults is modestly better than version 1, and holidays help on the holiday days.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjjb9ay42ga2p8n9ef6gm.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjjb9ay42ga2p8n9ef6gm.png" alt="A table with the East and West mean absolute error for five forecasts. Version 1 with defaults is 40.38 and 33.17. Version 2 with defaults is 36.94 and 32.53. Version 2 with US holidays is 33.77 and 31.29. Version 2 with promo as a future covariate is 21.74 and 16.68. Version 2 with promo as a past-only covariate is 37.67 and 32.75." width="800" height="361"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Mean absolute error for each forecast on the 60 held-out days, lower is better.&lt;/em&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Forecast&lt;/th&gt;
&lt;th&gt;East MAE&lt;/th&gt;
&lt;th&gt;West MAE&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Version 1, defaults&lt;/td&gt;
&lt;td&gt;40.38&lt;/td&gt;
&lt;td&gt;33.17&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Version 2, defaults&lt;/td&gt;
&lt;td&gt;36.94&lt;/td&gt;
&lt;td&gt;32.53&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Version 2 with holiday_region US&lt;/td&gt;
&lt;td&gt;33.77&lt;/td&gt;
&lt;td&gt;31.29&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Version 2 with promo as a future covariate&lt;/td&gt;
&lt;td&gt;21.74&lt;/td&gt;
&lt;td&gt;16.68&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Version 2 with promo as a past-only covariate&lt;/td&gt;
&lt;td&gt;37.67&lt;/td&gt;
&lt;td&gt;32.75&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The data in this guide is generated. It has a built-in promo effect, with promotion days selling 30% more, and built-in dips on US holidays, so these results show how the arguments behave, not the accuracy you will see on real data. Use them to compare the options, and run the same comparison on your own history. The full example, including setup and cleanup, is in the &lt;a href="https://github.com/ven-nbondala-tf/databricks-release-lab/tree/main/databricks/ai-forecast" rel="noopener noreferrer"&gt;ai-forecast&lt;/a&gt; folder of the repository.&lt;/p&gt;

&lt;h2&gt;
  
  
  Things to know
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Version 1 is the default. Pass &lt;code&gt;version =&amp;gt; '2'&lt;/code&gt; to use version 2, and do it even after the default changes so your queries state what they use.&lt;/li&gt;
&lt;li&gt;The version 2 arguments &lt;code&gt;covariate_col&lt;/code&gt;, &lt;code&gt;holiday_region&lt;/code&gt; and &lt;code&gt;positive_only&lt;/code&gt; work only with version 2. If you leave out &lt;code&gt;version&lt;/code&gt;, the query fails with &lt;code&gt;PYTHON_TVF_UNSUPPORTED_ARGS_FOR_VERSION&lt;/code&gt;, for example: "The function &lt;code&gt;AI_FORECAST&lt;/code&gt; was called with &lt;code&gt;version =&amp;gt; '1'&lt;/code&gt;, which does not support the following argument(s): holiday_region." Add &lt;code&gt;version =&amp;gt; '2'&lt;/code&gt; to fix it.&lt;/li&gt;
&lt;li&gt;The horizon is right-exclusive, and the output holds forecast rows only. To plot history and forecast together, UNION the history with the result.&lt;/li&gt;
&lt;li&gt;A future covariate needs rows for the horizon with the covariate filled in and the value column NULL. Without them the covariate is past-only and adds little.&lt;/li&gt;
&lt;li&gt;The frequency is the mode of the most recent observations in each group. If you set &lt;code&gt;frequency&lt;/code&gt;, it must match the granularity of your data.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Best practices
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Hold out the last part of your history, forecast it and compare, as the tables in this guide do, before you trust a forecast.&lt;/li&gt;
&lt;li&gt;Pass &lt;code&gt;holiday_region&lt;/code&gt; when your series follows public holidays, and &lt;code&gt;positive_only&lt;/code&gt; when it cannot go below zero.&lt;/li&gt;
&lt;li&gt;Add &lt;code&gt;group_col&lt;/code&gt; instead of writing one call per group. A grouped call returns the same values as separate calls.&lt;/li&gt;
&lt;li&gt;Save the forecast in a table and schedule the query, so &lt;a href="https://medium.com/@nagarajubru/databricks-dashboards-permissions-refresh-553e82f3e4a1" rel="noopener noreferrer"&gt;dashboards&lt;/a&gt; read a table instead of calling the function.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;ai_forecast&lt;/code&gt; returns a forecast with an upper and a lower bound from a single SQL query, and version 2 adds holidays, covariates and non-negative forecasts. The two things to remember: pass &lt;code&gt;version =&amp;gt; '2'&lt;/code&gt; to use the new options, and give it your planned covariate values, since that is where the biggest gain came from. Use it when your history is in a table and you want a forecast from SQL without training a model.&lt;/p&gt;

&lt;p&gt;Which series in your workspace would you forecast first?&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/functions/ai_forecast" rel="noopener noreferrer"&gt;ai_forecast function&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/large-language-models/ai-functions" rel="noopener noreferrer"&gt;Transform unstructured data using AI Functions&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/release-notes/product/2026/november" rel="noopener noreferrer"&gt;November 2026 release notes&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>databricks</category>
      <category>sql</category>
      <category>machinelearning</category>
      <category>dataengineering</category>
    </item>
    <item>
      <title>How Databricks Dashboards Handle Permissions and Refresh</title>
      <dc:creator>Nagaraju Bondala</dc:creator>
      <pubDate>Sun, 22 Nov 2026 20:35:31 +0000</pubDate>
      <link>https://dev.to/nagaraju_bondala/how-databricks-dashboards-handle-permissions-and-refresh-4di0</link>
      <guid>https://dev.to/nagaraju_bondala/how-databricks-dashboards-handle-permissions-and-refresh-4di0</guid>
      <description>&lt;p&gt;Databricks dashboards run every query with someone's permissions and answer it from one of several caches, and the choices you make when you publish decide which. This guide follows an AI/BI dashboard after it is built: you publish it, share it, schedule it and add subscribers. It explains whose credentials run each query, where the results come from, and how to keep the data fresh. The complete code for this guide is on GitHub: &lt;a href="https://github.com/ven-nbondala-tf/databricks-release-lab/tree/main/databricks/dashboard" rel="noopener noreferrer"&gt;dashboard&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/yQTWy0OZFVc" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;br&gt;
&lt;em&gt;A short visual tour of how publishing, permissions and refresh fit together.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  What happens behind the scenes when a dashboard runs
&lt;/h2&gt;

&lt;p&gt;A published AI/BI dashboard is a fixed version of a draft that viewers open, filter and refresh. Two things decide what each viewer sees: whose credentials run each query, and where the results come from.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Credentials: with Share data permission, the default, viewers run queries using the publisher's data permissions. With Individual data permission, viewers run queries using their own credentials, and compute access is always granted by the publisher's credentials.&lt;/li&gt;
&lt;li&gt;Results: a dashboard first checks the dashboard cache, then the query result cache. The SQL warehouse starts only when a dashboard runs a query that the caches cannot serve. For small datasets, filtering and aggregation also run in your browser after the first load.&lt;/li&gt;
&lt;li&gt;Cache audience: with shared permissions all viewers share one cache and see the same results. With individual permissions each viewer has a per user cache.&lt;/li&gt;
&lt;li&gt;Drafts and published versions: viewers always see the last published version. For a draft, the viewer's own data permissions always apply.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The diagram shows the life of a dashboard from draft to snapshot.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0hm0nfhv5wlo84o33a32.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0hm0nfhv5wlo84o33a32.png" alt="A flow diagram. A draft is published with either shared or individual data permissions. Both lead to sharing. From sharing, one path is a viewer opening the dashboard, which uses the dashboard cache, then the query result cache, then the SQL warehouse. The other path is a schedule run, which refreshes the dashboard cache and sends a PDF snapshot to subscribers." width="800" height="1210"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The life of a dashboard: draft, publish, share, and the three ways its data is refreshed.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  When to use each publish option
&lt;/h2&gt;

&lt;p&gt;Use Share data permission when many people view the same dashboard and you want one shared cache. Viewers see the same results, and they do not need access to the underlying tables. Use Individual data permission when each viewer must see only the data they are allowed to see, because queries then run with their own credentials and row filters and column masks apply to them.&lt;/p&gt;

&lt;p&gt;The shared option can expose data to people who have no direct access to it, so check who the dashboard is shared with before you publish. The table compares the two options.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw71hdkep473ewtt2e040.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw71hdkep473ewtt2e040.png" alt="A table with two columns, Share data permission and Individual data permission, and rows for who runs queries, whose data permissions apply, compute, who can view, cache and best for." width="800" height="359"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Share data permission and Individual data permission compared.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;A dashboard with at least one dataset, and a SQL warehouse to run its queries.&lt;/li&gt;
&lt;li&gt;CAN EDIT or CAN MANAGE on the dashboard to publish it.&lt;/li&gt;
&lt;li&gt;For account-wide sharing: users who a workspace administrator has registered with your Azure Databricks account.&lt;/li&gt;
&lt;li&gt;For email subscriptions: a workspace admin has left "Enable dashboard email subscriptions" on. Subscribers are workspace users or email notification destinations that a workspace admin has defined.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  Step 1: Publish the dashboard
&lt;/h2&gt;

&lt;p&gt;Publishing creates the version your viewers see. Click Publish in the draft. Edits you make to the draft afterward stay invisible to viewers until you publish again. The dialog also shows a Materialize datasets option, marked Beta, which adds a daily schedule.&lt;/p&gt;

&lt;p&gt;You can also publish with the API. Set &lt;code&gt;embed_credentials&lt;/code&gt; to &lt;code&gt;true&lt;/code&gt; for shared data permission and to &lt;code&gt;false&lt;/code&gt; for individual data permission.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;databricks api post /api/2.0/lakeview/dashboards/&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;dashboard-id&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/published \
  --json &lt;span class="s1"&gt;'{"embed_credentials": true, "warehouse_id": "&amp;lt;warehouse-id&amp;gt;"}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 2: Choose a data permission option
&lt;/h2&gt;

&lt;p&gt;This choice decides whose credentials run the queries and who can view. The Publish dialog offers Share data permission (default) and Individual data permission. With the default, the Publisher Credential field shows the credential used for every viewer's queries. It starts as your user account, and its menu lets you pick a service principal instead, so the dashboard does not depend on one person's account.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmtghwuliglujtzueg0s6.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fmtghwuliglujtzueg0s6.png" alt="The Publish dialog with two cards. The first card, Share data permission (default), shows a Publisher Credential field. The second card is Individual data permission." width="800" height="369"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The Publish dialog with Share data permission selected.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;If you republish a dashboard that was published with shared permissions and choose Individual data permission, the dialog warns that the action pauses all schedules for the dashboard and removes all subscribers.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhstyys8t5ujpz0r4hy5k.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fhstyys8t5ujpz0r4hy5k.png" alt="The Publish dialog with Individual data permission selected and a red warning that the action will pause all schedules for the dashboard and remove all subscribers." width="800" height="369"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Switching to Individual data permission warns that schedules are paused and subscribers are removed.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 3: Share the dashboard
&lt;/h2&gt;

&lt;p&gt;Sharing decides who can open the dashboard and what they can do. Click Share and add users, groups or service principals with one of four levels. CAN VIEW and CAN RUN can view, interact with and refresh the dashboard. CAN EDIT adds editing and publishing. CAN MANAGE adds changing permissions and deleting the dashboard.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9f1ozi77wwncmx356761.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9f1ozi77wwncmx356761.png" alt="The Sharing dialog with a permission menu open. The levels are Can Manage, Can Edit, Can Run and Can View." width="800" height="509"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The four permission levels in the Sharing dialog.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;To share with everyone in your Azure Databricks account, set Sharing settings to "Anyone in my account can view". Users who do not have access to the workspace are limited to CAN RUN, and they cannot open the draft. They use the publisher's credentials to reach compute.&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 4: Set a schedule
&lt;/h2&gt;

&lt;p&gt;A schedule is the reliable way to refresh the dashboard cache. Open the published dashboard, click Schedule and choose Add schedule. Pick the frequency, start time and time zone. Under Advanced settings, choose the SQL warehouse for the run and, if you want, select "Use current filter selections". Click Create. Run now runs the schedule once without changing its regular cadence.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz9ks5jd4x1jbgamm8mp3.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz9ks5jd4x1jbgamm8mp3.png" alt="The Add schedule dialog. It shows the frequency, time and time zone, and under advanced settings a name, a SQL warehouse, a filter selection checkbox, a custom email subject, subscribers and attachments." width="770" height="835"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The Add schedule dialog with its advanced settings.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Step 5: Add subscribers
&lt;/h2&gt;

&lt;p&gt;Subscribers receive a snapshot each time the schedule runs. In the schedule's advanced settings, choose subscribers and, if you want, attachments. Email subscribers receive a PDF snapshot, with optional CSV, TSV or Excel data attachments. A subscription list can hold up to 100 subscribers, and attachments over 9 MB are not included.&lt;/p&gt;

&lt;p&gt;Each run shows an indicator that reports whether it succeeded, failed or was skipped.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0tmupjexwsl8qblols3h.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0tmupjexwsl8qblols3h.png" alt="The Schedules panel with one schedule, one subscriber, four empty indicators and a green check for the last run." width="770" height="240"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;The Schedules panel after a run, with its subscriber count and run indicators.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Example: refresh a sales dashboard after new data arrives
&lt;/h2&gt;

&lt;p&gt;A sales table feeds a dashboard with two datasets, a Region filter and a Minimum amount parameter. New rows arrive, but viewers keep the numbers from the cache until it is refreshed. The setup SQL creates the table, and a second statement stands in for a pipeline run.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dash_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'North'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Laptop'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-20'&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After this insert, reopening the published dashboard still shows 50 orders and 14,600 in revenue for North. Clicking Refresh on the dashboard shows 60 orders and 24,600.&lt;/p&gt;

&lt;p&gt;For the scheduled path, a second insert adds 5 West rows.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;catalog&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dash_demo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sales&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="mi"&gt;2000&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'West'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Phone'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;DATE&lt;/span&gt;&lt;span class="s1"&gt;'2026-11-20'&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After Run now on the schedule, reopening the dashboard shows West at 55 orders and 22,950 without clicking Refresh.&lt;/p&gt;

&lt;p&gt;The full example, including setup and cleanup, is in the &lt;a href="https://github.com/ven-nbondala-tf/databricks-release-lab/tree/main/databricks/dashboard" rel="noopener noreferrer"&gt;dashboard&lt;/a&gt; folder of the repository.&lt;/p&gt;

&lt;h2&gt;
  
  
  Things to know
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Drafts always use the viewer's permissions. Even when a dashboard is published with shared permissions, anyone opening the draft runs queries with their own data permissions, so check the draft as an ordinary viewer before you publish.&lt;/li&gt;
&lt;li&gt;Share data permission can expose data. Folders pass their permissions down, so anyone with access to the folder that holds the dashboard can also open it. Keep dashboards that use shared permissions in a folder only the right people can reach, and grant access to the dashboard itself.&lt;/li&gt;
&lt;li&gt;Account users have limits. Users without workspace access are limited to CAN RUN, and account users cannot see data from workspace-bound catalogs. Widgets that query them show no data, so do not publish such dashboards to account-wide audiences.&lt;/li&gt;
&lt;li&gt;The dashboard cache can be up to 24 hours old. A change to the table does not refresh it, and a pipeline step that refreshes the data does not refresh it either. Add a schedule if viewers must see current data.&lt;/li&gt;
&lt;li&gt;Choose the permission option before you set up subscribers. Switching a published dashboard from Share data permission to Individual data permission pauses all its schedules and removes all its subscribers. The Publish dialog warns about this, and testing confirmed it. Settle on the option first, then add the schedule and the subscribers.&lt;/li&gt;
&lt;li&gt;The Refresh button on a published dashboard runs the queries again. The documentation pages read do not describe it, but in testing it showed new rows straight away. Use it to check the latest data yourself, and use a schedule for everyone else.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Best practices
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Schedule the dashboard right after the pipeline that feeds it, for example 15 minutes after the pipeline normally finishes, so the cache is refreshed with the new data.&lt;/li&gt;
&lt;li&gt;Publish production dashboards with a service principal as the Publisher Credential, so the dashboard does not stop working when one person's account changes.&lt;/li&gt;
&lt;li&gt;Choose Individual data permission for any dashboard that shows data with row filters or column masks.&lt;/li&gt;
&lt;li&gt;Test the dashboard as a viewer with the lowest permission level you plan to grant, and open the published version, not only the draft.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;A Databricks dashboard runs its queries with the publisher's credentials or the viewer's own, depending on the publish option, and it answers from the dashboard cache, the query result cache or the SQL warehouse. The main takeaway is that the dashboard cache can be up to 24 hours old and only a schedule refreshes it reliably. Publish with the option that matches who may see the data, then schedule the dashboard right after the pipeline that feeds it.&lt;/p&gt;

&lt;p&gt;What is the first dashboard in your workspace that should have a schedule?&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/dashboards/share/share" rel="noopener noreferrer"&gt;Share a dashboard&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/dashboards/concepts" rel="noopener noreferrer"&gt;Dashboard concepts&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/dashboards/caching" rel="noopener noreferrer"&gt;Dataset optimization and caching&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/dashboards/share/schedule-subscribe" rel="noopener noreferrer"&gt;Manage scheduled dashboard updates and subscriptions&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/dashboards/limits" rel="noopener noreferrer"&gt;Dashboard limits&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/security/auth/access-control/" rel="noopener noreferrer"&gt;Access control lists&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/ai-bi/admin/" rel="noopener noreferrer"&gt;AI/BI administration guide&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://learn.microsoft.com/en-us/azure/databricks/ai-bi/consumers/" rel="noopener noreferrer"&gt;What is consumer access?&lt;/a&gt;, Microsoft Learn&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>databricks</category>
      <category>dataengineering</category>
      <category>dashboards</category>
      <category>analytics</category>
    </item>
  </channel>
</rss>
