AWS re:Invent 2025 - Build, govern, and share Amazon Quick Suite dashboards with Amazon SageMaker

🦄 Making great presentations more accessible.
This project aims to enhances multilingual accessibility and discoverability while maintaining the integrity of original content. Detailed transcriptions and keyframes preserve the nuances and technical insights that make each session compelling.

Overview

📖 AWS re:Invent 2025 - Build, govern, and share Amazon Quick Suite dashboards with Amazon SageMaker

In this video, Luis Campos and Utkarsh demonstrate the integration between Amazon SageMaker Unified Studio and Amazon QuickSight (Quick Suite) to solve two critical problems: disconnected dashboard creation from data pipelines and lack of integrated governance for dashboards. The session shows how data analysts can discover data through SageMaker's catalog, request access via subscription workflows, analyze data using built-in query editors, and seamlessly transition to QuickSight with one click to create visualizations. Dashboards are automatically cataloged back in SageMaker for discovery and sharing. Mirko Buholzer from Natera shares how his company leverages this integration in a data mesh architecture, enabling distributed teams to independently manage data domains while maintaining centralized governance, resulting in 80% automated billing eligibility and improved lab operations insights.

; This article is entirely auto-generated while preserving the original presentation content as much as possible. Please note that there may be typos or inaccuracies.

Main Part

Introduction: The Paradox of Choice in Data Analytics

All right, good afternoon, everyone. This is the last session of the day in this room. My name is Luis Campos. I'm a Data and AI specialist from the Netherlands, and I'm here today to share with you how to build, manage, and share Amazon QuickSight dashboards using Amazon SageMaker. I'm going to be joined on stage by my colleague Utkarsh, product manager for Amazon SageMaker, and later on, Mirko, who leads software engineering at Natera.

The agenda that we have for you today is going to be a packed agenda. I'm first going to show you a bit of the map of the cave and lay out the problem and what is that problem, how do we intend to solve it. Later, Utkarsh is going to show you what happens behind the scenes and also an end-to-end demo of this integration. Mirko is going to come and tell his story and how his team is innovating using these technologies. So let's get started.

Now, one of the books by Barry Schwartz, The Paradox of Choice, talks about this notion that in order for someone to be happy, you need to have options. But after a certain threshold, too many options becomes a hindrance and creates decision fatigue. It becomes a problem. One thing I can assure you all here today, the choice to be here in this room was the right one. So thank you for your presence.

Now, this problem is not exclusive to technology. In many day-to-day situations, you will find decision fatigue because of this plethora of options, too many options. Well, retail consumers go through this either in in-store experiences or online with too many products, sometimes from the same category, and being very hard to distinguish what's the difference between them.

In the entertainment industry, we have the same, but we have a different problem. We're seeing more and more people replacing the consumption of selected content for simple swiping. And that has to do precisely with that decision fatigue. I need to choose content to consume, but there are too many options, so I will let the algorithm choose for me.

In companies, in enterprises, the organizations that have to manage data, you have a similar issue with data coming from different sources, separate systems, different technologies as well. And so, once you have all these vast amounts of data that is being sourced, being a consumer of that data becomes a much harder task, let alone if you need to consume that data, transform it, and then serve it to someone else.

From Filters to Recommendations: The Evolution of Data Discovery

Navigating through all these options has always the challenge to bridge these two worlds, the options that we have and the consumer. Now let's go back to the retail example or entertainment example, if you wish. Once you have curated these options and extracted the metadata from the products, let's say from an e-commerce website or let's say a travel agency that has an online website, that metadata is being extracted to create what we call the filters.

So I'll give you my personal example. Every year, I go on Booking.com, and I try to select one of these paradise islands in the Mediterranean, and I try to choose a villa in front of the sea with private pool. And I get so many options. And after that, I use a fourth filter, which is my budget, and it goes down to zero.

So this is the experience that still today a lot of people are grappling in terms of navigating through the options that exist. Like two decades ago, the last two decades, what we have been doing, and all our customers have been doing as well, is trying to detect the footprint of these users on their website to understand their behavior. Also try to capture their comments, their reviews, and then feed a new batch of metadata in order now, instead of being reactive and just give filters to the user, provide them recommendations.

This is not something new, yet it's something that a lot of organizations are still struggling with. And this is not a technology example, this is just from a consumer perspective. Now, let's get a little bit more, let's dive a little bit deeper. Let's transport to the enterprise data world, where not only you have several different sources from several different technologies, sometimes the same data in several different forms, and on the other side, you have humans, you have machines, you have agents, you have software, you have different sizes and different shapes of consumers.

The first step is to curate that data, extract that data, or simply Zero ETL that data over to a lakehouse. At the lakehouse, that's exactly where you're going to start doing your pre-processing. See if your data is clean, if it provides context, but most importantly, to correlate the data between all the sources, because now you have everything in one place. So far, so good, okay?

Once you have this data, you will serve it to different communities inside the organization. Data scientists will use that to create models. But by the way, data scientists, if they want to have access to the detail and raw data, they can always tap into the lakehouse as well. Same thing with BI analysts that will create dashboards, data stories, storyboards, pixel reports, et cetera, which by the way, can also tap into the lakehouse to get detailed data.

All of these actions generate metadata. And so in order to provide metadata to an agent, a piece of software, a human or a machine, you need to have this catalog. This catalog will provide a much needed discovery capability that users want and need, as well as the ability to let them sift through models, GenAI applications, dashboards, et cetera, either an agent or a consumer.

Two Critical Problems: Disconnected Dashboards and Missing Governance

This is an issue today because there are two missing links happening at the same time. First of all, the creation of a dashboard is disconnected from the data engineering pipeline, and what do I mean by this? I mean that when a BI analyst needs to create a dashboard, they need to first source the data and make sure that data is available for the tool that they need. Or sometimes it may happen that the tool will bring the pre-processing into its capabilities, which is a bit of an overkill if you already pre-processed the data.

This happens a lot. These tools getting fatter and fatter in terms of capability. They do more and more, but they are not supposed to. So, where is the data that I need to build my dashboard, and I need to move my data. Turns out that this hinders the time it takes to create a dashboard. The second issue is the opposite side of the same coin.

That plethora of options I was talking about happens for two reasons. First of all, the dashboards that are created, leaving the private workspaces of the BI analyst, or sometimes the business user. And there are a lot of duplications, we know that. And so what tends to happen is that when someone needs a dashboard, they don't know where to find the exact, maybe someone has built something similar, so what do they do?

They come back to square one, which is source the data and build it yet again. There's no integrated governance for dashboards, so we are here today to solve these two problems. And solving these two problems means creating a connected experience through all the three stages of the data life cycle, from collect, connect, and consume. We're going to talk a little bit about Amazon SageMaker and Amazon Quick Suite.

Amazon SageMaker Unified Studio: A Single Platform for All Data Personas

But the key topic for this talk today is the integration between both, okay. Let's just overview what Amazon SageMaker is. SageMaker is, SageMaker and SageMaker Unified Studio were launched in this same place last year, one year ago, as an integrated platform for all your data, AI and analytics needs, with three main layers. The first one is the face of the tool, which we call SageMaker Unified Studio, a single environment.

SageMaker Unified Studio would be used by all these personas: the data scientist, the engineer, the business analyst, and the Gen AI developer. They all will meet in one place, which is the SageMaker catalog. SageMaker catalog will provide them the data discovery capability that they all need to work together. They will have projects, they will have domains, they will have AI-generated metadata, and they will have all these capabilities like glossaries that they'll need to find what they need and to publish their data.

Every persona will use a different tool once they are in SageMaker Unified Studio, because for instance, a data scientist may use SageMaker AI to train and deploy AI models, while at the same time, a Gen AI developer can use the Bedrock IDE to create their custom Gen AI applications. A data engineer may prepare and integrate their data with EMR or Athena, or business analysts or data analysts may run their SQL queries using Redshift. So they all will enter through the same door, but at the end they will use different capabilities of the tool. SageMaker will integrate natively with Redshift, with EMR, with SageMaker AI, and with Bedrock.

Now we're about to bring a brand new member to this family, and that's the BI analyst, which now can also start in SageMaker, use the catalog, and solve those two issues that we were talking about. This means that in the catalog now, side by side with data, models, and Gen AI applications, there will be dashboards as well. You will still have the horizontal capabilities of SageMaker catalog of data lineage and integrated data quality from AWS Glue Data Quality. You also have the publish and subscribe mechanisms that exist for people to share data, models, and dashboards amongst them.

Amazon Quick Suite Integration: Seamless Dashboard Creation and Cataloging

I'm not sure if you were attentive two months ago, but we did a big launch, and that was introducing Amazon Quick Suite. It's a tool that supersedes our previous tool called QuickSight, our BI tool. And I'm going to expand a bit more on what Amazon Quick Suite does. It has three main pillars, and all of these pillars are made consistent by something called spaces. This means that you can create a space, provide data assets inside that space, provide files, dashboards, and something called actions. An action is an integration with a third-party tool. And once you have all that, you can create something new, a new colleague, and that would be your custom chat agent.

You can specialize a chat agent inside that space that will be an expert on all the things you've put inside. So now you can not only talk with your dashboard, you can also have a specialist that knows about that. And most of the times, this represents a big advancement in people that are already working together. Now, Amazon Quick Suite is comprised of several different tools, and I'm going to very quickly skim through them, and then we're going to see how the integration works.

First of them is Quick Index that will allow, through an extensive list of connectors, to connect Amazon Quick Suite to a plethora of external sources from SharePoint to Salesforce and others. The other interesting part would be QuickSight and Quick Research. QuickSight is the tool that you use to create dashboards, you use to create storyboards, you use to create pixel-perfect reports. And Quick Research, we have been using internally at AWS and it's a game changer. You will provide an input in terms of natural language, you will provide the spaces that you have created, and you will toggle something: go to the web or don't. And it will create the topics of a research report you would like to create. And after that, of course, you will be able to customize your own research paper.

The automation inside Amazon Quick Suite is done through the use of Quick Flows, so you will create several different flows. And in order for those flows to connect with the tools, you will have Quick Automate. Automate will be where you create those actions, where you connect to those external sources like Salesforce and SharePoint and whatever tool you use, for instance, for CRM. Now, this is where all of these users will work together inside Amazon Quick Suite,

and the option of the dashboards is where the BI analyst will start, right? Wrong, because the BI analyst needs first to source the data and needs to understand where the data is. So let's layer all these tools that I've talked about on top of this diagram and see how this works.

So of course, the first one is easy. Lakehouse is Amazon SageMaker Lakehouse. The second one would be the experience of Amazon SageMaker Unified Studio, which would be the same both for the builder and publisher persona, but also for the consumer or the agent. And they will all meet in the catalog as the metadata catalog. They will bring all these teams together. Once Amazon Quick Suite comes into play, this means that it will be accessible so that the same catalog will provide the data assets for the users of Amazon Quick Suite to start working. And that's what we're going to explore in this session. Because once you do that, you're going to have governed data governance across your dashboard experience as well.

This means that once, for instance, I'm a BI analyst and I connect, enter, and subscribe to this table that is called customer churn data, now I have access to that through a project. I want to create a dashboard. So where do I start? I just click on open in Amazon Quick Suite. That's it. In the back, Amazon SageMaker will take care of all that integration. You don't need to move data, you don't need to integrate tools. You will land in Amazon Quick Suite with that data asset immediately available to create a data analysis which will result in a dashboard. And once you create that dashboard, the dashboard will become cataloged automatically back in the Amazon SageMaker catalog. How cool is that? Yes, okay.

Demo Part 1: Data Discovery and Self-Service Access in SageMaker

Well, to show you how all this magic works in the background, let me call on stage my colleague Utkarsh. Thank you. Well, thank you so much, Luis. So, Luis talked about how Amazon SageMaker and Amazon Quick Suite together help you simplify the end-to-end analytics journey. Now, let's explore a real-life use case to see how it works in practice.

A data analyst has been asked to explore whether customer support interactions have an impact on customer churn. So it sounds like a simple problem at first, but where do you even begin? Your churn data might live in a Redshift data warehouse managed by a different team. Your customer support data might be in the AWS Glue Data Catalog in a completely different account with no business context at all. So first of all, how do you find this data? And once you find this data, how do you get access to this data? How do you understand this data with business context? And then how do you analyze and visualize this data using the tools of your choice?

Amazon SageMaker Unified Studio and Amazon Quick Suite help you do all of that in a single streamlined governed workflow that Luis talked about, and that is exactly what we'll explore in this demo. We'll start with data discovery. We'll show you how you can use the catalog in Amazon SageMaker Unified Studio to find data that you may not even have access to. We will then show you how you can access that data in a self-service manner. We will show you how you can analyze that data using some of the tools built in within Amazon SageMaker Unified Studio. And then once you have analyzed your data, how you can transition into Amazon Quick Suite so that you can visualize that data and build a dashboard. And once you have built that dashboard, how you can catalog and share that data so that the insights that you have generated do not just simply exist in a silo within your own personal workspace.

Now, before we start, let's take a look at the three personas who will be part of this demo. First, we have Daniel, who is a data analyst, and his responsibility is to turn raw data into meaningful insights. Then we have John, who is a business user. He heads the customer support team, and he's looking to understand how the customer support interactions of his team impact customer churn. And finally we have Priya, who is a data owner. She owns the data in the catalog, and she is responsible for managing access to the data that she owns. All right, so with that, let's jump into the demo and see how the data analyst can start with data discovery.

Here we are in SageMaker Unified Studio. We are logged in as Daniel, who is our data analyst. He's in this churn analysis project, and if you go to the data page, you can see that Daniel already has access to some of the data. For example, he has access to this churn labels dataset. He can see its technical metadata, like which account does the data exist in. He can look at the columns, he can look at the sample data to understand how the data looks like. But what he does not have is access to any data related to customer support metrics.

So let's see if he can find this data in the catalog. He starts by typing in the search bar, and immediately he can start seeing some search results. So let's look at the first search result. It's a table called Support chat metrics. It's a table in the AWS Glue Data Catalog. You can see that there is already some business context about this table. He can also see some technical metadata, like where does this table exist, which account, what is the S3 location of this table. And he can see some other information like the ownership, like who is the owner of this data. So in case he needs to reach out to the owner, he can do that.

On the schema tab, he can see all of the columns, he can see its data types, and he can see the descriptions. It's not just cryptic column names, but a lot of business context. So after all of this business context, Daniel realizes that this is the dataset he needs for his analysis. But how does he get access? All he has to do is just hit the subscribe button, provide a reason for why he needs access, and that's it. He can submit the request.

Now at this point, this request goes to the owner of the data, who is Priya in this case. So let's take a look at how Priya's experience would look like. We are again in SageMaker Unified Studio, logged in as Priya this time, and if we go to the subscription request page, we can see that there are a couple of pending requests. Priya can open one of the requests. She can see who has requested access for what data, for what purpose. And she can choose whether she wants to approve with full access, which means Daniel gets access to the full table, and she can simply approve the request.

Now what happens at this point is once the request is approved, Daniel should be able to access the data, but to be able to do that, we do a lot of orchestration behind the scenes. For example, when this request is approved, SageMaker goes into the underlying data source, which in this case is the table in the AWS Glue Data Catalog. So SageMaker goes into Lake Formation, which is used to manage the permissions on the AWS Glue Data Catalog, and creates a grant on behalf of Priya. And what that means is once that grant is created, Daniel would be able to access the data.

So let's go back to Daniel's project. And if we go to the assets page, you will see that under subscribed data, there's an asset. The same asset that Daniel subscribed to, and it shows up as accessible. Just to confirm that Daniel can actually access it, let's go to the data page, and you can see that the same table also shows up under the Data Explorer. Daniel can access it.

Great, so let's take a pause and revisit our demo flow. So far what we have seen is how Daniel was able to discover new data that he originally did not have access to, how he was able to understand that data with business context, and how he was able to request access and get access to this data in a self-service manner. From this point, Daniel can start analyzing this data. And for this, he's going to use a built-in query editor within SageMaker Unified Studio.

Demo Part 2: From Data Analysis to Dashboard Visualization in QuickSight

So let's take a look at that. He goes to the query editor and he starts exploring the data. Here he's writing a SQL query, he executes it, and he's basically seeing how the churn rate varies with CSAT score, how does the average sentiment look like, how does the average response time look like. Once he has done this exploratory analysis, he decides to create a table with all the dimensions and metrics that he needs to be able to visualize this data. So here, he again runs a query to create that table, and you can see in the Data Explorer on the left-hand side that the table is created.

Great. So with that, Daniel is now ready to visualize this data in Amazon QuickSight. Now, before we jump into QuickSight, let's first understand how this integration actually works between SageMaker and QuickSight. So, with this integration, like Luis was saying, we provide a one-click transition from SageMaker to QuickSight, so that as a data analyst who is working in SageMaker, you can go into QuickSight and visualize the same data that you're able to access in SageMaker.

Now to make that happen, we do a lot of orchestration behind the scenes. First of all, any project that you have in SageMaker, if that project has QuickSight enabled in it, we create behind the scenes a folder in QuickSight with the same name as the project, and any datasets or dashboards that you create in QuickSight will exist in this folder itself.

Any IAM roles or connections that you have created in SageMaker to access the data, we use the same IAM roles and connections, the credentials within those connections to create data sources in QuickSight. So for example, we create an Athena data source, a Redshift data source using the same IAM role and the same connection so that whatever data you are able to access as an analyst in SageMaker, you can access the same data in QuickSight as well.

Any tables that you want to visualize, we can automatically create a QuickSight dataset for that table. So when you land in QuickSight, you have a dataset ready for you to visualize. You can focus your energy on visualizing the data rather than trying to create a data source or go through this extensive setup. With this dataset, you create an analysis and then a dashboard in QuickSight, which is a standard way of creating a dashboard, and once that dashboard is created, behind the scenes, we automatically bring that dashboard as an asset in the catalog in SageMaker.

What that means is from that point you can add more business metadata to that dashboard and make it discoverable in the catalog so that other users who do not have access to your dashboard can find the dashboard. If they want to use the insights that you have generated, they can do that. Finally, we make sure that the membership, members in the project are always in sync with the members in the folder in QuickSight. So anytime you add a member to the project or remove a member from the project, we make sure that the same members are also added and removed from the folder as well.

Great. So with that understanding, let's see how Daniel can go from SageMaker to QuickSight. So he simply chooses this option to open in QuickSight, and we open QuickSight in a new browser tab, and he lands on this dataset in QuickSight, which is automatically created for him. It's the same with the same name as the table that he needs to visualize. Now all he needs to do is simply use this dataset in an analysis. He adds this, he creates an analysis using this dataset.

Now he wants to add a new calculated field for churn rate. So he adds a calculated field. He describes how the churn rate should be calculated in natural language, and QuickSight is able to figure out the right formula for him. All he has to do is just add this new field to his analysis. Now he creates a few charts to understand how the churn rate varies with support metrics. So here he's creating a chart to see how CSAT score impacts churn rate, and he can see that as the CSAT score is lower, the churn rate is higher.

Now he also wants to see how the sentiment in the chat impacts the churn rate. And this time instead of creating a chart manually, he describes it in natural language, and QuickSight is able to create the right chart for him. All he has to do is add this chart to his analysis. So you can see how easy it is to go from SageMaker to come to QuickSight and start building your analysis and charts.

Now, after doing all of this ad hoc analysis, Daniel realizes that customer support interactions are really impacting the customer churn and it's really important to monitor these metrics on a regular basis. So he goes and creates some more beautiful charts like these. Now I won't go into showing you how exactly he built those charts, but I guess you get the point. Once he has built this nice analysis with all these beautiful charts, he wants to make these charts available to other users, and for that, he can simply publish this analysis as a dashboard.

He gives it a name. He enables Q&A. I'm going to talk about the Q&A feature in a minute. And that's it. He can publish this dashboard, which means now he can start sharing and cataloging this dashboard. So let's come back to our demo flow. So far we have seen how the analyst was able to discover and access the data, how he was able to analyze the data using the query editor, and how he was able to go into QuickSight to visualize that data and build a dashboard.

Demo Part 3: Cataloging, Sharing, and Natural Language Insights

Now Daniel wants to make sure that the dashboard that he has built with all the insights don't remain in a silo within his own personal BI workspace. So how can he catalog this dashboard and make it discoverable in the catalog? Let's take a look at that. So, Daniel is back to his project in SageMaker. He's at exactly the same place where he left before moving to QuickSight. Now if he goes to the assets page, you will notice that there is an asset called Churn Dashboard automatically created for him. He did not have to do anything to create this asset in the catalog. It's automatically registered.

Once registered, from this point he can open this asset. You will see that a bunch of information about this dashboard is automatically imported over from Amazon QuickSight, like its URL, which QuickSight account it exists in. And on top of that, he can start adding business metadata. So as an example, he is adding a description to this dashboard. And once he is done adding this business metadata, he can make this dashboard discoverable for others by simply publishing it to the catalog. And once he's published it, you will see that if we start searching in the catalog search bar, this dashboard will start showing up in the search results, which means any user who has access to SageMaker can now find this dashboard.

Another thing that Daniel can do here is that he can also start sharing this dashboard with the other users and groups in the organization from within the catalog itself. He does not have to necessarily go into QuickSight to do that. So here you will see that he's sharing this dashboard with a business user. He can choose to share it with users or groups, but in this case, he's sharing it with a user. And once he has shared this dashboard, again, SageMaker goes into the underlying service, which is QuickSight in this case, and manages the permissions on the dashboard so that this business user can access the dashboard.

So let's see how the experience of a business user looks like. So the business user, of course, can access the dashboard, but it's not just that they can access the dashboard. They can also ask questions and dive deeper into this analysis by asking questions in natural language. So in this case, they're asking, how does the response time impact churn rate, and SageMaker can do all the analysis behind the scenes using the same data that's used to build the dashboard and come up with some charts, as well as a natural language description, explaining how the churn rate is impacted by the response time.

Great, so that concludes the demo. We saw in this demo how data analysts can go from having no access to data to finding the data, understanding it with business context, and getting access to that data in a self-service manner, and analyzing that data and visualizing it, and finally cataloging and sharing their insights in the catalog. Now with that, I'm going to hand over the mic to Mirko Buholzer, VP of Natera, to share how Natera is using these two services to drive innovation.

Natera's Journey: From Siloed Analytics to Data Mesh Architecture

Thank you, Akash. Thank you for being here. I'm going to represent Natera here for this functionality. It was great to partner with AWS to actually get this implemented. Early on we saw this as something we really wanted to have in our data analysis environments, and with this integration of data and visualizations into one environment, it really helped us quite a bit to get the value of our tools up and running. And we're going to talk about a little bit more on how we're using this and why this is quite important for us.

So, Natera was founded in 2004, pioneering the cell-free DNA technology, and we're basically building non-invasive tests which can give us critical information about health conditions early on in the journey of many, many patients. We have labs in the San Francisco Bay Area as well as in Austin, Texas. And it's really key for us on a mission to help as many patients as we can.

So with that, we can aim at transforming the management of disease worldwide, and as you can see from the statement, this is kind of a very moonshot type activity that we would like to achieve. And with that we know this is not something which is done within a month or two, right? We have quite a bit of tests that we're actually already providing from women's health all the way to oncology and organ health, and we have a deep portfolio of tests that we're providing to our patients. The key part here is we want to expand and enhance those tests as quick as possible.

So for us it's very important to be able to iterate very fast and have a quick innovation cycle across our portfolio and be able to basically bring new tests to market very quickly. So as you can see, the innovation is not just around being able to do the scientific innovation itself,

but then also being able to leverage the data that we have from all the other different tests that we're already doing and with that, introducing new tests. So alone in the last five years, we basically brought to market eight tests, and we're able to expand our portfolio quite a bit. For us, scalability is a key component. Signathera is one of the fastest growing genomic diagnostic tests in the world, and with that we need to have a good infrastructure to be able to actually handle all that.

So going a little bit back to the key aspects in regards to combining data with visualization, the journey of many companies basically starts probably with applications having siloed data and analytics tools within each of the applications. Once that stage is past, we are moving into an area where a lot of companies are basically building centralized analytics teams, and with that we're introducing a bottleneck in the organization where we're trying to have one team solve all the answers for everyone else. So what we moved into is really a data mesh model, which allows everyone to collaborate in a very distributed, very self-serve model where we use SageMaker Unified Studio and have the ability to really contribute all the data products and assets to that platform.

For example, here we have a data domain on the lab side which is combining all the DNA data, the clinical data, as well as some of the instrument and execution and operations data onto the platform and moving that and allowing everyone else to use that data across the organization. So this includes obviously interpretation, but then all the way to R&D to develop new tests.

Implementing Distributed Data Ownership and Self-Service Analytics at Natera

A key part from the data mesh standpoint is really having the ability for each of the data domains to be separate and independent. We don't want to have a centralized bottleneck, and with that we want to make sure that the different teams are able to manage their data on their own. This has a big disadvantage where we want to make sure that we have governance, we have security, we have access rights, we know where the data is, we don't copy data around. So to mitigate that, we basically want to have a centralized place with SageMaker Unified Studio to collaborate around the data and being able to do that for the full team, and until now that was mostly around the analytics person, but it wasn't really around visualization.

So with this new functionality, we are now able to centralize including the visualization aspects and the dashboards as well. Obviously on the platform itself, I can use all the different tools which are available. So the SQL UI, the Jupyter notebooks, I can go into MLOps and so on. I have all that capability at my fingertips and can use that, but I'm also now able to bring in the actual business user and the visual aspects and can combine all that together.

What we were doing initially when we introduced this was really making sure we're moving the data responsibility to the different applications. A key aspect that you see many times with analytics organizations is that they start to just copy databases over into the analytics environment, and then you have the central team trying to make sense out of it. What we changed here is really making sure that data is a first-class citizen within the application teams where they define the schema, where they define the actual shape of the data that we want to publish and are able to own that similar to an API or something else within the application where you really have a defined way of how you push that into the system.

The application generates events that are clearly formatted and then moves them into the data mesh. We have an option for serverless transformation as well if needed, but the goal is really to have the application shape the data so we don't have to do a lot of transformation anymore. Everything is basically done at the application level, and we just ingest it into the data mesh and have it available in the catalog.

The other big question we had was really how do we organize our infrastructure with Amazon QuickSight and SageMaker Unified Studio, and how do we make this an ability to really self-serve across the organization. Obviously, since looking at a couple of the items that we already saw, you might see that we really want to have small independent teams being able to run on their own pace. With that, we decided to have lab domains, a commercial domain, and a clinical domain, where each of those domains basically owns certain aspects of the data and manages their own catalog and visualizations. But then combined, we have a company-wide implementation where we can share the golden data sets as well as the golden visualizations that can be accessed everywhere.

With this approach, we don't really have to copy data, so the information is basically just accessed directly in the domain, which makes it very transparent. Obviously, you can make the argument that you want to have everything in one account, which is fine too, but I think this is an approach where we think the teams are really independent and can run at their own pace.

Similar to that, there's the question of how do we organize our teams. Usually you have a technology team which owns the application. As we mentioned, we are basically pushing and shaping the data already into the data mesh, so that's called the application owner. We have product owners who really take that data already, create visualizations, create metrics, create insights where we know what type of functionality or features we need to develop based on the insights we get, and have the ability to really run the application data-driven so that we can develop applications which meet the user's requirements and are valuable.

At the same time, obviously you have a domain owner which makes sure that the different data is quite coherent across the different applications. With that, you generate high quality data because you're basically drinking your own champagne and are making sure that the data that is coming in meets the metrics that you want to be able to develop your application. This model creates high quality data which then can be used by the business users.

On the business side, obviously they might have similar or different metrics. They probably look across different applications as well and combine some of this data. You have analysts doing this, combining some of the data sources, maybe creating new data assets and visualizations from it, and they are able to really work independently on that level. Then obviously sometimes you have the business users come in and say, oh, by the way, I have this idea, can you give me some data around it? That's when the data scientist can just run on their own, create a Jupyter notebook, run the analysis, and come back with a validation.

So the business side really is independent as well. They usually have very small analytics teams within the business organization and are able to just leverage the infrastructure that was already built from the technology organization. There might be some requests when some data is missing, but in general, the goal is really to have those two teams being able to run independently.

Real-World Impact: Lab Operations, Billing Optimization, and the Path to Gen AI

So here we're showing some of those visualizations that have been built. Our lab ops team together with our technology organization obviously has all the clinical metrics.

and execution metrics, but they really wanted to have shop floor level, instrument level visualization and insight of what's going on with each of the instruments. So not just test metrics, but really instrument metrics. This allows them to look at utilization, allows them to look at maintenance windows, and allows them to look at when or predict certain maintenance requests as well and have that available directly on their end. It was really incredible to see them do that in a self-serve model.

The other thing that they can do now with that is really plan demand. So with the utilization of the different instruments, they can now go through what-if scenarios and they can go through planning new lab space or lab requirements based on the insights they already get from the production lab.

Another good example that we have here is around billing eligibility. This is a big topic for our patients. They want to know if their insurance company is actually paying for the test or what they need to pay out of pocket. We worked closely together with the business team, the billing team, the RCM team in this case, together with technology, kind of having three in a box where they're able to take the data that is coming in from the different eligibility responses and then optimizing the rules and the executions to get highly automated billing eligibility responses back from our vendors.

We can see we're close to 80% fully automated where we're getting eligibility responses directly from our system, and believe me, that wasn't at 80% when we started. So it was really great to see the collaboration between the different teams to get up to that level.

Key insights really for us is creating reusable components within the organization where not everyone has to recreate everything from scratch. So starting with data assets or data products that we have in the organization, being able to share that in the catalog, being able to reuse that across the organization is a big advantage. And now with the integration into Amazon QuickSight and Amazon Quick Suite, we have the ability now to also have dashboards with it. A lot of times some of our business users just created their whole catalog of own dashboards which weren't shared across the organization.

And then the other part is really the distributed data ownership. So having the ability to have small teams run on their own pace, being able to innovate, being able to have quick answers, and having the ability to get back to the business owners in a good way is a key aspect that we now have and allows us to really walk and run fast. And obviously all of this is the platform that we need for our Gen AI journey that we're going through. I mean, it's all great to have agents and everything else running, but if you don't really know and have a good data foundation, you're not going to be able to get the results that you want. So with this we're able to combine all of that.

All right, that's it from my end, handing it over to Luis. Just wrapping it up. So I would like to thank your presence here today. Wish you the rest of a good week. If you enjoyed this session, please give us five stars in the app.

---

; This article is entirely auto-generated using Amazon Bedrock.