AWS re:Invent 2025 - Master AI model development with Amazon SageMaker AI (AIM272)

🦄 Making great presentations more accessible.
This project enhances multilingual accessibility and discoverability while preserving the original content. Detailed transcriptions and keyframes capture the nuances and technical insights that convey the full value of each session.

Note: A comprehensive list of re:Invent 2025 transcribed articles is available in this Spreadsheet!

Overview

📖 AWS re:Invent 2025 - Master AI model development with Amazon SageMaker AI (AIM272)

In this video, Ankur and Mark Andrews present AI model development with Amazon SageMaker AI, focusing on creating differentiation through custom models. They explain model learning stages from pre-training to post-training, introducing SageMaker HyperPod's checkpointless training that recovers from faults in seconds and achieves 95% GPU utilization. Key features include elastic training, task governance for cluster optimization, and comprehensive observability. The session covers fine-tuning techniques (supervised fine-tuning, reinforcement learning, DPO) and announces serverless training jobs, model evaluation capabilities, and serverless MLflow. Mark introduces Amazon Nova Forge, enabling customers to build proprietary foundation models by blending their data with Amazon Nova curated data while preventing catastrophic forgetting. Reddit achieved a 26-point precision lift using Nova Forge for moderation. The platform offers code-based, UI-guided, and agent-guided paths with full lineage tracking and governance.

; This article is entirely auto-generated while preserving the original presentation content as much as possible. Please note that there may be typos or inaccuracies.

Main Part

The Challenge of Differentiation: Moving Beyond Generic AI Models

All right. Welcome everyone, and good afternoon. My name is Ankur, and I'm joined by my colleague here today, Mark Andrews, and we're going to be talking to you about AI model development with Amazon SageMaker AI. So let's dive in with the key challenge. These days, when I talk to customers, they often tell me how easy it has become for them to access generative AI models and start using them. But they also tell me that because of the same fact, now they feel that everybody has access to the same models. In fact, their competitors have access to the same models as they do. So they ask me, well, how do we build differentiation?

Now we think that differentiation is a key component of innovation, as is creating value for customers. But how do you create differentiation when the models that you work with are generic? We don't think that creating differentiation will happen through building better AI with the same commodity foundation models. We think that to create differentiation, what you need are models that deeply understand you and your business. But how do you get that competitive advantage?

Well, first, let's acknowledge that the generative AI models that are out there today are amazing. They can create, write, summarize, reason, you name it. They're genuinely intelligent and they hold a vast amount of information. But they don't know about your business or your domain. They don't know about your data. They don't understand the hidden patterns in your transactional history or your support tickets or the domain expertise that you have accumulated over the years. They also don't know about your constraints in your business, what's considered safe and compliant in your business, and what's considered high quality.

So ultimately, building models that can deeply understand you really boils down to how you make the model learn what you want it to learn. So it's important to understand how a model learns. Well, it turns out models learn in a step by step process. In the first stages of model learning, they learn the generalized world knowledge. Then they start learning more specialized knowledge, and then finally, they get better at executing tasks. This is actually not much different than how humans learn.

How Models Learn: From Pre-training to Real-World Application

So let's actually take a look at the human learning journey and how it compares to how models learn. So let's start with pre-training, which is the first stage of model learning. At this stage, the model basically mimics a child who is learning language by immersion. It consumes a vast amount of text, images, and other data, and it learns how to predict the next token. At this stage, the model is not learning any specific tasks, but it is the foundation that is created at this stage that it draws upon later during inference, much like how humans recall early language skills in daily conversations.

Now, after having mastered basic language skills, the model is ready to learn how to follow instructions. So it is exposed to pairs of input and output, questions with good answers, and through that, it learns how to stay on topic, provide structured explanations, and be generally helpful. This is much like middle school students learning how to write essays and follow directions.

Now the next stage is preference optimization. At this stage, the model learns how to not just be correct, but how to be good. So it's exposed to ranked and rated examples of responses that it learns from, and it understands what humans think is concise, clear, and aligned with social norms. This is similar to how high school students refine their communication and tone through feedback from teachers. And in our model's case, the model is learning judgment and empathy.

Now college represents a shift from memorization to reasoning. So this is where the model practices multi-step thought, problem decomposition, and thinking through and providing structured answers.

This is commonly done through techniques such as reinforcement learning, and this is quite similar to how college students are able to synthesize ideas across multiple disciplines. After graduation, in the workforce, the model has to switch from learning to doing. At this point, the model has to apply whatever it has learned to real-world tasks. Much like many professionals who like to continue taking educational courses while working, models can still continue to learn. Through techniques such as LoRA and parameter-efficient fine-tuning, models can continue to learn even when they're deployed in production without having to retrain the base model.

Pre-training Challenges: Scaling, Resiliency, and Infrastructure Complexity

So let's talk about pre-training in more detail, which is the first stage of the model's learning journey. During pre-training, the model is trained on a massive amount of tokens from diverse data sources, and through this it learns patterns in the data. It learns how to predict the next token based on a sequence of tokens, and it also discovers grammar and syntax and facts, and it builds internal representations. Also, while there are different scaling laws today, generally speaking, with larger models with more training steps, more compute, and more data, models achieve stronger generalization and capabilities. Note that at the pre-training stage, the model is not learning how to execute a particular task or how to become an expert in any one topic.

Well, this sounds simple, right? But in reality, the pre-training stage is quite complicated to execute due to the multiple challenges that you face when you're pre-training a model. The first one is efficient scaling. These models are large and there's a lot of data being used to train them, so they often don't fit onto a single AI accelerator. So you have to pre-train over a large cluster of AI accelerators such as GPUs or AWS training instances, and you need solutions to be able to efficiently scale across a large number of accelerators.

The next challenge you run into is resiliency. Unlike distributed data computing workloads where every worker is doing its own thing, in distributed training, every process, every training process across all your GPUs in a cluster, is collaboratively training one model. So whenever there is a failure in the system, even if a single GPU fails, the entire training process stops. So resiliency of your infrastructure is extremely important.

Next comes utilization. Let's face it, these AI accelerator-based clusters are expensive. So if they're not fully utilized, it can lead to wastage and cost overruns, so maintaining high utilization is extremely important. You also want to make sure that the engineers and data scientists who are working on your pre-training projects are able to focus on value-added work rather than dealing with infrastructure. So having the right tools to ensure that they are able to be highly productive is important.

While your training is running, it's also important for you to have a good amount of insight into what's happening within your training job and have observability across your training stack. And lastly, there are new open source tools that are becoming available and popular very quickly these days, so you want to make sure that your infrastructure gives you the ability to be able to use new tools as they become available. So the composability of your AI stack is important.

SageMaker Training Options: HyperPod's Persistent Cluster Architecture

Before we move forward, I want to talk to you about the two big ways you can train models on SageMaker. The first one is SageMaker training jobs. This is an ephemeral training job capability where you have your training code and you submit a training job, and it goes and spins up the infrastructure to train your model, does the training, monitors the model, and then spins down the infrastructure. The second option is SageMaker HyperPod. HyperPod provides a persistent cluster of AI accelerators for you to train your models.

HyperPod automatically handles downtime failover and also provides you full control and flexibility in your infrastructure. We're talking a little bit more about HyperPod. So, in HyperPod, you can spin up a cluster of AI accelerators quickly, and you can orchestrate your workloads using either Kubernetes or using SLURM. Many of our customers who have run HPC workloads before prefer to use SLURM as an interface, which is why we provide that choice. In addition, HyperPod is integrated with tools such as SageMaker TensorBoard, as well as provides capabilities for real-time scheduling and prioritization of your training jobs. Also, it has built-in fault tolerance, so it's able to recover from failures quickly, which we'll talk a little bit more about in the next few minutes.

It's also highly customizable. At the top layer, as I mentioned, it supports different orchestration options through Amazon EKS as well as SLURM for job submission. You can use the HyperPod CLI, or you could also directly use kubectl, Ray, the SLURM client, or Composer. It has out-of-the-box support for observability through integrations with Prometheus, Grafana, and CloudWatch. In addition, it supports all popular AI frameworks such as PyTorch, TensorFlow, NeMo, and JAX. And lastly, it also supports all the latest AI accelerated instances on AWS, both GPUs and Trainium, and a variety of different storage options that you can use in your pre-training workload.

Checkpointless Training: Revolutionary Fault Recovery in Seconds

Now one challenge that we've seen with pre-training, as we talked about resiliency earlier, is that as your cluster size grows, the frequency of failures increases. And why is that relevant? It's relevant because, as I mentioned earlier, in distributed training, even if a single GPU fails, your entire training process across the cluster comes to a halt. And as the cluster size grows, the probability of a failure increases because you have more GPUs in the cluster. So as the frequency of failures increases, you're spending more and more time dealing with those failures than actually making training progress.

And why is this a problem? Because when a failure occurs, you have to spend time finding the root cause. Then you have to spend time taking remediation actions such as restarting or even replacing the failed node with a healthy node, and then restarting the entire training cluster. Then you have to restore from the previous checkpoints from durable storage, and then restart all the training frameworks and processes on every GPU in the cluster. And this process can actually take hours.

So in HyperPod, we've created a self-healing capability to handle such faults. So when your training is running and a node fails, HyperPod automatically detects and diagnoses the issue. If a node needs to be replaced, then it does that automatically, and then it automatically restores your training from the last safe checkpoint, and then it automatically resumes training. But there's still a key challenge.

The first one is that there is an all-or-nothing cascade problem here. When the training is stopping because of a fault, then it's stopping on all GPUs in your cluster. Your cluster is completely stopped. It's not doing any work, so that leads to a lot of wastage. Two, all the recovery steps are sequential. First, you're replacing a failed node with a healthy node. Then you are restoring from the last safe checkpoint, and then you're restarting all the training processes. All of that is sequential in nature and blocks every step of the way.

The second thing is the checkpoint-based recovery is expensive because you're loading the checkpoints from persistent storage onto compute nodes, and that can take tens of minutes based on the size of your checkpoint. And at scale, this has a big impact because overall recovery from faults can take up to an hour. So, yesterday in the keynote, we announced checkpointless training on HyperPod. So this capability completely changes the paradigm for distributed training at scale. The way it works is when a failure occurs during a distributed training job,

checkpointless training is able to keep the training processes alive rather than stopping all the training processes. It's able to restore model state through a peer-to-peer mechanism from nearby nodes that have model state. That way it's able to recover from failures without having to restart any infrastructure or training processes and without relying on any checkpoint in durable storage. So essentially you get a restartless and checkpointless system. With that, what that means is that you're able to recover from faults in mere seconds.

Let's take an example of how this works. When a failure occurs in a traditional checkpoint-based recovery, you're diagnosing and triggering a recovery, and then you're reinitializing your cluster. That happens on all GPUs in the cluster. Then you're downloading your checkpoint from S3. Finally, you're also then resuming training. So there's a big difference between the steps that a traditional process follows and how quickly checkpointless training is able to recover from the fault.

Now, all the latest Nova models that we announced this week were all trained using this capability. What we found during those training projects was that using this capability we were able to recover from faults in many cases in seconds and at larger scale within one to two minutes. It also helped us consistently achieve 95% GPU utilization on our training cluster.

Checkpointless training under the hood is underpinned by four important innovations. The first one is optimized collective communication initialization. When a distributed training job is initializing, it has to establish communication channels and every node has to do that. Traditionally the way it happens is that there's a central root server that coordinates establishing all those communication channels, and that creates a big bottleneck. With this capability we've gotten rid of that centralized root server and we've developed a new peer-to-peer mechanism for nodes to discover each other and establish communication channels. That reduces the communication initialization from minutes to seconds.

The second is memory-mapped data loading. When you're resuming training, your training process needs access to training data, and many times you have data pipelines that require preprocessing and it can take several minutes for you to preprocess that data and actually make it available to your training process. This new innovation caches the training data in shared memory through memory-mapped files that persist across training failures. So when your training resumes, it has instant access to training data while your data pipeline is initializing.

The third is in-process recovery. This means that when a failure occurs, this capability does not have to stop the training processes. It keeps the training processes running, so your training basically just pauses and resumes. The last is checkpointless recovery where HyperPod is able to swap failed components with healthy ones through a peer-to-peer recovery of model and optimizer state from healthy AI accelerators rather than relying on an explicit checkpoint.

Maximizing Infrastructure Utilization: Task Governance, Elastic Training, and Observability

Now let's talk about another problem or challenge we face during pre-training, which is infrastructure underutilization. Idle compute happens to be a key challenge here because if your jobs, the way your jobs are running, if they are not utilizing your infrastructure fully, then it leads to compute being wasted and that leads to cost overruns. It also leads to suboptimal prioritization because if you have high priority jobs waiting in the queue because your low priority jobs are running, then obviously you're not working on the most important things. This wastage also increases your cost and reduces productivity for your developers because oftentimes they're waiting for infrastructure availability rather than being able to make progress.

So we've solved this in SageMaker through HyperPod task governance, which automatically runs your jobs according to their priority. And you can also adjust these priorities in real time. It also offers you real-time observability, so you can monitor your overall cluster utilization both at the cluster level as well as at job and team level. This helps you reduce idle compute and minimize task wait time. And it helps you increase and maximize your overall compute utilization in your cluster.

So the way it works is with this capability, you can go into HyperPod and you can set up different types of tasks, such as training, fine-tuning, experimentation, or even inference, and you can give them weights. Then you can set up compute allocations. You can define your teams or projects and allocate compute limits to them. And finally, you can also specify how idle compute is allocated whenever there is free compute capacity available within the cluster. You can control whether it's distributed to different teams or projects or jobs on a first-come-first-serve basis or using a fair-share way, which uses the weights that you define for different tasks to allocate capacity.

In addition, you can set up lend and borrow rules. So here you can define whether a team can automatically borrow compute capacity from another team when it is available, or can a team lend its compute capacity to another team when it's available, and all of this happens automatically. You can also control whether higher priority tasks can preempt lower priority tasks, so you're always working on the most important things.

So let's talk about another challenge related to infrastructure underutilization during pre-training. Now, distributed training jobs are quite inflexible. What I mean is that once you've started the job with a set of nodes, let's say you're running a job with eight nodes, then until your job has completed, you have to continue with the same node configuration. But oftentimes the compute availability changes. Sometimes more compute may become available and you may want to use it in your existing training job to make it go faster. Or other times there might be another high priority job that may need some compute, and you may want to take some compute out of the training job that you're running to make that job go faster.

However, today, if you need to do that, you need to completely stop the training workload that you're running, need to completely reconfigure it with the new node configuration, and then completely restart and rerun it. So that obviously leads to a lot of wastage and delay. So to solve this, yesterday we also announced elastic training on HyperPod. What this does is, when your training job is running, it helps you define scaling rules for your training job, and when there is extra capacity available, it helps your training job automatically scale up, and when you need more capacity out of this training job, it can also scale down and give you capacity back while continuing to execute. This is also done in a way that does not impact model convergence because it shrinks and grows in terms of data parallel replicas.

So let's see how it works. So you've got a training job running. You have four nodes become available and so your training job automatically scales up and then further scales up. And then, when you need capacity out of this training job, because let's say your inference is scaling up and you need capacity to scale inference up, it can also easily scale down and the training continues to make progress throughout this process.

Let's talk about another key challenge of observability. So, when you're running your training job, it's very important for you to have a very deep understanding of what's going on in your cluster, right? So I'll give you one example. Some time ago, before this capability existed, our science team at AWS, they were working on some training projects, and they saw a dip in the model training performance. And they were wondering why that's happening, and it took us days to figure out that that was happening because of certain temperature fluctuations at the GPU level. And there wasn't good tooling to actually root cause that quickly.

Your training stack is like a layered cake. It has so many different layers from your training code to your training frameworks to compute, networking, and the actual hardware. So it's very important that you have high quality observability in your training solution. That's why we built HyperPod observability. This gives you one-click and out-of-the-box support for Amazon Managed Prometheus and Grafana. It offers preconfigured metrics across all layers of the stack, from your job level metrics to compute layer to networking layer and hardware. And this scales automatically as your cluster scales.

Post-Training Techniques: Supervised Fine-Tuning, Reinforcement Learning, and Model Evaluation

OK, so we've talked about how our model learns more general information through pre-training. Let's now talk about post-training or model customization, because that's where we create real value and real differentiation. So at a high level, there are many different fine-tuning or model customization techniques available today. Let's talk about each of them and see where they are relevant.

So the first one is supervised fine-tuning. With supervised fine-tuning, the model directly learns from labeled input-output data, and it can use this to improve overall task performance and understand your domain. And this is actually quite useful when you have clear ground truth available with you. Commonly used use cases for fine-tuning include customer support Q&As and summarization tasks where you have gold standard references, or even domain adaptation when you need to adapt your model to jargon or terms from the medical field or legal or finance.

Next, let's talk about reinforcement learning. Reinforcement learning is interesting. The way it works is the model generates multiple outputs, and then it receives a reward or a penalty, and based on that it knows what to do and what not to do. And so it's extremely useful for long-form reasoning and adherence to multi-rule policies. Now, the way RL works is there's a policy model in the system, in the RL system, which is basically the model that you're fine-tuning. It generates rollouts, which are basically different outputs for a given prompt. Then a component computes rewards using a reward function or a model, and then an optimization algorithm updates the model based on those rewards.

And there are a few different variations of this. When the reward is computed based on labeled human data, this is known as RLHF or RL from human feedback. When an AI judge or AI model acts as a judge and computes these rewards, then it is known as RL from AI feedback, and this is quite useful when there's a large amount of reward computation that needs to happen and you don't have enough human resources to label that data. And lastly, for cases where you can programmatically verify the accuracy of model output, you can also write a reward function to do RL, and this is known as RL from verifiable rewards. And this is easy because it's scalable, it's objective, it has less bias, and for specific tasks such as coding and tasks with clear correctness signals, this is quite a popular technique.

And the last one is direct preference optimization. So DPO is a close cousin of RL but not quite RL in the sense that with DPO you're telling the model, you show the model that hey, I prefer this response for a given prompt, I prefer this response over the other, and through that the model learns your preferences. And so this technique is quite useful for subjective or style-driven learning. So let's say you want to make your model learn about your brand voice or values or the tone that you want in your chatbot.

And it's also easy to implement because it doesn't require the full reinforcement learning setup. But here's the reality. These techniques are great, but to actually implement them and put them to use, you have to wrestle with a lot of different things. First, you have to deal with a lot of infrastructure setup because there are different components involved, from accumulating data to running these different techniques, and the setup for all these different techniques is actually quite different. And then you have to evaluate models, and if you're doing this at enterprise scale, then you also have to worry about governance and who has access to what and whether you have full lineage tracking in place. So all in all, you're spending more time plumbing than actually doing value-added work.

So this is where SageMaker comes in. SageMaker acts as a unified runway for model customization. First, it offers a broad choice of models that you can customize out of the box, including the Amazon Nova models and other open weights models such as GPT OSS, Llama models, Qwen, DeepSeek, and more. And it supports all the popular different fine-tuning techniques, from reinforcement learning to supervised fine-tuning and direct preference optimization. And the entire process for model customization in SageMaker is fully managed, so you don't have to deal with any infrastructure management.

So yesterday, we also announced serverless training jobs on SageMaker, which can help you customize these models without having to deal with any compute capacity. It's very simple. You basically pick your data and pick the model that you want to customize, and you can either use an API or the UI to kick off a completely serverless training job.

Now, once you've customized your model, you also want to make sure that it's actually better than the model that you started with. To do that, evaluation of the model that you've customized is extremely important. So for that, we've also launched a new model evaluation capability. Again, this is available both through the API as well as our UI, which is SageMaker Studio. And it supports multiple modes of evaluation. You can evaluate a model using standard industry benchmarks, or you can use an LLM as a judge, where an AI model acts as a critic for the model you're evaluating, and you can also define your own custom metrics for evaluation.

And then when you're running your training, you also want to make sure that you're able to capture and look at different metrics such as training loss or your reward metrics. MLflow has emerged as a popular tool for doing such experiment tracking and looking at different metrics. So this week we also announced a serverless MLflow in SageMaker, and so you can use serverless MLflow to start logging metrics from your training experiments and review them easily. And also, when you customize a model using our serverless training jobs in SageMaker now, your metrics from your training job automatically show up in MLflow.

Three Paths to Model Customization: Code, UI, and Agent-Guided Approaches

Okay, so we talked about the tools and techniques, but how do you use them, right? Well, there are three ways in SageMaker. One is the code-based path where you have access to an SDK to easily customize a model. The second is a UI path where you can use an easy-to-use UI in a point-and-click fashion to customize a model. And the third one is an agent-guided path. So this is where an agent can work with you and collaborate with you to design a customization task and actually execute on it.

So this is an example of the code-based path. Using the SageMaker AI SDK, you can start customizing models in a few minutes, in a few lines of code. Also, you can write these codes in whatever IDE you want, but you can also use built-in JupyterLab and open source VS Code IDEs within SageMaker. And also, you can start with code and then jump to UI or vice versa.

So next is the UI-guided fine-tuning. So the way this works is that you can go to models,

you can select a model and say you want to customize with the UI. Let's say you give it a name, select your fine-tuning technique. In this case, we are using RLVR. And then you can select a reward function that you previously created. You select your data and then you can simply just submit the training job and off it goes and executes.

Now, once your training job is running, you can monitor its performance. And once the model is done, you can also evaluate the model. You can create a new evaluation job. You can select, let's say in this case, if you're selecting an LLM as a judge approach, you can select the judge model. You can also select the metrics that you want to evaluate the model on. You can also provide your own custom datasets and prompts for evaluation. And then when you kick off the evaluation job, it's completely serverless and it just executes in the background.

Now going back to the custom model that we've created here, you can monitor different metrics as the job is running. You can also go back to your evaluation job and look at different evaluation steps and their progress. And then once the evaluation job has completed, you can look at evaluation results on the UI or you can also download the results via S3. In addition, when you've evaluated the job and now you want to deploy it, you can easily just click the deploy button and deploy either on a SageMaker inference endpoint or import it into Bedrock. So here, we're choosing a SageMaker endpoint to deploy this model to.

So in this case, right now it's deploying, now it's done, and once the model is deployed, you can open the playground and you can actually start chatting with the model to test the output. Here we're just going to put some sample text and see that the model actually responds, which is great. Now we can go back to our model page. And in addition to the metrics, what we also have is other information such as lineage. So with our lineage tool, you can see all the changes that the model has gone through, what data was used, what job was run, what approvals were used to run those jobs, and what evaluations were done. You can also see all the training logs, and this is available as the training job is executing.

All right, so now let's move and move to the agent-guided path and see how that works. So again, you can choose the option to customize a model with an AI agent. You can give your conversation a name, and then you can in simple language describe your use case. And then the agent asks you questions to clarify your use case. It asks you to select a base model, and then it generates a use case specification which describes the use case and success tenets, which will later be used to evaluate the model.

Finally, it also generates some, based on your use case, it also generates some data examples which you can edit or improve. And then it generates a data generation specification, which determines how your data for this model customization will be generated. It also then goes ahead and generates synthetic data for you using the examples or using contextual data that you provide. And once that's done, you can look at the samples of the synthetic data generated and you can also get data quality metrics such as data diversity and other statistics as well as Responsible AI metrics.

Once that's done, because the agent recommended RLVR, it also gives you a starting reward function template based on your use case which you can further edit and approve.

Once you've done that, now you're ready to kick off your training job. You can start the job just by talking to the agent, and then you can also review and monitor the training progress within the same screen. Once your training is done, you can also review the evaluation criteria in tenets and make any revisions to it. Once you approve it, the agent goes and runs the evaluation.

Once that's done, you can easily see the evaluation results. So here you see that the fine-tuned model actually performed much better than the base model on a number of different metrics. And after that, you can also go and deploy the model from the models page to SageMaker or Bedrock.

SageMaker AI Platform: Choice, Efficiency, and Security at Scale

So to summarize, there are three important pillars when it comes to model development on SageMaker AI. The first is choice, and we use this as a core design tenet in all our capabilities. We offer choice of different models that you can use and customize, choice of different customization techniques, and also choice of different interfaces from UI to SDK to agent. We also focus on efficiency, so with the new serverless experience, you don't have to worry about infrastructure management, and it reduces the time for customizing your models from months to days.

When you're doing AI at scale, doing it safely and securely is really important. So with built-in governance and lineage capabilities, you don't have to worry about how your models are being used and who has access to what, and you have full visibility into the changes that have gone into the model. Plus, you can securely control access with IAM and run these jobs in your VPC.

All right, so now let's move on to talk about how you can build with Amazon Nova on SageMaker, and for that I'll call upon my colleague Mark Andrews. Awesome, thank you, Ankur. All right, hey folks, hopefully you may have seen the announcement on Tuesday about Amazon Nova, and we had a talk yesterday at 3:25 if I'm remembering correctly, so we're excited to go into this in a little detail here again too. This is a bit of a refresh if any of you were in that session yesterday, but we'll blast through it. There is a little bit of new content.

Okay, so roughly, you all operate in an environment where you have lots of intellectual property, your own proprietary data, which is unique to your company. So this is the whole concept of Nova Forge. Essentially, we're allowing you to bridge this gap here between foundation model knowledge on the left and organizational knowledge, which is essentially central to your company. So we want to essentially bridge that divide through the program we're calling Nova Forge.

Just to kind of recap, to help you kind of level set on why is Forge useful, there are many different techniques you can use. On the top left here, this is like the RAG-based approach where you're essentially bringing knowledge context from an external source into the model. On the top right, we have adaptations, which is LoRA-based fine-tuning is the example here, where you're essentially enhancing the foundation model a small bit in various dimensions.

On the bottom left, we have expansion. This is essentially where continued pre-training with open weights models might come in, but as you can see from the graphic here, the risk there is it catastrophically forgets basic things that the model needs to be pretty proficient in, like instruction following. And then on the bottom right-hand side, building your own model from scratch. However, building your own model from scratch is enormously expensive and time-consuming.

So that's why we are excited to announce Nova Forge. This bridges that gap. It essentially allows you to accelerate building your own foundation model, leveraging all of the benefits of the SageMaker platform that Ankur went into. SageMaker is the best infrastructure platform in the world to build your own foundation model, and that's why Nova Forge is built on top of it.

Amazon Nova Forge: Building Custom Foundation Models with Proprietary Data

So just as a quick recap, on Tuesday, Matt Garman basically announced the newer generation models, so the second-generation models that are here on the slide. That's Nova 2 Lite, and on the bottom, we have the early access models, Nova 2 Pro and Nova 2 Omni. As you, if you have seen it, you will also see that the performance of these models on the third-party validated benchmarks are very equivalent to the leading models from all of the other leading model providers.

So if you start with Nova Forge, especially our Nova 2 family, you're starting from a great baseline. Okay, so roughly Ankur went into this in detail, so I'll breeze through some of this, essentially.

You start with an empty model. This is ultimately giving you a precursor with respect to how you build a model from scratch. You start with the scaffolding, essentially the foundation model scaffolding. The base model is built through the pre-training, which Ankur explained earlier, and then mid-training where you refine the context. Think of it like pre-training being like the Gray's Anatomy medical text, and mid-training might be specializing in dermatology. And then on the right-hand side is your fully fine-tuned model where you might tweak various clinical practices and so on, based on your business.

This is the same thing again. It's essentially taking your long-form context text, like the Gray's Anatomy medical text, plus the second is your focus into dermatology, and the third is post-training fine-tuning, as Ankur also explained. You have Supervised Fine-Tuning and Reinforcement Learning. One example I like to think of just to keep it simple is dog training. Supervised Fine-Tuning is like your obedience school training. This is teaching the dog the basic concepts like sit, roll, and so on. Reinforcement Learning is essentially you take the dog to the park. Now there are lots of different distractions. Maybe you want to teach the dog new techniques. This is where you essentially provide reward functions. In this case, if the dog does something good, you give it a treat. If it doesn't do something that you want, you say no. Ultimately, the end result is you have a very aligned post-trained model that behaves exactly how you want it to behave.

So building with Nova Forge, the exciting part here, there are five key things that we've done. Access to model checkpoints across all phases of model development, which I'll show briefly on the second slide after this. The ability to blend your proprietary data with Amazon Nova curated training data. Performing reinforcement learning with reward functions in your environment, which is actually a key one that we will also go through. Using push-button recipes, so this is accelerating the model development cycle itself, and then using our Responsible AI toolkit if you wish. As we announced yesterday, Reddit was a key development partner for us, and they built one of the world's leading moderation solutions using Nova Forge. They couldn't do this with any other model off the shelf. They used Nova Forge because it gave them the flexibility to create the most highly accurate proficient model for their use case.

So building a model with Forge, essentially this is exactly as we described earlier. You have large volumes of unstructured data. This is like the Gray's Anatomy text example in the pre-training. You have your specialist domain knowledge like the dermatology in your mid-training, and then you have instruction following to essentially have a highly performant post-trained model. So in looking at this, the reason why we see a lot of advantage to your businesses being able to bring in your intellectual property, your assets in terms of what your company is proficient in, is most of the learning model, most of the, again using the brain analogy from earlier, neuroplasticity is highest earlier in the process. This is where much more data is coming into it. I think Matt Garman used on Tuesday the analogy of a child learning a new language. It's much easier for a child to learn a new language than it is for an adult to use a new language. And as a result, before we get to the post-training phase, there's much more neuroplasticity earlier in the process of training, which is why we believe bringing your data in at that point is actually much more proficient. It imbues the model with the competence of your domain data. And then post-training is still a very useful function. This is where you refine the model and tweak it for your specific performance reasons.

So one of the key points here is I mentioned earlier, catastrophic forgetting is a big risk with respect to open weights models. The way we've helped prevent that is through our proprietary data mixing technique, which essentially allows you to blend your proprietary data with Amazon Nova curated data. We'll show a slide in a moment where it essentially indicates that you can blend across different domains like math, science, and so on, and you can dial up or dial down the data that you want to reinforce or want it to forget. But the important part is this data mixing is actually critical to helping prevent catastrophic forgetting. Without it, what can easily happen is if you bring in too much data, models can forget very fundamental tasks with respect to instruction following and others.

This is not just a hypothetical. Just take an example of Amazon Stores. They improved their shopping MMLU, the Massive Multitask Language Understanding, by 9.8%, and indeed it actually improved by 2.7% on standard MMLU tasks.

So building your model with Nova Forge, there are two things here that are worth mentioning with respect to reinforcement learning. You can essentially bring your reward functions into the AWS environment yourself, but the key interesting part here is that you can actually incorporate your own reward functions in your very own environment through an API that we provide. This gives you much more flexibility within your own applications to essentially send the signals back into the Forge development environment and into SageMaker to reinforce the model live, and then you can essentially iterate as much as you want with respect to how often you want that reinforcement learning data to be incorporated into the core model.

Again, we've been working very closely with key partners here, so Cosign AI, we worked with them to develop this API reinforcement learning solution. This was one big thing that we learned from them, that it wasn't enough just simply to incorporate their reinforcement learning data into Nova Forge, into the AWS environment. They wanted an API to be able to have it basically push that data in real time.

Just talking briefly about Reddit, which I mentioned as well, they had amazing results with their moderation solution. They saw a 26-point precision lift, which is huge in the moderation use case. They've been a key development partner here again, very successful results from Reddit using Nova Forge.

So building your model with Nova Forge, there are two things here that are worth pointing out. We do provide push-button recipes so that it's as easy as possible and fast as possible for you to actually get started, and then you have the maximum flexibility to tweak and adjust as appropriate. The other thing, you can do it all through the CLI, which is on the left, or you can use our UI, which is on the right. The other thing is you can actually interface between the two. So if you want to start with the UI and go to the CLI, you can. You can also go back to the UI if you start with the CLI later too, so they're fully reinforcing one another.

So that gives you maximum flexibility to be able to speed up or slow down or leverage the more advanced techniques as appropriate. And then actually on the right-hand side of the UI, you can see this is our data mixing UI, so you can see for entertainment, you can dial it up to 5% or whatever. For factual, you can have it at a higher percentage. For legal, you can turn it off if you wish, et cetera. And that's a pretty long list of data mixing categories that you can actually dial up or down as appropriate.

The other thing we have, again, you can monitor, of course, the training process itself, and again this is reinforced by the SageMaker platform. So you have the full status of your training jobs as they're running through the process end to end. So building with Nova Forge.

The main thing here is this is built on the same code used to train Nova, so essentially you're building on a very powerful foundation model as it stands. The training code that is used if you bring in your intellectual property is essentially reinforcing your domain knowledge into the Nova model itself, and then we're actually calling these Novellas in terms of the unique models that you may develop as companies based on your data. And it is actually very important to point out that these models that you develop are your models exclusively. We do not have access to them. No one else has access to them. It's completely your flexibility to do with them as you wish.

You can, if you wish, obviously, if you have customers, if you're an ISV or whatnot, actually expose those to ISVs and allow them to further fine-tune and so on. So there is maximum flexibility in what you do with the model, but Amazon does not have access to the model itself. It's completely within your control. The data is also yours. We do not see the data that goes into the model training itself. That is your intellectual property, and that is protected for you.

So as I mentioned before, one of the other component parts that you can leverage if you wish is the customizable content moderation settings. As I mentioned in the Reddit example, they essentially are a moderation platform, or at least the model they developed is a moderation solution. So they want to dial down all of these controls because essentially they're building that capability from scratch. Another example, just a simple example, is if you're law enforcement, you don't want Responsible AI controls to be on because that's the whole point of what you're doing. You're trying to get the model to process potentially disturbing subject matter and whatnot, and that's actually part of the solution that you may want to build. So you have complete flexibility to turn this on or off and dial it up and down as appropriate.

The other thing that's worth pointing out, if you wish to have some assistance or whatnot to be able to help identify the right data and whatnot that you may want to incorporate into building your own model, the AWS Gen AI Innovation Center is there for you. Essentially this allows you to work with them in a professional services context to integrate your business DNA into your own customized model.

So I think just from there, very excited to talk about Nova Forge. Again, we couldn't do it without the benefit of the SageMaker platform. So thank you all for listening, and if you have any questions, we're happy to take them afterwards. Thank you.

---

; This article is entirely auto-generated using Amazon Bedrock.