DEV Community

Karthik Sakthivel
Karthik Sakthivel

Posted on

3

Amazon Bedrock Model Evaluation now supports evaluating custom model import models

What's new at AWS πŸ“’

⚜️ Amazon Bedrock Model Evaluation now supports evaluating custom model import models

⚜️ This feature allows customer to evaluate, compare, and select the best foundation models for your use case.

⚜️ Amazon Bedrock also offers a choice of automatic evaluation and human evaluation.

⚜️ This automatic evaluation with predefined algorithms for metrics such as accuracy, robustness, and toxicity.

⚜️ Additionally, for those metrics or subjective & custom metrics, such as friendliness, style, and alignment to brand voice, you can set up a human evaluation workflow with a few clicks.

⚜️ Human evaluation workflows can leverage your own employees or an AWS-managed team as reviewers. Model evaluation provides built-in curated datasets or you can bring your own datasets.

⚜️ It enables customers to evaluate their own models they imported to Amazon Bedrock through the Custom Model Import feature.

⚜️ Importantly, it allows customers to complete the cycle of selecting a base model, customizing it, evaluating it, and customizing it again or continuing to production if they are satisfied.

⚜️ To evaluate an imported model, simply select the custom model from the list of models to evaluate in the model selector tool when creating an evaluation job.

πŸ“Œ https://aws.amazon.com/bedrock/developer-experience/
Explore more about Model Evaluation on Amazon Bedrock:

πŸ“Œ Evaluate best foundation models in Amazon Bedrock
https://aws.amazon.com/blogs/aws/evaluate-compare-and-select-the-best-foundation-models-for-your-use-case-in-amazon-bedrock-preview/

Image of Datadog

How to Diagram Your Cloud Architecture

Cloud architecture diagrams provide critical visibility into the resources in your environment and how they’re connected. In our latest eBook, AWS Solution Architects Jason Mimick and James Wenzel walk through best practices on how to build effective and professional diagrams.

Download the Free eBook

Top comments (0)

Image of Docusign

πŸ› οΈ Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more