DEV Community: Novelvista

UNESCO AI Ethics Principles: A Practical Guide for Responsible AI

Novelvista — Thu, 24 Sep 2026 12:29:20 +0000

Artificial intelligence is moving from experimentation into everyday business operations. It supports hiring decisions, customer service, fraud detection, healthcare research, education, public services, and more. But as AI systems become more influential, the consequences of poor design, weak oversight, and biased data become far more serious.

UNESCO’s Recommendation on the Ethics of Artificial Intelligence provides a global framework for ensuring that AI benefits people while protecting human rights, dignity, fairness, and the environment. It gives governments, organizations, developers, and leaders a common ethical foundation for building and deploying AI responsibly.

What are the UNESCO AI Ethics Principles?
UNESCO adopted its Recommendation on the Ethics of Artificial Intelligence in 2021. It is the first global standard-setting framework on AI ethics, endorsed by 193 Member States.
The framework is not simply about making AI “fair.” It recognizes that responsible AI requires action across the full lifecycle: from data collection and model development to deployment, monitoring, incident response, and retirement.

For organizations, these principles can become a practical operating model for AI governance rather than a policy document that sits unread in a shared drive.

Human Rights and Human Dignity AI systems must respect fundamental human rights, freedoms, and dignity. This includes privacy, equality, freedom of expression, access to information, and protection from discrimination. For example, an AI-powered recruitment system should not unfairly exclude candidates based on gender, age, ethnicity, disability, or other protected characteristics. Similarly, a customer-facing chatbot should not expose personal information or manipulate users into decisions they do not understand.

The key question is simple: does the AI system strengthen people’s ability to make informed choices, or does it quietly take that ability away?

Human Oversight and Accountability UNESCO emphasizes that humans must remain accountable for significant AI-driven decisions. AI can support decision-making, but responsibility cannot be handed over to an algorithm.

Organizations should define:
• Who owns each AI system.
• Which decisions require human review.
• When an AI system must be paused or overridden.
• How users can challenge or appeal an AI-generated decision.
• How incidents and failures are investigated.
Human oversight is especially important in high-impact areas such as finance, healthcare, education, employment, law enforcement, and public services. A model may produce an answer in milliseconds; fixing a harmful decision can take months.

Fairness and Non-Discrimination
AI learns from data, and data often reflects historical inequality, incomplete representation, and human bias. If these issues are ignored, AI can scale unfairness faster than any manual process.
Fairness requires more than removing obvious sensitive fields from a dataset. Teams should evaluate whether outcomes differ across groups, whether the data is representative, and whether the system creates unequal access or treatment.
Practical actions include bias testing before release, periodic fairness reviews, representative data sampling, and documented remediation plans when harmful patterns are found.
Privacy and Data Protection
AI systems often depend on large volumes of personal, commercial, and sensitive data. UNESCO’s framework makes privacy a core ethical requirement, not an afterthought.
Organizations should collect only the data needed for the defined purpose, protect it throughout its lifecycle, and clearly explain how it is used. Sensitive information should not be copied into public models, exposed through prompts, or retained longer than necessary.
Privacy controls should cover data access, encryption, retention, consent, third-party sharing, and the ability to delete or correct information where appropriate.
Transparency and Explainability
People should know when they are interacting with AI and understand, at an appropriate level, how important decisions are made.
This does not always mean exposing model source code or complex mathematical details. It means providing meaningful explanations: what data was considered, what the system can and cannot do, where human review occurs, and how users can raise concerns.
Transparency is also essential internally. Business teams, risk teams, and executives need an accurate view of which AI systems are in use, what models they rely on, what data they access, and what risks they create.
Safety, Security, and Reliability
An AI system must be reliable enough for its intended use and resilient against misuse, failure, and attacks. This includes technical risks such as hallucinations, prompt injection, data poisoning, model theft, insecure tool access, and unexpected behavior after model updates.
A responsible AI program should include pre-deployment testing, security reviews, red-team exercises, continuous monitoring, and a clear incident-response process. A system that works beautifully in a demo but fails under real-world pressure is not ready for production.
Environmental and Societal Well-Being
UNESCO also highlights AI’s impact on society and the environment. Training and operating advanced models can consume significant energy and computing resources. AI can also affect employment, public trust, cultural diversity, and access to essential services.
Organizations should consider whether an AI solution is proportionate to the problem. A smaller, efficient model may sometimes deliver the required outcome with lower cost, lower energy use, and less operational risk.
Responsible AI means optimizing for long-term societal value, not just short-term automation gains.

Turning Principles into Practice
The real challenge is operationalizing these principles. Organizations need governance structures that convert ethical intentions into repeatable controls.
A practical approach includes maintaining an AI system inventory, classifying systems by risk, documenting intended use, testing for bias and security issues, assigning accountable owners, monitoring production performance, and creating escalation paths for incidents.
Teams looking to build these capabilities can explore AI governance practices that connect ethical principles with real-world policies, risk controls, lifecycle monitoring, and organizational accountability.

Conclusion
UNESCO’s AI Ethics Principles provide a valuable reminder: AI should serve people, not merely optimize processes. Responsible AI requires more than a policy statement. It requires accountability, transparency, security, fairness, and continuous oversight.
Organizations that embed these principles early will be better positioned to earn trust, reduce risk, meet emerging regulatory expectations, and deploy AI with confidence. In the long run, ethical AI is not a constraint on innovation. It is what makes innovation sustainable.

How to Calculate the Cost of an LLM Application

Novelvista — Wed, 23 Sep 2026 06:22:05 +0000

Building with a large language model (LLM) can look inexpensive at first: send a prompt, receive an answer, pay a fraction of a cent. The surprise arrives when that single call becomes thousands of prompts, long documents, retrieval, retries, monitoring, and users expecting fast answers.

Building reliable, cost-aware applications is a core part of AI Engineering.

The right question is not, “What does one model call cost?” It is, “What does one successful user outcome cost, end to end?” This guide shows how to estimate that number before launch and keep it under control after launch.

Start with the basic LLM cost formula
Most hosted LLM providers charge separately for input and output tokens. A token is a small unit of text; it is not the same as a word. The exact token count varies by language and formatting.

Use this calculation for one request:
Cost per request =
(input tokens / 1,000,000 × input price per million tokens)

(output tokens / 1,000,000 × output price per million tokens) For example, assume an application sends 2,500 input tokens and generates 500 output tokens. If the model costs $2 per million input tokens and $8 per million output tokens: Input: 2,500 / 1,000,000 × $2 = $0.005 Output: 500 / 1,000,000 × $8 = $0.004 Total model cost per request = $0.009 That looks small. At 100,000 requests per month, it becomes about $900 in model inference alone. This is why usage assumptions matter more than the price of an individual prompt.

Measure every token source, not only the user message
The user’s question is usually only part of the input. A production LLM request may include:
• System instructions and safety rules
• Conversation history
• Retrieved RAG documents
• Tool definitions or function schemas
• Structured output instructions
• Few-shot examples
• The user message itself
If your user writes a 30-token question but your application sends a 7,000-token prompt, your cost is driven by application design—not user behaviour.

Create a token budget for each request type. For a customer-support RAG bot, it could look like this:
Prompt component Typical tokens
System prompt and policies 800
Conversation summary 500
Retrieved knowledge chunks 3,000
Tool instructions 700
User question 100
Expected answer 400
Total 5,500

This table is the foundation of a useful cost forecast.
Forecast monthly cost using user behaviour
Use an activity-based model rather than a vague monthly estimate:
Monthly cost =
active users × requests per user × cost per request

ingestion cost
infrastructure cost
evaluation and operations cost Imagine a knowledge assistant with 5,000 monthly active users. Each user asks 12 questions each month. The end-to-end cost per answer—including model, retrieval, and guardrails—is $0.018.

5,000 × 12 × $0.018 = $1,080 per month
Add $250 for vector storage and search, $400 for application infrastructure, and $270 for scheduled evaluations and monitoring. The expected monthly operating cost becomes $2,000.

Now calculate unit economics:
Cost per active user = $2,000 / 5,000 = $0.40 per month
This is the number product, finance, and sales teams can use to decide pricing, limits, and margins.
Plan for the costs that spike unexpectedly
The average request is not the dangerous request. Costs commonly jump because of long conversation history, large file uploads, repeated retries, tool loops, broad RAG retrieval, or a sudden usage increase.

Build three scenarios:
• Expected: normal user behaviour and target adoption.
• High usage: more requests, longer prompts, and greater retrieval volume.
• Failure mode: retries, fallback-model calls, and runaway agent loops.
For agentic applications, set hard limits on tool calls, maximum turns, maximum output tokens, and total tokens per task. An agent without budgets is a small financial experiment with a keyboard.

Reduce cost without damaging the user experience
Cost optimisation should improve the product, not merely shrink outputs. Start with these high-impact controls:

Route requests by complexity. Use a smaller, cheaper model for classification, extraction, and simple answers. Reserve the strongest model for reasoning-heavy work.
Keep prompts lean. Remove duplicate instructions, summarize history, and pass only relevant context.
Improve retrieval. Retrieve fewer, more relevant chunks; apply metadata filters before semantic search.
Cache repeatable results. Cache embeddings, common answers, and deterministic tool outputs where appropriate.
Set output limits. Ask for the shortest useful answer and cap maximum completion tokens.
Evaluate before switching models. A cheaper model that increases escalations or wrong answers may cost more overall.
Monitor cost by feature. Track tokens, latency, errors, and quality by customer, workflow, model, and prompt version.
Create a cost dashboard before launch
At minimum, your dashboard should show daily and monthly spend, cost per request, input versus output tokens, cost per active user, cost per successful task, retrieval volume, error/retry rate, and spend by model. Set alerts for sudden increases in prompt size, request volume, or cost per task.
Also attach business context. If an AI support assistant costs $0.03 per resolved case but avoids a $4 human-handled case, the decision is different from a feature that costs $0.03 and creates no measurable value.

Final takeaway
An LLM application is a system, not a single API call. Calculate its cost by measuring model tokens, retrieval, infrastructure, evaluations, and operational overhead. Then translate that total into cost per request, cost per successful task, and cost per active user.
The teams that manage LLM spend well do not guess. They set budgets, measure real usage, test quality, and design the application so that every extra token earns its place.

OECD AI Principles: A Practical Guide for Responsible AI

Novelvista — Mon, 21 Sep 2026 08:26:21 +0000

Artificial intelligence now influences lending, customer support, hiring, security operations, healthcare triage, and public services. As adoption accelerates, organisations face a practical question: how can they gain AI’s value without compromising people, trust, or accountability? The OECD AI Principles provide a widely recognised answer.

Adopted as an international framework for trustworthy AI, these principles establish a human-centred foundation for designing, deploying, and governing AI systems. They are not a one-time compliance checklist. Instead, they help leaders, product teams, and risk owners make sound decisions throughout the AI lifecycle.

What Are the OECD AI Principles?
The framework is built around five principles for responsible stewardship of trustworthy AI. It asks organisations to ensure that AI benefits people and society, respects human rights, remains transparent, operates safely and securely, and has clear accountability.

For an enterprise, this means responsible AI cannot sit only with legal or data science teams. It must be embedded in architecture decisions, data practices, vendor assessment, model monitoring, and incident response. In short, AI governance must be operational, not merely aspirational.

Inclusive Growth, Sustainable Development and Well-being AI should create beneficial outcomes for individuals, society, and the planet. This expands success beyond efficiency alone. A model that reduces handling time but excludes a customer group, increases unnecessary energy use, or drives harmful behaviour is not truly successful just because it improves a KPI.

Organisations can apply this principle by defining success measures that include customer impact, accessibility, workforce effects, and sustainability alongside financial outcomes. During use-case selection, teams should ask who benefits, who may be disadvantaged, and whether a lower-risk non-AI solution could achieve the same result.

Human-Centred Values and Fairness
AI systems should respect human rights, diversity, fairness, and the rule of law. This is especially important where models influence a person’s access to employment, finance, education, healthcare, or public services.
Historical datasets may contain bias, and even accurate models can produce unfair outcomes when applied in the wrong context. Practical controls include representative data reviews, fairness testing across meaningful user groups, accessibility design, human review for high-impact decisions, and a clear route for people to challenge an outcome.
Teams should also define when automation must stop and a qualified person must take responsibility.
Transparency and Explainability
People should understand when they are interacting with AI and receive information appropriate to the system’s impact. Transparency does not require exposing every line of code. It requires useful disclosure: what the system does, what information it uses, where its limitations lie, and how its outputs are reviewed.
For generative AI, this can include labelling AI-generated content, documenting prompt and retrieval controls, and being transparent about hallucination risk. For predictive AI, model cards, decision logs, and business-friendly explanations can help users, auditors, and stakeholders understand how the system is being used.
Robustness, Security and Safety
AI systems must operate reliably under expected conditions and fail safely when conditions change. This includes protection against adversarial inputs, data poisoning, prompt injection, privacy leakage, model theft, and third-party supply-chain weaknesses.

A model that performs well in a controlled demonstration but fails silently in production is not robust. Organisations need lifecycle controls such as pre-release testing, red teaming, access management, fallback paths, drift monitoring, and incident response.
For AI agents, this means applying least-privilege tool access, approval gates for consequential actions, and audit trails that show what the agent did and why.

Accountability Accountability turns the other principles into action. Someone must own the AI system’s outcomes, risks, and corrective actions. This responsibility cannot simply be transferred to a cloud provider or vendor. Effective governance assigns clear roles: a business owner responsible for intended value and acceptable use, a technical owner responsible for performance and controls, risk and compliance reviewers for high-impact systems, and an incident owner for post-deployment issues. Approval records, risk assessments, model inventories, and scheduled reviews convert those responsibilities into evidence.

From Principles to Enterprise Practice
Leading organisations translate OECD principles into repeatable operating mechanisms. They maintain an inventory of AI use cases, classify risk before deployment, establish stage-gate approvals, document data and model decisions, test for safety and fairness, and monitor systems after launch.
This is particularly important for large language models and AI agents. These systems are flexible and valuable, but their non-deterministic behaviour makes guardrails, monitoring, and human oversight essential. Governance should be designed into the workflow from the beginning—not attached after an incident.

Conclusion
The OECD AI Principles remain relevant because they focus on a durable truth: trustworthy AI is a business capability, not a public-relations statement. Organisations that embed fairness, transparency, security, and accountability into everyday decisions are better positioned to scale AI confidently, build stakeholder trust, and respond effectively when systems behave unexpectedly.

The next step is to turn these principles into policies, technical controls, defined roles, and measurable review processes. That is how responsible AI moves from a policy document into reliable business practice.

The Hidden Cost of Building AI Without Guardrails

Novelvista — Sat, 19 Sep 2026 06:34:52 +0000

Most AI projects begin with a promising demo.

A model answers questions, summarizes documents, creates content, or triggers an API call. The team sees early value, stakeholders get excited, and the pressure to move the prototype into production begins.

But a working demo is not the same as a reliable AI system.

The hidden cost of building AI Engineering without guardrails is rarely visible at the beginning. It emerges later through incorrect outputs, unsafe tool calls, sensitive-data exposure, compliance issues, customer dissatisfaction, and rushed engineering rework.

Why AI guardrails matter

Traditional software follows predefined logic. An AI system works differently: its output can vary depending on prompts, context, model behaviour, connected tools, retrieved data, and user input.

That flexibility is powerful, but it introduces a new category of engineering risk.

For example, an AI assistant connected to internal tools might:

Call the wrong API because it misunderstood user intent
Return a confident but incorrect answer
Reveal information from a restricted knowledge source
Execute an action that should have required human approval
Produce output that violates a business, regulatory, or security policy

These failures can be small in isolation. At scale, they become operational problems.

The cost is bigger than a bad answer

A hallucinated answer can be corrected. But if an AI system gives incorrect advice to customers, initiates the wrong workflow, or exposes confidential information, the consequences grow rapidly.

Teams often pay through:

Engineering rework

Without guardrails, teams may discover issues only after release. Fixing prompt design, permissions, data handling, tool workflows, logging, and approval processes in production is slower and more expensive than designing them early.

Security and privacy risk

AI applications frequently interact with documents, databases, APIs, and enterprise tools. Uncontrolled access can lead to data leakage, unauthorized actions, or prompt-injection vulnerabilities.

Loss of customer trust

Users may forgive one imperfect answer. They are less forgiving when an AI system behaves unpredictably, makes critical errors, or handles their data carelessly.

Compliance exposure

Organizations operating in regulated sectors need to know how AI decisions are made, what data is used, and who approved important actions. A system with no traceability creates unnecessary audit risk.

Slower innovation over time

It may seem that guardrails slow development. In reality, a lack of controls often slows the entire roadmap later. Teams become hesitant to expand automation because they do not trust the system.

Essential guardrails for production AI

Guardrails are not a single feature. They are a set of controls across the AI application lifecycle.

Define action boundaries

Clearly define what the AI is allowed to do. A model should not have unrestricted access to every database, tool, or workflow simply because it can technically call them.

Use allowlists, scoped permissions, and purpose-specific tools.

Validate inputs and outputs

User inputs can be ambiguous, malicious, or outside the application’s intended scope. Validate inputs before they reach the model, and validate outputs before they are shown to users or passed to downstream systems.

For structured workflows, prefer strict JSON schemas over free-form responses.

Add human approval for critical actions

High-impact actions should not rely on model confidence alone.

Actions such as sending external communications, changing customer data, approving transactions, or deleting records should require a human checkpoint. AI can recommend and prepare actions, while people retain final control.

Secure tool calling

Tool calling is where AI applications become useful—and where risks become real.

Each tool should have a clear schema, limited permissions, parameter validation, timeout controls, and audit logs. The model should never have the ability to execute arbitrary commands or access sensitive systems without safeguards.

Monitor and log behaviour

Production AI needs observability.

Track model inputs, outputs, latency, token usage, tool calls, retrieval quality, errors, overrides, and user feedback. These signals make it possible to identify failures early and improve the system continuously.

Design fallback paths

AI systems will fail sometimes. The goal is to ensure that a failure does not become an incident.

A reliable application should know when to ask for clarification, escalate to a human, return a safe response, retry a bounded operation, or stop an unsafe workflow.

Build AI that can be trusted

The strongest AI systems are not the ones that automate everything. They are the ones that automate the right things with clear limits, visible controls, and accountable decision-making.

Guardrails are not a brake on innovation. They are the seatbelt that allows innovation to travel faster and farther.

Govern, Map, Measure, and Manage: A Practical Framework for AI Governance

Novelvista — Fri, 18 Sep 2026 13:10:05 +0000

AI systems are moving from experimentation into real business workflows: customer support, fraud detection, hiring, content generation, software development, and internal decision-making.

That shift creates a simple but important question: who is responsible when an AI system produces an incorrect, unsafe, biased, or non-compliant outcome?

The answer is not “the model.” It is the organization deploying it.

A practical AI governance framework can be built around four actions: Govern, Map, Measure, and Manage.

Govern: Establish accountability before deployment

Governance starts with ownership.

Every AI initiative should have clear answers to questions such as:

Who owns the business outcome?
Who is accountable for technical quality and security?
Who approves high-risk use cases?
What data can the system access?
When is human approval mandatory?

For engineering teams, this means governance should be embedded into delivery workflows—not introduced after launch. Define approval gates for model selection, data use, production deployment, and material changes to prompts, tools, or retrieval sources.

Map: Know where AI exists and what it affects

You cannot govern what you cannot see.

Maintain an AI inventory that records each system’s purpose, model provider, data sources, integrations, user groups, risk level, and owner. This is especially important for AI agents that can call APIs, access documents, or take actions across enterprise systems.

Mapping should also include:

Data flows and retention practices
Third-party model and tool dependencies
Potentially affected stakeholders
Security and privacy risks
Regulatory or contractual obligations

An accurate map turns an unknown AI landscape into a manageable operating environment.

Measure: Turn trust into evidence

A model that performed well in a demo may behave differently in production. Data changes, user behavior evolves, and edge cases inevitably appear.

Teams should define measurable controls for:

Accuracy and task completion
Hallucination and unsafe output rates
Bias and fairness indicators
Latency, reliability, and cost
Prompt-injection and tool-use failures
Privacy, security, and compliance outcomes

Measurement is not only a data science responsibility. Product, engineering, security, risk, and business teams all need visibility into what “good” and “acceptable” look like.

Manage: Keep governance active throughout the lifecycle

AI governance is not a one-time compliance checklist.

Models, prompts, policies, datasets, and integrations change over time. A mature organization continuously monitors production behaviour, investigates incidents, retrains teams, reviews controls, and updates documentation.

The goal is not to prevent every issue. It is to detect issues early, respond clearly, and learn from them.

Final thought

Responsible AI does not mean slower AI. It means AI that can scale with fewer surprises.

Govern accountability. Map the landscape. Measure what matters. Manage the lifecycle.

NIST AI Risk Management Framework (AI RMF): A Practical Guide to Responsible AI

Novelvista — Thu, 17 Sep 2026 13:35:06 +0000

Artificial Intelligence is transforming how businesses operate, make decisions, and serve customers. From chatbots and recommendation systems to generative AI and automated decision-making, organizations are adopting AI faster than ever.

But with this growth comes an important question:
How can organizations manage AI risks while continuing to innovate?
This is where the NIST AI Risk Management Framework (AI RMF) becomes valuable.

The NIST AI RMF provides a structured approach to help organizations identify, assess, and manage risks associated with artificial intelligence. It supports the development and use of trustworthy and responsible AI systems.
In this article, we’ll explore what NIST AI RMF is, its four core functions, and how organizations can use it to strengthen AI governance.
What Is the NIST AI Risk Management Framework?
The NIST AI Risk Management Framework, commonly known as AI RMF, is a voluntary framework developed by the National Institute of Standards and Technology (NIST).

The framework was released on January 26, 2023, as AI RMF 1.0. It is designed to help organizations manage AI-related risks throughout the lifecycle of AI systems.
Unlike a framework focused only on cybersecurity or technical performance, AI RMF also considers broader trustworthiness characteristics, including:
• Validity and reliability
• Safety
• Security and resilience
• Accountability and transparency
• Explainability and interpretability
• Privacy enhancement
• Fairness and harmful bias management
The framework is flexible and can be applied across industries, including finance, healthcare, education, retail, manufacturing, and technology.
NIST AI RMF 1.0 is intended for voluntary use and is not a certification standard by itself.

Why Is AI Risk Management Important?
AI systems can create business value, but they can also introduce risks that traditional software testing may not fully address.
For example, an AI-powered hiring system might produce biased recommendations. A generative AI chatbot might share inaccurate information. A machine learning model might perform well during testing but fail when real-world data changes.

Common AI risks include:

Bias and unfair outcomes AI models can produce unfair results when training data, model design, or deployment conditions introduce bias.
Privacy and data protection AI systems may process personal, confidential, or sensitive information. Poor data handling can create privacy risks.
Security threats AI applications can face threats such as prompt injection, data poisoning, unauthorized access, and model manipulation.
Lack of transparency Some AI systems make decisions that are difficult for users and organizations to understand.
Reliability and inaccurate outputs AI models can generate incorrect predictions, misleading recommendations, or hallucinated information.
Third-party and supply-chain risks Organizations may depend on external AI models, APIs, datasets, and vendors. These dependencies introduce additional risks. A structured risk management process helps organizations identify these issues before they become serious business problems. Understanding the Four Core Functions of NIST AI RMF The AI RMF Core is organized around four functions: Govern → Map → Measure → Manage These functions help organizations establish governance, understand risks, evaluate performance, and take action.
GOVERN: Establish AI Governance Governance is the foundation of AI risk management. This function focuses on creating the policies, responsibilities, processes, and organizational culture needed to manage AI risks.

Key activities include:
• Defining AI governance policies
• Assigning roles and responsibilities
• Establishing accountability
• Creating risk management processes
• Promoting a culture of responsible AI
• Aligning AI practices with organizational goals
• Managing legal, regulatory, and ethical considerations
For example, an organization deploying an AI customer-support chatbot should define who owns the system, who approves its deployment, and who responds if the chatbot generates harmful or inaccurate information.
Practical takeaway: AI governance should not be limited to the IT department. It requires collaboration between leadership, technical teams, legal, compliance, security, and business stakeholders.

MAP: Identify and Understand AI Risks The MAP function focuses on understanding the AI system, its intended purpose, its users, and the risks associated with its context. Before deploying an AI solution, organizations should ask: • What problem is the AI system solving? • Who will use or be affected by it? • What data does it process? • What decisions does it influence? • What could go wrong? • What are the potential impacts on individuals and the organization? For example, consider an AI system used to evaluate loan applications. Potential risks may include: • Biased recommendations • Inaccurate applicant information • Privacy concerns • Lack of explainability • Incorrect decisions caused by poor-quality data Mapping these risks helps the organization understand where controls and further evaluation are needed. Practical takeaway: You cannot effectively manage an AI risk if you do not first understand the system and its potential impacts.
MEASURE: Evaluate AI Performance and Trustworthiness The MEASURE function focuses on assessing AI risks using appropriate methods, metrics, testing, and monitoring. This is where organizations evaluate whether their AI systems perform as expected.

Depending on the use case, measurement activities may include:
• Accuracy and reliability testing
• Bias and fairness assessments
• Security testing
• Privacy risk assessments
• Explainability evaluations
• Robustness testing
• Monitoring real-world performance
• Testing generative AI outputs
For a generative AI application, an organization might measure:
• Response accuracy
• Relevance of generated answers
• Frequency of hallucinations
• Harmful or unsafe outputs
• Resistance to prompt injection
• Performance across different user inputs
Measurement should continue after deployment because AI systems can change as models, data, users, and operating environments change.
Practical takeaway: AI testing is not a one-time activity. Continuous evaluation helps organizations identify emerging risks and performance issues.

What Happens When an LLM API Hits Its Rate Limit?

Novelvista — Wed, 16 Sep 2026 13:45:13 +0000

Large Language Models (LLMs) are now used in chatbots, AI assistants, content generation tools, RAG applications, and AI agents. Most of these applications communicate with an LLM through an API.

But what happens when an application sends more requests than the API allows?

The answer is rate limiting.

Understanding LLM API rate limits is important for developers and AI engineers because hitting a limit can cause failed requests, delayed responses, and interrupted AI workflows.

What Is an LLM API Rate Limit?

An API rate limit is a restriction placed on the number of requests or tokens an application can use within a specific period.

For example, an API provider may limit:

Requests per minute (RPM)
Requests per day
Tokens per minute (TPM)
Concurrent requests
Overall usage based on an account or subscription

The exact limits vary depending on the API provider, model, account, and pricing plan.

Rate limits are generally used to manage system capacity, maintain service reliability, and ensure fair resource usage.

What Happens When You Reach the Rate Limit?

When your application exceeds an API's allowed usage, the provider may reject additional requests.

A common response is an HTTP 429 error, which indicates that too many requests have been sent within the permitted period.

For an AI application, this can result in:

Failed API Requests

The immediate effect is often a failed request. If the application does not have proper error handling, the user may simply see an error message or receive no response.

Delayed Responses

Some applications automatically retry failed requests. While retries can help, they may also increase response time if the application repeatedly waits before sending another request.

Interrupted AI Workflows

Modern AI applications often involve multiple API calls.

For example:

User → Retrieval → LLM → Tool Call → LLM → Final Response

If one of these calls is rejected because of a rate limit, the entire workflow may be interrupted.

This can be particularly important for RAG systems and AI agents that make multiple model calls for a single user request.

Poor User Experience

Repeated API failures can result in slow responses, incomplete answers, or temporary unavailability.

For applications used by many users simultaneously, uncontrolled API usage can become a significant reliability problem.

Rate Limits vs. Token Limits

It is important to understand that rate limits are not always about the number of API requests.

An API may also limit the number of tokens processed during a specific period.

Consider an application making only a few requests, but each request contains a very large context.

Even though the request count is low, token consumption may still reach the provider's limit.

Therefore, AI engineers should monitor both request volume and token usage.

How Can You Handle LLM API Rate Limits?

Implement Exponential Backoff

Instead of immediately sending the same request again, the application can wait before retrying.

For example:

Retry 1 → Wait 1 second
Retry 2 → Wait 2 seconds
Retry 3 → Wait 4 seconds
Retry 4 → Wait 8 seconds

This approach reduces the chance of repeatedly overwhelming the API.

A small amount of randomization, often called jitter, can also help prevent many clients from retrying at exactly the same time.

Monitor API Usage

Monitoring helps identify when your application is approaching its limits.

Useful metrics include:

Requests per minute
Tokens per minute
Error rates
Response latency
Concurrent requests
Retry frequency

With proper monitoring, teams can identify usage spikes before they become major production problems.

Optimize Token Usage

Reducing unnecessary tokens can improve both performance and cost.

Developers can consider:

Removing unnecessary prompt instructions
Reducing duplicated context
Limiting conversation history when appropriate
Optimizing retrieved documents in RAG systems
Choosing an appropriate model for each task

Efficient prompts can help an application accomplish the same task with fewer resources.

Use Caching

If an application repeatedly requests the same information, caching can reduce unnecessary API calls.

For example, frequently requested static information could be cached instead of generating a new response every time.

Caching is especially useful for applications with repeated or predictable queries.

Use Request Queues

A request queue can help control how many requests are sent to an LLM API at a given time.

Instead of allowing hundreds of requests to reach the API simultaneously, the application can process them according to defined limits.

This approach is useful for high-volume applications and background AI workloads.

Design Graceful Error Handling

Production applications should assume that API failures can happen.

Instead of displaying a technical error to users, the application can provide a useful message such as:

“The AI service is temporarily busy. Please try again shortly.”

Behind the scenes, the application can log the error and apply an appropriate retry strategy.

A Simple Example

Imagine an AI chatbot receives 1,000 requests within a short period, but its API configuration only allows a smaller number of requests during that window.

Without rate-limit handling:

1,000 requests → API → Many requests rejected → User errors

With proper request management:

1,000 requests → Queue → Controlled API requests → Retry when necessary → Improved reliability

The goal is not simply to send requests faster. The goal is to manage API resources efficiently.

Why Rate-Limit Management Matters for AI Engineering

Building an LLM application involves more than connecting a model to an API.

A production-ready AI application needs to consider:

Reliability
Scalability
Cost
Latency
Error handling
API quotas
Token consumption
Monitoring

As AI applications grow from prototypes to production systems, these engineering considerations become increasingly important.

Key Takeaways

When an LLM API reaches its rate limit:

API requests may be rejected.
Applications may receive HTTP 429 errors.
Response times can increase when retries are required.
Multi-step AI workflows can be interrupted.
Token limits can matter in addition to request limits.
Exponential backoff, caching, queues, monitoring, and token optimization can improve reliability.

Understanding rate limits is an important part of building scalable and dependable LLM applications.

Learn AI Engineering

Want to develop practical skills for building modern AI applications?

Explore NovelVista's Certified AI Engineering Professional course to learn more about LLMs, RAG, AI agents, and production-focused AI engineering.

Why the NIST AI RMF Matters for Modern Organizations

Novelvista — Tue, 15 Sep 2026 10:35:27 +0000

Companies are adopting artificial intelligence faster than many of their internal controls can evolve. A model may perform well in a demonstration but behave differently when exposed to real customers, unfamiliar data, changing business conditions, or malicious input. The consequences may include unfair decisions, confidential-data exposure, inaccurate advice, operational disruption, or loss of public trust. The NIST AI Risk Management Framework helps organizations move from broad promises about responsible AI to a structured risk-management process.

The framework is not a law and does not guarantee that an AI system is safe. Its value comes from helping organizations ask better questions: What is the system intended to do? Who could be affected? How will performance and harm be measured? Who has authority to stop the system? Which risks can be accepted, and which require treatment? These questions connect technical development with business accountability.

What the Framework Is Designed to Achieve
The framework helps organizations identify, assess, prioritize, and manage AI risks throughout design, development, deployment, use, and evaluation. It considers potential harm to people as well as operational, financial, legal, security, and reputational effects on organizations. Its goal is not to eliminate all uncertainty. Instead, it helps decision-makers understand uncertainty, determine whether a use is appropriate, establish safeguards, and revisit decisions as evidence changes.

Trustworthy AI Characteristics
NIST identifies several interconnected characteristics of trustworthy AI: systems should be valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. No characteristic should be evaluated in isolation. A highly accurate system may still be unacceptable if it exposes personal information, produces systematically unfair outcomes, or cannot be safely overridden. Organizations must examine trade-offs according to the system's context and impact.
Organizations investing in AI Governance practices can use these functions to establish ownership and oversight.

Govern creates the foundation for the entire program. It covers policies, roles, accountability, organizational culture, documentation, legal and regulatory considerations, and engagement with relevant stakeholders. Teams should know who owns the system, who accepts risk, who approves deployment, who monitors performance, and who can suspend its use. Governance is cross-cutting because it shapes how the other functions are performed throughout the lifecycle.
Map
Map develops a clear understanding of the AI system and the environment in which it will operate. Organizations define the intended purpose, expected benefits, users, affected groups, data dependencies, limitations, foreseeable misuse, and consequences of failure. A recruitment model, for example, must be considered in relation to applicants, employment rules, historical data, human reviewers, and the ability to challenge a decision. Context determines which risks matter most.

Measure
Measure evaluates identified risks using evidence. This may include testing accuracy, reliability, robustness, cybersecurity, privacy, explainability, accessibility, and performance across relevant groups and conditions. Measurement should include realistic scenarios rather than relying only on laboratory benchmarks. Teams should document uncertainty, limitations, assumptions, and the quality of the evidence. Human interaction also matters because users may misunderstand, over-trust, ignore, or deliberately misuse an AI system.

Manage
Manage turns the findings into prioritized action. An organization may reduce a risk through technical controls, restrict the use case, introduce human review, strengthen monitoring, transfer part of the risk, accept it within defined limits, or avoid the activity entirely. Higher-impact risks deserve stronger controls and clearer escalation. Management also includes incident response, recovery, communication, and decisions about whether a system should remain in operation.

Putting AI RMF into Practice
A practical starting point is to create an inventory of AI systems and classify them according to purpose, sensitivity, autonomy, and potential impact. Each system should have an owner, documented intended use, risk assessment, approval route, testing evidence, monitoring plan, and retirement process. The NIST AI RMF Playbook offers suggested actions aligned with the four functions, while profiles allow organizations to tailor the framework to particular technologies, sectors, or use cases. The Playbook is guidance rather than a universal checklist.

The Role of People and Continuous Oversight
Effective risk management requires multidisciplinary participation. Developers understand technical limitations, business owners understand operational goals, security specialists assess threats, legal and privacy teams identify obligations, and affected users reveal practical concerns that internal teams may overlook. Oversight must continue after deployment because data can drift, behaviour can change, threats can evolve, and users can apply the system in ways the designers did not expect. Monitoring, feedback, incident reporting, and periodic reassessment keep controls aligned with reality.

Final Thoughts
NIST AI RMF is valuable because it recognizes that AI risk is both technical and human. A system can be accurate yet unfair, secure yet opaque, or efficient yet unsuitable for a high-impact decision. Organizations that use the framework thoughtfully can build stronger oversight while continuing to innovate.

When an AI Assistant Takes the Wrong Action

Novelvista — Tue, 15 Sep 2026 07:08:53 +0000

The most important difference between a chatbot and an AI agent is not intelligence; it is agency. A chatbot produces text. An agent can use software tools to change the world around it. It may book a meeting, update a sales opportunity, reset a password, create a ticket, run a query, or deploy an application. That power makes agents useful, but it also means an incorrect decision can escape the chat window and enter a real business process.

Suppose an employee asks an agent to ‘clean up duplicate customer records.’ The system has tools for merging records, archiving records, and permanently deleting them. The agent may decide that deletion is the fastest interpretation. Technically, the tool works exactly as designed. Operationally, the result is a failure because the agent selected the wrong capability and acted on the wrong meaning.
This kind of failure usually begins with ambiguity. Tool descriptions may be brief, outdated, or almost identical. Parameters may not clearly distinguish a customer ID from an account ID. The user’s wording may also leave room for interpretation. Humans fill such gaps using organizational knowledge, tone, and experience. An AI model may fill them using probability. Ai Engineer The answer can look confident even when the underlying choice is uncertain.

Context can disappear between systems as well. The agent may know that a document should be shared, but not that the recipient is external. It may know that a server needs a restart, but not that the system is supporting a live payroll run. It may see a refund request without knowing that the amount exceeds the employee’s approval limit. A correct tool in the wrong context can be just as damaging as the wrong tool.
The immediate effects range from mild to severe. A mistaken search may simply return the wrong file. A mistaken update can corrupt a record or interrupt a workflow. A mistaken communication can disclose confidential information or confuse a customer. A mistaken administrative action can revoke access, stop a service, or alter security settings. In highly automated environments, one call can trigger several downstream actions before anyone reviews the first one.

The security implications deserve special attention. Attackers can deliberately write instructions that encourage an agent to misuse its tools. Content retrieved from emails, websites, or documents may contain hidden or misleading instructions. If the agent treats that content as trusted guidance, it could send data, change settings, or call an external service without valid authorization. Tool access therefore becomes part of the organization’s security perimeter.

A responsible system separates reasoning from execution. The model can propose an action, but a policy layer should decide whether the action is allowed. That layer can check the user’s identity, role, target resource, data sensitivity, transaction value, environment, and current workflow stage. High-risk requests should require human approval. Prohibited requests should be blocked even if the model strongly recommends them.
The interface should also make consequences visible. Before a destructive action, the agent can show the exact tool, target, parameters, and expected effect. Instead of asking ‘Are you sure?’, it can ask ‘Delete invoice INV-2048 permanently from the production finance system?’ Specific confirmation reduces accidental approval and gives the user a meaningful opportunity to catch errors.

Teams should test agent behaviour using realistic failure scenarios. They should try vague commands, conflicting instructions, similar tool names, invalid identifiers, prompt injection, unavailable services, and partial API responses. Monitoring should identify unusual tool sequences, repeated failures, unexpected destinations, and actions outside normal operating patterns. Logs must support investigation and accountability without exposing sensitive information unnecessarily.

Organizations should also pay attention to user-interface design. People often approve agent actions quickly because the system appears confident. A good preview slows the user down only when necessary and highlights the fields that matter most. It can show what changed, identify unusual values, and make irreversible effects prominent. Approval fatigue is a genuine risk, so teams should avoid asking for confirmation on every harmless search. Controls work best when friction is proportional to consequence. When users learn that an approval request signals real risk, they are more likely to examine it carefully instead of clicking through automatically.

When an AI agent calls the wrong tool, the model is only one part of the problem. The larger issue is a system that allowed uncertain reasoning to trigger an inadequately controlled action. Safe agent design accepts that mistakes will occur and builds barriers around their consequences. The goal is not a flawless model. It is a resilient operating environment in which a wrong choice can be detected, contained, reversed, and learned from.

A Practical Guide to ISO/IEC 42001 Requirements

Novelvista — Sat, 12 Sep 2026 06:21:04 +0000

Artificial intelligence often enters an organization gradually. One department introduces a chatbot, another purchases an AI-powered analytics platform, and employees begin using generative AI to prepare reports. Before long, AI is influencing business operations without a common governance structure.
ISO/IEC 42001:2023 helps organizations bring these activities under one coordinated management system. The standard establishes requirements for an Artificial Intelligence Management System, commonly called an AIMS. Its purpose is to help organizations manage AI responsibly while continuing to benefit from innovation.

The Management System Approach
ISO/IEC 42001 does not prescribe one particular technology, model or programming method. Instead, it focuses on how an organization manages AI.
It follows the Plan–Do–Check–Act cycle:
• Plan AI objectives, risks and controls.
• Implement the required processes.
• Check whether those processes are effective.
• Correct weaknesses and continually improve.
This structure also makes ISO/IEC 42001 compatible with other management system standards, including ISO 9001 and ISO/IEC 27001.
Understanding Organizational Context
The first major requirement is to understand the organization’s context. A business must identify the internal and external conditions that influence its use of AI.

For example, a healthcare organization may need to consider patient safety and medical regulations. A financial organization may focus on fairness, explainability and regulatory reporting. An educational institution may need to examine student privacy and the effect of automated decisions.
The organization must also identify interested parties and understand their expectations. These parties may include employees, customers, regulators, vendors and people affected by AI decisions.
Leadership and AI Policy
ISO/IEC 42001 expects senior management to play an active role. Leaders must establish an AI policy, assign responsibilities and ensure that adequate resources are available.
The policy should explain the organization’s approach to responsible AI. It may address fairness, transparency, safety, security, privacy, human oversight and compliance.

Leadership must also make accountability visible. Every important AI system should have an owner with the authority to manage risks and make decisions.
Risk and Impact Assessments
Risk-based planning is central to the standard. Organizations must identify possible events that could prevent their AIMS from achieving its objectives.
AI-related risks may include:
• Discriminatory recommendations
• Inaccurate or misleading outputs
• Unauthorized use of personal data
• Security vulnerabilities
• Poor-quality training data
• Model drift
• Insufficient human supervision
• Unclear responsibility for decisions
Risk treatment measures should be selected according to the seriousness and likelihood of each risk.
ISO/IEC 42001 also requires a process for assessing the broader impact of AI systems. An impact assessment considers how an AI system could affect individuals, communities and society.
Resources, Competence and Awareness
Organizations need employees with appropriate skills. Technical teams may require knowledge of testing, data quality and model monitoring. Business teams may need to understand the limitations of AI-generated recommendations. Procurement teams should know how to evaluate AI suppliers.

Training should be relevant to each person’s responsibilities. A single awareness presentation for everyone is unlikely to address every risk.
The organization must also maintain controlled documentation, including system inventories, policies, risk assessments, impact assessments and monitoring records.

Operational Control
Operational requirements cover how AI systems are developed, acquired, deployed and monitored.
Controls should apply throughout the AI lifecycle. Before deployment, an AI system may require testing, risk review and formal approval. After deployment, the organization should monitor accuracy, unexpected behaviour, user complaints and changes in data.
Third-party AI services also require oversight. Vendor contracts, data practices, service limitations and incident responsibilities should be reviewed before adoption.

Measuring and Improving Performance
Organizations must determine what they will monitor and how results will be evaluated. Useful measures may include incident frequency, model error rates, bias indicators, complaint volumes and overdue risk treatments.
Internal audits provide an independent check of whether required processes are being followed. Management reviews allow leaders to examine performance and decide whether changes are needed.
When weaknesses are found, corrective action must address their causes—not merely their symptoms.

Conclusion
ISO/IEC 42001 turns responsible AI from a broad intention into a managed business practice. Its requirements connect leadership, risk management, operational control, monitoring and improvement.
The result is not simply another folder of policies. When implemented properly, an AIMS gives organizations a living framework for making better decisions about AI.

From AI User to AI Engineer: Skills You Need to Make the Transition

Novelvista — Fri, 11 Sep 2026 09:05:01 +0000

Many professionals begin their AI journey by using chatbots, content generators and coding assistants. These tools provide an accessible introduction, but becoming an Ai Engineer requires a broader set of capabilities.
An AI user focuses on communicating effectively with a tool. Important skills include clear prompting, domain knowledge, critical thinking and output verification.

An AI engineer must understand how to turn a model into a working application.
The transition commonly begins with programming. Python is widely used for AI applications because it supports APIs, data processing and popular AI frameworks. Learners should also understand JSON, HTTP requests, authentication and basic software design.
The next step is working with language-model APIs. This includes sending prompts, managing parameters, processing responses and handling errors.
Data and retrieval skills then become important. Enterprise AI applications often need access to private information that a general model does not know. AI engineers use embeddings, vector databases and retrieval-augmented generation to provide relevant context.

Evaluation is another essential capability. A response that sounds convincing may still be inaccurate. Engineers must measure relevance, groundedness, safety and task completion.

Production skills complete the journey. These include deployment, monitoring, security, latency management and cost control.
A practical learning sequence is:

Learn Python and API fundamentals.
Build applications using LLM APIs.
Add structured outputs and tool calling.
Build a RAG application.
Create an AI-agent workflow.
Add evaluation and safety controls.
Deploy and monitor the application. Moving from AI user to AI engineer does not require becoming a machine-learning researcher. It requires learning how to combine models, data, tools and software into dependable solutions.

ISO/IEC 42001: The Management System Every AI-Driven Organization Should Understand

Novelvista — Fri, 11 Sep 2026 06:37:37 +0000

Artificial intelligence rarely enters an organization through one carefully controlled doorway.
It may begin with a customer-service chatbot, an automated recruitment tool or a forecasting model. Soon, employees are using generative AI to write reports, developers are connecting language models to internal data and business teams are purchasing AI-enabled software from external vendors.

The technology spreads quickly. Governance usually arrives later.
This creates a difficult situation for business leaders. They may know that AI is being used, but they cannot always answer basic questions:
• Which AI systems does the organization currently use?
• What data can those systems access?
• Who approved each use case?
• Who is responsible for the outcome?
• What happens when an AI system produces a harmful or inaccurate result?
• How can the organization demonstrate that reasonable controls exist?
ISO/IEC 42001 was introduced to help organizations address this management gap.

What is ISO/IEC 42001?
ISO/IEC 42001 is an international standard for Artificial Intelligence Management Systems. It provides requirements for creating, operating, maintaining and continually improving an AI Management System, commonly known as an AIMS.
The standard is designed for organizations that develop, provide or use artificial intelligence.
Its focus is not limited to algorithms or technical model performance. It examines how the entire organization manages AI through leadership, policies, risk assessments, operational controls, monitoring and improvement.

In simple terms, ISO/IEC 42001 helps an organization move from:
“We have several AI policies.”
to:
“We have a structured system for deciding how AI is approved, governed, monitored and improved.”
That distinction matters.
An AIMS is more than an AI policy
Many organizations begin AI governance by writing an acceptable-use policy. The policy might tell employees not to enter confidential information into public AI tools or require approval before deploying an AI application.
Such policies are useful, but they are only one part of governance.
A complete AI Management System connects the policy with real operational activities. It defines who is accountable, how AI systems are recorded, how risks are assessed, which controls must be applied and how compliance is monitored.
For example, imagine that a company wants to use AI to shortlist job candidates. An AIMS would encourage the company to consider:
• The intended purpose of the system
• The data used by the system
• Potential bias or discrimination
• The level of human oversight
• Candidate transparency
• Supplier responsibilities
• Testing and performance requirements
• Complaint and escalation processes
• Monitoring after deployment
The objective is not to block the project automatically. It is to ensure that the organization makes an informed and documented decision.
Why businesses need an AI inventory
One of the first practical challenges in AI governance is discovering where AI is already being used.
Different departments may purchase tools independently. Employees may use publicly available generative AI services without informing security or compliance teams. Existing software products may introduce AI features through routine updates.
An AI inventory creates visibility.
A useful inventory can record:
• System or tool name
• Business purpose
• Accountable owner
• Provider or supplier
• Data being processed
• Affected users and stakeholders
• Risk classification
• Lifecycle stage
• Required assessments
• Current approval status
Without this information, an organization cannot govern AI consistently. It is difficult to manage risks that no one has formally identified.
Risk should determine the level of control
Not every AI system creates the same level of risk.
An internal tool that summarizes non-sensitive meeting notes does not have the same potential impact as an AI system used to evaluate loan applications, diagnose medical conditions or recommend disciplinary action against employees.