DEV Community: Novita AI

How to Use Kimi-K2 in Claude Code on Windows and Mac

Novita AI — Tue, 15 Jul 2025 09:46:32 +0000

Claude Code offers more powerful agent capabilities than traditional code editors like Cursor. By integrating Kimi-K2 through Novita AI’s platform, developers can access enterprise-grade AI functionality at a fraction of the cost. This guide covers setting up Kimi-K2 with Claude Code on both Windows and Mac systems.

What is Claude Code

Source: http://www.anthropic.com/claude-code

Claude Code is an agentic command line tool that revolutionizes the way developers interact with AI for coding tasks. Unlike traditional code editors, Claude Code offers more powerful agent abilities than Cursor.

This innovative tool enables developers to delegate complex coding tasks directly from their terminal. It transforms natural language descriptions into fully functional code, making it an indispensable asset for modern development workflows.

The tool operates as an interactive session where developers can describe their requirements in plain English. Claude Code intelligently generates, modifies, and optimizes code accordingly. Its advanced understanding of context and project structure allows it to make informed decisions about code architecture, dependencies, and implementation patterns.

Why Use Kimi-K2 in Claude Code

Kimi-K2 presents a compelling alternative to traditional Claude models, offering similar capabilities at significantly reduced costs. The economic advantages are substantial:

Kimi-K2 on Novita AI: $0.57 per 1M input tokens and $2.3 per 1M output tokens
Claude Sonnet: $3 per 1M input tokens and $15 per 1M output tokens

This represents an 81% cost reduction for input tokens and an 85% reduction for output tokens.

Beyond cost savings, Kimi-K2 through Novita AI provides an anthropic-compatible LLM API with higher rate limits than official channels. This compatibility ensures seamless integration with existing Claude Code workflows while offering improved performance and reliability.

The combination delivers enterprise-grade AI capabilities without the premium pricing. This makes advanced AI development accessible to a broader range of developers and organizations.

Getting Your API Key on Novita AI

Sign up for a Novita AI account to get started with free trial credits. Navigate to the Key Management page in your dashboard and click “Create New Key.”

Copy the generated API key immediately and store it securely – it won’t be displayed again. You’ll need this key for the configuration steps below.

Installing Claude Code

Before installing Claude Code, ensure your system meets the minimum requirements. Node.js 18 or higher must be installed on your local environment. You can verify your Node.js version by running node --version in your terminal.

For Windows

Open Command Prompt and execute the following commands:

cmd

npm install -g @anthropic-ai/claude-code

npx win-claude-code@latest

The global installation ensures Claude Code is accessible from any directory on your system. The npx win-claude-code@latest command downloads and runs the latest Windows-specific version.

For Mac

Open Terminal and run:

bash

npm install -g @anthropic-ai/claude-code

Mac users can proceed directly with the global installation without requiring additional platform-specific commands. The installation process automatically configures the necessary dependencies and PATH variables.

Setting Up Environment Variables

Environment variables configure Claude Code to use Kimi-K2 through Novita AI’s API endpoints. These variables tell Claude Code where to send requests and how to authenticate.

For Windows

Open Command Prompt and set the following environment variables:

cmd

set ANTHROPIC_BASE_URL=https://api.novita.ai/anthropic

set ANTHROPIC_AUTH_TOKEN=<Novita API Key*>*

set ANTHROPIC_MODEL=moonshotai/kimi-k2-instruct

set ANTHROPIC_SMALL_FAST_MODEL=moonshotai/kimi-k2-instruct

Replace <Novita API Key> with your actual API key obtained from the Novita AI platform. These variables remain active for the current session and must be reset if you close the Command Prompt.

For Mac

Open Terminal and export the following environment variables:

bash

export ANTHROPIC_BASE_URL="https://api.novita.ai/anthropic"

export ANTHROPIC_AUTH_TOKEN="<Novita API Key>"

export ANTHROPIC_MODEL="moonshotai/kimi-k2-instruct"

export ANTHROPIC_SMALL_FAST_MODEL="moonshotai/kimi-k2-instruct"

Starting Claude Code

With installation and configuration complete, you can now start Claude Code in your project directory. Navigate to your desired project location using the cd command:

cd <your-project-directory*>*

claude .

The dot (.) parameter instructs Claude Code to operate in the current directory. Upon startup, you’ll see the Claude Code prompt appear in an interactive session.

This indicates the tool is ready to receive your instructions. The interface provides a clean, intuitive environment for natural language programming interactions.

Building Your First Project

Claude Code excels at transforming detailed project descriptions into functional applications. After entering your prompt, press Enter to begin the task. Claude Code will analyze your requirements, create the necessary files, implement the functionality, and provide a complete project structure with documentation.

Here’s an example of how to create a Python Flask web app with MBTI personality guessing game:

Using Claude Code in VSCode or Cursor

Claude Code integrates seamlessly with popular development environments. It enhances your existing workflow rather than replacing it.

You can use Claude Code directly in the terminal within VSCode or Cursor. This maintains access to your familiar development tools while leveraging AI assistance.

Additionally, Claude Code plugins are available for both VSCode and Cursor. These plugins provide deeper integration with these editors, offering inline AI assistance, code suggestions, and project management features directly within your IDE interface.

The terminal integration allows you to run Claude Code commands without leaving your development environment. This creates a streamlined workflow for AI-assisted development.

Help and Documentation Resources

Claude Code includes comprehensive help documentation accessible through the /help command. This command displays available commands, usage examples, and troubleshooting information.

The help system is context-aware, providing relevant information based on your current project and session state.

For additional support, Novita AI provides extensive documentation . This covers advanced configuration options, API usage patterns, and best practices.

The Anthropic documentation offers detailed information about Claude Code’s capabilities and features.

Conclusion

Kimi-K2 integration with Claude Code through Novita AI delivers enterprise-grade capabilities at significantly reduced costs. The combination transforms natural language descriptions into functional code, dramatically accelerating development workflows. Start your journey with Kimi-K2 and Claude Code today to experience the future of AI-assisted programming.

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing the affordable and reliable GPU cloud for building and scaling.

Access Free DeepSeek R1 0528 API Now

Novita AI — Thu, 29 May 2025 08:48:55 +0000

We’re excited to announce that DeepSeek AI’s latest model, DeepSeek R1 0528, released today, is officially available in the Novita AI Model Library. We are also the official inference partner for DeepSeek R1 0528 on Hugging Face, supporting the community in bringing advanced models to production.

For a limited time, new users can claim $10 in free credits to explore and build with DeepSeek-R1 0528’s advanced reasoning capabilities.

Here’s the current DeepSeek-R1 0528 pricing on Novita AI:

DeepSeek-R1–0528: $0.7 / M input tokens, $2.5 / M output tokens

How to Access DeepSeek R1 0528 on Novita AI

Getting started with DeepSeek R1 0528 is fast, simple, and risk-free on Novita AI. Thanks to the Referral Program, you’ll receive $10 in free credits — enough to fully explore DeepSeek R1 0528’s power, build prototypes, and even launch your first use case without any upfront cost.

Use the Playground (No Coding Required)

Instant Access: Sign up, claim your free credits, and start experimenting with DeepSeek R1 0528 and other top models in seconds.
Interactive UI: Test prompts, chain-of-thought reasoning, and visualize results in real time.
Model Comparison: Effortlessly switch between Qwen 3, Llama 4, DeepSeek, and more to find the perfect fit for your needs.

Explore DeepSeek R1 0528 Demo Now

Integrate via API (For Developers)

Seamlessly connect DeepSeek R1 0528 to your applications, workflows, or chatbots with Novita AI’s unified REST API — no need to manage model weights or infrastructure. Novita AI offers multi-language SDKs (Python, Node.js, cURL, and more) and advanced parameter controls for power users.

Option 1: Direct API Integration (Python Example)

To get started, simply use the code snippet below:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    api_key="session_Ntg-O34ZOS-q5bNnkb3IcixmWnmxEQBxwKWMW3es3CD7KG4PEhFE1yRTRMGS3s8zZ52hrMdz14MmI4oalaDJTw==",
)
model = "deepseek/deepseek-r1-0528"
stream = True # or False
max_tokens = 2048
system_content = ""Be a helpful assistant""
temperature = 1
top_p = 1
min_p = 0
top_k = 50
presence_penalty = 0
frequency_penalty = 0
repetition_penalty = 1
response_format = { "type": "text" }
chat_completion_res = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "system",
            "content": system_content,
        },
        {
            "role": "user",
            "content": "Hi there!",
        }
    ],
    stream=stream,
    max_tokens=max_tokens,
    temperature=temperature,
    top_p=top_p,
    presence_penalty=presence_penalty,
    frequency_penalty=frequency_penalty,
    response_format=response_format,
    extra_body={
      "top_k": top_k,
      "repetition_penalty": repetition_penalty,
      "min_p": min_p
    }
  )
if stream:
    for chunk in chat_completion_res:
        print(chunk.choices[0].delta.content or "", end="")
else:
    print(chat_completion_res.choices[0].message.content)

Key Features:

Unified endpoint:/v3/openai supports OpenAI’s Chat Completions API format.
Flexible controls: Adjust temperature, top-p, penalties, and more for tailored results.
Streaming & batching: Choose your preferred response mode.

Option 2: Multi-Agent Workflows with OpenAI Agents SDK

Build advanced multi-agent systems by integrating Novita AI with the OpenAI Agents SDK:

Plug-and-play: Use Novita AI’s LLMs in any OpenAI Agents workflow.
Supports handoffs, routing, and tool use: Design agents that can delegate, triage, or run functions, all powered by Novita AI’s models.
Python integration: Simply point the SDK to Novita’s endpoint (https://api.novita.ai/v3/openai) and use your API key.

Connect DeepSeek R1 0528 API on Third-Party Platforms

Hugging Face: Use DeepSeek R1 0528 in Spaces, pipelines, or with the Transformers library via Novita AI endpoints.
Agent & Orchestration Frameworks: Easily connect Novita AI with partner platforms like Continue, AnythingLLM, LangChain, Dify and Langflow through official connectors and step-by-step integration guides.
OpenAI-Compatible API: Enjoy hassle-free migration and integration with tools such as Cline and Cursor, designed for the OpenAI API standard.

Novita AI is the All-in-one cloud platform that empowers your AI ambitions. Integrated APIs, serverless, GPU Instance — the cost-effective tools you need. Eliminate infrastructure, start free, and make your AI vision a reality.

Access Free DeepSeek R1 0528 API Now

Novita AI — Thu, 29 May 2025 08:42:14 +0000

For a limited time, new users can claim $10 in free credits to explore and build with DeepSeek-R1 0528’s advanced reasoning capabilities.

Here’s the current DeepSeek-R1 0528 pricing on Novita AI:

DeepSeek-R1–0528: $0.7 / M input tokens, $2.5 / M output tokens

How to Access DeepSeek R1 0528 on Novita AI
Getting started with DeepSeek R1 0528 is fast, simple, and risk-free on Novita AI. Thanks to the Referral Program, you’ll receive $10 in free credits — enough to fully explore DeepSeek R1 0528’s power, build prototypes, and even launch your first use case without any upfront cost.

Use the Playground (No Coding Required)
Instant Access: Sign up, claim your free credits, and start experimenting with DeepSeek R1 0528 and other top models in seconds.
Interactive UI: Test prompts, chain-of-thought reasoning, and visualize results in real time.
Model Comparison: Effortlessly switch between Qwen 3, Llama 4, DeepSeek, and more to find the perfect fit for your needs.
Explore DeepSeek R1 0528 Demo Now

Integrate via API (For Developers)
Seamlessly connect DeepSeek R1 0528 to your applications, workflows, or chatbots with Novita AI’s unified REST API — no need to manage model weights or infrastructure. Novita AI offers multi-language SDKs (Python, Node.js, cURL, and more) and advanced parameter controls for power users.

Option 1: Direct API Integration (Python Example)
To get started, simply use the code snippet below:

from openai import OpenAI

client = OpenAI(
base_url="https://api.novita.ai/v3/openai",
api_key="session_Ntg-O34ZOS-q5bNnkb3IcixmWnmxEQBxwKWMW3es3CD7KG4PEhFE1yRTRMGS3s8zZ52hrMdz14MmI4oalaDJTw==",
)
model = "deepseek/deepseek-r1-0528"
stream = True # or False
max_tokens = 2048
system_content = ""Be a helpful assistant""
temperature = 1
top_p = 1
min_p = 0
top_k = 50
presence_penalty = 0
frequency_penalty = 0
repetition_penalty = 1
response_format = { "type": "text" }
chat_completion_res = client.chat.completions.create(
model=model,
messages=[
{
"role": "system",
"content": system_content,
},
{
"role": "user",
"content": "Hi there!",
}
],
stream=stream,
max_tokens=max_tokens,
temperature=temperature,
top_p=top_p,
presence_penalty=presence_penalty,
frequency_penalty=frequency_penalty,
response_format=response_format,
extra_body={
"top_k": top_k,
"repetition_penalty": repetition_penalty,
"min_p": min_p
}
)
if stream:
for chunk in chat_completion_res:
print(chunk.choices[0].delta.content or "", end="")
else:
print(chat_completion_res.choices[0].message.content)
Key Features:

Unified endpoint:/v3/openai supports OpenAI’s Chat Completions API format.
Flexible controls: Adjust temperature, top-p, penalties, and more for tailored results.
Streaming & batching: Choose your preferred response mode.
Option 2: Multi-Agent Workflows with OpenAI Agents SDK
Build advanced multi-agent systems by integrating Novita AI with the OpenAI Agents SDK:

Plug-and-play: Use Novita AI’s LLMs in any OpenAI Agents workflow.
Supports handoffs, routing, and tool use: Design agents that can delegate, triage, or run functions, all powered by Novita AI’s models.
Python integration: Simply point the SDK to Novita’s endpoint (https://api.novita.ai/v3/openai) and use your API key.
Connect DeepSeek R1 0528 API on Third-Party Platforms
Hugging Face: Use DeepSeek R1 0528 in Spaces, pipelines, or with the Transformers library via Novita AI endpoints.
Agent & Orchestration Frameworks: Easily connect Novita AI with partner platforms like Continue, AnythingLLM, LangChain, Dify and Langflow through official connectors and step-by-step integration guides.
OpenAI-Compatible API: Enjoy hassle-free migration and integration with tools such as Cline and Cursor, designed for the OpenAI API standard.
Showcase
Prompt: Write a Python program that shows a ball bouncing inside a spinning hexagon. The ball should be affected by gravity and friction, and it must bounce off the rotating walls realistically

Prompt: Build a pilot game

Prompt: Build a PDF summary web app + UI concept

Novita AI is the All-in-one cloud platform that empowers your AI ambitions. Integrated APIs, serverless, GPU Instance — the cost-effective tools you need. Eliminate infrastructure, start free, and make your AI vision a reality.

Qwen 3 Now Available on Novita AI — Claim Your $10 Free Credits

Novita AI — Fri, 23 May 2025 03:30:00 +0000

We’re excited to announce a strategic partnership with SGLang, a fast serving engine for large language models and vision language models. Through this collaboration, Novita AI will provide high-performance GPU cloud resources for SGLang’s ongoing research, benchmarking, and optimization efforts.

SGLang is a leading inference engine that co-designs a structured generation language with a highly optimized runtime, enabling powerful performance gains such as efficient RadixAttention cache reuse and zero-overhead batch scheduling for large language and vision-language models. By aligning language-level control with backend optimizations, it empowers developers to build complex generation workflows, multi-modal applications, and parallel inference pipelines with reliability and scale. SGLang is supported by leading institutions including NVIDIA, AMD, xAI, Oracle Cloud, Google Cloud, LinkedIn, Cursor, alongside research groups at Stanford, University of California, Berkeley, and University of California, Los Angeles — evidence of strong community engagement and broad industry adoption.

“SGLang’s integration of language-level primitives with runtime optimizations demonstrates the value of aligning software and hardware to unlock new performance levels,” said Junyu Huang, Co-Founder & COO at Novita AI. “By contributing our infrastructure and expertise, we’ve already supported the development of SGLang’s first end-to-end multi-turn reinforcement learning (RL) framework and the Prism multi-large language model serving system, and remain committed to fueling its ongoing innovations for developers everywhere.”

“We’re thrilled to partner with the SGLang team,” added Junyu Huang. “Having supported their RL framework and multi-LLM serving system, we’re excited to see these achievements accelerate their work and bring powerful inference performance to applications across industries.”

Novita AI is also collaborating on SGLang’s large-scale expert parallelism project, an open-source implementation designed to approach the throughput benchmarks detailed in the official DeepSeek blog, partnering to bring this milestone to fruition.

This collaboration reflects Novita AI’s ongoing commitment to advancing an open ecosystem of inference engines and supporting diverse research initiatives through shared infrastructure and joint development efforts.

Through collaborations with pioneering open-source projects like SGLang, Novita AI continues to advance its mission of democratizing AI, making cutting-edge inference capabilities readily available to developers worldwide.

About Novita AI

Novita AI is an AI cloud platform that helps developers easily deploy AI models through a simple API, backed by affordable and reliable GPU cloud infrastructure. By supporting open-source libraries for LLM inference and serving — Novita AI is driving the future of AI and encouraging innovation across the industry.

LLM Dedicated Endpoint on Novita AI: Custom Models, Usage-Based Pricing, and DevOps-Free Scaling

Novita AI — Wed, 14 May 2025 04:00:00 +0000

Want to ship your own fine-tuned LLMs, without babysitting GPUs or racking up idle costs?

Novita AI’s LLM Dedicated Endpoint gives you true flexibility: run your custom models, pay only for tokens used, and let Novita handle deployment and scaling.

Compared to LLM Public APIs, it’s your stack, your way. Compared to raw GPU hosting, you get predictable pricing and a pro team to keep your models running smoothly.

What is an LLM Dedicated Endpoint?

A LLM Dedicated Endpoint is your own private API for running any model you want — fine-tuned, proprietary, or mainstream. No noisy neighbors, no shared resources. Novita AI handles all the infra, you just send requests. Learn more

Key Features

Bring Your Own Model: Deploy your fine-tuned or custom LLMs.
No Idle GPU Bills: Pay only for tokens used (usage-based, not hourly).
Auto-Scales Instantly: Handles spikes, no manual scaling.
Full Isolation: Dedicated compute, your data only.
Enterprise Uptime, Low Latency: SLAs for mission-critical apps.
Zero-DevOps: Monitoring, scaling, and patching done for you.

LLM Public Endpoints vs LLM Dedicated Endpoint

Novita AI offers two LLM API flavors—pick what fits your workflow:

1. LLM Public Endpoints

What:

Plug-and-play APIs for open-source models like Llama, DeepSeek, Qwen, Gemma, and more.
When to use:

Prototyping, hackathons, projects with standard LLMs.
Why:
Fast to integrate
No servers or infra
Scale to production

2. LLM Dedicated Endpoint

What:

Your own API for custom/fine-tuned models, including proprietary LLMs.
When to use:

When you need control, privacy, or custom models (think: internal tools, production SaaS, unique data).
Why:
Private, dedicated resources
Custom SLAs and scaling
Usage-based pricing
Expert deployment and monitoring

TL;DR:

Need standard models, fast? Go Public Endpoints.

Need your own model, full control, and pro support? Go LLM Dedicated Endpoint.

Why Developers Love It

Drop-in API: Keep your code—just update the endpoint URL.
No Cloud Headaches: No need for Dockerfiles, GPU quotas, or on-call alerts.
Transparent Pricing: No surprises. Billed for tokens, with optional daily minimums.
24/7 Support: Hit a snag? Ping Novita’s support team.

How to Get Started

Ready to deploy?

Contact Novita AI Sales
Share your requirements (QPS, latency, model type)
Novita sets up your endpoint—no DevOps needed
Update your API URL and ship!

Conclusion

LLM Dedicated Endpoint on Novita AI is the dev-friendly way to run custom models with no ops, no idle GPU costs, and no guesswork. You focus on building, Novita keeps your models running—secure, scalable, and fast.

Ready to launch your own LLM? Book a Demo.

Frequently Asked Questions

How does Novita handle scaling during traffic spikes?

Resources auto-scale based on real-time demand. You’re only billed for actual usage, not reserved capacity.

Can I migrate from a Novita public API to a Dedicated Endpoint?

Yes—just update the endpoint URL. 100% API compatibility means no code changes are required.

What if I need guaranteed uptime and latency?

Novita offers custom SLAs for uptime, latency, and throughput, tailored to your needs.

How is billing handled?

You pay only for tokens processed, with a minimum daily token commitment. No idle GPU bills.

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing the affordable and reliable GPU cloud for building and scaling.

Qwen 3 Now Available on Novita AI - Claim Your $10 Free Credits

Novita AI — Tue, 29 Apr 2025 10:09:20 +0000

Alibaba’s cutting-edge Qwen 3 large language models are now live on Novita AI’s Model API platform! Instantly access the latest Qwen3–235B-A22B, Qwen3–30B-A3B, and Qwen3–32B models — all with a massive 128,000 context window and industry-leading performance.

For a limited time, new users can claim $10 in free credits to explore and build with Qwen 3.

Here’s the current Qwen 3 lineup and pricing on Novita AI:

Qwen3–235B-A22B: $0.20 / M input tokens, $0.80 / M output tokens
Qwen3–30B-A3B: $0.10 / M input tokens, $0.45 / M output tokens
Qwen3–32B: $0.10 / M input tokens, $0.45 / M output tokens

Power your chatbots, apps, and workflows with state-of-the-art language models — Qwen 3 is just an API call away.

What is Qwen 3?

Qwen 3 is the latest and most advanced family of large language models developed by Alibaba Cloud’s Qwen team. Building on the experience of QwQ and Qwen2.5, Qwen 3 sets a new standard for open-source AI with major improvements in reasoning, multilingualism, and agentic abilities.

Key Features of Qwen 3

Dense and Mixture-of-Experts (MoE) models in various sizes: Qwen 3 is available in both dense and MoE architectures, ranging from lightweight 0.6B and 1.7B models up to large-scale 32B (dense) and flagship 30B-A3B and 235B-A22B (MoE) variants.
Hybrid thinking modes: The model allows seamless switching between thinking mode (for complex, step-by-step logical reasoning, math, and code generation) and non-thinking mode (for fast, efficient, general-purpose chat).
Significantly enhanced reasoning: Qwen 3 surpasses previous Qwen models in mathematics, code generation, and commonsense logical reasoning. It also offers more stable and controllable reasoning budgets for different tasks.
Superior human preference alignment: The model excels in creative writing, role-playing, multi-turn dialogues, and instruction following, resulting in more natural, engaging conversations.
Advanced agentic capabilities: Qwen 3 is designed for agent-based workflows, supporting seamless integration with external tools and precise function calling in both reasoning modes. This enables state-of-the-art performance in complex, agent-driven tasks.
Robust multilingual support: Supporting 119 languages and dialects, Qwen 3 is capable of high-quality multilingual instruction following and translation, opening the door for truly global applications.

Benchmarks and Performance

The Qwen 3 series demonstrates industry-leading performance across a comprehensive suite of AI benchmarks, excelling in coding, mathematics, general reasoning, and multilingual understanding.

Flagship Model: Qwen3–235B-A22B

The flagship model, Qwen3–235B-A22B, consistently achieves top or near-top results when compared with the most advanced models available today, such as DeepSeek-R1, OpenAI-01, OpenAI-o3-mini, Grok-3 Beta, and Gemini-2.5-Pro.

Complex Reasoning: Highest scores on ArenaHard (95.6), outperforming or matching all competitors.
Mathematics: Leading results on AIME’24 (85.7) and AIME’25 (81.5), well ahead of most commercial and open-source models.
Coding: Exceptional performance on LiveCodeBench (70.7) and CodeForces Elo (2056), confirming its strength in software and algorithmic tasks.
Multilingual & General Capabilities: Qwen3–235B-A22B achieves strong results on LiveBench and MultiF, demonstrating robust real-world and multilingual understanding.

Model Efficiency and Scalability

Qwen 3’s architectural innovations also translate to outstanding performance at smaller model sizes:

Qwen3–32B (Dense): Delivers results just behind the flagship, still outperforming most alternative models across all categories.
Qwen3–30B-A3B (MoE): Outperforms QwQ-32B, despite using only a tenth of the activated parameters — showcasing Qwen’s efficiency and smart scaling.
Qwen3–4B (Dense): Even this compact model can rival the performance of much larger models like Qwen2.5–72B-Instruct, especially on reasoning and multilingual tasks.

How to Access Qwen 3 on Novita AI

Getting started with Qwen 3 is fast, simple, and risk-free on Novita AI. Thanks to the Referral Program, you’ll receive $10 in free credits — enough to fully explore Qwen 3’s power, build prototypes, and even launch your first use case without any upfront cost.

Use the Playground (No Coding Required)

Instant Access: Sign up, claim your free credits, and start experimenting with Qwen 3 and other top models in seconds.
Interactive UI: Test prompts, chain-of-thought reasoning, and visualize results in real time.
Model Comparison: Effortlessly switch between Qwen 3, Llama 4, DeepSeek, and more to find the perfect fit for your needs.

Integrate via API (For Developers)

Seamlessly connect Qwen 3 to your applications, workflows, or chatbots with Novita AI’s unified REST API — no need to manage model weights or infrastructure. Novita AI offers multi-language SDKs (Python, Node.js, cURL, and more) and advanced parameter controls for power users.

Option 1: Direct API Integration (Python Example)

To get started, simply use the code snippet below:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    api_key="<YOUR Novita AI API Key>",
)

model = "qwen/qwen3-235b-a22b-fp8"
stream = True # or False
max_tokens = 2048
system_content = """Be a helpful assistant"""
temperature = 1
top_p = 1
min_p = 0
top_k = 50
presence_penalty = 0
frequency_penalty = 0
repetition_penalty = 1
response_format = { "type": "text" }

chat_completion_res = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "system",
            "content": system_content,
        },
        {
            "role": "user",
            "content": "Hi there!",
        }
    ],
    stream=stream,
    max_tokens=max_tokens,
    temperature=temperature,
    top_p=top_p,
    presence_penalty=presence_penalty,
    frequency_penalty=frequency_penalty,
    response_format=response_format,
    extra_body={
      "top_k": top_k,
      "repetition_penalty": repetition_penalty,
      "min_p": min_p
    }
  )

if stream:
    for chunk in chat_completion_res:
        print(chunk.choices[0].delta.content or "", end="")
else:
    print(chat_completion_res.choices[0].message.content)

Option 2: Multi-Agent Workflows with OpenAI Agents SDK

Build advanced multi-agent systems by integrating Novita AI with the OpenAI Agents SDK:

Plug-and-play: Use Novita AI’s LLMs in any OpenAI Agents workflow.
Supports handoffs, routing, and tool use: Design agents that can delegate, triage, or run functions, all powered by Novita AI’s models.
Python integration: Simply point the SDK to Novita’s endpoint (https://api.novita.ai/v3/openai) and use your API key.

Connect Qwen 3 API on Third-Party Platforms

Hugging Face: Use Qwen 3 in Spaces, pipelines, or with the Transformers library via Novita AI endpoints.
Agent & Orchestration Frameworks: Easily connect Novita AI with partner platforms like Continue, AnythingLLM, LangChain, Dify and Langflow through official connectors and step-by-step integration guides.
OpenAI-Compatible API: Enjoy hassle-free migration and integration with tools such as Cline and Cursor, designed for the OpenAI API standard.

Best Practices for Optimal Qwen 3 Performance

Sampling Parameter Settings

Thinking Mode

enable_thinking=True

Temperature: 0.6

TopP: 0.95

TopK: 20

MinP: 0

Tip: Avoid greedy decoding to prevent degraded performance or repetitive outputs.

Non-Thinking Mode

enable_thinking=False

Temperature: 0.7

TopP: 0.8

TopK: 20

MinP: 0

Repetition Control

For supported frameworks, adjust presence_penalty between 0 and 2 to reduce repetitions.

Note: Higher values may cause some language mixing or a slight decrease in model performance.

2. Output Length Recommendations

For most queries, set the output length to 32,768 tokens.
For complex benchmarking tasks (such as math or programming competitions), increase the max output length to 38,912 tokens for more comprehensive responses.

3. Standardizing Output Format

Math Problems: Include this in your prompt: “Please reason step by step, and put your final answer within \boxed{}.”
Multiple-Choice Questions: Standardize responses using a JSON field: “Please show your choice in the answer field with only the choice letter, e.g., “answer”: “C”.”

4. Conversation History Management

In multi-turn conversations, include only the final output in the chat history. Omit any intermediate “thinking” content.
If using a Jinja2 chat template, this is handled automatically. For other frameworks, ensure this practice is followed manually.

By following these recommendations, you’ll ensure Qwen 3 consistently delivers accurate, high-quality results across all use cases.

Conclusion

Qwen 3 delivers best-in-class performance for coding, reasoning, and multilingual tasks — no matter the project size. Ready to see it in action?

Try the Qwen 3 demo on Novita AI now and claim your free credits!

Originally published on Novita AI

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing the affordable and reliable GPU cloud for building and scaling.

Earn $500 Free Credits: Build Faster with Deepseek, Llama & Qwen on Novita AI

Novita AI — Thu, 24 Apr 2025 05:46:08 +0000

Novita AI is offering an exclusive, limited-time opportunity! With the Referral Program, you can earn up to $500 in LLM API credits by simply referring your friends. Here’s the best part: both you and your referral will receive $10 in credits, unlocking access to top-tier models like DeepSeek, Llama and Qwen.

These credits can power your next big project, whether you’re working with Hugging Face, Anything LLM, Langflow, Continue, Helicone, Dify, Cursor, LobeChat, and many more.
Don’t miss out on the opportunity to supercharge your AI applications.

👉 Sign up for the Novita AI Referral Program and begin earning credits now.

Why Novita AI is the Trusted Choice
Artificial Analysis, a leading AI model evaluation platform, ranks Novita AI alongside industry leaders such as Together AI and Fireworks AI, reinforcing Novita AI’s reputation as a trusted choice for developers worldwide.

Additionally, OpenRouter recognizes Novita AI as one of the most cost-effective LLM API providers.

Novita AI also serves as the official inference provider on Hugging Face.

4 Easy Steps to Claim $10 in API Credits
Visit the Referral Program Page
Head to the official page to begin.
Enter Your Invite Code
Use either the official invite code 5W10UA or your personal one to get started.
Create Your Novita AI Account
Sign up using your email, Google, Hugging Face or GitHub account.
Verify Your GitHub Account
Complete the verification process to unlock your credits.
3 Ways to Share and Earn Up to $500 in Credits
Earn up to $500 in LLM API credits by referring others. Here’s how you can share and earn:

Copy Your Referral Link:
https://novita.ai/referral?invited_code=xxx
Copy your own Referral Code
Share on Social Media:
Post your referral link on platforms like Twitter (X), LinkedIn, Facebook, or anywhere else developers are hanging out.
The more you share, the more you earn!

LLM API on Novita AI
You can use your credits across the entire range of LLM APIs available on Novita AI. Below is a comprehensive list of all supported LLM APIs on the platform.

LLM API
Integrated Projects & SDKs
Novita AI supports seamless integration with many leading open-source projects and developer tools. Once you get the LLM API credits, you can call the API on the following platforms.

Novita AI & OpenAI Agents SDK
Novita AI & AnythingLLM
Novita AI & Dify
Novita AI & Helicone
Novita AI & Hugging Face
Novita AI & Langflow
Novita AI & Continue
Novita AI & Cursor
Novita AI & LangChain
Novita AI & Skyvern
Novita AI & LobeChat
Novita AI & ai-gradio
Novita AI & Langfuse
Novita AI & Verba
Novita AI & Portkey
Novita AI & DocsGPT
Novita AI & LlamaIndex
Novita AI & LoLLMS WebUI
Novita AI & CodeCompanion.nvim
Novita AI & Page Assist
Novita AI & DeepSearcher
Start Earning and Building with Novita AI Today!
Don’t miss out on the chance to earn up to $500 in credits, unlock powerful LLM API models, and supercharge your projects with Novita AI. Whether you’re building AI-powered tools, developing advanced agents, or creating the next big thing in AI, Novita AI is your trusted partner.

👉 Sign up now, share your link, and start building!

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing an affordable and reliable GPU cloud for building and scaling.

What is Agent2Agent (A2A)? A New Era of AI Agent Interaction

Novita AI — Sat, 12 Apr 2025 04:00:00 +0000

In the world of artificial intelligence (AI), the ability for different AI agents to communicate and collaborate is essential for streamlining processes and maximizing productivity. Google has taken a major step forward in this area with the introduction of the Agent2Agent (A2A) protocol. This open protocol allows AI agents to interact with each other across different platforms, breaking down silos and enabling more efficient collaboration. In this article, we’ll explore what A2A is, how it works, and the real-world applications of this groundbreaking protocol.

What is Agent2Agent (A2A)?

Agent2Agent (A2A) is an open protocol developed by Google that allows AI agents to communicate and collaborate, regardless of their underlying frameworks or platforms. It enables seamless interaction between diverse AI agents, making it possible for them to perform complex tasks together without being constrained by system boundaries.

Think of A2A as a universal translator for AI agents — it allows different agents to “speak” the same language, regardless of the technology they’re built on. With A2A, AI agents can now share information, update each other, and perform tasks collaboratively without requiring a complete overhaul of existing systems.

This protocol is set to radically change how businesses operate by allowing AI agents to integrate more easily into everyday workflows, without needing to be customized for every different platform or framework.

How Does A2A Work?

A2A works by facilitating communication between two key types of agents: the client agent and the remote agent. Let’s break down the process:

Discovery: The client agent first discovers the capabilities of the remote agent by fetching its “Agent Card,” a JSON-based file that contains metadata about the remote agent, including its skills, authentication requirements, and endpoint URL.
Task Assignment: Once a suitable agent has been identified, the client agent can assign a task to the remote agent. This task is represented by a unique Task ID, which helps both agents keep track of the progress and state of the task.
Communication: After the task is initiated, the two agents communicate with each other by sending messages, which may include text, files, or other data. These messages are referred to as “Parts,” and each part can contain different types of content such as plain text, files, or structured data.
Task Progress: A2A supports long-running tasks and can provide real-time updates using Server-Sent Events (SSE) or Push Notifications. This allows agents to continuously update each other on the progress of the task.
Completion: Once the task is completed, the results are referred to as Artifacts. These artifacts represent the outputs or results generated by the remote agent in response to the client’s task.

Key Features of Agent2Agent

1. Interoperability Across Platforms

A2A is designed to be platform-agnostic, allowing AI agents built on different frameworks to work together seamlessly. This breaks down traditional system silos, enabling businesses to integrate their existing systems with AI agents without needing to overhaul their entire infrastructure.

For example, a business might use Atlassian for project management, Box for file storage, and Salesforce for customer relationship management. With A2A, these systems can now work together by allowing their respective agents to communicate and share data.

2. Task Management and Flexibility

A2A provides a robust mechanism for managing tasks. Each task has a lifecycle, and agents can communicate with each other to ensure the task progresses smoothly. Whether the task is a simple query or a complex, multi-step process, A2A handles it efficiently.

Additionally, the protocol is designed to support a wide range of tasks, from quick operations to long-running research projects. This flexibility is crucial in industries like healthcare or research, where tasks can vary dramatically in terms of time and complexity.

3. Security and Authentication

A2A supports enterprise-level authentication and authorization, ensuring that data exchanges between agents are secure and comply with regulations. This makes it ideal for businesses that need to ensure sensitive data is handled appropriately during agent interactions.

By supporting existing authentication mechanisms such as OpenAPI, A2A ensures that enterprises can quickly implement it into their current systems without worrying about security issues.

4. Multi-Modal Support

Unlike traditional text-based AI agents, A2A supports multiple modes of interaction. This means agents can communicate not only through text, but also through images, videos, and audio. This opens up new possibilities for tasks that require richer forms of communication, such as training simulations, customer service interactions, and multimedia content generation.

5. Real-Time Updates and Notifications

For tasks that take a long time to complete, A2A offers real-time updates via Server-Sent Events (SSE) or Push Notifications. These updates allow the client agent to track the status of the task and receive feedback about the task’s progress, artifacts, or issues in real-time. This feature is especially useful for long-running tasks in industries like pharmaceuticals, where real-time progress updates are crucial for timely decision-making.

Real-World Applications of A2A

Enterprise Software Integration

Many enterprises use a range of platforms for different functions — for example, Atlassian for project management, Box for file storage, and Salesforce for customer relationship management. Traditionally, these systems don’t communicate with each other. With A2A, however, these platforms can integrate seamlessly, allowing their respective agents to share data and automate tasks.

For instance, an e-commerce company could use A2A to link its order management system with intelligent agents that provide real-time logistics updates. This integration would streamline operations without requiring the company to rebuild its existing infrastructure.

Research and Development

A2A is also useful in research environments, where tasks can range from simple data retrieval to complex simulations. A research organization working on drug development might use A2A to connect various agents, such as those responsible for data analysis, database querying, and simulations of molecular structures.

In this context, A2A provides real-time progress updates, helping researchers stay informed about the status of long-running tasks, such as simulating the interaction of drug molecules with human cells.

Healthcare

In the healthcare industry, A2A can be used to integrate AI agents that handle patient records, diagnosis, treatment recommendations, and follow-ups. By using A2A, healthcare systems can ensure that these various agents can work together seamlessly, improving patient care and operational efficiency.

For example, an agent in a hospital’s scheduling system might use A2A to communicate with an AI-powered diagnostic agent to schedule and follow up on appointments related to specific health conditions.

Conclusion

The Agent2Agent (A2A) protocol represents a major leap forward in AI interoperability, allowing AI agents to communicate and collaborate seamlessly across different platforms and frameworks. By breaking down traditional system silos, A2A enables more efficient, secure, and flexible workflows, making it easier for businesses and developers to integrate AI into their operations.

As A2A continues to gain adoption across industries, it will likely lead to significant advancements in AI-powered automation, communication, and collaboration. Whether you’re a developer looking to integrate AI into your applications, or a business seeking to optimize your workflows, A2A has the potential to revolutionize the way we interact with AI agents.

By embracing A2A, businesses can unlock new possibilities, improve operational efficiency, and pave the way for a more interconnected future in AI development.

Frequently Asked Questions

What distinguishes A2A from other interoperability protocols?

A2A focuses specifically on agent-to-agent communication and collaboration across different platforms and vendors, providing a networking layer for agents to discover, negotiate, and interact securely.

Can A2A work with agents built on any framework?

Yes, A2A is framework-agnostic, enabling communication between agents regardless of their underlying technology or vendor.

How does A2A ensure security?

A2A uses token-based security for function calling, leverages DNS security for discovery, and specifies authentication requirements through Agent Cards.

How does A2A complement Anthropic’s Model Context Protocol (MCP)?

While MCP enhances individual agent capabilities through plugins, A2A facilitates communication and collaboration between different agents, serving as a networking layer for multi-agent systems.

About Novita AI

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing an affordable and reliable GPU cloud for building and scaling.

Novita AI Evaluates FlashMLA on H100 and H200

Novita AI — Fri, 11 Apr 2025 08:17:59 +0000

DeepSeek has officially kicked off its five-day open source release initiative, with the first featured project being FlashMLA. FlashMLA is an optimized, high-efficiency MLA decoding kernel specifically designed for NVIDIA Hopper GPUs (e.g., H800 SXM5). Its primary goal is to accelerate computations for large-scale models, particularly enhancing performance on NVIDIA's high-end GPUs.

As a leading provider of AI infrastructure, Novita AI was among the first to evaluate FlashMLA's performance across mainstream Hopper GPUs (H100, H200).

What is MLA?

Before diving into the evaluation results, let’s take a moment to understand some relevant background concepts.

Hopper GPU: NVIDIA's next-generation high-performance GPU architecture, engineered for AI and high-performance computing (HPC). Built with advanced process technologies and an innovative architecture, Hopper GPUs deliver exceptional performance and energy efficiency for complex computational tasks. The mainstream Hopper GPUs include H100 and H200.
Decoding Kernel: A hardware or software module specifically designed to accelerate decoding tasks. In AI inference, decoding kernels significantly enhance the speed and efficiency of model inference, particularly when processing sequential data.
Key-Value (KV) Pairs
- Key:
  - Represents a compressed version of the input data, used to compute attention weights (how much focus to place on different parts of the input).
  - Example: In text generation, keys help the model identify which words in a sentence are most relevant to the current word being generated.
- Value:
  - Contains the actual information associated with each input token, weighted by the attention scores.
  - Example: Values store the semantic meaning of words, which are combined based on attention weights to produce the output.
MLA (Multi-head Latent Attention): A novel attention mechanism that requires lighter KV (key-value) caching, making it more scalable for long-sequence processing. MLA outperforms traditional Multi-Head Attention (MHA) mechanisms in both scalability and performance.

MHA VS MQA VS GQA VS MLA

Module	Technical Logic	Inference Speed	Model Performance
MHA	Multiple heads independently generate keys and values with no sharing (full-dimensional computation).	⭐️	⭐️⭐️⭐️
MQA	All query heads share a single key-value pair (single KV group).	⭐️⭐️⭐️	⭐️
GQA	Query heads share key-value pairs in groups (multiple KV groups).	⭐️⭐️	⭐️⭐️
MLA	Key-value pairs are compressed into low-dimensional latent vectors and decoded with decoupled RoPE to retain positional information.	⭐️⭐️⭐️⭐️	⭐️⭐️⭐️⭐️

MQA/GQA: A "simplified version" of MHA, focusing on efficiency at the cost of information loss.
MLA: An "upgraded compressed version" that balances memory efficiency and information retention, even outperforming MHA.
Architectural Innovation: MLA is not a mere optimization but a reimagining of attention mechanisms, leveraging latent variables to reconstruct them mathematically. It achieves the best of both worlds: efficiency and capability.

FlashMLA Performance Evaluation by Novita AI

DeepSeek has announced that FlashMLA achieves a memory bandwidth limit of 3000 GB/s and a compute limit of 580 TFLOPS on the H800 SXM5 GPU. To validate these claims, Novita AI conducted a comprehensive evaluation, testing FlashMLA under various parameter configurations.

To present the results more intuitively, the horizontal axis in the performance charts represents the following parameter configurations:

Batch Size
Sequence Length
Number of Attention Heads

Note

These results are based on the official test scripts. Without knowledge of the optimal parameter configurations, the data may not fully reflect theoretical maximums.

What Impact Will FlashMLA Have?

The release of FlashMLA has not only captured the interest of developers but also garnered positive responses from mainstream inference frameworks, vLLM and SGLang.

vLLM Integration:

The vLLM team has announced plans to integrate FlashMLA soon. Technically, FlashMLA is built on PagedAttention, making it highly compatible with vLLM's technology stack. Once integrated, FlashMLA is expected to further enhance vLLM's inference performance.
SGLang Adoption:

SGLang will continue utilizing the already integrated FlashInferMLA, which has been evaluated to deliver performance comparable to FlashMLA.

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing the affordable and reliable GPU cloud for building and scaling.

💡
Get $20 credits and Try DeepSeek now!

Recommend Reading

Llama 4 Now Available on Novita AI: Unleashing Multimodal MoE Power

Novita AI — Tue, 08 Apr 2025 06:33:39 +0000

Meta has just unveiled its groundbreaking Llama 4 family of models, marking a significant leap in AI capabilities with native multimodality and mixture-of-experts (MoE) architecture.

Today, we’re excited to announce that Llama 4 Scout and Llama 4 Maverick are now available on Novita AI, enabling business and developers to harness these powerful models through simple API integration.

Novita AI is offering the first of the Llama 4 model herd at the following pricing:

Llama 4 Scout: $0.1 / M input tokens and $0.5 / M output tokens

Llama 4 Maverick: $0.2 / M input tokens and $0.85 / M output tokens

Understanding the Llama 4 Herd

The Llama 4 release introduces three distinct models, each designed for different needs and computational constraints:

Llama 4 Scout features 16 experts and delivers state-of-the-art performance for its class. It supports an industry-leading 10M token context length, making it ideal for processing large amounts of data, including entire codebases or extensive documentation.

Llama 4 Maverick is Meta’s product workhorse, incorporating 128 experts to deliver superior performance across a wide range of tasks. It excels at precise image understanding and creative writing while supporting up to 1M tokens in context.

Llama 4 Behemoth serves as the teacher model for the Llama 4 family with 16 experts. While not yet publicly released as it’s still in training, Behemoth outperforms GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM-focused benchmarks.

Note: The context window for Llama 4 Scout on Novita AI is 131,072 tokens, while the context window for Llama 4 Maverick is 1,048,576 tokens.

Key Features and Capabilities

Native Multimodality

Llama 4 models incorporate early fusion to seamlessly integrate text and vision tokens into a unified model backbone. This enables joint pre-training with large amounts of unlabeled text, image, and video data.

The enhanced vision encoder, based on MetaCLIP but further optimized for LLM integration, allows the models to process multiple images alongside text prompts without additional engineering.

Extended Context Length

One of the most significant advancements in Llama 4 is its support for extraordinarily long contexts:

Llama 4 Scout: 10 million tokens
Llama 4 Maverick: 1 million tokens

This leap in context length enables applications that were previously impractical, such as:

Multi-document summarization and analysis
Reasoning over extensive codebases
Parsing vast amounts of user activity for personalized experiences
Processing entire research archives in a single prompt

Multilingual and Reasoning Capabilities

Llama 4 models have been pre-trained on 200 languages — with dedicated fine-tuning support for 12, including Arabic, Spanish, German, and Hindi. Over 100 of these languages have more than 1 billion training tokens each — offering 10 times more multilingual coverage than Llama 3.

This extensive training enables superior performance across languages, making the models suitable for global applications.

The models also demonstrate enhanced reasoning abilities thanks to specialized training recipes. For Maverick, this included a continuous online RL strategy with adaptive data filtering, focusing on medium-to-hard difficulty prompts.

Performance Benchmarks and Use Cases

Benchmarks

According to Meta’s official benchmark data, Llama 4 models demonstrate exceptional performance across various tasks, as shown in the tables below:

Llama 4 Scout Benchmarks

Llama 4 Maverick Benchmarks

Llama 4 Scout is best suited for long-context applications, while Llama 4 Maverick excels at complex reasoning and creative tasks that involve multimodal understanding. Here are the ideal use cases for each model based on their strengths:

Llama 4 Scout:

Multi-document summarization for legal or financial analysis
Personalized task automation using extensive user data
Efficient image processing for lightweight multimodal applications

Explore Llama 4 Scout Demo Now

Llama 4 Maverick:

Multilingual customer support with visual context
Generating marketing content based on multimodal inputs
Advanced document intelligence combining text, diagrams, and tables
Creative writing and content generation with precise image understanding

Explore Llama 4 Maverick Demo Now

Both models excel in situations requiring multimodal understanding, reasoning over extensive context, and multilingual capabilities.

Getting Started with Llama 4 on Novita AI

Integrating Llama 4 models into your applications via Novita AI’s model library is straightforward, requiring just a few lines of code. Here’s how to get started:

Setting Up Your Environment

First, ensure you have an API key from Novita AI. If you don’t have one yet, sign up and create an API key through the Novita AI dashboard.

Integrating with Python

Novita AI provides OpenAI-compatible endpoints for seamless integration. Here’s a simple example using the Python client:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    api_key="<YOUR Novita AI API Key>",
)

model = "meta-llama/llama-4-maverick-17b-128e-instruct-fp8"
stream = True # or False
max_tokens = 2048
system_content = """Be a helpful assistant"""
temperature = 1
top_p = 1
min_p = 0
top_k = 50
presence_penalty = 0
frequency_penalty = 0
repetition_penalty = 1
response_format = { "type": "text" }

chat_completion_res = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "system",
            "content": system_content,
        },
        {
            "role": "user",
            "content": "Hi there!",
        }
    ],
    stream=stream,
    max_tokens=max_tokens,
    temperature=temperature,
    top_p=top_p,
    presence_penalty=presence_penalty,
    frequency_penalty=frequency_penalty,
    response_format=response_format,
    extra_body={
      "top_k": top_k,
      "repetition_penalty": repetition_penalty,
      "min_p": min_p
    }
  )

if stream:
    for chunk in chat_completion_res:
        print(chunk.choices[0].delta.content or "", end="")
else:
    print(chat_completion_res.choices[0].message.content)

For more detailed examples and comprehensive integration guides, visit our LLM API documentation.

Conclusion

The arrival of Llama 4 on Novita AI represents a significant milestone in the democratization of advanced AI capabilities.

With native multimodality, extended context lengths, and an efficient MoE architecture, these models enable new classes of applications that were previously impractical or prohibitively expensive.

Whether you’re building applications for document processing, multilingual communication, or creative content generation, Llama 4 provides the tools you need to create intelligent, responsive experiences.

Get started today with Novita AI’s simple integration process and competitive pricing to bring the power of Llama 4 to your applications and users.

About Novita AI
Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing an affordable and reliable GPU cloud for building and scaling.

DeepSeek V3 0324 Available on Novita AI

Novita AI — Thu, 27 Mar 2025 04:00:00 +0000

DeepSeek V3 0324, the latest iteration of the powerful DeepSeek AI series, is now seamlessly accessible via Novita AI.

This advanced version brings substantial enhancements in logical reasoning, mathematical problem-solving, function-calling accuracy, and specialized language proficiency, especially for Chinese.

This technical guide comprehensively covers DeepSeek V3 0324’s upgrades, detailed benchmarks, comparative analyses, and practical integration methods.

For quick hands-on experimentation, developers can use Novita AI’s interactive LLM Playground and follow the clear Quick Start Guide.

What’s New: DeepSeek V3 0324 vs. DeepSeek V3

Enhanced Reasoning Capabilities

DeepSeek V3 0324 significantly outperforms its predecessor in logical and mathematical reasoning benchmarks, although available at the same price:

MMLU-Pro: Improved from 75.9% to 81.2% (+5.3%).
GPQA Diamond: Increased from 59.1% to 68.4% (+9.3%).
MATH-500: Enhanced accuracy from 90.2% to 94.0%, excelling in mathematical reasoning.
AIME 2024: Improved considerably from 39.6% to 59.4% (+19.8%).
LiveCodeBench: Improved coding performance from 39.2% to 49.2% (+10.0%).

These improvements ensure greater accuracy and reliability for complex reasoning tasks.

Try DeepSeek V3 0324 now with $0.5 free credits

Improved Front-End Web Development

DeepSeek V3 0324 generates cleaner, executable, and professionally structured front-end code.

This improvement allows developers to prototype interactive and visually appealing web solutions faster, reducing debugging and accelerating project timelines.

Refined Chinese Writing and Interaction

DeepSeek V3 0324 aligns closely with DeepSeek-R1’s sophisticated writing style.

The model produces superior-quality, contextually relevant medium-to-long-form Chinese content, ideal for chatbots, customer support, and content generation tools.

Accurate Function Calling

Function-calling accuracy has significantly improved, resolving reliability issues observed in earlier versions.

Developers can now confidently integrate structured outputs into API-driven applications and complex workflows.

In-Depth Benchmarking Analysis

Below is a detailed benchmarking analysis comparing DeepSeek V3 0324 to other prominent models:

Source from: DeepSeek-V3–0324 Release

Key Takeaways:

Mathematical Reasoning (MATH-500):

DeepSeek V3 0324 achieves the highest score (94.0%) in the MATH-500 benchmark, outperforming GPT-4.5 (90.7%), Qwen-Max (82.6%), and Claude-Sonnet-3.7 (82.2%), indicating superior math-solving capabilities.
General Knowledge (MMLU-Pro):

DeepSeek V3 0324 (81.2%) exhibits significant improvement compared to DeepSeek V3 (75.9%), closely approaching GPT-4.5 (86.1%) and surpassing Qwen-Max (76.1%) and Claude-Sonnet-3.7 (80.7%).
Coding Performance (LiveCodeBench):

DeepSeek V3 0324 demonstrates notable improvement (49.2%) over its predecessor (39.2%) in the LiveCodeBench, closely competitive with GPT-4.5 (44.4%) and surpassing Claude-Sonnet-3.7 (42.2%) and Qwen-Max (38.7%).
Complex Problem-Solving (AIME 2024):

With a remarkable score of 59.4%, DeepSeek V3 0324 significantly exceeds GPT-4.5 (36.7%), Claude-Sonnet-3.7 (23.3%), and Qwen-Max (26.7%), showcasing its strength in solving advanced problems.

DeepSeek V3 0324 vs. Claude 3.7 vs. GPT-4.5

To better understand how DeepSeek V3 0324 compares in the broader AI ecosystem, let’s take a closer look at its performance and features alongside two other leading models — Claude 3.7 and GPT-4.5:

Key Takeaways:

Cost Efficiency: DeepSeek V3 0324 offers significant cost savings compared to GPT-4.5 and Claude-Sonnet-3.7, making it highly accessible for startups and large-scale projects.
Specialization: DeepSeek V3 0324 excels in scenarios requiring advanced mathematical reasoning, coding, and specialized Chinese language applications.
Integration Simplicity: Novita AI simplifies integration significantly by maintaining compatibility with the OpenAI API standard.

Getting Started with DeepSeek V3 0324 on Novita AI

To quickly leverage DeepSeek V3 0324:

Step 1: Go to Novita AI and log in using your Google, GitHub account, or email address.

Step 2: Try the DeepSeek-V3–0324 Demo.

Step 3: Monitor the LLM Metrics Console of the model on Novita AI.

Step 4: Get your API Key:

Navigate to “Key Management” in the settings.
A default key is created upon your first login.
To generate additional keys, click on “+ Add New Key.”

Step 5: Set up your development environment and configure options such as content, role, name, and prompt

Accessing DeepSeek V3 0324 API via Novita AI

Novita AI simplifies DeepSeek V3 0324 integration via an intuitive, OpenAI-compatible API.

For python users

from openai import OpenAI

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    api_key="<YOUR Novita AI API Key>",
)
model = "deepseek/deepseek-v3-0324"
stream = True # or False
max_tokens = 2048
system_content = """Be a helpful assistant"""
temperature = 1
top_p = 1
min_p = 0
top_k = 50
presence_penalty = 0
frequency_penalty = 0
repetition_penalty = 1
response_format = { "type": "text" }
chat_completion_res = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "system",
            "content": system_content,
        },
        {
            "role": "user",
            "content": "Hi there!",
        }
    ],
    stream=stream,
    max_tokens=max_tokens,
    temperature=temperature,
    top_p=top_p,
    presence_penalty=presence_penalty,
    frequency_penalty=frequency_penalty,
    response_format=response_format,
    extra_body={
      "top_k": top_k,
      "repetition_penalty": repetition_penalty,
      "min_p": min_p
    }
  )
if stream:
    for chunk in chat_completion_res:
        print(chunk.choices[0].delta.content or "", end="")
else:
    print(chat_completion_res.choices[0].message.content)

For JavaScript users:

import OpenAI from "openai";
const openai = new OpenAI({
  baseURL: "https://api.novita.ai/v3/openai",
  apiKey: "<YOUR Novita AI API Key>",
});
const stream = true; // or false
async function run() {
  const completion = await openai.chat.completions.create({
    messages: [
      {
        role: "system",
        content: "Be a helpful assistant",
      },
      {
        role: "user",
        content: "Hi there!",
      },
    ],
    model: "deepseek/deepseek-v3-0324",
    stream,
    response_format: { type: "text" },
    max_tokens: 2048,
    temperature: 1,
    top_p: 1,
    min_p: 0,
    top_k: 50,
    presence_penalty: 0,
    frequency_penalty: 0,
    repetition_penalty: 1
  });
  if (stream) {
    for await (const chunk of completion) {
      if (chunk.choices[0].finish_reason) {
        console.log(chunk.choices[0].finish_reason);
      } else {
        console.log(chunk.choices[0].delta.content);
      }
    }
  } else {
    console.log(JSON.stringify(completion));
  }
}
run();

For Curl users:

curl "https://api.novita.ai/v3/openai/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR Novita AI API Key>" \
  -d @- << 'EOF'
{
    "model": "deepseek/deepseek-v3-0324",
    "messages": [
        {
            "role": "system",
            "content": "Be a helpful assistant"
        },
        {
            "role": "user",
            "content": "Hi there!"
        }
    ],
    "response_format": { "type": "text" },
    "max_tokens": 2048,
    "temperature": 1,
    "top_p": 1,
    "min_p": 0,
    "top_k": 50,
    "presence_penalty": 0,
    "frequency_penalty": 0,
    "repetition_penalty": 1
}
EOF

For detailed instructions, refer to Novita AI’s comprehensive Quick Start Guide.

Conclusion

DeepSeek V3 0324, now available on Novita AI, significantly upgrades reasoning, coding, and specialized language processing capabilities.

Its competitive pricing, powerful features, intuitive API integration, and scalable infrastructure offer developers unmatched efficiency and cost-effectiveness.

Start leveraging DeepSeek V3 0324 on Novita AI today and elevate your AI projects effectively and affordably.

About Novita AI

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing an affordable and reliable GPU cloud for building and scaling.

High-Frequency CPU + RTX 4090: Optimizing AI Image Generation by 150%+

Novita AI — Tue, 18 Mar 2025 04:00:00 +0000

In AI image generation workloads, the relationship between CPU and GPU plays a crucial role in overall system performance. Our comprehensive testing reveals that CPU frequency is a more significant factor than core count when paired with an RTX 4090 GPU. This finding challenges conventional wisdom that favors multi-core enterprise processors for AI tasks and demonstrates how high-frequency consumer CPUs can dramatically improve generation speed while reducing costs.

This article details our benchmarks showing how high-frequency CPUs dramatically reduce generation times, explores ComfyUI optimization techniques that further enhance performance, and provides a step-by-step guide to accessing these optimized configurations through Novita AI’s platform.

CPU Frequency Impact on Image Generation Performance

For text-to-image generation tasks, the CPU prepares data for the GPU to process. Higher CPU frequencies enable faster preparation and transfer of these instruction sets, allowing the GPU to operate at maximum efficiency rather than waiting for data. Our testing demonstrates that consumer CPUs with higher frequencies can increase GPU utilization by over 150% compared to lower-frequency enterprise processors.

Testing Frameworks

ComfyUI running Stable Diffusion 1.8.0, tested on NVIDIA RTX 4090 (24GB VRAM) across multiple CPU configurations

CPU Configuration Performance: StableDiffusion Generation Speed

Key Finding: The consumer-grade high-frequency CPU completes the same task in less than half the time required by the enterprise CPU, demonstrating over 150% performance improvement.

ComfyUI Optimization Modes Impact on Image Generation Speed

After establishing the superior performance of high-frequency CPUs, we explored additional optimization methods to further enhance generation speed.

Testing Framework:

ComfyUI running Flux1.dev fp8 model on a high-frequency CPU + RTX 4090 system

Average Image Generation Time (seconds)

Understanding ComfyUI Functions

Fast Mode: Accelerates resource loading through preloading and caching mechanisms. Reduces unnecessary checks and optimizes resource loading for faster startup and improved efficiency when generating multiple images.
HighVRAM Mode: Keeps more models and data in GPU VRAM, reducing data transfer overhead. Optimizes memory management by avoiding frequent allocation/release operations, improving generation efficiency. Can process multiple batches when sufficient VRAM is available.
Accelerator Node: A custom plugin for ComfyUI that further enhances processing pipeline efficiency.

Key Finding:

When using RTX 4090 + high-frequency CPU configurations, optimizing your workflow with specialized modes delivers significant performance gains while maintaining image quality:

Basic Model Workflow: Using flux.dev-fp8 with fast+highVRAM modes reduces generation time from 10.05s to 6.59s
Single LoRA Workflow: Implementing flux.dev-fp8 with fast+accelerator nodes cuts generation time from 12.63s to 9.68s
Four LoRA Switching Workflow: Applying flux.dev-fp8 with fast mode decreases generation time from 14.07s to 11.10s

Across all scenarios, these optimizations save approximately 3 seconds per image while maintaining full quality and reliability.

How to Access the Image Generation Friendly GPU on Novita AI

For those looking to implement these findings, Novita AI offers pre-configured instances with the optimal hardware combination:

Step 1: Go to Novita AI and log in using your Google, GitHub account, or email address.

Step 2: Navigate to GPU instances page.

Step 3: Select the appropriate GPU template:

StableDiffusion:v1.8.0 for Stable Diffusion model optimization
Comfyui:flux1-fp8 for Flux model optimization

Step 4: Select 24 vCPUs per GPU in the bottom right corner, then choose the ‘RTX 4090 (High-Freq CPU)’ configuration.

Technical Specifications for RTX 4090 (High-Freq CPU)

GPU: 1× NVIDIA RTX 4090 with 24GB VRAM
CPU: High-frequency CPU (13th Gen Intel Core i7–13790F)
System Memory: 58GB RAM
Processing Cores: 24 vCPUs
Cost-Efficiency: $0.69/hour (on-demand pricing)

Conclusion

Our research conclusively demonstrates that high-frequency consumer CPUs significantly outperform lower-frequency enterprise CPUs when paired with an RTX 4090 for AI image generation tasks. This combination delivers up to 150% faster performance while potentially reducing hardware costs.

By implementing the additional ComfyUI optimization techniques outlined in this article, users can further enhance their generation speed and throughput.

Ready to transform your AI image generation workflow? Consider upgrading to an RTX 4090 with a high-frequency CPU to immediately improve your generation speeds and output quality.

About Novita AI

Novita AI is an AI cloud platform that offers developers an easy way to deploy AI models using our simple API, while also providing an affordable and reliable GPU cloud for building and scaling.

Originally published at Novita AI