Retrieval and RAG

Retrieval-augmented generation answers a question from your own documents. Your system searches an index, picks the passages most likely to hold the answer, and sends them to the model along with the question. The model reads them and writes the answer. The search is not the expensive part. The model reads every retrieved passage as input tokens on every question, so cost follows how much you retrieve.

The cost of one call is the input tokens multiplied by the input price, plus the output tokens multiplied by the output price, divided by one million. Clarke costs $1.50 per million input tokens and $7.00 per million output tokens. Current rates are on the pricing page, and the tokenomics page explains where the cost behind a token comes from.

Three worked examples to replace with your own numbers

The first is one question answered from eight retrieved passages.

The retrieved passages are 82 percent of the input, and input is 75 percent of the cost. A product that answers 200,000 questions a month pays about $1,960 on Clarke. The number of passages is the lever you control. Sending more of them raises the cost in a straight line, and it does not always improve the answer.

The second example is the work before the model reads anything. A follow-up question like "and what about the enterprise plan?" has to be rewritten into a search query that stands on its own. A first search that returns 20 candidates is then narrowed to the eight that matter.

Rewriting the question and narrowing the candidates costs $0.00222 on Tycho. Passing the 12 extra passages to Clarke would cost $0.0090 in input alone, on every question.

The third is a one-off. Tagging every chunk with a topic and a short summary at ingestion makes retrieval easier to filter.

Prompt caching helps with the part of the prompt that repeats. The instructions and answer format are identical on every call, and Radium's model pages put the saving on repeated prompts at up to 90 percent. The retrieved passages change on every question, so they do not cache. Anthropic's caching documentation describes the same principle, and the arithmetic applies on any provider.

Measure the passages your own retriever sends before you plan around any of these numbers.

Clarke is the default for RAG. Radium built it for RAG, copilots, tool calling and structured outputs, and positions it as comparable to Sonnet 5.5 and GPT-6 Sol. It follows instructions about when to answer and when to say the passages do not contain the answer, returns JSON in a schema you define, and has a context window of 1,000,000 tokens. Radium publishes matched-tier benchmark results against Claude on its own task suite.

A large context window does not replace retrieval. Research on long-context models, such as Lost in the Middle, found that models use information less reliably when it sits in the middle of a long prompt, and every extra token is billed on every question. Retrieve first, then send the passages that matter.

Hal takes the questions that need reasoning across several sources, such as comparing two contracts, reconciling figures from different reports or answering a question that depends on a chain of documents. It costs $2.25 per million input tokens and $13.75 per million output tokens, so the question above costs about $0.0158 on Hal against $0.0098 on Clarke. Route to it with a classifier or a button for a deeper answer, and keep Clarke as the default.

Tycho does the work around the answer. Use it to rewrite follow-up questions, rerank candidates, tag chunks at ingestion, and check whether an answer is supported by its sources. It reads text only, so a chart or scanned page needs Clarke or Hal. The pre-processing in the second example costs $0.00222 per question on Tycho.

Start on Clarke and promote the hard questions to Hal.

The model is one half of a RAG system. Radium serves the generation half, so confirm what carries over and what stays with your current stack.

Four things do not carry over unchanged. Radium does not offer embeddings, so the index and the search stay where they are. Radium does not support the Responses API or hosted file search, so a RAG system built on those keeps its retrieval in your own stack first. Model names from other providers return a not-found error until you change them to a Radium model ID. Schemas with recursive references or union types may not be fully supported, so keep the answer schema flat and test it.

Compare all three models on the model overview. The switching guide walks through a four-step evaluation against Anthropic and OpenAI, and the migration guide lists what changes and what stays the same. Support bots and internal copilots are the most common front ends for retrieval, and each has its own guide: Customer Support Chat and Internal Copilots. The RAG page covers the models for RAG pipelines. More workloads are in the use cases index.

Replace only the generation call, and leave retrieval alone.

Radium serves OpenAI-compatible and Anthropic-compatible APIs, so an existing RAG system keeps its SDK, its prompts and its retrieval code. Three values change in the generation call, which are the base URL, the API key and the model name.

Pick the SDK you already use, then your language. The Anthropic SDK adds the /v1 path itself, so its base URL does not include it. Replace YOUR_API_KEY with your key.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.radium.cloud/v1",
)

response = client.chat.completions.create(
    model="clarke-1.0",
    messages=[
        {"role": "user", "content": "Hello, Radium!"}
    ],
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.radium.cloud/v1",
});

const response: OpenAI.Chat.ChatCompletion =
  await client.chat.completions.create({
    model: "clarke-1.0",
    messages: [{ role: "user", content: "Hello, Radium!" }],
  });

console.log(response.choices[0].message.content);

curl https://api.radium.cloud/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "clarke-1.0",
    "messages": [{"role": "user", "content": "Hello, Radium!"}]
  }'

package main

import (
	"context"
	"fmt"

	openai "github.com/sashabaranov/go-openai"
)

func main() {
	cfg := openai.DefaultConfig("YOUR_API_KEY")
	cfg.BaseURL = "https://api.radium.cloud/v1"
	client := openai.NewClientWithConfig(cfg)

	resp, err := client.CreateChatCompletion(
		context.Background(),
		openai.ChatCompletionRequest{
			Model: "clarke-1.0",
			Messages: []openai.ChatCompletionMessage{
				{Role: openai.ChatMessageRoleUser, Content: "Hello, Radium!"},
			},
		},
	)
	if err != nil {
		panic(err)
	}

	fmt.Println(resp.Choices[0].Message.Content)
}

import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.chat.completions.*;

OpenAIClient client = OpenAIOkHttpClient.builder()
    .apiKey("YOUR_API_KEY")
    .baseUrl("https://api.radium.cloud/v1")
    .build();

ChatCompletion completion = client.chat().completions().create(
    ChatCompletionCreateParams.builder()
        .model("clarke-1.0")
        .addUserMessage("Hello, Radium!")
        .build());

System.out.println(
    completion.choices().get(0).message().content().orElseThrow());

using OpenAI;
using OpenAI.Chat;

var client = new OpenAIClient(
    new ApiKeyCredential("YOUR_API_KEY"),
    new OpenAIClientOptions
    {
        Endpoint = new Uri("https://api.radium.cloud/v1"),
    });

var chatClient = client.GetChatClient("clarke-1.0");
var response = await chatClient.CompleteChatAsync(
    new UserChatMessage("Hello, Radium!"));

Console.WriteLine(response.Value.Content[0].Text);

import anthropic

client = anthropic.Anthropic(
    api_key="YOUR_API_KEY",
    base_url="https://api.radium.cloud",
)

message = client.messages.create(
    model="clarke-1.0",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Hello, Radium!"}
    ],
)

print(message.content[0].text)

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.radium.cloud",
});

const message: Anthropic.Message = await client.messages.create({
  model: "clarke-1.0",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hello, Radium!" }],
});

console.log(message.content[0].text);

curl https://api.radium.cloud/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "clarke-1.0",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello, Radium!"}]
  }'

A grounded answer with sources

This is the generation step of most RAG systems. Your retriever returns passages with ids. Clarke answers from them, returns the ids it used in a JSON schema, and sets answerable to false when the passages do not contain the answer. The last lines drop any source id that was not in the retrieved set, so a made-up citation never reaches a user.

import json
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.radium.cloud/v1",
)

ANSWER_SCHEMA = {
    "type": "object",
    "properties": {
        "answerable": {"type": "boolean"},
        "answer": {"type": "string"},
        "sources": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["answerable", "answer", "sources"],
    "additionalProperties": False,
}

SYSTEM = (
    "Answer the question using only the passages provided. "
    "Cite the id of every passage you use. "
    "If the passages do not contain the answer, set answerable to false."
)


def retrieve(question, k=8):
    return []  # call your own search here: [{"id": "doc-12", "text": "..."}]


def answer(question):
    passages = retrieve(question)
    context = "\n\n".join(f"[{p['id']}]\n{p['text']}" for p in passages)
    response = client.chat.completions.create(
        model="clarke-1.0",
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": f"Passages:\n{context}\n\nQuestion: {question}"},
        ],
        response_format={
            "type": "json_schema",
            "json_schema": {"name": "answer", "strict": True, "schema": ANSWER_SCHEMA},
        },
        max_tokens=800,
    )
    result = json.loads(response.choices[0].message.content)
    known = {p["id"] for p in passages}
    result["sources"] = [s for s in result["sources"] if s in known]
    return result


print(answer("What is our refund window for annual plans?"))

Steps

1. Create an account at deploy.radium.cloud and generate a key at deploy.radium.cloud/keys. New accounts get $25 in credit with no card required.
2. Keep the key on your server, next to your retriever.
3. Leave embeddings and the vector database where they are. Change only the call that writes the answer.
4. Send a share of questions to Clarke behind a feature flag while the rest stay on your current provider.
5. Compare the two groups on the same retrieved passages for a week, then widen the flag.

Switching back means turning the flag off. Guides for the OpenAI and Anthropic SDKs, LangChain, LangGraph, LlamaIndex, the Vercel AI SDK and PydanticAI are in the Radium docs.

Test the generation step on your own questions. Collect 100 questions your users have actually asked, and for each one record the passages that hold the answer. Run them through your current model and through Clarke with the same retrieved passages and the same prompt, so the only change is the model.

1. Check the answers against the passages. For each answer, record whether it is correct and whether every claim appears in a cited passage. A right answer with an unsupported claim is a failure.
2. Check the citations. Count how often the cited ids contain the answer. A correct answer that points to the wrong page sends a user to the wrong place.
3. Ask questions the corpus cannot answer. Include 20 questions with no answer in your documents. The model should set answerable to false instead of guessing.
4. Sweep the passage count. Run the same questions with 4, 8 and 16 passages. Pick the smallest number that holds accuracy, because that number is also your cost per question.
5. Measure what users feel. Record time to first token and total time per answer at the 50th and 95th percentiles with streaming on, and read the cost per question from the usage export.

Decide what counts as a pass before you look at the results.

Radium fits when you run your own RAG system through a standard SDK and the answers are a meaningful share of your AI spend. There are cases where switching is the wrong move.

• You need embeddings from the same provider. Radium does not offer an embeddings endpoint. You can keep your current embedding provider and still move generation, but if you want one vendor for both, Radium is not that vendor today.
• Your RAG system is built on hosted file search or the Responses API. Radium supports neither, so retrieval and conversation state have to live in your own stack before anything moves.
• Your answers are wrong because retrieval is wrong. If the right passage is not in the set the model reads, a different model will not fix it. Fix chunking, search and reranking first.
• Your corpus is small and stable. A few pages of policy can sit in the prompt and benefit from caching, with no retrieval at all.
• Your usage is small. If the bill is a few hundred dollars a month, the evaluation will cost more than the switch saves.
• Your security review requires a certification Radium has not published. Radium lists GDPR compliance and handles SOC 2 Type II, ISO 27001 and HIPAA questions through its security package, so request it before regulated documents flow through the system.

If none of these apply, run the 100 questions.

Can I run my RAG system on Radium without rebuilding it?

Yes, if your system calls a model API through the OpenAI or Anthropic SDK. Radium serves OpenAI-compatible and Anthropic-compatible APIs, so you change the base URL, the API key and the model name in the generation call. Your retriever, your prompts and your answer format stay the same. A system built on the Responses API or hosted file search needs to keep its own retrieval first, because Radium does not support them.

Does Radium offer embeddings for RAG?

No. Radium does not offer an embeddings endpoint today. Keep your current embedding provider and vector database for search, and use Radium for the step that reads the retrieved passages and writes the answer.

Which Radium model should I use for RAG?

Use Clarke (clarke-1.0) for answering from retrieved passages. Use Hal (hal-1.0) for questions that reason across several sources, and for charts and scanned pages alongside Clarke. Use Tycho (tycho-1.0) for query rewriting, reranking, chunk tagging and answer checks, since it reads text only and costs the least.

How much does RAG cost to run on Radium?

Clarke costs $1.50 per million input tokens and $7.00 per million output tokens. A question answered from 8 passages of 500 tokens, at about 4,900 input tokens and 350 output tokens, costs about $0.0098. A product answering 200,000 questions a month would pay about $1,960 before prompt caching. The number of passages sent is the largest lever.

Should I use the long context window instead of retrieval?

Usually not. Clarke accepts 1,000,000 tokens, but every token in the prompt is billed on every question, and research on long-context models has found they use information less reliably when it sits in the middle of a long prompt. Retrieve the passages that matter and send those. A small, stable set of documents can sit in the prompt and use prompt caching.

Does Radium train on the documents I send for RAG?

No. Radium does not train on customer data, and prompts and completions are used only to service the request. Radium lists GDPR compliance, and SOC 2 Type II, ISO 27001 and HIPAA questions go through its security package.