completion( )
Generates completion from a language model based on conversation history.
function completion(params: CompletionParams): { stats: Promise<{ cacheTokens: number; timeToFirstToken: number; tokensPerSecond: number } | —>; text: Promise<string>; tokenStream: AsyncGenerator<string>; toolCalls: Promise<ToolCallWithCall[]>; toolCallStream: AsyncGenerator<{ call: { arguments: Record<string, unknown>; id: string; name: string; raw?: string }; type: "toolCall" } | { error: { code: "PARSE_ERROR" | "VALIDATION_ERROR" | "UNKNOWN_TOOL"; message: string; raw?: string }; type: "toolCallError" }> }Parameters
| Name | Type | Required? | Description |
|---|---|---|---|
params | CompletionParams | ✓ | The completion parameters |
Returns
{ stats: Promise<{ cacheTokens: number; timeToFirstToken: number; tokensPerSecond: number } | —>; text: Promise<string>; tokenStream: AsyncGenerator<string>; toolCalls: Promise<ToolCallWithCall[]>; toolCallStream: AsyncGenerator<{ call: { arguments: Record<string, unknown>; id: string; name: string; raw?: string }; type: "toolCall" } | { error: { code: "PARSE_ERROR" | "VALIDATION_ERROR" | "UNKNOWN_TOOL"; message: string; raw?: string }; type: "toolCallError" }> }| Field | Type | Description |
|---|---|---|
stats | Promise | Resolves with inference performance metrics (cache tokens used, time to first token, tokens per second), or undefined if profiling is not enabled |
text | Promise | Resolves with the full completion text once generation is finished |
tokenStream | AsyncGenerator | Yields individual tokens as they are generated in real-time |
toolCalls | Promise | Resolves with all parsed tool calls once generation is complete. Each entry includes an invoke() method to execute the tool handler |
toolCallStream | AsyncGenerator | Yields tool call events and tool call errors as they are parsed during generation |
Example
import { z } from "zod";
const result = completion({
modelId: "llama-2",
history: [
{ role: "user", content: "What's the weather in Tokyo?" }
],
stream: true,
tools: [{
name: "get_weather",
description: "Get current weather",
parameters: z.object({
city: z.string().describe("City name"),
}),
handler: async (args) => {
return { temperature: 22, condition: "sunny" };
}
}]
});
for await (const token of result.tokenStream) {
process.stdout.write(token);
}
for (const toolCall of await result.toolCalls) {
if (toolCall.invoke) {
const toolResult = await toolCall.invoke();
console.log(toolResult);
}
}close( )
Closes the SDK client connection and releases all associated resources (RPC worker, IPC server). Safe to call multiple times — subsequent calls are a no-op if already closed.
defineHandler( )
Helper function to define a handler with full type inference. This is an identity function that provides type checking.