nimgent/providers/provider

This page is generated from the module’s exported API and ## documentation comments.

Internal representation of messages, tools and provider traffic.

Provider adapters translate between this representation and their own wire format. The representation is deliberately close to a superset of what the supported APIs need. Low-level callers can reach provider-specific wire fields through ProviderRequest.options; the high-level generation API resolves GenerationOptions and ProviderOptions.

Types

Role

Message author role sent to the model.

Role = enum
  roleUser = "user", roleAssistant = "assistant"

View source

ContentKind

Kind of content stored in a message or response.

ContentKind = enum
  ckText, ckToolUse, ckToolResult, ckThinking, ckImage, ckFile, ckSource

View source

ImageContent

Base64-encoded image content and optional source path.

ImageContent = object
  mimeType*: string
  data*: string              ## base64, no data: prefix
  path*: string              ## optional caller-owned source/reference path

View source

FileContent

Base64-encoded file content and optional file metadata.

FileContent = object
  mimeType*: string
  data*: string              ## base64, no data: prefix
  path*: string
  filename*: string

View source

SourceContent

Citation or source metadata returned by a provider.

SourceContent = object
  url*: string
  title*: string
  id*: string
  citedText*: string
  raw*: JsonNode             ## provider citation object; required for Anthropic replay

View source

ContentBlock

One provider-neutral text, tool, media, thinking, or source block.

ContentBlock = object
  googlePart*: JsonNode ## Original native Gemini part, retained for signed replay.
                        ## Non-empty when the provider already ran this tool use/result
                        ## (`web_search`, …). `toolCalls` skips these; generateText must not
                        ## execute them. Value is the logical tool name, used to replay results.
  hosted*: string
  case kind*: ContentKind
  of ckText:
    text*: string
  of ckThinking:
    thinking*: string
    signature*: string
  of ckToolUse:
    id*: string
    name*: string
    input*: JsonNode
    parseError*: string      ## set when the provider got invalid tool JSON; do not execute
    thoughtSignature*: string ## Google tool-call signature, replayed unchanged
  of ckToolResult:
    toolUseId*: string
    output*: string
    isError*: bool ## Structured value returned by a local tool. `output` remains the
                   ## provider-facing rendering of this value.
    value*: JsonNode         ## Structured local tool failure, when `isError` is true.
    errorCode*: string
    errorMessage*: string
    errorDetails*: JsonNode
    errorRetryable*: bool
    images*: seq[ImageContent]
  of ckImage:
    mimeType*: string
    data*: string
    path*: string
  of ckFile:
    file*: FileContent
  of ckSource:
    source*: SourceContent

View source

Message

A role and ordered content blocks sent to or returned by a model.

Message = object
  role*: Role
  content*: seq[ContentBlock]

View source

ToolDefinition

Provider-facing description of a callable or hosted tool.

ToolDefinition = object
  name*: string
  description*: string
  inputSchema*: JsonNode     ## Non-empty: provider-hosted tool (`web_search`, …). No execute.
  hosted*: string
  hostedOptions*: JsonNode

View source

ToolChoiceKind

Strategy for selecting a tool during a model call.

ToolChoiceKind = enum
  tckAuto, tckRequired, tckNone, tckSpecific

View source

ToolChoice

Tool-selection strategy, optionally naming one required tool.

ToolChoice = object
  case kind*: ToolChoiceKind
  of tckSpecific:
    name*: string
  else:
    nil

View source

FinishReason

Why a model step stopped.

FinishReason = enum
  frUnknown, frEndTurn, frToolUse, frMaxTokens, frStop, frStepLimit

View source

Usage

Token usage reported by a provider.

Usage = object
  inputTokens*: int
  outputTokens*: int
  cacheReadTokens*: int
  cacheWriteTokens*: int ## True when the provider reports cache statistics at all; without this we
                         ## cannot distinguish "zero cached" from "not reported".
  cacheReported*: bool

View source

ProviderRequest

Provider-neutral request passed to a provider adapter.

ProviderRequest = object
  model*: string
  conversationId*: string    ## Optional execution identity and metadata for local tool context.
  turnId*: string
  metadata*: JsonNode
  system*: seq[string]
  messages*: seq[Message]
  tools*: seq[ToolDefinition] ## Provider-neutral tool selection. The default is automatic selection.
  toolChoice*: ToolChoice
  maxTokens*: int ## Raw wire escape hatch for low-level provider integrations. High-level
                  ## calls use GenerationOptions and ProviderOptions instead.
  options*: JsonNode ## When >= 0, emit seWake when this fd becomes readable during streaming.
                     ## Default -1 means no side-channel wake (stdin is 0 when used).
  wakeFd*: cint = -1

View source

StepResult

Result from one model call in a multi-step run.

StepResult = object
  model*: string
  content*: seq[ContentBlock]
  usage*: Usage
  finishReason*: FinishReason ## Results produced locally for this step, in tool-call order.
  toolResults*: seq[ContentBlock]

View source

ProviderResponse

Provider response, including content, usage, and step history. Provider-reported model, which may differ from the requested alias after routing or fallback. It is more trustworthy than asking the model.

ProviderResponse = object
  model*: string
  content*: seq[ContentBlock]
  usage*: Usage              ## Usage across every model call in `steps`. For a single call this equals usage.
  totalUsage*: Usage
  finishReason*: FinishReason ## Request ID returned by the provider, when available.
  requestId*: string         ## Every model call made by the high-level tool loop.
  steps*: seq[StepResult]

View source

ProviderError

Error raised for provider, transport, or request failures. Raised for transport and API errors. overflow marks the specific case of exceeding the context window, which the agent can recover from.

ProviderError = object of CatchableError
  overflow*: bool
  retryable*: bool           ## 429 / 5xx / transport; generateText may retry
  aborted*: bool             ## caller abort() returned true
  status*: int               ## HTTP status, or 0 when there was no response
  retryAfterMs*: int         ## from Retry-After; 0 if the server did not send one
  requestId*: string         ## Provider request ID, when the server returned one.

View source

ObjectIssue

Best-effort JSON path and diagnostic message for one validation issue.

ObjectIssue = object
  path*: string
  message*: string

View source

ObjectError

Error raised when structured output does not match its schema. generateObject could not produce a value that matches the schema.

ObjectError = object of ProviderError
  issueDetails*: seq[ObjectIssue]
  raw*: string

View source

CancelledError

Error raised when a caller cancels a request.

CancelledError = object of ProviderError

View source

AbortCheck

Return true to cancel. Checked before each attempt and tool call.

AbortCheck = proc (): bool {.closure.}

View source

ToolContext

Context passed to a local tool invocation. Per-invocation state supplied to context-aware tool handlers.

ToolContext = object
  callId*: string
  conversationId*: string
  turnId*: string
  abort*: AbortCheck
  metadata*: JsonNode

View source

ToolError

Machine-readable failure produced by a local tool.

ToolError = object
  code*: string
  message*: string
  details*: JsonNode
  retryable*: bool

View source

ToolResult

Structured result produced by a local tool. output is the text sent back to the model; value is retained for application inspection.

ToolResult = object
  value*: JsonNode
  output*: string
  isError*: bool
  error*: ToolError
  images*: seq[ImageContent]

View source

Tool

Callable local tool or provider-hosted tool definition.

Tool = object
  name*: string
  description*: string
  inputSchema*: JsonNode     ## When set, generateText/streamText can run the tool and continue (maxSteps).
  execute*: proc (context: ToolContext; input: JsonNode): ToolResult {.closure.}
  executeAsync*: proc (context: ToolContext; input: JsonNode): Future[ToolResult] {.
      closure.}              ## Overlap executeAsync when every runnable tool in the batch sets this.
  parallel*: bool
  hosted*: string
  hostedOptions*: JsonNode

View source

Provider

Base type implemented by a model provider adapter.

Provider = ref object of RootObj
  name*: string

View source

LanguageModel

Provider and model identifier used for text generation.

LanguageModel = object
  provider*: Provider
  id*: string

View source

EmbeddingModel

Provider and model identifier used for embeddings.

EmbeddingModel = object
  provider*: Provider
  id*: string

View source

EmbeddingUsage

Embedding token usage reported by a provider.

EmbeddingUsage = object
  tokens*: int

View source

AgentEventKind

Lifecycle event kind emitted during an agent run.

AgentEventKind = enum
  aeRunStart, aeStepStart, aeTextDelta, aeThinkingDelta, aeToolCall,
  aeToolApprovalRequired, aeToolResult, aeStepFinish, aeRunFinish, aeError

View source

ToolApprovalMode

Default decision returned by a tool approval policy.

ToolApprovalMode = enum
  tamAllow, tamAsk, tamDeny

View source

ToolApproval

Approval policy decision and optional explanation.

ToolApproval = object
  mode*: ToolApprovalMode
  reason*: string

View source

ToolApprovalDecision

Decision resolved on a pending tool approval request.

ToolApprovalDecision = enum
  tadApprove, tadDeny

View source

ToolApprovalRequest

Pending request that an application can approve or deny.

ToolApprovalRequest = ref object
  callId*: string
  toolName*: string
  input*: JsonNode
  reason*: string

View source

AgentEvent

Normalized lifecycle event emitted during an agent run.

AgentEvent = object
  runId*: string
  conversationId*: string
  turnId*: string
  step*: int
  case kind*: AgentEventKind
  of aeRunStart:
    prompt*: string
    model*: string
  of aeStepStart:
    stepModel*: string
  of aeTextDelta, aeThinkingDelta:
    text*: string
  of aeToolCall:
    call*: ContentBlock
  of aeToolApprovalRequired:
    approval*: ToolApprovalRequest
  of aeToolResult:
    toolResult*: ContentBlock
    durationMs*: int
  of aeStepFinish:
    stepResult*: StepResult
  of aeRunFinish:
    response*: ProviderResponse
  of aeError:
    error*: ref CatchableError

View source

AgentEventCallback

Callback that receives agent lifecycle events; return false to cancel.

AgentEventCallback = proc (event: AgentEvent): bool {.closure.}

View source

ToolApprovalPolicy

Callback that decides whether a tool call may run.

ToolApprovalPolicy = proc (step: int; call: ContentBlock; tool: Tool): ToolApproval {.
    closure.}

View source

EmbeddingRequest

Provider-neutral request for one or more embeddings.

EmbeddingRequest = object
  model*: string
  values*: seq[string]       ## Escape hatch for provider-specific embedding settings (dimensions, user, ...).
  options*: JsonNode

View source

EmbeddingResponse

Embedding vectors and usage returned by a provider.

EmbeddingResponse = object
  model*: string
  embeddings*: seq[seq[float]]
  usage*: EmbeddingUsage     ## Request ID returned by the provider, when available.
  requestId*: string

View source

StreamEventKind

Kind of normalized streaming event.

StreamEventKind = enum
  seTextDelta, seThinkingDelta, seToolCallDelta, seFinished, seWake ## Input or periodic cancellation check while waiting on the provider

View source

StreamEvent

Text, thinking, tool-call, completion, or wake event from a stream.

StreamEvent = object
  case kind*: StreamEventKind
  of seTextDelta, seThinkingDelta:
    text*: string
  of seToolCallDelta:
    toolCallId*: string
    toolName*: string
    toolArgs*: string        ## argument fragment; empty when only the name arrived
  of seFinished, seWake:
    nil

View source

ThinkingWire

Provider wire format used for reasoning or thinking options.

ThinkingWire = enum
  twEffort,                 ## reasoning.effort / thinking.budget_tokens
  twToggle,                 ## reasoning.enabled / reasoning.effort=medium / thinking high
  twMaxTokens                ## reasoning.max_tokens / reasoning.effort / thinking.budget_tokens

View source

StreamCallback

Return false to cancel the stream early.

StreamCallback = proc (ev: StreamEvent): bool {.closure.}

View source

RequestMapper

Return a new request. Do not mutate req in place: the tool loop and generateObject repairs reuse one request across turns.

RequestMapper = proc (req: ProviderRequest): ProviderRequest {.closure.}

View source

ResponseMapper

Inspect or update a provider response after a request completes.

ResponseMapper = proc (req: ProviderRequest; resp: var ProviderResponse) {.
    closure.}

View source

WrapProvider

Provider middleware: forwards every call to inner through optional request and response hooks (inject defaults, drop images, redact, log).

WrapProvider = ref object of Provider
  inner*: Provider
  mapRequest*: RequestMapper
  mapResponse*: ResponseMapper

View source

ProviderRouter

Provider serving model, or nil to fall back to the router’s default.

ProviderRouter = proc (model: string): Provider {.closure.}

View source

RouteProvider

Sends each model to whichever provider serves it. Use it when one logical provider - a gateway - exposes different models on different wire formats, so a model added or retired upstream needs no code change in the caller.

RouteProvider = ref object of Provider
  default*: Provider
  route*: ProviderRouter

View source

Procedures

newToolApprovalRequest

Create a pending approval request for a tool call.

proc newToolApprovalRequest(call: ContentBlock; reason: string): ToolApprovalRequest {.
    raises: [], tags: [], forbids: [].}

Returns: ToolApprovalRequest.

Arguments

Name	Type	Default
call	ContentBlock
reason	string

View source

approve

Resolve a pending approval request as approved.

proc approve(request: ToolApprovalRequest) {.raises: [ValueError, Exception],
    tags: [RootEffect], forbids: [].}

Arguments

Name	Type	Default
request	ToolApprovalRequest

View source

deny

Resolve a pending approval request as denied.

proc deny(request: ToolApprovalRequest) {.raises: [ValueError, Exception],
    tags: [RootEffect], forbids: [].}

Arguments

Name	Type	Default
request	ToolApprovalRequest

View source

waitDecision

Wait for an approval request to be resolved.

proc waitDecision(request: ToolApprovalRequest): Future[ToolApprovalDecision] {.
    raises: [], tags: [], forbids: [].}

Returns: Future[ToolApprovalDecision].

Arguments

Name	Type	Default
request	ToolApprovalRequest

View source

raiseProviderError

Raise a provider error.

proc raiseProviderError(msg: string; overflow = false; retryable = false;
                        aborted = false; status = 0; retryAfterMs = 0;
                        requestId = "") {.raises: [ProviderError], tags: [],
    forbids: [].}

Arguments

Name	Type	Default
msg	string
overflow	inferred	false
retryable	inferred	false
aborted	inferred	false
status	inferred	0
retryAfterMs	inferred	0
requestId	inferred	""

View source

addUsage

Add the counters in b to a.

proc addUsage(a: var Usage; b: Usage) {.raises: [], tags: [], forbids: [].}

Arguments

Name	Type	Default
a	var Usage
b	Usage

View source

model

Create a language model reference for a provider and model ID.

proc model(provider: Provider; id: string): LanguageModel {.
    raises: [ProviderError], tags: [], forbids: [].}

Returns: LanguageModel.

Arguments

Name	Type	Default
provider	Provider
id	string

View source

embeddingModel

Create an embedding model reference for a provider and model ID.

proc embeddingModel(provider: Provider; id: string): EmbeddingModel {.
    raises: [ProviderError], tags: [], forbids: [].}

Returns: EmbeddingModel.

Arguments

Name	Type	Default
provider	Provider
id	string

View source

embed

Execute an embedding request synchronously.

proc embed(p: Provider; request: EmbeddingRequest): EmbeddingResponse {.
    raises: [ValueError, Exception, OSError], tags: [TimeEffect, RootEffect],
    forbids: [].}

Returns: EmbeddingResponse.

Arguments

Name	Type	Default
p	Provider
request	EmbeddingRequest

View source

contextTokens

Tokens occupying the context window on the last request. OpenAI/OpenRouter prompt_tokens already includes cached tokens; Anthropic splits them (input + cache_read + cache_write).

proc contextTokens(u: Usage): int {.raises: [], tags: [], forbids: [].}

Returns: int.

Arguments

Name	Type	Default
u	Usage

View source

formatUsageLabels

Plain usage fragments shared by console, TUI, and status bar.

proc formatUsageLabels(usage: Usage): seq[string] {.raises: [], tags: [],
    forbids: [].}

Returns: seq[string].

Arguments

Name	Type	Default
usage	Usage

View source

generate

Execute a generation request synchronously.

proc generate(p: Provider; request: ProviderRequest): ProviderResponse {.
    raises: [ValueError, Exception, OSError], tags: [TimeEffect, RootEffect],
    forbids: [].}

Returns: ProviderResponse.

Arguments

Name	Type	Default
p	Provider
request	ProviderRequest

View source

text (overload 1)

Create a text content block.

proc text(s: string): ContentBlock {.raises: [], tags: [], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
s	string

View source

image (overload 1)

Create an image block from base64 data.

proc image(mimeType, data: string; path = ""): ContentBlock {.raises: [],
    tags: [], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
mimeType	string
data	string
path	inferred	""

View source

image (overload 2)

Create an image block from ImageContent.

proc image(img: ImageContent): ContentBlock {.raises: [], tags: [], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
img	ImageContent

View source

imageFromPath

Read and base64-encode an image. path is retained as source metadata.

proc imageFromPath(path, mimeType: string): ContentBlock {.raises: [IOError],
    tags: [ReadIOEffect], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
path	string
mimeType	string

View source

toImage

Extract image data from an image content block.

proc toImage(part: ContentBlock): ImageContent {.raises: [], tags: [],
    forbids: [].}

Returns: ImageContent.

Arguments

Name	Type	Default
part	ContentBlock

View source

file (overload 1)

Create a file block from base64 data.

proc file(mimeType, data: string; path = ""; filename = ""): ContentBlock {.
    raises: [], tags: [], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
mimeType	string
data	string
path	inferred	""
filename	inferred	""

View source

file (overload 2)

Create a file block from FileContent.

proc file(f: FileContent): ContentBlock {.raises: [], tags: [], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
f	FileContent

View source

fileFromPath

Read and base64-encode a file. Defaults filename to the path’s basename.

proc fileFromPath(path, mimeType: string; filename = ""): ContentBlock {.
    raises: [IOError], tags: [ReadIOEffect], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
path	string
mimeType	string
filename	inferred	""

View source

source

Create a source or citation content block.

proc source(url: string; title = ""; id = ""; citedText = "";
            raw: JsonNode = nil): ContentBlock {.raises: [], tags: [],
    forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
url	string
title	inferred	""
id	inferred	""
citedText	inferred	""
raw	JsonNode	nil

View source

fileLabel

Return the filename, path basename, or a generic file label.

proc fileLabel(f: FileContent): string {.raises: [], tags: [], forbids: [].}

Returns: string.

Arguments

Name	Type	Default
f	FileContent

View source

fileDataUri

Return file data in a data: URI.

proc fileDataUri(f: FileContent): string {.raises: [], tags: [], forbids: [].}

Returns: string.

Arguments

Name	Type	Default
f	FileContent

View source

takeFollowingSources

Consume ckSource blocks after content[i].

proc takeFollowingSources(content: openArray[ContentBlock]; i: var int): seq[
    ContentBlock] {.raises: [], tags: [], forbids: [].}

Returns: seq[ContentBlock].

Arguments

Name	Type	Default
content	openArray[ContentBlock]
i	var int

View source

applyCacheBreakpoints

Last tool, last system block, last message content - Anthropic’s 4-breakpoint budget.

proc applyCacheBreakpoints(body: JsonNode) {.raises: [KeyError], tags: [],
    forbids: [].}

Arguments

Name	Type	Default
body	JsonNode

View source

parseToolArguments

Parse tool arguments, returning an error string for invalid JSON.

proc parseToolArguments(raw: string): tuple[input: JsonNode, parseError: string] {.
    raises: [], tags: [ReadIOEffect, WriteIOEffect], forbids: [].}

Returns: tuple[input: JsonNode, parseError: string].

Arguments

Name	Type	Default
raw	string

View source

toolUse

Create a tool-call content block.

proc toolUse(id, name: string; input: JsonNode; parseError = ""; hosted = ""): ContentBlock {.
    raises: [], tags: [], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
id	string
name	string
input	JsonNode
parseError	inferred	""
hosted	inferred	""

View source

toolUseFromArgs

Parse raw tool arguments and create a tool-call content block.

proc toolUseFromArgs(id, name, raw: string): ContentBlock {.raises: [],
    tags: [ReadIOEffect, WriteIOEffect], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
id	string
name	string
raw	string

View source

toolResult

Create a tool-result content block.

proc toolResult(toolUseId, output: string; isError = false;
                images: seq[ImageContent] = @[]; hosted = "";
                value: JsonNode = nil; errorCode = ""; errorMessage = "";
                errorDetails: JsonNode = nil; errorRetryable = false): ContentBlock {.
    raises: [], tags: [], forbids: [].}

Returns: ContentBlock.

Arguments

Name	Type	Default
toolUseId	string
output	string
isError	inferred	false
images	seq[ImageContent]	@[]
hosted	inferred	""
value	JsonNode	nil
errorCode	inferred	""
errorMessage	inferred	""
errorDetails	JsonNode	nil
errorRetryable	inferred	false

View source

userMessage (overload 1)

Create a user message containing text.

proc userMessage(s: string): Message {.raises: [], tags: [], forbids: [].}

Returns: Message.

Arguments

Name	Type	Default
s	string

View source

userMessage (overload 2)

Create a user message from content blocks.

proc userMessage(parts: seq[ContentBlock]): Message {.raises: [], tags: [],
    forbids: [].}

Returns: Message.

Arguments

Name	Type	Default
parts	seq[ContentBlock]

View source

assistantMessage (overload 1)

Create an assistant message containing text.

proc assistantMessage(s: string): Message {.raises: [], tags: [], forbids: [].}

Returns: Message.

Arguments

Name	Type	Default
s	string

View source

assistantMessage (overload 2)

Create an assistant message from content blocks.

proc assistantMessage(parts: seq[ContentBlock]): Message {.raises: [], tags: [],
    forbids: [].}

Returns: Message.

Arguments

Name	Type	Default
parts	seq[ContentBlock]

View source

dropImages

Replace image blocks with a text note. Conversation storage is unchanged.

proc dropImages(messages: seq[Message]): seq[Message] {.raises: [], tags: [],
    forbids: [].}

Returns: seq[Message].

Arguments

Name	Type	Default
messages	seq[Message]

View source

toolCalls (overload 1)

Return local tool calls in a provider response.

proc toolCalls(r: ProviderResponse): seq[ContentBlock] {.raises: [], tags: [],
    forbids: [].}

Returns: seq[ContentBlock].

Arguments

Name	Type	Default
r	ProviderResponse

View source

toolCalls (overload 2)

Return local tool calls in one model step.

proc toolCalls(s: StepResult): seq[ContentBlock] {.raises: [], tags: [],
    forbids: [].}

Returns: seq[ContentBlock].

Arguments

Name	Type	Default
s	StepResult

View source

textContent

Join the text blocks with newline separators.

proc textContent(blocks: openArray[ContentBlock]): string {.raises: [],
    tags: [], forbids: [].}

Returns: string.

Arguments

Name	Type	Default
blocks	openArray[ContentBlock]

View source

text (overload 2)

Return the text content of a provider response.

proc text(r: ProviderResponse): string {.raises: [], tags: [], forbids: [].}

Returns: string.

Arguments

Name	Type	Default
r	ProviderResponse

View source

text (overload 3)

Return the text content of one model step.

proc text(s: StepResult): string {.raises: [], tags: [], forbids: [].}

Returns: string.

Arguments

Name	Type	Default
s	StepResult

View source

mergeRequestOptions

Merge a JSON options object into a provider request body.

proc mergeRequestOptions(body, options: JsonNode) {.raises: [ProviderError],
    tags: [], forbids: [].}

Arguments

Name	Type	Default
body	JsonNode
options	JsonNode

View source

generateStream

Synchronously execute a streaming provider request.

proc generateStream(p: Provider; request: ProviderRequest;
                    onEvent: StreamCallback): ProviderResponse {.
    raises: [ValueError, Exception, OSError], tags: [TimeEffect, RootEffect],
    forbids: [].}

Returns: ProviderResponse.

Arguments

Name	Type	Default
p	Provider
request	ProviderRequest
onEvent	StreamCallback

View source

isContextOverflow

True for known context-window overflow messages (not generic “token” noise).

proc isContextOverflow(detail: string): bool {.raises: [], tags: [], forbids: [].}

Returns: bool.

Arguments

Name	Type	Default
detail	string

View source

isRetryableStatus

Return whether an HTTP status normally warrants a retry.

proc isRetryableStatus(code: int): bool {.raises: [], tags: [], forbids: [].}

Returns: bool.

Arguments

Name	Type	Default
code	int

View source

parseRetryAfter

Milliseconds from a Retry-After header. Integer seconds only; HTTP-date is ignored (caller falls back to jittered backoff). Capped at 30s.

proc parseRetryAfter(value: string): int {.raises: [], tags: [], forbids: [].}

Returns: int.

Arguments

Name	Type	Default
value	string

View source

apiErrorMessage

error.message from an OpenAI-family JSON body, including the top-level array form Google’s OpenAI-compatible endpoint returns ([{"error": {"message": ...}}]). Falls back to the raw text.

proc apiErrorMessage(raw: string): string {.raises: [],
    tags: [ReadIOEffect, WriteIOEffect], forbids: [].}

Returns: string.

Arguments

Name	Type	Default
raw	string

View source

raiseCancelledError

Raise a provider error marked as caller-cancelled.

proc raiseCancelledError(msg = "aborted") {.noreturn, raises: [CancelledError],
    tags: [], forbids: [].}

Arguments

Name	Type	Default
msg	inferred	"aborted"

View source

raiseObjectError

Raise an object error with validation issues and optional raw output.

proc raiseObjectError(msg: string; issues: seq[string]; raw = "") {.
    raises: [ObjectError], tags: [], forbids: [].}

Arguments

Name	Type	Default
msg	string
issues	seq[string]
raw	inferred	""

View source

rawTool

Escape hatch for runtime-defined schemas.

proc rawTool(name, description: string; inputSchema: JsonNode; execute: proc (
    context: ToolContext; input: JsonNode): ToolResult {.closure.} = nil;
             parallel = false; hosted = ""; hostedOptions: JsonNode = nil): Tool {.
    raises: [], tags: [], forbids: [].}

Returns: Tool.

Arguments

Name	Type	Default
name	string
description	string
inputSchema	JsonNode
execute	proc (context: ToolContext; input: JsonNode): ToolResult {.closure.}	nil
parallel	inferred	false
hosted	inferred	""
hostedOptions	JsonNode	nil

View source

toolChoiceAuto

Let the model decide whether to call a tool.

proc toolChoiceAuto(): ToolChoice {.raises: [], tags: [], forbids: [].}

Returns: ToolChoice.

View source

toolChoiceRequired

Require the model to call one of the available tools.

proc toolChoiceRequired(): ToolChoice {.raises: [], tags: [], forbids: [].}

Returns: ToolChoice.

View source

toolChoiceNone

Prevent the model from calling tools.

proc toolChoiceNone(): ToolChoice {.raises: [], tags: [], forbids: [].}

Returns: ToolChoice.

View source

toolChoiceSpecific

Require the model to call the named tool.

proc toolChoiceSpecific(name: string): ToolChoice {.raises: [], tags: [],
    forbids: [].}

Returns: ToolChoice.

Arguments

Name	Type	Default
name	string

View source

validateToolChoice

Validate a tool choice against the tools available to a request.

proc validateToolChoice(choice: ToolChoice; tools: openArray[ToolDefinition]) {.
    raises: [ProviderError], tags: [], forbids: [].}

Arguments

Name	Type	Default
choice	ToolChoice
tools	openArray[ToolDefinition]

View source

rawAsyncTool

Async escape hatch for runtime-defined schemas.

proc rawAsyncTool(name, description: string; inputSchema: JsonNode; execute: proc (
    context: ToolContext; input: JsonNode): Future[ToolResult] {.closure.};
                  parallel = false): Tool {.raises: [], tags: [], forbids: [].}

Returns: Tool.

Arguments

Name	Type	Default
name	string
description	string
inputSchema	JsonNode
execute	proc (context: ToolContext; input: JsonNode): Future[ToolResult] {.closure.}
parallel	inferred	false

View source

hostedTool

Provider-executed tool (web_search, …). OpenAI Responses, Anthropic, native Gemini.

proc hostedTool(name: string; options: JsonNode = nil): Tool {.raises: [],
    tags: [], forbids: [].}

Returns: Tool.

Arguments

Name	Type	Default
name	string
options	JsonNode	nil

View source

toDefinitions

Convert executable tools into provider-facing definitions.

proc toDefinitions(tools: openArray[Tool]): seq[ToolDefinition] {.raises: [],
    tags: [], forbids: [].}

Returns: seq[ToolDefinition].

Arguments

Name	Type	Default
tools	openArray[Tool]

View source

thinkingBudgetTokens

Map a thinking level to a standard token budget.

proc thinkingBudgetTokens(level: string): int {.raises: [], tags: [],
    forbids: [].}

Returns: int.

Arguments

Name	Type	Default
level	string

View source

thinkingOptions

Provider-body knobs for a thinking/reasoning level. Empty or none is {}. Catalog snapping (which rungs exist) stays with the caller.

proc thinkingOptions(provider, level: string; wire = twEffort): JsonNode {.
    raises: [], tags: [], forbids: [].}

Returns: JsonNode.

Arguments

Name	Type	Default
provider	string
level	string
wire	inferred	twEffort

View source

wrapProvider

Wrap inner so generate, stream, and generateObject all pass through the hooks. Structured-output methods forward to the inner provider.

proc wrapProvider(inner: Provider; name = ""; mapRequest: RequestMapper = nil;
                  mapResponse: ResponseMapper = nil): WrapProvider {.
    raises: [ProviderError], tags: [], forbids: [].}

Returns: WrapProvider.

Arguments

Name	Type	Default
inner	Provider
name	inferred	""
mapRequest	RequestMapper	nil
mapResponse	ResponseMapper	nil

View source

routeProvider

Create a provider that routes requests by model ID.

proc routeProvider(default: Provider; name = ""; route: ProviderRouter = nil): RouteProvider {.
    raises: [ProviderError], tags: [], forbids: [].}

Returns: RouteProvider.

Arguments

Name	Type	Default
default	Provider
name	inferred	""
route	ProviderRouter	nil

View source

servingProvider

The provider a request for model goes to.

proc servingProvider(p: RouteProvider; model: string): Provider {.
    raises: [Exception], tags: [RootEffect], forbids: [].}

Returns: Provider.

Arguments

Name	Type	Default
p	RouteProvider
model	string

View source

Methods

embedAsync (overload 1)

Execute an embedding request asynchronously.

method embedAsync(p: Provider; request: EmbeddingRequest): Future[
    EmbeddingResponse] {.base, stackTrace: false,
                         raises: [Exception, ValueError, CatchableError],
                         tags: [RootEffect], forbids: [].}

Returns: Future[EmbeddingResponse].

Arguments

Name	Type	Default
p	Provider
request	EmbeddingRequest

View source

generateAsync (overload 1)

Execute a generation request asynchronously.

method generateAsync(p: Provider; request: ProviderRequest): Future[
    ProviderResponse] {.base, stackTrace: false,
                        raises: [Exception, ValueError, CatchableError],
                        tags: [RootEffect], forbids: [].}

Returns: Future[ProviderResponse].

Arguments

Name	Type	Default
p	Provider
request	ProviderRequest

View source

generateStreamAsync (overload 1)

Default: non-streaming fallback that emits thinking, text, then tool calls.

method generateStreamAsync(p: Provider; request: ProviderRequest;
                           onEvent: StreamCallback): Future[ProviderResponse] {.
    base, stackTrace: false, raises: [Exception, ValueError],
    tags: [RootEffect], forbids: [].}

Returns: Future[ProviderResponse].

Arguments

Name	Type	Default
p	Provider
request	ProviderRequest
onEvent	StreamCallback

View source

nativeObjectOptions (overload 1)

Provider-body knobs for native structured output. nil means none. model is available because a gateway may serve one model on another wire format, which needs a different native shape.

method nativeObjectOptions(p: Provider; model, name, description: string;
                           schema: JsonNode): JsonNode {.base, raises: [],
    tags: [], forbids: [].}

Returns: JsonNode.

Arguments

Name	Type	Default
p	Provider
model	string
name	string
description	string
schema	JsonNode

View source

nativeObjectSchemaIssues (overload 1)

Provider-specific native-schema restrictions. Empty means compatible. model is available for gateways whose models use different wire formats.

method nativeObjectSchemaIssues(p: Provider; model: string; schema: JsonNode): seq[
    string] {.base, raises: [], tags: [], forbids: [].}

Returns: seq[string].

Arguments

Name	Type	Default
p	Provider
model	string
schema	JsonNode

View source

generateAsync (overload 2)

Forward generation through the wrapper hooks.

method generateAsync(p: WrapProvider; request: ProviderRequest): Future[
    ProviderResponse] {.stackTrace: false, raises: [Exception, ValueError],
                        tags: [RootEffect], forbids: [].}

Returns: Future[ProviderResponse].

Arguments

Name	Type	Default
p	WrapProvider
request	ProviderRequest

View source

generateStreamAsync (overload 2)

Forward streaming through the wrapper hooks.

method generateStreamAsync(p: WrapProvider; request: ProviderRequest;
                           onEvent: StreamCallback): Future[ProviderResponse] {.
    stackTrace: false, raises: [Exception, ValueError], tags: [RootEffect],
    forbids: [].}

Returns: Future[ProviderResponse].

Arguments

Name	Type	Default
p	WrapProvider
request	ProviderRequest
onEvent	StreamCallback

View source

embedAsync (overload 2)

Forward embeddings to the wrapped provider. Text-generation mappers intentionally do not alter embedding requests.

method embedAsync(p: WrapProvider; request: EmbeddingRequest): Future[
    EmbeddingResponse] {.stackTrace: false, raises: [Exception, ValueError],
                         tags: [RootEffect], forbids: [].}

Returns: Future[EmbeddingResponse].

Arguments

Name	Type	Default
p	WrapProvider
request	EmbeddingRequest

View source

nativeObjectOptions (overload 2)

Forward native structured-output options to the wrapped provider.

method nativeObjectOptions(p: WrapProvider; model, name, description: string;
                           schema: JsonNode): JsonNode {.raises: [Exception],
    tags: [RootEffect], forbids: [].}

Returns: JsonNode.

Arguments

Name	Type	Default
p	WrapProvider
model	string
name	string
description	string
schema	JsonNode

View source

nativeObjectSchemaIssues (overload 2)

Forward native schema checks to the wrapped provider.

method nativeObjectSchemaIssues(p: WrapProvider; model: string; schema: JsonNode): seq[
    string] {.raises: [Exception], tags: [RootEffect], forbids: [].}

Returns: seq[string].

Arguments

Name	Type	Default
p	WrapProvider
model	string
schema	JsonNode

View source

generateAsync (overload 3)

Route an asynchronous generation request by model ID.

method generateAsync(p: RouteProvider; request: ProviderRequest): Future[
    ProviderResponse] {.stackTrace: false, raises: [Exception, ValueError],
                        tags: [RootEffect], forbids: [].}

Returns: Future[ProviderResponse].

Arguments

Name	Type	Default
p	RouteProvider
request	ProviderRequest

View source

generateStreamAsync (overload 3)

Route an asynchronous streaming request by model ID.

method generateStreamAsync(p: RouteProvider; request: ProviderRequest;
                           onEvent: StreamCallback): Future[ProviderResponse] {.
    stackTrace: false, raises: [Exception, ValueError], tags: [RootEffect],
    forbids: [].}

Returns: Future[ProviderResponse].

Arguments

Name	Type	Default
p	RouteProvider
request	ProviderRequest
onEvent	StreamCallback

View source

embedAsync (overload 3)

Route an asynchronous embedding request by model ID.

method embedAsync(p: RouteProvider; request: EmbeddingRequest): Future[
    EmbeddingResponse] {.stackTrace: false, raises: [Exception, ValueError],
                         tags: [RootEffect], forbids: [].}

Returns: Future[EmbeddingResponse].

Arguments

Name	Type	Default
p	RouteProvider
request	EmbeddingRequest

View source

nativeObjectOptions (overload 3)

Route native structured-output options by model ID.

method nativeObjectOptions(p: RouteProvider; model, name, description: string;
                           schema: JsonNode): JsonNode {.raises: [Exception],
    tags: [RootEffect], forbids: [].}

Returns: JsonNode.

Arguments

Name	Type	Default
p	RouteProvider
model	string
name	string
description	string
schema	JsonNode

View source

nativeObjectSchemaIssues (overload 3)

Route native schema checks by model ID.

method nativeObjectSchemaIssues(p: RouteProvider; model: string;
                                schema: JsonNode): seq[string] {.
    raises: [Exception], tags: [RootEffect], forbids: [].}

Returns: seq[string].

Arguments

Name	Type	Default
p	RouteProvider
model	string
schema	JsonNode

View source

Constants

imageOmitted

Text used when an image is omitted.

imageOmitted = "[image omitted: model does not accept images]"

View source

retryAfterCapMs

Maximum Retry-After delay accepted, in milliseconds.

retryAfterCapMs = 30000

View source