Inference

inference

Methods

Run Inference With Free Form Payload -> |
post/v5/inference

Runs a model using a free-form, vendor-native request payload rather than a fixed OpenAI schema.

Use this endpoint when the target model does not fit the OpenAI chat, responses, or text-completion contracts: the args field is an arbitrary dict passed straight through to the selected vendor gateway, and the reply is returned inside response as arbitrary JSON (object, array, string, number, or boolean). Prefer /v5/chat/completions, /v5/responses, or /v5/completions when your request matches one of those OpenAI-standard shapes, since those return typed OpenAI response objects. The model is chosen from the model field formatted as vendor/name, which selects the per-vendor inference gateway. When the request enables streaming, the response is sent as server-sent events with each chunk wrapped as a generic_inference.chunk object; otherwise a single generic_inference object is returned.

Domain types

InferenceCreateResponse = |
InferenceResponse = { response, object }
InferenceResponseChunk = { response, object }
LaunchInferenceConfiguration = { num_retries, timeout_seconds }