Documentation | Models › List Models

Models

models

Methods

Create Model ->

InferenceModel

post/v5/models

Create Model

List Models -> CursorPage<

InferenceModel

get/v5/models

List Models

query Parameters

ending_before: string

Optional

limit: number

Optional

(maximum: 10000, minimum: 1, default: 100)

model_vendor:

InferenceModelVendor

Optional

name: string

Optional

sort_by: string

Optional

sort_order:

SortOrder

Optional

starting_after: string

Optional

Response fields

has_more: boolean

Whether there are more items left to be fetched.

items: Array<

InferenceModel

total: number

The total of items that match the query. This is greater than or equal to the number of items returned.

limit: number

Optional

(default: 100)

The maximum number of items to return.

object: "list"

Optional

(default: "list")

Request example

curl https://api.egp.scale.com/v5/models \
    -H "x-api-key: $SGP_API_KEY"

200Example

{
  "has_more": true,
  "items": [
    {
      "id": "id",
      "created_at": "2019-12-27T18:11:19.117Z",
      "created_by_identity_type": "user",
      "created_by_user_id": "created_by_user_id",
      "model_type": "generic",
      "model_vendor": "openai",
      "name": "name",
      "status": "failed",
      "model_availability": "unknown",
      "model_metadata": {
        "foo": "bar"
      },
      "object": "model",
      "vendor_configuration": {
        "model_image": {
          "command": [
            "string"
          ],
          "registry": "registry",
          "repository": "repository",
          "tag": "tag",
          "env_vars": {
            "foo": "bar"
          },
          "healthcheck_route": "healthcheck_route",
          "predict_route": "predict_route",
          "readiness_delay": 0,
          "request_schema": {
            "foo": "bar"
          },
          "response_schema": {
            "foo": "bar"
          },
          "streaming_command": [
            "string"
          ],
          "streaming_predict_route": "streaming_predict_route"
        },
        "model_infra": {
          "cpus": "string",
          "endpoint_type": "async",
          "gpu_type": "nvidia-tesla-t4",
          "gpus": 0,
          "high_priority": true,
          "labels": {
            "foo": "string"
          },
          "max_workers": 0,
          "memory": "memory",
          "min_workers": 0,
          "per_worker": 0,
          "public_inference": true,
          "storage": "storage"
        }
      }
    }
  ],
  "total": 0,
  "limit": 0,
  "object": "list"
}

Update Model ->

InferenceModel

patch/v5/models/{model_id}

Update Model

Delete Model -> { id, deleted, object }

delete/v5/models/{model_id}

Delete Model

Get Model ->

InferenceModel

get/v5/models/{model_id}

Get Model

Domain types

InferenceModel = { id, created_at, created_by_identity_type, 9 more... }

InferenceModelAvailability = "unknown" | "available" | "unavailable"

InferenceModelType = "generic" | "completion" | "chat_completion"

LaunchVendorConfiguration = { model_image, model_infra }

LlmEngineVendorConfiguration = { model, checkpoint_path, cpus, 19 more... }

ModelDeleteResponse = { id, deleted, object }