Documentation | Evaluation Items

Evaluation Items

EvaluationItemsResource

Methods

client.evaluation_items.retrieve(, ) ->

get/v5/evaluation-items/{evaluation_item_id}

Retrieve a single evaluation item by its ID within the caller's account.

By default only non-archived items are returned; pass include_archived=true to also retrieve an item that has been archived. The response merges the item's cached task results into its data field and exposes a task_errors map keyed by task alias, so a task that failed on this item surfaces as an entry there rather than as a request error. Use this to inspect one item's input data and per-task results; to page through many items, use the list endpoint instead. The request fails if no item with the given ID exists in the caller's account.

client.evaluation_items.list() -> SyncCursorPage[

EvaluationItem

]

get/v5/evaluation-items

Return a paginated list of evaluation items belonging to the caller's account.

Pass evaluation_id to restrict the results to a single evaluation's items. The completion_status filter selects items by whether any of their tasks errored: failed returns only items that have task errors, passed returns only items with no task errors, and all (or omitting the parameter) returns every item. Archived items are excluded unless include_archived=true. Each returned item has its cached task results merged into data and its errors exposed in task_errors, identically to the single-item endpoint.

Parameters

completion_status: Optional[Literal["failed", "passed", "all"]]

Filter items by completion status. Pass 'failed' to return only items with errors, 'passed' for items without errors. Pass 'all' or omit to return all items.

"failed"

"passed"

"all"

ending_before: Optional[str]

evaluation_id: Optional[str]

include_archived: Optional[

bool

]

limit: Optional[int]

(maximum: 10000, minimum: 1, default: 100)

sort_by: Optional[str]

sort_order: Optional[

SortOrder

]

starting_after: Optional[str]

Returns

EvaluationItem

id: str

The unique identifier of the entity.

created_at:

datetime

(format: date-time)

The date and time when the entity was created in ISO format.

created_by:

Identity

The identity that created the entity.

data: Dict[str,

object

]

evaluation_id: str

archived_at: Optional[datetime]

(format: date-time)

The date and time when the entity was archived in ISO format.

dataset_item_id: Optional[str]

dataset_item_version_num: Optional[int]

files: Optional[Dict[str, str]]

object: Optional[Literal["evaluation.item"]]

(default: "evaluation.item")

task_errors: Optional[Dict[str,

TaskError

]]

Map of task alias to error info.

Request example

import os
from scale_gp_beta import SGPClient

client = SGPClient(
    api_key=os.environ.get("SGP_API_KEY"),  # This is the default and can be omitted
)
page = client.evaluation_items.list()
page = page.items[0]
print(page.id)

200Example

{
  "has_more": true,
  "items": [
    {
      "id": "id",
      "created_at": "2019-12-27T18:11:19.117Z",
      "created_by": {
        "id": "id",
        "type": "user",
        "object": "identity"
      },
      "data": {
        "foo": "bar"
      },
      "evaluation_id": "evaluation_id",
      "archived_at": "2019-12-27T18:11:19.117Z",
      "dataset_item_id": "dataset_item_id",
      "dataset_item_version_num": 0,
      "files": {
        "foo": "string"
      },
      "object": "evaluation.item",
      "task_errors": {
        "foo": {
          "message": "message",
          "type": "type"
        }
      }
    }
  ],
  "total": 0,
  "limit": 0,
  "object": "list"
}

client.evaluation_items.export() ->

EvaluationItemExport

post/v5/evaluation-items/export

Export all evaluation items for a single evaluation as a downloadable file.

The evaluation is specified by evaluation_id in the request body. export_format selects CSV, JSON, or JSONL; for CSV the per-item data and files fields are flattened into individual columns and metric-like result columns are expanded. export_method controls delivery: direct returns the file contents inline in the response, while signed_url uploads the file to object storage and returns a pre-signed download URL. Requesting signed_url in an environment where object storage is not configured fails with a 501, so use direct there instead. Set include_archived=true to include archived items in the export. This endpoint reads items only and does not modify the evaluation.

Domain types

class Component: ...

class Container: ...

class EvaluationItem: ...

class EvaluationItemExport: ...

Response model for exporting evaluation items. This class represents the response when users export evaluation items. It contains either a signed URL to download the exported data from object storage, or the actual content bytes when direct download is used (in environments where object storage is not configured).

Literal["json", "jsonl", "csv"]

Literal["signed_url", "direct"]

class TaskError: ...