Answering many texts

There is no batch endpoint. Each call answers one text, so a list of texts is a loop of calls. The app's own file runs work this way too. They send eight rows at a time, each with its own idempotency key.

The loop needs three things:

A few calls at a time. Start with three in flight. The API has no rate limit, but each call stays open until the model answers, which can take up to about 30 seconds.
A client timeout above 30 seconds. Sending a text explains why. The examples use 45 seconds.
An idempotency key per record. Run the same list again and every record that already succeeded comes back from storage without asking the model, while every failed one is retried. A run that stopped halfway resumes where it left off.

The loop

Each example reads tickets.json, sends each ticket with format=jsonl, and writes the rows to answers.jsonl. With jsonl, every response is one line of JSON, so the output file is the responses joined together.

[
  {
    "id": "8812",
    "subject": "Charged twice this month",
    "body": "Hi, I was billed twice for September. Please refund the second charge before Friday."
  },
  {
    "id": "8813",
    "subject": "Calendar sync stopped",
    "body": "Since this morning none of our appointments show up in Google Calendar."
  }
]

The shell version needs jq to turn tickets.json into one request body per line, then xargs runs three curl calls at a time.

jq -c '.[] | {
  ref: ("TCK-" + .id),
  idempotency_key: ("tck-" + .id + "-1"),
  fields: { subject: .subject, text: .body }
}' tickets.json |
  tr '\n' '\0' |
  xargs -0 -n 1 -P 3 curl -sS --fail --max-time 45 \
    -H "Authorization: Bearer $PRISMLET_API_KEY" \
    -H "Content-Type: application/json" \
    "https://api.prismlet.com/v1/prisms/support-routing/results?format=jsonl" \
    -d > answers.jsonl

xargs puts each body after -d. With --fail, only answered rows reach answers.jsonl, and curl names each failing status on stderr. Run the same command again to fill the gaps. Answered tickets come back from storage and failed ones are retried. A 400 repeats on every run until you fix that ticket.

import { readFile, writeFile } from 'node:fs/promises';

const API = 'https://api.prismlet.com/v1';
const CONCURRENCY = 3;

const tickets = JSON.parse(await readFile('tickets.json', 'utf8'));

async function answer(ticket) {
  const response = await fetch(`${API}/prisms/support-routing/results?format=jsonl`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.PRISMLET_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      ref: `TCK-${ticket.id}`,
      idempotency_key: `tck-${ticket.id}-1`,
      fields: { subject: ticket.subject, text: ticket.body },
    }),
    signal: AbortSignal.timeout(45_000),
  });
  const text = await response.text();
  // 200 and 502 both carry the result's row; any other status is an error body.
  if (response.status !== 200 && response.status !== 502) {
    throw new Error(`${response.status} ${text}`);
  }
  return text;
}

// Rows go in input order, whatever order the calls finish in.
const rows = new Array(tickets.length).fill('');
const problems = [];
let next = 0;

async function worker() {
  while (next < tickets.length) {
    const index = next++;
    const ticket = tickets[index];
    try {
      rows[index] = await answer(ticket);
    } catch (error) {
      problems.push({ id: ticket.id, error: String(error) });
    }
  }
}

await Promise.all(Array.from({ length: CONCURRENCY }, worker));
await writeFile('answers.jsonl', rows.join(''));
console.log(`${tickets.length - problems.length} rows, ${problems.length} problems`, problems);

import json
import os
from concurrent.futures import ThreadPoolExecutor

import requests

API = "https://api.prismlet.com/v1"
CONCURRENCY = 3

with open("tickets.json", encoding="utf-8") as f:
    tickets = json.load(f)


def answer(ticket):
    response = requests.post(
        f"{API}/prisms/support-routing/results",
        params={"format": "jsonl"},
        headers={"Authorization": f"Bearer {os.environ['PRISMLET_API_KEY']}"},
        json={
            "ref": f"TCK-{ticket['id']}",
            "idempotency_key": f"tck-{ticket['id']}-1",
            "fields": {"subject": ticket["subject"], "text": ticket["body"]},
        },
        timeout=45,
    )
    # 200 and 502 both carry the result's row; any other status is an error body.
    if response.status_code not in (200, 502):
        raise RuntimeError(f"{response.status_code} {response.text}")
    return response.text


rows, problems = [], []
with ThreadPoolExecutor(max_workers=CONCURRENCY) as pool:
    futures = [(ticket, pool.submit(answer, ticket)) for ticket in tickets]
    for ticket, future in futures:
        try:
            rows.append(future.result())
        except Exception as error:
            problems.append({"id": ticket["id"], "error": str(error)})

with open("answers.jsonl", "w", encoding="utf-8", newline="") as f:
    f.write("".join(rows))
print(f"{len(rows)} rows, {len(problems)} problems", problems)

Reading the output

Each line of answers.jsonl is one row: result_id, ref, version, status, error, created_at, the fields, then the Lens columns. A row with "status": "failed" carries its error and has null Lens columns. Run the script again to retry those. Their keys retry them, and the rows that were already answered come back unchanged, without asking the model.

problems holds calls that got no row: a status other than 200 or 502, a timeout or a network error. Running again fixes timeouts, network errors and 500s. A 400, such as a text over 100,000 characters, repeats until you fix the record. The JavaScript and Python versions write rows in the order of tickets.json, and the curl version writes them as calls finish.

If you'd rather have a spreadsheet, use format=csv and keep only the first response's header row.

What a key stands for

A key stands for one answer to one record. The examples use tck-<id>-1. If a ticket's text changes and you want it answered again, change the suffix to -2. The old key would return the old answer.

Don't reuse a key scheme between prisms. Errors and retries explains why.

Running a file from the app

The Try tab of a prism can run a whole file without any code.

The file is CSV, TSV or JSON, told apart by its extension. A .txt file is read as TSV or CSV, whichever it looks like. In CSV and TSV the first row is the column names. A JSON file is an array of objects, and its keys are the column names.
A file holds at most 10,000 rows and 200 MiB. Blank lines are skipped. A file that is not valid UTF-8 is read as Windows-1252. Keep the tab open while it runs; a large file of long texts can take most of an hour.
You choose a column for every field, and optionally one for the reference. Without one, or where its cell is blank, the reference is the row number.
Optionally, a column for the context. Each row's cell is sent as that call's context, after the prism's own. An empty cell sends none.
Each row is saved as a result with the file as its source, so it shows in Results like any other.
Running the same file again with the same columns chosen, on the same prism version, resumes: answered rows come back from storage and failed ones are retried. Choosing other field, reference or context columns, or saving a new version of the prism, answers every row again.
The answered file is the original columns, then the Lens columns, then result_id, in the file's own format. CSV and TSV cells hold labels. A JSON file holds keys, and every original value comes back as a string.
A Lens column or result_id that has the same name as an original column gets a number suffix, such as urgent_2, so your own columns are kept.
A row the app could not send, for example one with a reference over 200 characters, is counted as not saved and has no result.

Checking answers against your own labels

If the file already holds the answers you expect, from your QA team for example, choose that column under Expected answers for each question you want checked. After the run the app shows how often its answers match yours.

Yes / No cells can be yes, no, y, n, true, false, 1 or 0. Choice cells can be an option's key or label, including Other and Not enough information. Scale cells can be a level's key, its label or its number counting from 1. Case and surrounding spaces don't matter.
A blank cell leaves that row out of the check. Any other value the app can't read is left out and counted, with a few examples shown.
Yes / No shows agreement, recall and precision for Yes at cutoffs from 0.1 to 0.9, plus a row for the Lens's own cutoffs, which also shows the share in the Unsure band. Unsure rows are left out of the Lens row's agreement, recall and precision, and they are listed as misses.
Choice shows agreement, precision and recall per option, and agreement when only answers with a top probability from 0.5 to 0.9 are counted, with the share of rows that leaves.
Scale shows exact agreement and agreement within one level.
Group results by splits the headline figures by a column, such as language, site or team. A Yes / No group also shows its share in the Unsure band, which its agreement leaves out.
Every row where the answers differ is listed as a miss. Download evaluation saves the figures as CSV, and Download misses saves every miss with its reference and result_id.
Your expected answers and the group column stay in the browser. Only the field columns, and the context column when you choose one, are sent.
Changing the expected-answer or group columns after a run updates the figures without asking the model again. To compare two versions of the prism, run the file once on each.