The API

The same pass, fail, or defer cards as the local tool, queued on tenali.dev for when your agent runs elsewhere or the reviewer isn't you.

Tenali runs on your machine first; running it locally needs none of this. These are the same cards, queued here for when your agent runs elsewhere or the reviewer isn't you.

The local tool is the whole product. This is for the day your agent lives on a server and the reviewer isn't you: same cards, still nothing vaguer than pass, fail, or defer.

1Check your key

GET/api/v1/wallet

Every call sends your API key. Name your client too: it shows up in our analytics as how you use the service, and helps when something breaks. Base path /api/v1; JSON in, JSON out. The first call worth making reads your wallet.

Headers

Authorizationrequired
Bearer and your key. tenali.dev is invite-only for now, so your key is issued to you, and shown once.
Tenali-Clientrecommended
Your client's name and version, recorded with every call.
Idempotency-Keyoptional
On POST /batches: the same key returns the batch it already made, never a duplicate.

Your wallet starts with a proof-of-concept grant of 250,000 GOLD; the response shows the balance and what's in escrow.

Request
curl https://tenali.dev/api/v1/wallet \
  -H "Authorization: Bearer $TENALI_KEY" \
  -H "Tenali-Client: my-agent/0.1"
Response
{"address": "gp1…", "token": "GOLD",
 "balance": 250000, "in_escrow": 0,
 "escrowed_total": 0, "refunded_total": 0,
 "transactions": [ … ]}

2Post a batch

POST/api/v1/batches

The reward is per seat. Posting moves gold × seats into the batch's escrow wallet. The request sends the required fields and a model verdict to check; the lists below name every field.

Fields

capabilitystring
A key from /capabilities. Exactly one of capability or criterion.
criterionstring
A key from /criteria: your own question, private to your workspace.
titlestring, required
1 to 120 characters.
audienceobject
Default {"mode": "open"}. {"mode": "me"} keeps the batch private to the key's owner.
goldinteger
Reward per seat, whole GOLD. Required for open, refused for me. At least 1,800 GOLD an hour at median_seconds.
median_secondsinteger
Your estimate of the median time for the whole batch. Required for open, refused for me.
seatsinteger
1 to 5, default 1. Each seat is one reviewer deciding every card. me allows only 1.
closes_in_hoursnumber
1 to 336, default 72.
tasksarray, required
1 to 16 cards, each with the fields below.

Each task

instructionsstring, required
The question the reviewer answers about this card.
model_verdictpass | fail
What your model said, so the result can say whether a person overturned it.
confidencenumber
0 to 1: how sure the model was.
stratumrandom | gray | flagged
Why the card was sampled; gray is the zone where the model was least sure.
artifactsarray, required
1 to 4 of {"kind", "label", "data_base64"}, up to 5 MB each: audio (WAV or MP3), text, or json. Text and JSON can send "text" instead of base64.
Request
curl -X POST https://tenali.dev/api/v1/batches \
  -H "Authorization: Bearer $TENALI_KEY" \
  -H "Content-Type: application/json" -d '{
  "capability": "transcript",
  "title": "Support calls, week 37",
  "gold": 400, "median_seconds": 360,
  "tasks": [{
    "instructions":
      "Does the transcript say what the caller says?",
    "model_verdict": "pass",
    "artifacts": [
      {"kind": "audio", "label": "Call audio",
       "data_base64": "SUQzBAAAAA…"},
      {"kind": "text", "label": "Transcript",
       "text":
         "I'd like to change my billing address."}
    ]
  }]
}'
Response
201 Created
{"batch_id": 41, "replayed": false, "status": "open",
 "escrow": {"address": "gp1…", "balance": 400,
            "committed": 0},
 "wallet_balance": 249600,
 "url": "/api/v1/batches/41"}

3Get notified

Pick either. A webhook gets a signed POST for every event, retried with backoff for about seven hours. The events feed works from anywhere, including a laptop behind a firewall: long-poll it with wait.

PUT/api/v1/webhook

Fields

urlstring, required
https on port 443, on a public address. The secret is shown once, in this response; setting the webhook again rotates it.

Each delivery carries a Tenali-Signature header. Check it before trusting the body, as the Python sample does.

Request
curl -X PUT https://tenali.dev/api/v1/webhook \
  -H "Authorization: Bearer $TENALI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/tenali"}'
Response
{"webhook": {"url": "https://example.com/tenali",
             "secret": "whsec_…"}}
Verify a delivery
import hashlib, hmac, time

def verify(secret: str, header: str, body: bytes,
           tolerance: int = 300) -> bool:
    # Tenali-Signature: t=…,v1=…
    parts = dict(p.split("=", 1)
                 for p in header.split(","))
    if abs(time.time() - int(parts["t"])) > tolerance:
        return False
    signed = f"{parts['t']}.".encode() + body
    mac = hmac.new(secret.encode(), signed,
                   hashlib.sha256).hexdigest()
    return hmac.compare_digest(mac, parts["v1"])

GET/api/v1/events

Query

afterinteger
The last event id you saw. Keep next_after and pass it back.
limitinteger
Default 100.
waitinteger
0 to 25 seconds. The call returns as soon as an event exists.
EventWhen
batch.seat_acceptedA reviewer took a seat.
batch.seat_releasedA reviewer let a seat go, or their 24-hour hold ran out. The seat is back on the board.
batch.seat_finishedA seat decided every card.
batch.completedEvery seat finished. Carries the full results and any refund.
batch.cancelledYou cancelled it. Unused escrow is refunded; seats already working can finish.
batch.expiredIt closed before every seat finished. Unused escrow is refunded.
Request
curl -G https://tenali.dev/api/v1/events \
  -d after=0 -d wait=25 \
  -H "Authorization: Bearer $TENALI_KEY"
Response
{"events": [{"id": 88, "type": "batch.completed",
             "batch_id": 41, "data": {…}}],
 "next_after": 88}

4Read the results

GET/api/v1/batches/{id}

Every seat's verdicts with rationale and timing, the consensus per card, and each reviewer as a wallet address with a reputation tier. Verdicts from an account removed for abuse come back with "excluded": true and don't count toward consensus.

Each card

consensuspass | fail | split | null
What the decided seats said. Deferrals and excluded verdicts don't count; null until a seat decides.
agreementboolean | null
Whether the decided seats agree. It needs two, so a one-seat batch reports null.
overturned_modelboolean
True when every decided verdict differs from your model_verdict.
state_hashstring
The rendered question plus the artifact bytes the card showed, so a verdict on a reworded question or a changed artifact can be told apart.
Request
curl https://tenali.dev/api/v1/batches/41 \
  -H "Authorization: Bearer $TENALI_KEY"
Response
{"batch_id": 41, "status": "complete", "done": true,
 "seats": {"total": 1, "finished": 1, "live": 0},
 "cards": [{
   "position": 1, "consensus": "fail",
   "agreement": null, "overturned_model": true,
   "verdicts": [{
     "seat": 1, "verdict": "fail",
     "rationale": "Says shipping, not billing",
     "duration_ms": 21400, "reviewer": "gp1…",
     "excluded": false}]}]}

Endpoints

Every path is under https://tenali.dev/api/v1.

CallWhat it does
GET/capabilitiesKinds of work, rubric versions, qualified reviewers, the token, limits, event types.
PUT/criteria/{key}Define your own question, or version it. Private to your workspace.
GET/criteriaYour criteria: every rubric version, and what's been asked against each.
GET/criteria/{key}One criterion and its versions.
POST/batchesPost and escrow a batch. Honors Idempotency-Key.
GET/batchesYour batches; filter with status, external_ref; page with after.
GET/batches/{id}Status, seats, escrow, and full results.
POST/batches/{id}/cancelStop new seats and refund unused escrow.
GET/eventsThe events feed: after, limit, wait (up to 25 s).
GETPUTDELETE/webhookRead, set (rotates the secret), or remove your webhook.
GET/walletYour address, balance, GOLD in escrow, recent transactions.

Your own questions

PUT/api/v1/criteria/{key}

A capability is a kind of work reviewers qualified for: few, curated by us, public. A criterion is your own question: unlimited, private to your workspace, and never on the board. Writing down what right means for your app is the whole point, so a criterion is where you do it.

Fields

keypath
1 to 64 characters: lowercase letters, digits, hyphens, underscores.
titlestring, required
What the question is called on the card.
questionstring, required
The statement a reviewer passes or fails. A changed question is the next version.
summarystring
One line on what you send against it.
kindbinary | choice | score
Default binary. Can't change later; a different kind is a different question, so give it a new key.
Request
curl -X PUT \
  https://tenali.dev/api/v1/criteria/receipt-total \
  -H "Authorization: Bearer $TENALI_KEY" \
  -H "Content-Type: application/json" -d '{
  "title": "Receipt total matches",
  "question": "The extracted total equals the receipt's final total, including tax."
}'
Response
201 Created
{"key": "receipt-total", "kind": "binary",
 "reviewable": true, "rubric_version": 1,
 "spec_hash": "80674949…", "batches": 0, "cards": 0,
 "url": "/api/v1/criteria/receipt-total"}

Then post a batch against it. The response has the same shape as in Post a batch, with nothing escrowed.

The same question again is the same version; a changed one is the next, and no version is ever edited, so a batch you already posted keeps the question it actually asked. A batch names a criterion that must already exist and carries no rubric of its own — a typo would otherwise quietly split the answers you're accumulating into two half-built sets.

A criterion batch is always "me": your own question is yours to answer. Reaching reviewers means a capability they qualified for.

Request
curl -X POST https://tenali.dev/api/v1/batches \
  -H "Authorization: Bearer $TENALI_KEY" \
  -H "Content-Type: application/json" -d '{
  "criterion": "receipt-total",
  "title": "Receipts, week 37",
  "audience": {"mode": "me"}, "tasks": [ … ]
}'

Audiences

A batch takes an audience saying who can see and take it. Leave it out and nothing changes: the batch is open work, exactly as before audiences existed.

"me" is a batch only you review, signed in. It never appears on the board, in the demand meters, or to another reviewer, and it's kept out of everyone's reputation and accuracy — including yours, since scoring you against your own answers would mean nothing. Nobody is paid for it, so gold, median_seconds and a second seat are refused rather than ignored, and no GOLD moves. Your key needs an owner: the account that reviews its private batches, which we set when we issue the key.

Not built yet, and refused with AUDIENCE_NOT_BUILT so you can tell: "team" for colleagues in your workspace, "invited" for one guest with a scoped link, and "targeted", which waits on an honest answer about pay. See what's built, and what isn't.

Request body
{"capability": "transcript",
 "title": "Support calls, week 37",
 "audience": {"mode": "me"},
 "tasks": [ … ]}

Limits

  • Rewards are whole GOLD and at least 1,800 GOLD an hour at median_seconds.
  • 1 to 16 cards a batch, 1 to 5 seats, 1 to 4 artifacts a card, each up to 5 MB: WAV or MP3 audio, UTF-8 text, or JSON.
  • Inline text fields and metadata are 4 KB at most; pack anything bigger as an artifact with only the range a reviewer needs.
  • Content labels: Mild language, Mentions alcohol, Medical terms, Money and prices. Graphic, sexual, violent, and self-harm content can't be posted.
  • Errors are {"error": "CODE", "message": "…"}. Every response carries a Tenali-Request-Id.
Error
422 Unprocessable Entity
{"error": "UNDER_FLOOR", "message": "…"}

Capabilities open now

KeyWhat reviewers judgeQualified
transcriptRubric v1Transcript matches the audioReal audiobook clips beside a speech-recognition transcript0
same_readerRubric v1Same reader?Two short clips from real audiobooks: is one person reading both0
entity_linkRubric v1Right entity?A real Wikipedia sentence and a proposed link for a name in it0

GOLD is a token with no cash value. Rewards, escrow, and refunds move GOLD, never money. See docs/API.md in the repository for every field.