Docs

Farsight

Farsight watches streaming video or ingests images and produces typed data based on its analysis. It's used for scientific research, security, embedded devices, among other things. If you need clean typed analysis of images or video streams, farsight has you covered.

Concepts

  • Chamber: one physical viewing area. It has questions, camera views, an optional context, and an optional webhook.
  • Context: free text about the chamber (species, layout, what matters), up to 600 words. It is added to every analysis prompt. Each image can also carry its own context (up to 200 words). Answers still come only from what is visible.
  • Question (an "eval" in the dashboard): {key?, prompt, type, min?, max?}. type is boolean or integer. No floats, no strings. An answer is null when the image cannot decide it.
  • Key: the snake_case answer name. Omit it and Farsight makes one from the prompt: filler words (a, the, is, how many, ...) are dropped and the rest joined, so "Is a fish visible?" becomes fish_visible. A clash gets a number: fish_visible_2.
  • View: one camera position in a chamber. Pass view (id or name) with an image to pin it; a new name creates the view. Omit it and Farsight matches the image to a known view, or creates one.
  • Set camera orientation in your camera or encoder. Farsight saves and analyses source image pixels and displays source video without rotation or mirroring.
  • Each paired device has its own key. One device can send several named views. Each view can have one live stream; a chamber can have several live views.
  • Metadata: your own key/value pairs on a chamber or a view, for example {"site": "lab 2", "rack": 4}. Values are strings (up to 500 chars), numbers, or booleans. Up to 50 keys of 1 to 40 chars, 8 KB in all. Farsight stores and shows metadata; it never goes to the analysis prompt. Every chamber and view response has metadata ({} when empty).
  • Observation: one image plus its analysis. status is queued, processing, complete, or failed.

Quick start

# 0. Pair this device (no key needed). Show the code to the owner, who enters it at https://farsight.observer/device.
curl -s -X POST https://farsight.observer/v1/device/start -H "Content-Type: application/json" -d '{"name": "lab pi"}'
# Poll every 5 s with the returned device_token until status is "approved"; it carries api_key.
curl -s -X POST https://farsight.observer/v1/device/token -H "Content-Type: application/json" -d '{"device_token": "fd_..."}'
KEY=fs_your_key
# 1. Create a chamber with questions
curl -s https://farsight.observer/v1/chambers -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d '{
  "name": "Moth box A",
  "context": "Silkworms on mulberry. Cocoons are white and oval. Adults are pale grey moths.",
  "questions": [
    {"key": "cocooning_stage", "prompt": "Is any caterpillar spinning or inside a cocoon?", "type": "boolean"},
    {"prompt": "How many adult moths are visible?", "type": "integer", "min": 0}
  ],
  "webhook_url": "https://example.com/farsight"
}'

# 2. Send an image (raw bytes; view is optional)
curl -s "https://farsight.observer/v1/chambers/CHAMBER_ID/observations?view=top" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: image/jpeg" --data-binary @frame.jpg

# 3. Read the result (or wait for the webhook)
curl -s https://farsight.observer/v1/observations/OBSERVATION_ID -H "Authorization: Bearer $KEY"

Endpoints

Support

  • POST /v1/support sends a report to the Farsight team. Use an API key. JSON body: {title, message, logs?}. Title is one line, 1 to 120 characters. Message is 1 to 10,000 characters. Logs is optional and may be up to 10,000 characters. The response is HTTP 201 with {id, status: "sent"}.
  • GET /v1/support lists your reports, newest activity first: {data: [{id, title, message, logs, status, created_at, updated_at, resolved_at, comment_count}]}. status is "open" or "resolved". resolved_at is null while open.
  • GET /v1/support/{id} returns one report with comments: [{id, author, body, created_at}], oldest first. author is "customer" (you) or "team" (Farsight).
  • POST /v1/support/{id}/comments adds your reply. JSON body: {body}, 1 to 10,000 characters. The response is HTTP 201 with the comment. A reply to a resolved report reopens it. The Farsight team is emailed, and you are emailed when the team replies.
  • Remove API keys, SRT passphrases, and other secrets from logs before sending. A person can use https://farsight.observer/support after signing in to see the same reports and replies.
  • To report a failed stream, include the FFmpeg version, command with the SRT URL removed, full error log, and time of failure.

Chambers

  • GET /v1/chambers list.
  • POST /v1/chambers body {name, context?, metadata?, questions?, webhook_url?}. context is up to 600 words; send null or "" to clear it.
  • GET /v1/chambers/{id} full chamber with questions, views, and webhook. Each view includes its own stream.
  • PATCH /v1/chambers/{id} any of the create fields. questions replaces the whole list. metadata replaces the whole object; send {} or null to clear it.
  • DELETE /v1/chambers/{id} deletes the chamber, its images, and all camera streams.

Questions

  • GET /v1/chambers/{id}/questions
  • POST /v1/chambers/{id}/questions body {key?, prompt, type, min?, max?}. Same key updates it. No key makes a new one from the prompt and returns it. min and max are optional for integers.
  • DELETE /v1/chambers/{id}/questions/{key}

Observations

  • POST /v1/chambers/{id}/observations returns 202 and the queued observation. Send the image one of three ways:
    - raw bytes with Content-Type: image/jpeg|png|webp|gif, options in the query string: ?view=top&context=...&captured_at=2026-01-01T00:00:00Z
    - multipart/form-data with field image, plus optional view, context, and captured_at fields
    - JSON {"image_base64": "...", "view": "top", "context": "Lid was opened at 09:00."} or {"image_url": "https://..."}
    context is optional, up to 200 words, and applies to this image only.
    Max 10 MB and 60 images per chamber per minute (422, field rate). captured_at is ISO 8601 or unix seconds/ms; default is receipt time.
  • GET /v1/chambers/{id}/observations?limit=20&cursor=...&view=VIEW_ID newest first; follow next_cursor.
  • GET /v1/observations/{id}
  • GET /v1/observations/{id}/image

Observation shape:

{
  "id": "ob_...", "chamber_id": "ch_...", "status": "complete", "source": "api",
  "view": {"id": "vw_...", "name": "top"},
  "context": null,
  "answers": {"cocooning_stage": true, "moth_count": 0},
  "note": "Two caterpillars on the left branch; one partly wrapped in silk.",
  "error": null,
  "image_url": "https://farsight.observer/v1/observations/ob_.../image",
  "captured_at": "...", "created_at": "...", "completed_at": "..."
}

Analysis usually finishes in 3 to 15 seconds. Poll GET /v1/observations/{id} about every 2 seconds, or use the webhook.

Views

  • GET /v1/chambers/{id}/views
  • POST /v1/chambers/{id}/views body {name} creates a camera view before its first image or stream.
  • PATCH /v1/chambers/{id}/views/{view_id} body {name?, metadata?, snapshot_minutes?}, at least one. snapshot_minutes is 1 to 1440, default 120. metadata replaces the whole object; send {} or null to clear it. Rotation and mirror fields are rejected.
  • DELETE /v1/chambers/{id}/views/{view_id} deletes the camera, its live input, and its clips.
  • GET /v1/views/{view_id}/image the reference frame used for matching.

Webhook

  • Set webhook_url on the chamber (https). Events: observation.completed, observation.failed, ping.
  • Body: {"type": "observation.completed", "created_at": "...", "data": <observation>}.
  • Header Farsight-Signature: t=<unix>,v1=<hex> where hex is HMAC-SHA256 of <t>.<raw body> with the chamber's webhook.secret. Reject if t is more than 5 minutes old.
  • Retries with backoff for up to 6 attempts on a non-2xx response.
  • POST /v1/chambers/{id}/webhook/test sends a ping. POST /v1/chambers/{id}/webhook/rotate returns a new secret.

Stream

  • Each view is one camera. A chamber can have many views streaming at once. POST /v1/chambers/{id}/views/{view_id}/stream enables SRT ingest and returns {srt_url, state, snapshot_minutes, public, public_url}. DELETE pauses it. The SRT URL uses ingest.farsight.observer:778 and stays the same when you pause and resume that view. GET /v1/chambers/{id} returns each view's current stream.srt_url, including while paused.
  • A chamber that had a stream before camera streams were added also keeps its old /v1/chambers/{id}/stream and /stream/public routes. They address its migrated camera and keep its SRT credentials across pause and resume. Its chamber response also includes the old top-level stream field. Use view routes for new cameras.
  • Publish with any SRT caller, for example:
    ffmpeg -i INPUT -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 -map 0:v:0 -map 1:a:0 -vf fps=10,setsar=1 -c:v libx264 -preset ultrafast -tune zerolatency -b:v 1000k -g 30 -keyint_min 30 -sc_threshold 0 -pix_fmt yuv420p -c:a aac -b:a 96k -f mpegts "SRT_URL"
  • For a video file, add -re before -i INPUT so the file plays at real-time speed. A live camera sets its own capture pace. The fps=10 filter duplicates frames when a camera delivers fewer than 10 per second, so the publisher sends a steady 10 fps stream.
  • A steady 10 fps is enough for monitoring; 30 fps gives smoother motion. The frame rate does not have to be 30 fps, but video timestamps must advance by one second for each second of real time. The publisher must encode and send at that pace. Check FFmpeg's speed= value during a long run: it should stay at or above 1.0x. If it stays below 1.0x, lower the capture resolution, frame rate, or bitrate, or use a faster encoder. Otherwise viewers can run out of playable video and see a spinner even while the SRT input says connected. Slowing the player would show slow motion; it does not fix the publisher's media clock.
  • state is live only while playback media is available and advances near real time. If media advances below 80% of wall time, state is slow and the dashboard shows a still frame instead of a buffering player. A stream that has no playable video after 180 seconds, or whose media requests fail, reports error. The dashboard shows the most recent saved clip when live media fails.
  • Streams are private by default. POST /v1/chambers/{id}/views/{view_id}/stream/public makes one camera public and returns its public_url: a signed-out page with only that player. DELETE on the same path makes it private; the old link stops working and the next public link is new. Pausing the stream also makes it private. A public page can stay cached for up to 30 seconds after it is made private.
  • Every snapshot_minutes for a view, Farsight takes a frame from that camera and analyses it as an observation with source: "stream" and that view's ID.
  • Rolling MP4 clips are off by default. The server can enable a window of at most two minutes with STREAM_RETENTION_MINUTES; existing clips are deleted when it is off. Agents can use GET /v1/chambers/{id}/views/{view_id}/clips for one camera or GET /v1/chambers/{id}/clips for all, then GET /v1/clips/{clip_id}.

Credits

  • New accounts start with 1,000 credits. The ledger models 1,000 credits as $1 for cost estimates; this is not a customer purchase price. Payment and refills are not enabled yet.
  • GET /v1/credits returns a balance rounded down to a whole credit and recent signed whole-credit changes. Measured charges accumulate internally. GET /v1/me includes the balance under credits.
  • AI analysis uses actual input and output tokens at 6 times the standard uncached model price. Stream recording capacity is 30 credits per retained video minute per month. R2 storage is 45 credits per GB-month, a 200% markup on its published unit cost. Copying Stream clips is 6 credits per minute. Playback is 6 credits per viewer-minute when provider analytics is configured.
  • Usage is posted after work and may be delayed. Playback usage is checked each minute and rechecked after the day closes. A balance can go below zero. Credit balance does not block uploads, image analysis, or streams. There is no way to buy or refill credits yet.

Keys

  • GET /v1/keys, POST /v1/keys body {name} returns the full key, DELETE /v1/keys/{id} revokes. The dashboard at https://farsight.observer/keys can show any active key again.
  • Device pairing needs no existing key. POST /v1/device/start body {name?} returns a 9-character code (four characters, dash, four characters), secret device_token, verification_uri, expires_at (Unix milliseconds), and interval (seconds). Show the code to the owner. The owner signs in and enters it at the verification URI within 10 minutes; the dash is optional. Poll POST /v1/device/token with {"device_token":"..."} every 5 seconds. HTTP 202 means pending; HTTP 200 with {"status":"approved","api_key":"fs_..."} supplies the key. Store it locally. HTTP 410 means expired; start again. HTTP 401 means an unknown token, and status: "revoked" means the owner revoked the key. HTTP 409 means five requests from this address are already pending. The code only approves the request; only the device token collects the key. The owner can revoke that device key at https://farsight.observer/keys; revoked keys stop working immediately.

Tips for agents

  • Write each question prompt as a plain visual check a person could answer from one frame.
  • Use integer questions for counts and levels. Add min/max when there is a real range (for example 0 to 5); answers are clamped to it.
  • Put what you know about the animals and setup in the chamber context. It helps the model read the frame; it never replaces looking.
  • Keep one camera per view name, and send view when you know it; matching is best effort.
llms.txtdocs.md