# Farsight

Farsight watches streaming video or ingests images and produces typed data based on its analysis. It's used for scientific research, security, embedded devices, among other things. If you need clean typed analysis of images or video streams, farsight has you covered.

- Quickstart for agents: https://farsight.observer/quickstart.md
- Linux/macOS camera examples: https://farsight.observer/observe.py (observations) and https://farsight.observer/stream.py (SRT). Both use FFmpeg and read `FARSIGHT_API_KEY` or `~/.config/farsight/api_key`; set `FARSIGHT_KEY_FILE` for another key path. See the quickstart for use.

- Base URL: https://farsight.observer/v1
- OpenAPI: https://farsight.observer/openapi.json (support report schema; other endpoints are described below).
- Auth: `Authorization: Bearer <api key>` (keys start with `fs_`). Pair a device at https://farsight.observer/device or create keys at https://farsight.observer/keys.
- Errors: `{"error": {"code": "...", "message": "...", "field": "..."}}` with HTTP 401, 404, 409, 422, or 503. A stream setup already in progress returns 409; retry shortly.

## Concepts

- Chamber: one physical viewing area. It has questions, camera views, an optional `context`, and an optional webhook.
- Context: free text about the chamber (species, layout, what matters), up to 600 words. It is added to every analysis prompt. Each image can also carry its own `context` (up to 200 words). Answers still come only from what is visible.
- Question (an "eval" in the dashboard): `{key?, prompt, type, min?, max?}`. `type` is `boolean` or `integer`. No floats, no strings. An answer is null when the image cannot decide it.
- Key: the snake_case answer name. Omit it and Farsight makes one from the prompt: filler words (a, the, is, how many, ...) are dropped and the rest joined, so "Is a fish visible?" becomes `fish_visible`. A clash gets a number: `fish_visible_2`.
- View: one camera position in a chamber. Pass `view` (id or name) with an image to pin it; a new name creates the view. Omit it and Farsight matches the image to a known view, or creates one.
- Set camera orientation in your camera or encoder. Farsight saves and analyses source image pixels and displays source video without rotation or mirroring.
- Each paired device has its own key. One device can send several named views. Each view can have one live stream; a chamber can have several live views.
- Metadata: your own key/value pairs on a chamber or a view, for example `{"site": "lab 2", "rack": 4}`. Values are strings (up to 500 chars), numbers, or booleans. Up to 50 keys of 1 to 40 chars, 8 KB in all. Farsight stores and shows metadata; it never goes to the analysis prompt. Every chamber and view response has `metadata` (`{}` when empty).
- Observation: one image plus its analysis. `status` is `queued`, `processing`, `complete`, or `failed`.

## Quick start

```sh
# 0. Pair this device (no key needed). Show the code to the owner, who enters it at https://farsight.observer/device.
curl -s -X POST https://farsight.observer/v1/device/start -H "Content-Type: application/json" -d '{"name": "lab pi"}'
# Poll every 5 s with the returned device_token until status is "approved"; it carries api_key.
curl -s -X POST https://farsight.observer/v1/device/token -H "Content-Type: application/json" -d '{"device_token": "fd_..."}'
KEY=fs_your_key
# 1. Create a chamber with questions
curl -s https://farsight.observer/v1/chambers -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d '{
  "name": "Moth box A",
  "context": "Silkworms on mulberry. Cocoons are white and oval. Adults are pale grey moths.",
  "questions": [
    {"key": "cocooning_stage", "prompt": "Is any caterpillar spinning or inside a cocoon?", "type": "boolean"},
    {"prompt": "How many adult moths are visible?", "type": "integer", "min": 0}
  ],
  "webhook_url": "https://example.com/farsight"
}'

# 2. Send an image (raw bytes; view is optional)
curl -s "https://farsight.observer/v1/chambers/CHAMBER_ID/observations?view=top" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: image/jpeg" --data-binary @frame.jpg

# 3. Read the result (or wait for the webhook)
curl -s https://farsight.observer/v1/observations/OBSERVATION_ID -H "Authorization: Bearer $KEY"
```

## Endpoints

### Support

- POST /v1/support sends a report to the Farsight team. Use an API key. JSON body: {title, message, logs?}. Title is one line, 1 to 120 characters. Message is 1 to 10,000 characters. Logs is optional and may be up to 10,000 characters. The response is HTTP 201 with {id, status: "sent"}.
- GET /v1/support lists your reports, newest activity first: {data: [{id, title, message, logs, status, created_at, updated_at, resolved_at, comment_count}]}. status is "open" or "resolved". resolved_at is null while open.
- GET /v1/support/{id} returns one report with comments: [{id, author, body, created_at}], oldest first. author is "customer" (you) or "team" (Farsight).
- POST /v1/support/{id}/comments adds your reply. JSON body: {body}, 1 to 10,000 characters. The response is HTTP 201 with the comment. A reply to a resolved report reopens it. The Farsight team is emailed, and you are emailed when the team replies.
- Remove API keys, SRT passphrases, and other secrets from logs before sending. A person can use https://farsight.observer/support after signing in to see the same reports and replies.
- To report a failed stream, include the FFmpeg version, command with the SRT URL removed, full error log, and time of failure.

### Chambers

- `GET /v1/chambers` list.
- `POST /v1/chambers` body `{name, context?, metadata?, questions?, webhook_url?}`. `context` is up to 600 words; send null or "" to clear it.
- `GET /v1/chambers/{id}` full chamber with questions, views, and webhook. Each view includes its own `stream`.
- `PATCH /v1/chambers/{id}` any of the create fields. `questions` replaces the whole list. `metadata` replaces the whole object; send `{}` or null to clear it.
- `DELETE /v1/chambers/{id}` deletes the chamber, its images, and all camera streams.

### Questions

- `GET /v1/chambers/{id}/questions`
- `POST /v1/chambers/{id}/questions` body `{key?, prompt, type, min?, max?}`. Same key updates it. No key makes a new one from the prompt and returns it. `min` and `max` are optional for integers.
- `DELETE /v1/chambers/{id}/questions/{key}`

### Observations

- `POST /v1/chambers/{id}/observations` returns 202 and the queued observation. Send the image one of three ways:
  - raw bytes with `Content-Type: image/jpeg|png|webp|gif`, options in the query string: `?view=top&context=...&captured_at=2026-01-01T00:00:00Z`
  - `multipart/form-data` with field `image`, plus optional `view`, `context`, and `captured_at` fields
  - JSON `{"image_base64": "...", "view": "top", "context": "Lid was opened at 09:00."}` or `{"image_url": "https://..."}`
  `context` is optional, up to 200 words, and applies to this image only.
  Max 10 MB and 60 images per chamber per minute (422, field `rate`). `captured_at` is ISO 8601 or unix seconds/ms; default is receipt time.
- `GET /v1/chambers/{id}/observations?limit=20&cursor=...&view=VIEW_ID` newest first; follow `next_cursor`.
- `GET /v1/observations/{id}`
- `GET /v1/observations/{id}/image`

Observation shape:
```json
{
  "id": "ob_...", "chamber_id": "ch_...", "status": "complete", "source": "api",
  "view": {"id": "vw_...", "name": "top"},
  "context": null,
  "answers": {"cocooning_stage": true, "moth_count": 0},
  "note": "Two caterpillars on the left branch; one partly wrapped in silk.",
  "error": null,
  "image_url": "https://farsight.observer/v1/observations/ob_.../image",
  "captured_at": "...", "created_at": "...", "completed_at": "..."
}
```
Analysis usually finishes in 3 to 15 seconds. Poll `GET /v1/observations/{id}` about every 2 seconds, or use the webhook.

### Views

- `GET /v1/chambers/{id}/views`
- `POST /v1/chambers/{id}/views` body `{name}` creates a camera view before its first image or stream.
- `PATCH /v1/chambers/{id}/views/{view_id}` body `{name?, metadata?, snapshot_minutes?}`, at least one. `snapshot_minutes` is 1 to 1440, default 120. `metadata` replaces the whole object; send `{}` or null to clear it. Rotation and mirror fields are rejected.
- `DELETE /v1/chambers/{id}/views/{view_id}` deletes the camera, its live input, and its clips.
- `GET /v1/views/{view_id}/image` the reference frame used for matching.

### Webhook

- Set `webhook_url` on the chamber (https). Events: `observation.completed`, `observation.failed`, `ping`.
- Body: `{"type": "observation.completed", "created_at": "...", "data": <observation>}`.
- Header `Farsight-Signature: t=<unix>,v1=<hex>` where hex is HMAC-SHA256 of `<t>.<raw body>` with the chamber's `webhook.secret`. Reject if t is more than 5 minutes old.
- Retries with backoff for up to 6 attempts on a non-2xx response.
- `POST /v1/chambers/{id}/webhook/test` sends a ping. `POST /v1/chambers/{id}/webhook/rotate` returns a new secret.

### Stream

- Each view is one camera. A chamber can have many views streaming at once. `POST /v1/chambers/{id}/views/{view_id}/stream` enables SRT ingest and returns `{srt_url, state, snapshot_minutes, public, public_url}`. `DELETE` pauses it. The SRT URL uses `ingest.farsight.observer:778` and stays the same when you pause and resume that view. `GET /v1/chambers/{id}` returns each view's current `stream.srt_url`, including while paused.
- A chamber that had a stream before camera streams were added also keeps its old `/v1/chambers/{id}/stream` and `/stream/public` routes. They address its migrated camera and keep its SRT credentials across pause and resume. Its chamber response also includes the old top-level `stream` field. Use view routes for new cameras.
- Publish with any SRT caller, for example:
  `ffmpeg -i INPUT -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 -map 0:v:0 -map 1:a:0 -vf fps=10,setsar=1 -c:v libx264 -preset ultrafast -tune zerolatency -b:v 1000k -g 30 -keyint_min 30 -sc_threshold 0 -pix_fmt yuv420p -c:a aac -b:a 96k -f mpegts "SRT_URL"`
- For a video file, add `-re` before `-i INPUT` so the file plays at real-time speed. A live camera sets its own capture pace. The `fps=10` filter duplicates frames when a camera delivers fewer than 10 per second, so the publisher sends a steady 10 fps stream.
- A steady 10 fps is enough for monitoring; 30 fps gives smoother motion. The frame rate does not have to be 30 fps, but video timestamps must advance by one second for each second of real time. The publisher must encode and send at that pace. Check FFmpeg's `speed=` value during a long run: it should stay at or above `1.0x`. If it stays below `1.0x`, lower the capture resolution, frame rate, or bitrate, or use a faster encoder. Otherwise viewers can run out of playable video and see a spinner even while the SRT input says connected. Slowing the player would show slow motion; it does not fix the publisher's media clock.
- `state` is `live` only while playback media is available and advances near real time. If media advances below 80% of wall time, state is `slow` and the dashboard shows a still frame instead of a buffering player. A stream that has no playable video after 180 seconds, or whose media requests fail, reports `error`. The dashboard shows the most recent saved clip when live media fails.
- Streams are private by default. `POST /v1/chambers/{id}/views/{view_id}/stream/public` makes one camera public and returns its `public_url`: a signed-out page with only that player. `DELETE` on the same path makes it private; the old link stops working and the next public link is new. Pausing the stream also makes it private. A public page can stay cached for up to 30 seconds after it is made private.
- Every `snapshot_minutes` for a view, Farsight takes a frame from that camera and analyses it as an observation with `source: "stream"` and that view's ID.
- Rolling MP4 clips are off by default. The server can enable a window of at most two minutes with `STREAM_RETENTION_MINUTES`; existing clips are deleted when it is off. Agents can use `GET /v1/chambers/{id}/views/{view_id}/clips` for one camera or `GET /v1/chambers/{id}/clips` for all, then `GET /v1/clips/{clip_id}`.

### Credits

- New accounts start with 1,000 credits. The ledger models 1,000 credits as $1 for cost estimates; this is not a customer purchase price. Payment and refills are not enabled yet.
- `GET /v1/credits` returns a balance rounded down to a whole credit and recent signed whole-credit changes. Measured charges accumulate internally. `GET /v1/me` includes the balance under `credits`.
- AI analysis uses actual input and output tokens at 6 times the standard uncached model price. Stream recording capacity is 30 credits per retained video minute per month. R2 storage is 45 credits per GB-month, a 200% markup on its published unit cost. Copying Stream clips is 6 credits per minute. Playback is 6 credits per viewer-minute when provider analytics is configured.
- Usage is posted after work and may be delayed. Playback usage is checked each minute and rechecked after the day closes. A balance can go below zero. Credit balance does not block uploads, image analysis, or streams. There is no way to buy or refill credits yet.

### Keys

- `GET /v1/keys`, `POST /v1/keys` body `{name}` returns the full key, `DELETE /v1/keys/{id}` revokes. The dashboard at https://farsight.observer/keys can show any active key again.
- Device pairing needs no existing key. `POST /v1/device/start` body `{name?}` returns a 9-character `code` (four characters, dash, four characters), secret `device_token`, `verification_uri`, `expires_at` (Unix milliseconds), and `interval` (seconds). Show the code to the owner. The owner signs in and enters it at the verification URI within 10 minutes; the dash is optional. Poll `POST /v1/device/token` with `{"device_token":"..."}` every 5 seconds. HTTP 202 means pending; HTTP 200 with `{"status":"approved","api_key":"fs_..."}` supplies the key. Store it locally. HTTP 410 means expired; start again. HTTP 401 means an unknown token, and `status: "revoked"` means the owner revoked the key. HTTP 409 means five requests from this address are already pending. The code only approves the request; only the device token collects the key. The owner can revoke that device key at https://farsight.observer/keys; revoked keys stop working immediately.

## Tips for agents

- Write each question prompt as a plain visual check a person could answer from one frame.
- Use integer questions for counts and levels. Add `min`/`max` when there is a real range (for example 0 to 5); answers are clamped to it.
- Put what you know about the animals and setup in the chamber `context`. It helps the model read the frame; it never replaces looking.
- Keep one camera per view name, and send `view` when you know it; matching is best effort.