# FrameFetch

> Agent-first API + MCP that turns any social-video URL (YouTube, YouTube Shorts, TikTok, Instagram Reels, Pinterest, Reddit) into metadata, insights, transcript, parametrically-sampled frames, and on-screen text (OCR). Pay per call; light calls are sub-cent; a free tier exists; agents can pay with x402 (USDC) without an account.

## Core

- [Get an API key]: `POST https://framefetch.net/v1/keys` — body `{ email }`. Returns `{ key }` (shown once) with a small free credit. Self-serve; no human in the loop.
- [Extract endpoint]: `POST https://framefetch.net/v1/extract` — body `{ url, fields[], frames{}, verbosity }`. `fields` ⊆ `["metadata","insights","transcript","frames","text_overlay"]` (default `["metadata"]`). Returns one JSON object with the requested data; `frames` are time-limited signed image URLs; `text_overlay` (on-screen text via OCR — captions burned into the video, price tags, signage) requires `frames` + a frames spec, one result per frame; auto-skipped with a warning (not an error) above 200 estimated frames — keep the frames spec narrow (fps/range) to use it. Auth: `Authorization: Bearer <key>`. Returns 402 when out of credit. Request only what you need — metadata is cheap, transcript/frames/text_overlay cost more.
- [Scoped shortcuts] (same auth/billing, preset fields): `POST /v1/metadata` (metadata+insights), `POST /v1/transcript`, `POST /v1/frames` (needs a `frames` spec). Body just `{ url, ... }`.
- [Buy credits with a card]: `POST /v1/checkout` (Bearer key) -> returns a Stripe Checkout URL (for humans). Body `{ pack? }`, one of `'5'` (default, $5, no bonus), `'20'` ($20, +5% bonus), `'100'` ($100, +12% bonus).
- [Top up credits]: `POST https://framefetch.net/v1/topup` — Bearer key + x402. Without payment returns HTTP 402 with x402 payment requirements (USDC on Base); pay and retry with `X-PAYMENT` to add credit. Agents can pay without a human.
- [Platform capabilities]: `GET https://framefetch.net/v1/platforms` — which data each platform supports.
- [Health]: `GET https://framefetch.net/healthz`.
- [MCP]: `POST https://framefetch.net/mcp` — Streamable HTTP MCP server (Bearer key). Tools: `framefetch_extract`, `framefetch_platform_capabilities`, `framefetch_account`.

## Account & billing transparency

- [Account]: `GET /v1/account` — email, signup date, balance_micros, spend_today_micros, spend_month_micros, limits, auto_topup. Read-only, never bills, works even at balance 0.
- [Usage]: `GET /v1/usage?from&to&group=day|endpoint` — usage history bucketed by day or endpoint (`{ items:[{bucket,calls,cost_micros}], total_micros }`). Defaults to the trailing 30 days; range capped at 366 days. Read-only, never bills.
- [MCP account tool]: `framefetch_account` — same balance/usage check via MCP, read-only, no input. Works at balance 0 (unlike `framefetch_extract`) so an agent can check affordability before/after a 402.
- [Manage billing]: `POST /v1/billing/portal` (Bearer key) -> Stripe Billing Portal link (update saved card, view invoices). Requires one prior `/v1/checkout` purchase.
- [Auto top-up]: `POST /v1/billing/auto-topup` — body `{ enabled, threshold_micros?, amount_micros? }`. Recharges a saved card automatically once balance drops below `threshold_micros` (min $0.01), adding `amount_micros` (must match a pack amount). Enabling requires a saved card from a prior checkout.
- Full account transparency is a USP: every balance, spend, and usage number an account has is available via both the API and MCP — nothing is hidden behind a dashboard-only view.

## Docs & machine specs

- Full single-file reference (everything inline, for ingestion): https://framefetch.net/llms-full.txt
- Docs (how to call every endpoint, what it returns): https://framefetch.net/docs
- OpenAPI 3.1 spec: https://framefetch.net/openapi.json
- Pricing (full rate card): https://framefetch.net/pricing
- Source code (open): https://github.com/MarvinRey7879/framefetch
- Per-platform guides: /youtube-transcript-api, /youtube-shorts-api, /tiktok-video-api, /instagram-reels-api, /reddit-video-transcript, /pinterest-video-api

## Pricing / payment (charged rates, 1.5x margin included)

- Free $0.05 credit on signup. No subscription. Paid credits never expire — no clock, no forced use-it-or-lose-it.
- Metadata + insights: $0.00015 per call. Transcript: $0.0015 per audio-minute. Translation (optional `translate` param): $0.0015 per audio-minute, same rate as transcript, billed only when used. Frames: $0.00012 per frame. Text overlay (OCR): $0.000225 per frame. Digest (LLM summary, `fields:["digest"]`): $0.00045 per call. Audio briefing (`fields:["audio_digest"]`, a spoken mp3 of the digest): $0.000375 per estimated spoken second (~3-4¢ for a 90s briefing), on top of the digest/transcript it's derived from — billed only when the mp3 is actually produced (a TTS failure degrades to a warning, never charged). Structured video understanding (`fields:["structured"]`, typed JSON — chapters/entities/products/claims/key moments): flat $0.03 per video, billed only when the analysis is actually produced. Comments (`fields:["comments"]`, up to `comments_cap` top-level comments, YouTube only — Reddit's unauthenticated comment API was deprecated on 2026-05-28): flat $0.0045 per call, billed only when comments were actually produced. Comment sentiment (`fields:["comment_sentiment"]`, aggregated audience mood over those comments): flat $0.006 per call on top of the comments call, billed only when produced (needs ≥5 fetched comments). Subtitles (`?format=srt|vtt` / `subtitle_format`): included at no extra cost beyond the transcript (and translation, if requested). Bandwidth (transcript/frames only): ~$6/GB downloaded. Minimum $0.002 per call.
- Watch (channel monitor): subscribing/listing/unsubscribing is free; each delivered `new_video` webhook costs $0.0001 per delivered notification, charged only on successful delivery.
- Credit packs via `/v1/checkout` (Bearer key, Stripe card): `'5'` = $5 for 5,000,000 micros (no bonus); `'20'` = $20 for 21,000,000 micros (+5% bonus); `'100'` = $100 for 112,000,000 micros (+12% bonus).
- Every response includes an exact `cost` block in micro-USD (1,000,000 = $1).
- Top up with USDC via x402 (`POST /v1/topup`, 1 USDC = $1) — no account or human; or Stripe card (`/v1/checkout`, optionally with balance-floor auto-topup via `/v1/billing/auto-topup`).

## Frames spec

- `mode`: `all` | `every_n` (with `n`) | `fps` (with `fps`, ≤60) | `range` (with `from`,`to`,`fps`)
- `format`: `jpg` | `png` | `webp`; `width`: 16–7680 (downscale)

## Translation & subtitles

- [Translate the transcript]: pass a top-level `translate` param (ISO-639-1 code, e.g. `"es"`, `"de"`, `"ja"`) to `/v1/extract` or `/v1/transcript` — 25 languages supported (en, es, de, fr, pt, it, nl, pl, tr, ru, uk, ar, hi, id, vi, th, ja, ko, zh, sv, da, no, fi, cs, ro). Requires `transcript` in `fields` (or `digest`, which pulls it in). Adds `transcript_translated: { lang, text, segments? }` alongside the original `transcript`. Billed per audio-minute like transcription — $0.0015/audio-minute — even for caption-sourced transcripts, since translation is a separate LLM pass.
- [Get subtitles directly]: add `?format=srt` or `?format=vtt` to any extract-family call (`/v1/extract`, `/v1/metadata`, `/v1/transcript`, `/v1/frames`) and get back a ready-to-save `.srt`/`.vtt` file (Content-Type `application/x-subrip` / `text/vtt`) built from the transcript's timed segments — instead of JSON. Uses the translated segments automatically when `translate` was also set. Subtitles are included at no extra cost beyond the transcript (and translation, if requested); requires `transcript` in `fields`. A result with no timed segments (e.g. an old cached entry) returns `409 SUBTITLES_UNAVAILABLE`.
- [MCP equivalent]: `framefetch_extract` takes the same `translate` arg plus `subtitle_format: "srt"|"vtt"`, which adds a top-level `subtitles` string to the JSON result instead of switching content type.
- [Timed segments]: `transcript.segments` (and `transcript_translated.segments`) is an array of `{ start, end, text }` in seconds — from Whisper `verbose_json` or caption cue timings — additive and best-effort (omitted, never `[]`, when the source has no usable timings).
- Useful for EU Accessibility Act (enforced since June 2025) caption requirements: one call gets you a translated, ready-to-publish subtitle file for any supported social-video URL.

## Audio briefings

- [Get a spoken mp3 briefing]: add `"audio_digest"` to `fields` on `/v1/extract` (or `/v1/transcript`) — auto-includes `digest` (and thus `transcript`). Returns `audio_digest: { url, seconds, voice, provider, lang? }`: `url` is a time-limited (24h) signed link to an mp3 rendering of the digest gist; `seconds` (digest text length / 15) is the billing basis, not a measurement of the mp3. Pick a voice with the top-level `voice` param: six OpenAI voices (`alloy` default, `echo`, `fable`, `onyx`, `nova`, `shimmer`) or Groq/PlayAI's `Fritz-PlayAI` (English/Arabic-only fallback, used when no OpenAI key is configured). Billed per estimated spoken second — $0.000375/s (~3-4¢ for a 90s briefing) — ONLY when the mp3 is actually produced; an unavailable TTS provider or unsupported language degrades the field to a warning (never charged, never a hard error), leaving the rest of the extraction intact.
- [Combine with translate — foreign-language audio]: pass `"translate": "<iso-639-1>"` alongside `fields: ["audio_digest"]` and the digest is generated directly in that target language, so the mp3 speaks it too — one call, YouTube link in, e.g. a Spanish spoken briefing out:
  ```
  curl -X POST https://framefetch.net/v1/extract \
    -H "Authorization: Bearer ff_xxx_yyy" -H "Content-Type: application/json" \
    -d '{ "url": "https://www.youtube.com/watch?v=jNQXAC9IVRw", "fields": ["audio_digest"], "translate": "es", "voice": "nova" }'
  ```
- [MCP equivalent]: `framefetch_extract` takes the same `fields: ["audio_digest"]`, `translate`, and `voice` arguments — identical behavior over MCP.

## Structured video understanding

- [Get typed JSON about the video]: add `"structured"` to `fields` on `/v1/extract` (or `/v1/batch`) — auto-includes `transcript`, and also triggers the engine's own keyframe pass over the source video (so it works even on caption-less sources). One vision LLM call (`gpt-4o-mini`, temperature 0) reads the transcript plus up to 12 sampled keyframes and returns `structured: { chapters, entities, products_shown, claims, key_moments, keyframe_mode? }`. Deterministic and cacheable — unlike `audio_digest` it carries no expiring signed URL, so repeat calls for the same URL hit the normal content-hash cache.
- [Fields]: `chapters` — `{ start_sec, end_sec, title, summary }[]`, timespan segments of the video. `entities` — `{ name, type }[]` where `type` is `person`|`org`|`place`|`product`|`other`. `products_shown` — `{ name, evidence }[]` where `evidence` is `visual` (only on-screen), `spoken` (only mentioned), or `both`. `claims` — `{ text, timestamp_sec? }[]`, factual statements made in the video. `key_moments` — `{ timestamp_sec, description }[]`, hooks/reveals/demo steps. `keyframe_mode` — `"scene"` (content-aware cut detection) or `"uniform"` (even-sampling fallback for static/short clips).
- [Example]:
  ```
  curl -X POST https://framefetch.net/v1/extract \
    -H "Authorization: Bearer ff_xxx_yyy" -H "Content-Type: application/json" \
    -d '{ "url": "https://www.youtube.com/watch?v=jNQXAC9IVRw", "fields": ["structured"] }'
  # -> { "structured": {
  #        "chapters": [ { "start_sec": 0, "end_sec": 19, "title": "At the zoo", "summary": "A first-person walk past the elephant enclosure." } ],
  #        "entities": [ { "name": "San Diego Zoo", "type": "place" }, { "name": "elephant", "type": "other" } ],
  #        "products_shown": [],
  #        "claims": [ { "text": "Elephants have really, really long trunks.", "timestamp_sec": 14 } ],
  #        "key_moments": [ { "timestamp_sec": 3, "description": "The narrator points the camera at the elephants." } ],
  #        "keyframe_mode": "uniform"
  #      },
  #      "cost": { "totalMicros": 30000 } }
  ```
- [No file upload — just a URL]: unlike upload-first video-understanding APIs (e.g. TwelveLabs), FrameFetch takes the same bare social-video URL it already accepts for metadata/transcript/frames — no separate ingest step, no polling an indexing job, no hosting the source file yourself.
- [Best-effort, never charged on failure]: like `digest`, any download/VLM/parse failure omits the whole field, adds a warning to the top-level `warnings` array, and is NOT charged — the rest of the extraction stays usable. Worker-internal codes (never an HTTP error, only a `warnings[]` string): `STRUCTURED_UNAVAILABLE` (no vision provider configured), `STRUCTURED_PROVIDER_ERROR` (non-2xx from the provider), `STRUCTURED_PARSE_ERROR` (unparseable model output).
- Billed a flat **$0.03 per video** (price-floor-pinned; measured COGS ~$0.005) — charged only when the analysis is actually produced.
- [MCP equivalent]: `framefetch_extract` takes the same `fields: ["structured"]` — identical behavior over MCP.

## Comments & audience sentiment

- [Get top-level comments]: add `"comments"` to `fields` on `/v1/extract` (or `/v1/batch`) — up to `comments_cap` (optional top-level param, integer 1-200, default 30) top-level comments for the video. Returns `comments: { items: [{ text, author?, like_count?, reply_count? }], total_fetched, cap_applied, sort: "top" }`. INDEPENDENT of `transcript` (no video download). Supported on **YouTube** only (via yt-dlp's info-json comments). Reddit's own public comment source — the unauthenticated thread JSON API — worked the same way until Reddit deprecated unauthenticated access to it on 2026-05-28; it now returns a hard 403, so Reddit degrades exactly like TikTok/Instagram/Pinterest, which never had a reliable public comment source. All four return a warning, never a charge (check `GET /v1/platforms`). Data-minimized by design: `author` is a DISPLAY handle only — never a user id, profile URL, or avatar. Billed a flat **$0.0045 per call** (independent of `comments_cap`), charged only when comments were actually produced; a fetch/parse failure degrades to a warning and is not charged. In batch, `comments` always uses the default cap of 30 (no top-level `comments_cap` in the batch body).
- [Get aggregated audience sentiment]: add `"comment_sentiment"` to `fields` — auto-includes `"comments"` (its input; same YouTube-only availability, since Reddit's comment API was deprecated). ONE Groq chat call (`llama-3.3-70b-versatile`, temperature 0, JSON mode) reads up to 100 comments and judges the mood of the comment section AS A WHOLE — not per-comment scoring. Returns `comment_sentiment: { positive_pct, neutral_pct, negative_pct, summary, top_themes, representative }`: the three percentages sum to exactly 100; `summary` is 2-3 plain English sentences; `top_themes` is up to 5 short recurring themes; `representative: { positive?, negative? }` is one exemplar quoted comment per polarity, when found. Needs at least 5 fetched comments — fewer, or a model/parse failure, omits the whole field with a warning and is NOT charged (never an HTTP error). Billed a flat **$0.006 per call**, on top of the $0.0045 `comments` call it rides on. No competitor offers this rollup.
- [Example]:
  ```
  curl -X POST https://framefetch.net/v1/extract \
    -H "Authorization: Bearer ff_xxx_yyy" -H "Content-Type: application/json" \
    -d '{ "url": "https://www.youtube.com/watch?v=jNQXAC9IVRw", "fields": ["comments", "comment_sentiment"], "comments_cap": 50 }'
  ```
- [MCP equivalent]: `framefetch_extract` takes the same `fields: ["comments", "comment_sentiment"]` plus a top-level `comments_cap` (integer, 1-200) — identical behavior over MCP.

## Trust & transparency

- Legal entity / imprint (TMG §5): https://framefetch.net/impressum
- Transparent, usage-based pricing with a per-call `cost` breakdown returned on every response — no hidden fees.
- Stable, typed errors: `{ "error": { "code", "message", "hint" } }` so agents self-correct.
- Supported hosts only (SSRF-guarded): youtube.com, youtu.be, tiktok.com, instagram.com, pinterest.*, reddit.com, redd.it.
- Privacy: only minimal data stored (email, hashed key, balance, usage); visitor IPs in analytics are stored as a salted hash, never raw.
- Payments are idempotent (no double-charge); x402 settlement verified on-chain.

## Notes

- Request only the fields you need — transcript and frames cost more than metadata.
- Up to 1000 frames per call; downscale with `width` to control cost.