Schemas

The shape

A schema is a JSON Schema, draft 2020-12, written in YAML or JSON. Top level is an object whose properties name the fields; the extractor returns a document that must validate against it — a string in a number field is an error, not a surprise three weeks later:

type: object
properties:
  title:
    type: string
  price:
    type: number
  in_stock:
    type: boolean
  upc:
    type: string
required: [title, price]

Two rules worth knowing:

• External $refs are refused — a schema is untrusted input, and the loader will not follow it to a file or URL. Internal references work: copy the definition under $defs and point at it with $ref: "#/$defs/money".
• Keep it one file. YAML and JSON are interchangeable; the editor on the Scrape screen accepts either.

Drafting one

On the Scrape screen, pick the schema chip and describe the record in one sentence — “book title, price, stock availability and UPC” — then press Draft schema. The LLM writes it fitted to the page you just scraped, and the badge on the draft names the provider, the tokens and the cost. A draft with no page scraped drafts from the description alone; scrape first for a tighter fit.

The draft lands in the editor. Edit freely — the edited badge marks unsaved changes, and a dropdown slip never discards them: New schema only detaches, keeping the text.

x-magpie hints

Each field takes an x-magpie block that steers extraction before the LLM is asked, and steers the selector synthesis that makes later runs free:

properties:
  price:
    type: number
    x-magpie:
      coerce: eur_decimal
      css_hint: ".price-now"
  published:
    type: string
    x-magpie:
      coerce: iso_date
      jsonld_path: "$.datePublished"
  authors:
    type: array
    x-magpie:
      multiple: true
      css_hint: ".byline a"

Hint	Effect
coerce	int, float, eur_decimal, iso_date, bool, trim, regex — normalizes the raw text into the field’s type
css_hint	A CSS selector to try first; when it hits, no LLM call is made for the field
regex	A regular expression the value must match
jsonld_path	Read the field straight off the page’s JSON-LD sidecar, $.a.b.0.c paths
trim	Trim whitespace
multiple	Collect every match, not the first

css_hint and jsonld_path are the budget levers: a schema whose fields resolve from the DOM or JSON-LD extracts with zero LLM calls from the first page. Unknown coerce values fail at load, naming the field.

The library

Save as a name and the schema becomes a plain file in the schemas folder beside config.yaml — see Data and cache for the path per OS. Saved schemas appear in the Scrape screen’s picker and the Crawl screen’s Extracted mode. They are ordinary files: version them, copy them between machines, edit them in any editor.

In Extracted-mode crawls, the first pages bill LLM calls until the selectors stabilize, then extraction runs free — the mechanism is on Extract. A crawl’s schema can also ride the CLI: magpie crawl <url> --schema page.yaml.