Time your own Mac

The benchmark pages quote one machine: a MacBook Pro with an M3 Max and 64 GB of unified memory. Yours is probably different. A base M4 mini, an M2 Air and an M3 Ultra Studio all run the same work at different speeds, and the published numbers describe only one Mac.

This skill times yours. Your agent identifies the Mac, checks which models are on disk, then runs the same probes the published figures come from: one line spoken, six lines spoken in a single call, a music bed written from a brief, and a video clip made at 704 by 1280. The prompts are the same word for word, and so is the geometry. It times the MCP tools an agent calls, so the numbers are what you actually wait for during a render.

Everything runs on your machine, against models already on your disk. It writes a folder with a run file per suite, the raw trials behind every median, the media each probe produced, and a report that puts your machine beside the reference one. There is no upload step, no account and no telemetry. The only thing it reads from the internet is our published pages, so the comparison uses today's reference numbers.

If you want your configuration counted, use the app's own benchmark, which has a Send to COEY control. Sent results appear as medians on what other Macs reported. This skill is different: it runs wider probes, keeps every trial, and never sends anything.

Quick depth takes minutes for voice and music, and about half an hour for video on a fast Mac. Full depth on video takes several hours: it makes twenty clips, one at a time, and one clip takes over seven minutes on the reference machine. Your agent gives you an estimate before it starts.

A suite whose model is not installed is skipped and recorded as absent. LTX 2.5 is a large download, so a Mac without it still produces a voice and music dataset.

Install it

Copy or download SKILL.md above, and save it as ~/.claude/skills/groundslate-benchmark/SKILL.md.

This one is for Claude Code only. It reads the machine with shell commands, measures durations with the ffprobe that ships inside GroundSlate, and writes JSON files to a folder you name, so it needs a terminal and a filesystem, not a chat window.

The skill

---
name: groundslate-benchmark
description: Time the models GroundSlate runs on this Mac and write a comparable dataset. Runs the published probes verbatim, one spoken line, six lines in a single call, a music bed and a video clip, through synthesize_speech, generate_music and generate_video, times each one from the queue's own event feed, and writes run files, raw trials, the media produced and a report comparing this machine to the reference Mac. Use when somebody wants to benchmark their Mac, measure local model speed, or contribute timings.
---

# Benchmark this Mac

You are taking a measurement, not producing media. Everything below exists to
stop a number being published that measures something other than what it
claims to.

**THIS IS A LOCAL BENCHMARK.** It runs on this machine, against models already
on this disk, and it ends with a folder here. There is no upload step and you
must not invent one: no API call carrying results, no attachment, no email, no
gist, no pastebin, no repository. The only thing you fetch is the published
reference pages named further down, and that is a read.

The human gets the folder. What they do with it is theirs to decide, and you do
not raise the subject unless they do.

## What you produce

One folder, named `groundslate-benchmark-<hardware-id>-<date>`:

```
hardware.json            one object describing this Mac
runs/voice/<run-id>.json
runs/sound/<run-id>.json
runs/video/<run-id>.json
raw/<suite>-<probe>.jsonl    every trial, one object per line
outputs/                     the media each probe made
REPORT.md                    what was measured, and how this Mac compares
```

`raw/` is not optional and it is not a debug artifact. A median with no trials
behind it cannot be checked by anybody, including you. Write every trial,
including the ones you discard, with the reason for discarding it.

## Ask three things, then stop asking

1. **Which suites.** Voice, music, video, or all three.
2. **Quick or full.** Quick is voice n=5 per probe, music n=7, video n=3. Full
   is voice n=10, music n=10, video n=20.
3. **Where the folder goes.**

Then work. Do not ask again mid run.

## Preflight, before you promise a duration

Do all of this before the first timed call.

- `hello` and `get_context`. If GroundSlate is not answering, say so and stop.
- `list_profiles`, then confirm which profile with the human if there is more
  than one. Pass `profile=` on every call after that. The benchmark writes real
  media into the profile it runs in.
- `list_models`. This says which weights are on disk. A suite whose model is
  missing is SKIPPED and recorded as absent. Never download a model to make a
  benchmark run; a 44 GB download is the human's decision, not yours.
- `voice_status`. Read the tier and whether that tier's weights are present.
  RECORD THE TIER. The published voice figures are the 1.7B tier. The tier is
  one machine-wide setting in the app, not a per-call argument, so if the human
  wants a like for like comparison they change it in GroundSlate and you say so
  rather than trying to set it yourself.
- `memory_status` and `storage_summary`. Report free memory and free disk
  before you start. Video at full depth writes twenty clips.
- `list_jobs`. **THE QUEUE MUST BE EMPTY.** If anything is running or queued,
  say what it is and wait, or ask the human to let it finish. If a watch folder
  is feeding the queue, tell them to pause watching in the app. Do not pause it
  yourself and do not cancel somebody else's job.
- `list_jobs` again for `throughput`. Where it carries a median for the kind
  you are about to run, use it to estimate the wall clock and SAY THE ESTIMATE
  OUT LOUD before you start. "Twenty clips at about eight minutes each is
  around two and a half hours" is the sentence that stops somebody starting a
  full video run at four in the afternoon. `throughput` is keyed inconsistently:
  some rows are tool names and some are the display label the queue writes, so a
  lookup for `generate_video` can come back empty while the history sits under
  "Generating video". Read the whole map before you conclude there is no history.

Every one of these calls takes `profile=`, and the generation tools also want
`agent=` and `label=`. Use one agent string for the whole run and a label that
names the probe and the trial, so the queue and the library both show what this
was.

Then tell the human, in one message: which suites will run, which are skipped
and why, the estimate, and roughly how much disk the outputs will take.

## Identify the Mac, and take only what you need

```bash
system_profiler SPHardwareDataType | grep -E 'Model Name|Chip|^ *Memory'
sysctl -n hw.memsize
sw_vers -productVersion
```

Read the model name, the chip and the memory. **The profiler output also
carries a serial number, a model number and a hardware UUID. Those never go in
the dataset, never go in the report, and never go in your message.** The file is
meant to be shareable, so it holds a machine description and nothing that
identifies one specific machine.

Memory is a single number in whole gigabytes, from `hw.memsize` divided by
1024 three times. It is the configured unified memory, never a band.

The hardware id is the model name, then the chip with `Apple ` removed, then
the memory, each lowercased with every run of non-alphanumeric characters
turned into a single hyphen: `MacBook Pro` plus `Apple M3 Max` plus `64` is
`macbook-pro-m3-max-64`.

`hardware.json` is one object:

```json
{
  "id": "mac-mini-m4-24",
  "product": "Mac mini",
  "chip": "Apple M4",
  "memoryGb": 24,
  "unifiedMemory": true,
  "kind": "desktop",
  "year": 2024,
  "placeholder": false
}
```

`kind` is `laptop` or `desktop`. `year` only if you are sure; leave it out
rather than guess. Never set `reference`.

## The shapes are fixed, and that is the whole point

A benchmark is comparable or it is anecdote. Do not improve these prompts, do
not shorten them, do not fix their punctuation, and do not swap in the human's
own material. They are the exact strings the published figures were taken with,
so a run that changes one is measuring something else.

**Voice, the `line` probe.** One `synthesize_speech` call, `text` set to:

```
The footage never left the folder it was already in. The faces were matched on this desk, and so was the voice you are listening to.
```

**Voice, the `batch` probe.** One `synthesize_speech` call with `lines` set to
these six, in this order:

```
The footage never left the folder it was already in.
The faces were matched on this desk, and so was the voice you are listening to.
No account to sign into.
No meter running.
No terms that could quietly change next quarter.
All of it came from one prompt, typed into Claude, by somebody who then went to make coffee.
```

Both use `temperature=0` and leave `speed` at 1.0. Anything other than 1.0 is
refused anyway.

**The voice itself.** `list_voices` first. If a voice named `bench-voice`
already exists, use it. Otherwise mint one once with
`create_voice(name="bench-voice", speaker="ryan")` and wait for it to finish
BEFORE the first timed call. Minting is a separate one-time cost and folding it
into a synthesis figure is the classic way to publish a wrong number. Do not
time the mint, do not report it as a probe, and never re-mint between trials:
the mint cache would hand you a file copy and you would publish that as the
cost of making a voice.

**Music, the `bed` probe.** One `generate_music` call, `seconds=64`,
`model="medium"` if the medium weights are installed and `model="small"` if
only the small ones are, `prompt` set to:

```
A cold open bed for a technology film, continuous and even from beginning to end. A muted electric piano figure repeating on a steady pulse at a relaxed tempo near one hundred and ten, warm analog synth bass holding underneath the whole way, soft brushed electronic percussion keeping quiet time. The level stays constant throughout with no silence, no dropout, no breakdown and no sudden drop. Restrained and curious rather than triumphant, deliberately understated so a voice sits clearly on top of it. Instrumental only.
```

Use a different `seed` per trial and record every one. Sound generation is
bit exact per seed, so the same seed twice measures a cache as much as a model.

**Video, the `clip` probe.** One `generate_video` call per trial, `width=704`,
`height=1280`, `num_frames=121`, `fps=24`, `prompt` set to:

```
magenta ink dropped into still water, blooming and folding against black, gold particles catching light as they sink
```

Use a different `seed` per trial and record every one. `num_frames` must be a
multiple of eight plus one, which 121 is; anything else is refused before a
queue slot is taken.

## Time the work, not the wait

**Never use the jobs ledger's `elapsed`, and never time a job with your own
clock.** `elapsed` starts when a job is accepted rather than when it starts
work, so under a busy queue it counts waiting as working and roughly doubles.
Your own clock measures your round trips as well as the render.

The instrument is `job_events`. Every job emits `queued`, then `started`, then
`done`, each with a millisecond timestamp `t`.

**The measured figure is `t(done) minus t(started)` for that job id.**

So, per trial:

1. `job_events(after=0)` and keep `latest` as your cursor.
2. Submit the one call. Keep the `job_id` it returns.
3. `job_wait(job_id)`. It blocks until the job is terminal or about fifty
   seconds pass, whichever is first. A timeout is not a failure. Branch on
   `state`, never on the call returning, and call again while the state is
   `queued` or `running`.
4. When `state` is `done`, walk `job_events` forward from your cursor and pull
   the `started` and `done` rows for that job id. Their difference is the
   trial's wall clock.
5. `probe` the output path, or run the bundled ffprobe, for the seconds of
   media the call produced.

**Submit one job at a time and let it reach `done` before you submit the next.**
Generation is serialised machine wide by a single guard, so several jobs in
flight do not run alongside each other, they queue, and a job can sit in
`started` while blocked on that guard. Serial submission is what makes
`started` to `done` pure work rather than work plus somebody else's render.

`state` of `done` with a non-empty `error`, or a `result.error`, is a REFUSAL.
Record it as a failed trial with the message, do not retry silently, and do not
let it into the median.

## Measure the media, never estimate it

The realtime factor and the cost per second are the useful figures and both
divide by the seconds the call actually produced. An estimate in that
denominator makes the ratio a measurement of the estimate.

```bash
FFPROBE=/Applications/GroundSlate.app/Contents/MacOS/ffprobe
[ -x "$FFPROBE" ] || FFPROBE=ffprobe
"$FFPROBE" -v error -show_entries format=duration -of default=nw=1:nk=1 "<path>"
```

For video also confirm the geometry came back as asked, because a clip that
quietly came back shorter invalidates the trial:

```bash
"$FFPROBE" -v error -select_streams v:0 -show_entries stream=width,height,nb_frames -of default=nw=1 "<path>"
```

704 by 1280 and 121 frames, or the trial is discarded with the reason written
into `raw/`.

## Warm and cold are different measurements

The first call of a session for any model reads weights off disk. Record
`warm: false` on the first trial of each suite and `warm: true` on the rest,
and never mix them silently into one median without saying so in `REPORT.md`.

Video at this size is the exception and it is worth knowing: at 121 frames the
weights are not resident, so they are freed after every render and every clip
reloads them. There is no warm clip to quote. If your video trials split into a
fast group and a slow one, something else was running.

## Write the run files

One per suite, at `runs/<suite>/<run-id>.json`, where the run id is the date
then the hardware id: `2026-09-14-mac-mini-m4-24`.

```json
{
  "id": "2026-09-14-mac-mini-m4-24",
  "suite": "voice",
  "hardware": "mac-mini-m4-24",
  "date": "2026-09-14",
  "placeholder": false,
  "methodology": {
    "engine": "mlx",
    "defaultRepo": "mlx-community/Qwen3-TTS-12Hz-1.7B-Base-4bit",
    "modelsLocked": true,
    "rows": [
      { "term": "Path", "def": "The MCP synthesize_speech tool, so the figure covers the queue, the machine-wide generation guard and the file landing in the library, not a library call in isolation." },
      { "term": "What was timed", "def": "The queue's own started to done stamps for each job, taken from job_events. One job in flight at a time, with an empty queue before each submission." },
      { "term": "Tier", "def": "1.7B, as voice_status reported it." },
      { "term": "Voice", "def": "bench-voice, minted once from the ryan preset before the first timed call. Minting is not in these numbers." }
    ]
  },
  "measurements": [
    {
      "model": "qwen3-tts-1-7b",
      "probes": [
        { "probe": "line", "medianS": 0, "meanS": 0, "minS": 0, "maxS": 0, "producedS": 0, "n": 5 },
        { "probe": "batch", "medianS": 0, "meanS": 0, "minS": 0, "maxS": 0, "producedS": 0, "n": 5 }
      ]
    }
  ],
  "findings": []
}
```

Rules for that file:

- Seconds to two decimals. The median is the headline; also carry `meanS`,
  `minS`, `maxS` and `n`, because a median with no spread hides a machine that
  was thermally throttling halfway through.
- `producedS` is the median of the measured durations, not the nominal length.
- Probe ids are fixed: voice is `line` and `batch`, music is `bed`, video is
  `clip`. Model ids are `qwen3-tts-1-7b`, `stable-audio-3-medium` and `ltx-2-5`.
  If the machine ran a tier or a checkpoint that is not one of those, use a
  clearly derived id, say so in `REPORT.md`, and carry the repo string in
  `defaultRepo` so there is no ambiguity about what ran.
- **`findings` stays empty.** A finding is an editorial claim on a published
  page. Anything you noticed goes in `REPORT.md` under Observations, in your own
  words, with the trials that support it.
- A suite you skipped gets no run file. Absence is recorded in `REPORT.md`, with
  the reason. Never write a run of zeroes and never write a placeholder run.

## Write the raw trials

`raw/<suite>-<probe>.jsonl`, one object per line, one line per trial including
failures:

```json
{"trial": 1, "job_id": "...", "started_t": 0, "done_t": 0, "wall_s": 0, "produced_s": 0, "warm": false, "seed": 0, "ok": true, "note": ""}
```

A discarded trial is `"ok": false` with the reason in `note`. It stays in the
file. Removing it is how a dataset stops being checkable.

## Keep the media, and keep it findable

The benchmark makes real files in the human's library. That is a genuine cost
and they should be told about it up front, not discover it later.

- `create_project` a project named `Benchmark <date>` and pass its
  `project_id` to `generate_video`, so the clips are filed somewhere obvious.
- Pass `out_path` on `synthesize_speech` and `generate_music` so the audio
  lands in the folder you are building.
- Copy or move each output into `outputs/` and write `outputs/outputs.json`:
  the probe, the trial, the file name, its duration, its seed and the wall
  clock for that call.
- `add_project_note` on the project saying what the run was and where the
  folder is, so it is not a mystery in a month.

Do not delete anything from the library, do not empty the scratch, and do not
cancel a job you did not submit.

## Vision is not in this run

The published vision figures are taken at the model server, with one image
fixed at 768 pixels on the long edge, which the shipped app does not expose. A
vision figure timed through the app would be measuring ingest, a different and
larger thing, and it would sit in a table under a heading that promised the
other measurement. So vision is out of scope here. Say that plainly if the
human asks, rather than substituting a number.

## Compare against the reference Mac

The reference numbers are published as markdown, so fetch them rather than
quoting them from memory:

```
https://coey.com/resources/groundslate/benchmarks/voice.md
https://coey.com/resources/groundslate/benchmarks/sound.md
https://coey.com/resources/groundslate/benchmarks/video.md
https://coey.com/resources/groundslate/benchmarks/machines/macbook-pro-m3-max-64.md
```

Compare probe against the same probe, and say the ratio in plain words: "a
line takes 1.4 times as long here" rather than a table of percentages. Where a
tier, a checkpoint or a model differs from the reference run, the comparison is
not like for like and you say so instead of drawing it as a gap.

If the pages cannot be reached, write the report without the comparison and say
why. Never fill it in from memory.

## Write the report

`REPORT.md`, for a person, in this order:

1. The machine, in one line: computer, chip, memory, macOS.
2. What ran, what was skipped, and why.
3. One table per suite: the probe, the median, the spread, `n`, and the
   produced seconds.
4. This Mac against the reference Mac, in sentences.
5. Observations. Anything you noticed and can point at a trial for. A machine
   that got slower through a long video run belongs here, with the numbers.
6. What would make the dataset better: a longer run, a tier change, a suite
   whose model is not installed.

No em-dashes anywhere in it. A comma, a colon, or two sentences.

## Report like an instrument

- Say the estimate before you start and hold to it, or say why it moved.
- Between trials, keep the human informed without making them watch a bar.
- Never report a number you did not take. A trial that failed is a failed
  trial, and a suite that did not run did not run.
- End with where the folder is, what is in it, and the one sentence that
  answers what they asked: how this Mac compares.
- Say where the folder is and leave it there. Do not upload it, do not attach
  it anywhere, do not put it in a repository, and do not offer to.
- ONLY IF THEY ASK how to get their machine onto the published pages: that is
  the app's own benchmark, which has a Send to COEY control. This skill never
  sends anything. If they do not ask, you do not mention it.