# Canine Demo — Rater Briefing

Thank you for helping evaluate the canine vocalization demo. **You are 1 of 2
raters.** Work **independently** — do not compare scores with the other rater
until you are both done. Expect ~45–60 minutes.

## What you are given

- **`clips/`** — 30 short dog video clips (`clip01.mp4` … `clip30.mp4`).
- **The system's predictions** for each clip. Two easy ways to see them:
  - open the **evaluation page** in the demo (`/evaluation`) — video + prediction
    side by side, and you can type scores right there and download your sheet; **or**
  - open **`predictions.csv`** next to the clips.
- **`sheet.csv`** — your scoring sheet. Your annotator letter (A or B) is already
  filled in on every row. You only type the five score columns.

For each clip the system predicts four things:

| Field | What it is |
|-------|------------|
| **environment** | where the dog seems to be (e.g. *Indoor Residential Spaces*, *Outdoor General Spaces*) |
| **activity** | what the dog/scene is doing (e.g. *Movement & Locomotion*, *Passive Actions*) |
| **intention** | the dog's apparent state/intent (e.g. *Calm & Relaxed*, *Alert*, *Playful & Excited*) |
| **interpretation** | one plain-language sentence summarizing the bark's likely meaning |

## How to score

**The video is the ground truth.** Watch it (with sound), then judge each
prediction against what you see and hear.

For **environment**, **activity**, **intention** — score each **0 / 1 / 2**:

- **2 = Correct** — clearly matches the video.
- **1 = Plausible** — reasonable and not contradicted, but you can't fully confirm it.
- **0 = Incorrect** — the video clearly contradicts it.

For the **interpretation sentence** — score **1–5**:

- **1** = nonsensical / wrong, … **5** = accurate *and* informative.

Rules of thumb:
- **Score every clip.** Don't leave blanks.
- When you're torn between two attribute scores, **use 1**.
- Judge only what the clip actually shows; don't score the dog's cuteness or the
  video quality.
- The `notes` column is optional — a few words on why, if you like.

## Filling in your sheet

Type into **your** `sheet.csv`, one row per clip. Columns:

```
clip_id,annotator,environment,activity,intention,interpretation,notes
```

Example row:

```
clip07,A,2,1,2,4,cat visible; movement brief
```

Save the file (keep it as CSV) and send it back. That's it — thank you!
