Pelican MapAN AI FIELD GUIDE
HF / OPENENV · REFERENCE ONLY

claude-opus-5

Upstream outputs, not a Pelican Map ranking

Upstream outputs, not a Pelican Map ranking

An independent HuggingFace / OpenEnv scoring reference, excluded from the main timeline, collection and case total.

Rows: 20 · Viewable images: 20 · Upstream mean reward: 0.868194

Species words in upstream blind captions

bird: 2/20 · duck: 5/20 · goose: 6/20 · pelican: 7/20

Matches species words in original captions; no new image judging. Multiple labels and missing/unidentified captions remain in the denominator. Original captions are retained below.

claude-opus-5__pelican_bicycle__004
Perfect upstream reward = 1

Upstream outputs, not a Pelican Map ranking · reward 1.000

A cartoon pelican is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 4,
  "reward": 1.0,
  "structure_score": 1.0,
  "semantic_score": 1.0,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a red bicycle.",
  "latency": 40.06,
  "svg": "svg/claude-opus-5__pelican_bicycle__004.svg",
  "png": "png/claude-opus-5__pelican_bicycle__004.png"
}
claude-opus-5__pelican_bicycle__000
Sample not called a pelican by the blind caption

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon goose is riding a red bicycle on a green ground with a blue sky and clouds in the background.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 0,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon goose is riding a red bicycle on a green ground with a blue sky and clouds in the background.",
  "latency": 44.02,
  "svg": "svg/claude-opus-5__pelican_bicycle__000.svg",
  "png": "png/claude-opus-5__pelican_bicycle__000.png"
}
Expand all samples (20)
claude-opus-5__pelican_bicycle__000
claude-opus-5__pelican_bicycle__000

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon goose is riding a red bicycle on a green ground with a blue sky and clouds in the background.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 0,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon goose is riding a red bicycle on a green ground with a blue sky and clouds in the background.",
  "latency": 44.02,
  "svg": "svg/claude-opus-5__pelican_bicycle__000.svg",
  "png": "png/claude-opus-5__pelican_bicycle__000.png"
}
claude-opus-5__pelican_bicycle__001
claude-opus-5__pelican_bicycle__001

Upstream outputs, not a Pelican Map ranking · reward 0.765

A cartoon goose with an orange beak and legs is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 1,
  "reward": 0.7652777777777777,
  "structure_score": 1.0,
  "semantic_score": 0.6388888888888888,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon goose with an orange beak and legs is riding a red bicycle.",
  "latency": 27.42,
  "svg": "svg/claude-opus-5__pelican_bicycle__001.svg",
  "png": "png/claude-opus-5__pelican_bicycle__001.png"
}
claude-opus-5__pelican_bicycle__002
claude-opus-5__pelican_bicycle__002

Upstream outputs, not a Pelican Map ranking · reward 0.494

A cartoon pelican is riding a red and yellow bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 2,
  "reward": 0.4944444444444444,
  "structure_score": 1.0,
  "semantic_score": 0.2222222222222222,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a red and yellow bicycle.",
  "latency": 43.9,
  "svg": "svg/claude-opus-5__pelican_bicycle__002.svg",
  "png": "png/claude-opus-5__pelican_bicycle__002.png"
}
claude-opus-5__pelican_bicycle__003
claude-opus-5__pelican_bicycle__003

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon duck wearing a red scarf is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 3,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon duck wearing a red scarf is riding a red bicycle.",
  "latency": 28.36,
  "svg": "svg/claude-opus-5__pelican_bicycle__003.svg",
  "png": "png/claude-opus-5__pelican_bicycle__003.png"
}
claude-opus-5__pelican_bicycle__004
claude-opus-5__pelican_bicycle__004

Upstream outputs, not a Pelican Map ranking · reward 1.000

A cartoon pelican is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 4,
  "reward": 1.0,
  "structure_score": 1.0,
  "semantic_score": 1.0,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a red bicycle.",
  "latency": 40.06,
  "svg": "svg/claude-opus-5__pelican_bicycle__004.svg",
  "png": "png/claude-opus-5__pelican_bicycle__004.png"
}
claude-opus-5__pelican_bicycle__005
claude-opus-5__pelican_bicycle__005

Upstream outputs, not a Pelican Map ranking · reward 0.801

A cartoon goose is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 5,
  "reward": 0.8013888888888889,
  "structure_score": 1.0,
  "semantic_score": 0.6944444444444444,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon goose is riding a red bicycle.",
  "latency": 47.29,
  "svg": "svg/claude-opus-5__pelican_bicycle__005.svg",
  "png": "png/claude-opus-5__pelican_bicycle__005.png"
}
claude-opus-5__pelican_bicycle__006
claude-opus-5__pelican_bicycle__006

Upstream outputs, not a Pelican Map ranking · reward 1.000

A cartoon pelican is riding a bicycle with its wings outstretched, and there is a yellow circle resembling the sun in the background.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 6,
  "reward": 1.0,
  "structure_score": 1.0,
  "semantic_score": 1.0,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a bicycle with its wings outstretched, and there is a yellow circle resembling the sun in the background.",
  "latency": 51.91,
  "svg": "svg/claude-opus-5__pelican_bicycle__006.svg",
  "png": "png/claude-opus-5__pelican_bicycle__006.png"
}
claude-opus-5__pelican_bicycle__007
claude-opus-5__pelican_bicycle__007

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon goose is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 7,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon goose is riding a red bicycle.",
  "latency": 43.12,
  "svg": "svg/claude-opus-5__pelican_bicycle__007.svg",
  "png": "png/claude-opus-5__pelican_bicycle__007.png"
}
claude-opus-5__pelican_bicycle__008
claude-opus-5__pelican_bicycle__008

Upstream outputs, not a Pelican Map ranking · reward 0.883

A cartoon bird with an orange beak is riding a red bicycle against a light blue background.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 8,
  "reward": 0.8826388888888889,
  "structure_score": 1.0,
  "semantic_score": 0.8194444444444444,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon bird with an orange beak is riding a red bicycle against a light blue background.",
  "latency": 25.03,
  "svg": "svg/claude-opus-5__pelican_bicycle__008.svg",
  "png": "png/claude-opus-5__pelican_bicycle__008.png"
}
claude-opus-5__pelican_bicycle__009
claude-opus-5__pelican_bicycle__009

Upstream outputs, not a Pelican Map ranking · reward 1.000

A cartoon pelican is riding a bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 9,
  "reward": 1.0,
  "structure_score": 1.0,
  "semantic_score": 1.0,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a bicycle.",
  "latency": 36.72,
  "svg": "svg/claude-opus-5__pelican_bicycle__009.svg",
  "png": "png/claude-opus-5__pelican_bicycle__009.png"
}
claude-opus-5__pelican_bicycle__010
claude-opus-5__pelican_bicycle__010

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon goose is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 10,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon goose is riding a red bicycle.",
  "latency": 35.34,
  "svg": "svg/claude-opus-5__pelican_bicycle__010.svg",
  "png": "png/claude-opus-5__pelican_bicycle__010.png"
}
claude-opus-5__pelican_bicycle__011
claude-opus-5__pelican_bicycle__011

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon duck is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 11,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon duck is riding a red bicycle.",
  "latency": 27.42,
  "svg": "svg/claude-opus-5__pelican_bicycle__011.svg",
  "png": "png/claude-opus-5__pelican_bicycle__011.png"
}
claude-opus-5__pelican_bicycle__012
claude-opus-5__pelican_bicycle__012

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon goose with an orange beak is riding a bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 12,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon goose with an orange beak is riding a bicycle.",
  "latency": 23.64,
  "svg": "svg/claude-opus-5__pelican_bicycle__012.svg",
  "png": "png/claude-opus-5__pelican_bicycle__012.png"
}
claude-opus-5__pelican_bicycle__013
claude-opus-5__pelican_bicycle__013

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon duck is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 13,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon duck is riding a red bicycle.",
  "latency": 36.71,
  "svg": "svg/claude-opus-5__pelican_bicycle__013.svg",
  "png": "png/claude-opus-5__pelican_bicycle__013.png"
}
claude-opus-5__pelican_bicycle__014
claude-opus-5__pelican_bicycle__014

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon duck is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 14,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon duck is riding a red bicycle.",
  "latency": 36.04,
  "svg": "svg/claude-opus-5__pelican_bicycle__014.svg",
  "png": "png/claude-opus-5__pelican_bicycle__014.png"
}
claude-opus-5__pelican_bicycle__015
claude-opus-5__pelican_bicycle__015

Upstream outputs, not a Pelican Map ranking · reward 1.000

A cartoon pelican is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 15,
  "reward": 1.0,
  "structure_score": 1.0,
  "semantic_score": 1.0,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a red bicycle.",
  "latency": 35.08,
  "svg": "svg/claude-opus-5__pelican_bicycle__015.svg",
  "png": "png/claude-opus-5__pelican_bicycle__015.png"
}
claude-opus-5__pelican_bicycle__016
claude-opus-5__pelican_bicycle__016

Upstream outputs, not a Pelican Map ranking · reward 1.000

A cartoon pelican is riding a bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 16,
  "reward": 1.0,
  "structure_score": 1.0,
  "semantic_score": 1.0,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a bicycle.",
  "latency": 33.39,
  "svg": "svg/claude-opus-5__pelican_bicycle__016.svg",
  "png": "png/claude-opus-5__pelican_bicycle__016.png"
}
claude-opus-5__pelican_bicycle__017
claude-opus-5__pelican_bicycle__017

Upstream outputs, not a Pelican Map ranking · reward 0.883

A cartoon bird with an orange beak is riding a bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 17,
  "reward": 0.8826388888888889,
  "structure_score": 1.0,
  "semantic_score": 0.8194444444444444,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon bird with an orange beak is riding a bicycle.",
  "latency": 37.33,
  "svg": "svg/claude-opus-5__pelican_bicycle__017.svg",
  "png": "png/claude-opus-5__pelican_bicycle__017.png"
}
claude-opus-5__pelican_bicycle__018
claude-opus-5__pelican_bicycle__018

Upstream outputs, not a Pelican Map ranking · reward 1.000

A cartoon pelican is riding a bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 18,
  "reward": 1.0,
  "structure_score": 1.0,
  "semantic_score": 1.0,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon pelican is riding a bicycle.",
  "latency": 49.62,
  "svg": "svg/claude-opus-5__pelican_bicycle__018.svg",
  "png": "png/claude-opus-5__pelican_bicycle__018.png"
}
claude-opus-5__pelican_bicycle__019
claude-opus-5__pelican_bicycle__019

Upstream outputs, not a Pelican Map ranking · reward 0.838

A cartoon duck is riding a red bicycle.

Upstream SVG ↗ · Local SVG

Original fields and blind caption
{
  "model": "claude-opus-5",
  "access": "closed",
  "provider": "Anthropic",
  "task": "pelican_bicycle",
  "sample": 19,
  "reward": 0.8375,
  "structure_score": 1.0,
  "semantic_score": 0.75,
  "gate_passed": true,
  "violations": [],
  "judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
  "judge_blind_caption": "A cartoon duck is riding a red bicycle.",
  "latency": 43.85,
  "svg": "svg/claude-opus-5__pelican_bicycle__019.svg",
  "png": "png/claude-opus-5__pelican_bicycle__019.png"
}

← Back to scoring reference