Matches species words in original captions; no new image judging. Multiple labels and missing/unidentified captions remain in the denominator. Original captions are retained below.
Perfect upstream reward = 1
Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a red bicycle on a road under a sunny sky with clouds.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 1,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a red bicycle under a blue sky with clouds and a sun.",
"latency": 68.88,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__001.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__001.png"
}
Expand all samples (20)
Qwen__Qwen3.6-27B__pelican_bicycle__000
Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a red bicycle on a road under a sunny sky with clouds.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 1,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a red bicycle under a blue sky with clouds and a sun.",
"latency": 68.88,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__001.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__001.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__002
Upstream outputs, not a Pelican Map ranking · reward 0.838
A cartoon stork is riding a red bicycle while holding an orange basket in its beak, with a blue sky, sun, and clouds in the background.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 2,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a red bicycle while holding an orange basket in its beak, with a blue sky, sun, and clouds in the background.",
"latency": 58.97,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__002.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__002.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__003
Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a red bicycle under a blue sky with clouds.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 5,
"reward": 1.0,
"structure_score": 1.0,
"semantic_score": 1.0,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a red bicycle under a blue sky with clouds and a sun.",
"latency": 72.08,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__005.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__005.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__006
Upstream outputs, not a Pelican Map ranking · reward 0.765
A cartoon stork is riding a red and black bicycle on a green road with a blue sky background.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 6,
"reward": 0.7652777777777777,
"structure_score": 1.0,
"semantic_score": 0.6388888888888888,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a red and black bicycle on a green road with a blue sky background.",
"latency": 116.07,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__006.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__006.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__007
Upstream outputs, not a Pelican Map ranking · reward 0.838
A cartoon stork is riding a red bicycle under a sunny blue sky with clouds and grass below.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 8,
"reward": 0.91875,
"structure_score": 1.0,
"semantic_score": 0.875,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon bird is riding a red bicycle with a basket on its back under a sunny sky with clouds.",
"latency": 123.31,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__008.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__008.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__009
Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a red bicycle under a blue sky with clouds and a sun.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 9,
"reward": 1.0,
"structure_score": 1.0,
"semantic_score": 1.0,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a red bicycle under a blue sky with clouds and a sun.",
"latency": 65.49,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__009.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__009.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__010
Upstream outputs, not a Pelican Map ranking · reward 0.454
A cartoon stork is riding a red bicycle under a sunny sky with clouds and grass below.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 12,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a red bicycle on grass under a blue sky with clouds and a sun.",
"latency": 97.27,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__012.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__012.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__013
Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a red bicycle on a road under a blue sky with clouds and a sun.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 13,
"reward": 1.0,
"structure_score": 1.0,
"semantic_score": 1.0,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a red bicycle on a road under a blue sky with clouds and a sun.",
"latency": 62.11,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__013.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__013.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__014
Upstream outputs, not a Pelican Map ranking · reward 0.964
A cartoon pelican is riding a red bicycle under a blue sky with clouds and a sun.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 14,
"reward": 0.9638888888888889,
"structure_score": 1.0,
"semantic_score": 0.9444444444444444,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a red bicycle under a blue sky with clouds and a sun.",
"latency": 153.52,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__014.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__014.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__015
Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a red bicycle against a blue sky with clouds.
{
"model": "Qwen/Qwen3.6-27B",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 17,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a red bicycle on a green ground with a blue sky and clouds in the background.",
"latency": 114.38,
"svg": "svg/Qwen__Qwen3.6-27B__pelican_bicycle__017.svg",
"png": "png/Qwen__Qwen3.6-27B__pelican_bicycle__017.png"
}
Qwen__Qwen3.6-27B__pelican_bicycle__018
Upstream outputs, not a Pelican Map ranking · reward 0.838
A cartoon goose is riding a red bicycle on grass under a blue sky with clouds.
