nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__000 Upstream outputs, not a Pelican Map ranking · reward 0.810
A bird with an orange beak is riding a bicycle with two wheels and handlebars.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 0,
"reward": 0.8104166666666666,
"structure_score": 1.0,
"semantic_score": 0.7083333333333333,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A bird with an orange beak is riding a bicycle with two wheels and handlebars.",
"latency": 268.67,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__000.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__000.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__001 Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a bicycle on a line under a blue sky with a cloud.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 1,
"reward": 1.0,
"structure_score": 1.0,
"semantic_score": 1.0,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a bicycle on a line under a blue sky with a cloud.",
"latency": 31.8,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__001.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__001.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__002 Upstream outputs, not a Pelican Map ranking · reward 0.838
A cartoon goose is riding a red bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 2,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon goose is riding a red bicycle.",
"latency": 68.4,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__002.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__002.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__003 Upstream outputs, not a Pelican Map ranking · reward 0.465
A cartoon bird with an orange beak and feet is riding a bicycle with two large wheels.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 3,
"reward": 0.46510416666666665,
"structure_score": 1.0,
"semantic_score": 0.17708333333333331,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon bird with an orange beak and feet is riding a bicycle with two large wheels.",
"latency": 35.8,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__003.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__003.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__004 Upstream outputs, not a Pelican Map ranking · reward 0.847
The image shows a cartoon-style drawing of a white bird with an orange beak riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 4,
"reward": 0.8465277777777778,
"structure_score": 1.0,
"semantic_score": 0.7638888888888888,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "The image shows a cartoon-style drawing of a white bird with an orange beak riding a bicycle.",
"latency": 69.25,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__004.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__004.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__005 Upstream outputs, not a Pelican Map ranking · reward 0.765
A stick figure with an orange beak is riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 5,
"reward": 0.7652777777777777,
"structure_score": 1.0,
"semantic_score": 0.6388888888888888,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A stick figure with an orange beak is riding a bicycle.",
"latency": 28.18,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__005.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__005.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__006 Upstream outputs, not a Pelican Map ranking · reward 0.765
A cartoon stork is riding a bicycle under a sunny sky with clouds.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 6,
"reward": 0.7652777777777777,
"structure_score": 1.0,
"semantic_score": 0.6388888888888888,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a bicycle under a sunny sky with clouds.",
"latency": 24.75,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__006.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__006.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__007 Upstream outputs, not a Pelican Map ranking · reward 0.847
A bird with an orange beak is riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 7,
"reward": 0.8465277777777778,
"structure_score": 1.0,
"semantic_score": 0.7638888888888888,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A bird with an orange beak is riding a bicycle.",
"latency": 101.63,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__007.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__007.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__008 Upstream outputs, not a Pelican Map ranking · reward 0.801
A cartoon stork is riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 8,
"reward": 0.8013888888888889,
"structure_score": 1.0,
"semantic_score": 0.6944444444444444,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon stork is riding a bicycle.",
"latency": 12.76,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__008.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__008.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__009 Upstream outputs, not a Pelican Map ranking · reward 0.950
A cartoon pelican is riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 9,
"reward": 0.95,
"structure_score": 0.8571428571428571,
"semantic_score": 1.0,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a bicycle.",
"latency": 10.91,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__009.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__009.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__010 Upstream outputs, not a Pelican Map ranking · reward 0.494
The image shows a bicycle with a pelican perched on its frame.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 10,
"reward": 0.4944444444444444,
"structure_score": 1.0,
"semantic_score": 0.2222222222222222,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "The image shows a bicycle with a pelican perched on its frame.",
"latency": 16.29,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__010.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__010.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__011 Upstream outputs, not a Pelican Map ranking · reward 0.883
A bird with an orange beak is riding a bicycle while holding a paper airplane in its beak.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 11,
"reward": 0.8826388888888889,
"structure_score": 1.0,
"semantic_score": 0.8194444444444444,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A bird with an orange beak is riding a bicycle while holding a paper airplane in its beak.",
"latency": 22.6,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__011.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__011.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__012 Upstream outputs, not a Pelican Map ranking · reward 0.838
The image depicts a stylized drawing of a bicycle with a large, rounded front and two wheels, but the rider is not clearly identifiable.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 12,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "The image depicts a stylized drawing of a bicycle with a large, rounded front and two wheels, but the rider is not clearly identifiable.",
"latency": 10.56,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__012.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__012.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__013 Upstream outputs, not a Pelican Map ranking · reward 0.914
A cartoon pelican is riding a bicycle with its wings spread out.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 13,
"reward": 0.913888888888889,
"structure_score": 0.8571428571428571,
"semantic_score": 0.9444444444444444,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a bicycle with its wings spread out.",
"latency": 129.18,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__013.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__013.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__014 Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 14,
"reward": 1.0,
"structure_score": 1.0,
"semantic_score": 1.0,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a bicycle.",
"latency": 21.0,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__014.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__014.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__015 Upstream outputs, not a Pelican Map ranking · reward 0.456
A bicycle with orange arrows pointing at its pedals and handlebars, and a bird-like figure above it.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 15,
"reward": 0.4560763888888889,
"structure_score": 1.0,
"semantic_score": 0.16319444444444445,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A bicycle with orange arrows pointing at its pedals and handlebars, and a bird-like figure above it.",
"latency": 296.77,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__015.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__015.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__016 Upstream outputs, not a Pelican Map ranking · reward 0.838
A cartoon duck is riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 16,
"reward": 0.8375,
"structure_score": 1.0,
"semantic_score": 0.75,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon duck is riding a bicycle.",
"latency": 16.48,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__016.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__016.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__017 Upstream outputs, not a Pelican Map ranking · reward 0.964
A cartoon pelican is riding a bicycle with large wheels on a grassy area against a light blue background.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 17,
"reward": 0.9638888888888889,
"structure_score": 1.0,
"semantic_score": 0.9444444444444444,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a bicycle with large wheels on a grassy area against a light blue background.",
"latency": 24.27,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__017.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__017.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__018 Upstream outputs, not a Pelican Map ranking · reward 1.000
A cartoon pelican is riding a bicycle on a line.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 18,
"reward": 1.0,
"structure_score": 1.0,
"semantic_score": 1.0,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon pelican is riding a bicycle on a line.",
"latency": 17.09,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__018.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__018.png"
} nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__019 Upstream outputs, not a Pelican Map ranking · reward 0.810
A cartoon bird with an orange beak is riding a bicycle.
Upstream SVG ↗ · Local SVG
Original fields and blind caption {
"model": "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4",
"access": "open",
"provider": "hf-inference-providers",
"task": "pelican_bicycle",
"sample": 19,
"reward": 0.8104166666666666,
"structure_score": 1.0,
"semantic_score": 0.7083333333333333,
"gate_passed": true,
"violations": [],
"judge_model": "Qwen/Qwen2.5-VL-72B-Instruct",
"judge_blind_caption": "A cartoon bird with an orange beak is riding a bicycle.",
"latency": 10.86,
"svg": "svg/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__019.svg",
"png": "png/nvidia__NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4__pelican_bicycle__019.png"
} 
