run_vlm_test_suite_batch

shallow

io.github.JcJamet/ia-qa-toolbox · Verify this server

Compare multiple VLMs on the same test suite in parallel — send an image (URL or base64) + N test cases to all models simultaneously. Returns per-model PASS/FAIL verdicts, pass rates, latency stats, and a comparison table. Assertion types: contains, not_contains, json_format, min_length, max_length, semantic_contains. BYOK: requires API keys for each provider.

100.0/100

1 trials · measured 8 days ago

run_vlm_test_suite_batch scores 100.0/100 on Vouch's measured behaviour index, from 1 real invocation trials against io.github.JcJamet/ia-qa-toolbox, measured 25 Aug 2026 under methodology v0.2.0. Every measured component scored 100.

Component breakdown

ComponentWeightValue
Reliability35%not applicable
Schema integrity25%100.0
Failure behaviour15%not applicable
Latency15%not applicable
Concurrency10%not applicable

Tool details

Transport
remote
Credential class
open
Input schema
not declared
Output schema
not declared
Side-effect classification
unclassified

Score history

DayScoreTierMethodology
2026-08-25100.0shallowv0.2.0

Probe evidence

ProbeOutcomes
schema_integritypass: 1

Raw request/response logs are not archived yet — the outcome counts above are drawn directly from every recorded trial.

Embed this score

Available for every tool, scored or not — not a verification perk. Always links back to this page.

Vouch score: run_vlm_test_suite_batch
[![Vouch score](https://vouch.tools/api/tools/57ec6127-d474-46fa-98d8-03f59bf0ea0e/badge.svg)](https://vouch.tools/tools/57ec6127-d474-46fa-98d8-03f59bf0ea0e)
run_vlm_test_suite_batch — Vouch