Schema compliance
Checks JSON validity, required fields and task-specific structures instead of relying only on free-form output review.
A reproducible toolkit for source-grounded Turkish generation, structured JSON, and translation consistency across English, German, Spanish, French, and Dutch.
Switch task and language to preview how the public toolkit presents quality, reliability and efficiency signals.
The toolkit combines deterministic validators with operational metrics so model comparisons remain inspectable and repeatable.
Checks JSON validity, required fields and task-specific structures instead of relying only on free-form output review.
Verifies expected source URLs and highlights missing or malformed evidence links in source-grounded tasks.
Runs language heuristics across Turkish and five translation targets to identify mixed-language or incomplete outputs.
Records endpoint latency, failures, retries and response-format fallbacks for operational comparison.
Exports machine-readable JSON and CSV alongside Markdown summaries suitable for reviews and public reports.
Includes deterministic synthetic fixtures so the evaluation workflow can be demonstrated without API keys or production data.
The evaluation code is public. Future GPU-backed work is intended to add reproducible configurations, datasets and anonymized reports.