Four metric kinds
Pointwise for one number and a reason, rubric for several named criteria at once, custom_schema for typed structured output, and pairwise to pick the stronger of two candidates.
Mizan
Turn the judgment you already make by eye into an explicit, repeatable, shareable check. Mizan runs LLM-as-a-judge evals on Google's Vertex AI across text, images, audio, video, and music.
What is Mizan?
Section titled “What is Mizan?”Mizan is a command-line tool for evaluating generative AI output with an LLM-as-a-judge. You state the criteria a response has to meet, hand a response and those criteria to a judge model, and get back a score and an explanation you can store, rerun, and share. It runs on Google’s Vertex AI Gen AI Evaluation Service across text, images, audio, video, and music.
Written down, not by eye
Your standard lives in a template file you can commit, hand to a teammate, or import — the criteria travel with the work instead of staying in one reviewer’s head.
Repeatable & stored
Runs are persisted by default, so mizan results list and
mizan results show let you look back at what you scored and why.
Start in ten minutes
go install github.com/ghchinoy/mizan/cmd/mizan@v0.1.0, point it at a Google
Cloud project, and run your first eval. See the
user guide.
Where to go next
Section titled “Where to go next”- User guide — install, configure, walk the registry, and run evals end to end.
- Blog — the Build Evals with Mizan series, starting with why evals matter and what an LLM-as-a-judge really is.
- Source on GitHub — the code, the authoritative docs, and releases.