Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.
The overall quant rating is not an average of the factor grades listed. Instead, it gives greater weight to the metrics with the strongest predictive value.
Some results have been hidden because they may be inaccessible to you
Show inaccessible results