Jupyter Notebook
Benchmark models using OpenAI-compatible APIs
Learn to benchmark language models with OpenAI-compatible APIs using our updated Jupyter Notebook for optimal performance evaluation.
Language model benchmarks only tell half a story
When it comes to language models, we tend to look at benchmarks to decide which model is the best to use in our application. But benchmarks only tell half a story. Unless you're building an all-purpose chat application, what you should be actually looking at is how well a model works for your application.
Calculate the number of language model tokens for a string
Here's an easy way to calculate the number of language model tokens for a string.