Session Outline
During model development, we must evaluate the performance and improve model quality based on the metrics. However, model evaluation can be time-consuming and expensive, especially for large language models (LLM), which usually need human raters to provide ratings. The session discusses the most advanced way of using LLM as an autorater to expedite the evaluation process and greatly reduce costs.
Key Takeaways
- A typical model development cycle and how important the model evaluation is, but how difficult it could be
- What is LLM-as-a-service autorater
- Autorater use cases (demonstrated by exact samples such as Search)
- How to build an autorater and what are the risks to avoid
