Evaluate Assistant Answer — assistant

Evaluates the correctness and completeness of a response from an assistant or RAG system by comparing it against a ground truth answer.

Usage

assistant_evaluate(question, answer, ground_truth_answer)

Arguments

question: The original question that was asked
answer: The answer to evaluate (from assistant or RAG system)
ground_truth_answer: The correct/expected answer to compare against

Value

List with http response, content (evaluation metrics), and status_code. Content includes: - correctness: Precision of the answer (0 to 1) - completeness: Recall of the answer (0 to 1) - alignment: Harmonic mean of correctness and completeness

Details

This endpoint evaluates answers based on: - Correctness: How precise is the answer? (Are the facts stated correct?) - Completeness: How complete is the answer? (Are all expected facts present?) - Alignment: Combined score (harmonic mean of correctness and completeness)

Examples

if (FALSE) { # \dontrun{
# Evaluate an answer
assistant_evaluate(
  question = "What are the capital cities of France, England and Spain?",
  answer = "Paris is the capital of France and London is the capital of England.",
  ground_truth_answer = "Paris is the capital of France, London is the capital of England, and Madrid is the capital of Spain."
)
} # }