Package index • vitals

Evaluation tasks

Task: Creating and evaluating tasks

Solvers

generate(): Convert a chat to a solver function

Scorers

model_graded_qa() model_graded_fact(): Model-based scoring
detect_includes() detect_match() detect_pattern() detect_exact() detect_answer(): Scoring with string detection

Store, view, deploy, and compare evaluation logs

vitals_log_dir() vitals_log_dir_set(): The log directory
vitals_view(): Interactively view local evaluation logs
vitals_bundle(): Prepare logs for deployment
vitals_bind(): Concatenate task samples for analysis

Example Evals

are: An R Eval