Package index • vitals

Evaluation tasks

Task: Creating and evaluating tasks

Solvers

generate(): Convert a chat to a solver function

Scorers

model_graded_qa() model_graded_fact(): Model-based scoring

detect_includes() detect_match() detect_pattern() detect_exact() detect_answer(): Scoring with string detection

Store, view, deploy, and compare evaluation logs

vitals_log_dir() vitals_log_dir_set(): The log directory

vitals_view(): Interactively view local evaluation logs

vitals_bundle(): Prepare logs for deployment

vitals_bind(): Concatenate task samples for analysis

Example Evals

are: An R Eval