eval / profiles

Eval Profiles

Define scoring rubrics — what "good" looks like for each workflow.

New Profile

Enabled

Profile Name *

Pass Threshold

70%

Score ≥ threshold → pass. Below → review or fail.

Input slots — flag the ones the output is judged against

Derived from the first target in Watches below. Each input gets a typed token you can reference in the rubric. "Original" = the output is judged against this; "context" = visible to the judge but not a comparison anchor.

Rubric — prose scoring rules; click a token to insert it

Goals *

Scoring Dimensions — each axis is scored 0–100, weighted average = final score

NameDescriptionWeight

Good Criteria — signs of a quality output

Bad Criteria — signs of a poor output

WATCHES — pipelines this profile automatically scores

Add the pipeline slugs this profile should score. Empty = no auto-eval (the profile is still available for manual scoring and monitors).

Loading profiles…

eval / results

Eval Results

All scored outputs across your workflows.

Request

Loading…

eval / analytics

Analytics

Quality trends across your profiles.

Total Evals

—

Pass Rate

—

Avg Score

—

Fail Count

—

By Profile

Loading…

eval / monitors

Monitors

Watch a URL or RSS feed — auto-score new or changed content against a profile.

New Monitor

Name *

Profile *

Source type

Source URL *

Interval

Mode

Max items per run

Loading…

eval / notifications

Notifications

Email alerts when an eval result is marked review or fail.

New recipient

Name *

Email *

Triggers

Fail Review

Pick one or both verdicts that should send this address an email. Pass results never trigger emails.

Active

Recipients

Loading…

Recent activity

Loading…

Quality Scoring

Eval Profiles

Eval Results

Analytics

By Profile

Monitors

Notifications