samples/python: Add evaluation learning path for hosted agents

[aprilk-ms]

What

Adds an evaluation learning path to the Python hosted-agent samples
(samples/python/hosted-agents/), per the three deliverables requested:

1. General evaluation learning path at
   agent-framework/responses/14-evaluation/ — a tiny demo agent (main.py,
   agent.yaml) + 8 eval scripts + 4 seed datasets + a beginner-focused
   nav-hub README.md.
2. Multi-turn evaluation co-located in 01-basic/ — two scripts
   (simulation + traces) + seed datasets + a new "Evaluating multi-turn
   conversations" section in 01-basic/README.md so the multi-turn learning
   path can stand on its own.
3. "Related: Evaluate this agent" links added to 31 hosted-agent
   READMEs (parent index, agent-framework table, every responses/01..13
   sample, invocations/01-basic, every bring-your-own/** sample).

The 8 scripts in 14-evaluation/

Script	What it teaches
evaluate_basic.py	Easiest first run — built-in evaluators (task_adherence, fluency, relevance) against 4 inline questions
evaluate_custom_rubric.py ⭐	Generates a 5-7 dimension rubric tailored to your agent from a short prompt — the script you'd actually use on a real project
evaluate_multiturn_simulation.py	Foundry simulates full multi-turn conversations from seed scenarios
evaluate_multiturn_traces.py	Same multi-turn evaluators over real traced conversations
generate_dataset_from_traces.py	Materialize recent traces into a reusable registered dataset
generate_dataset_synthetic.py	Bootstrap a dataset from topic seeds when you have no traffic
evaluate_scheduled.py	Score every new agent response continuously
evaluate_redteam.py	Content-safety / red-team eval (violence, self-harm, hate, sexual; 0-7 severity)

Shared helpers live in eval_common.py:

• API version pinned (2025-11-15-preview) in one place
• target_agent() default + env-var overrides (EVAL_AGENT_NAME, EVAL_AGENT_VERSION)
• print_friendly_output() — Pydantic-safe per-row summary (single-turn / multi-turn / dataset-row shapes), with EVAL_DEBUG=1 to also dump the raw payload
• build_testing_criteria(model, response_source=) — "sample" for live agent runs ({{sample.output_text}}), "item" for dataset rows that already contain a response ({{item.response}})

The demo agent ships with ENABLE_INSTRUMENTATION=true and
ENABLE_SENSITIVE_DATA=true so trace-based + continuous scripts work
out of the box. The README has an explicit privacy callout about this.

Beginner-friendliness passes

Two rubber-duck rounds (correctness + score scales) and one voice-and-verbosity
sweep were applied before this PR:

• Score-shape table distinguishes quality (1-5, higher better),
  agent-task (Pass/Fail + optional score), safety (0-7 severity, higher
  worse), attack-detection (boolean), custom rubric (weighted 1-5)
• builtin.indirect_attack documented as the prompt-injection
  evaluator (the non-existent builtin.jailbreak was removed in an
  earlier pass)
• evaluate_custom_rubric.py now polls to completion + prints
  friendly output, with CHANGE-THIS-FIRST markers on the rubric prompt
  and CHANGE-THIS-TOO on the inline eval questions
• generate_dataset_synthetic.py defaults to running the generated
  questions through the deployed agent; EVAL_AGAINST_DATASET_ONLY=true
  opts out
• "If a score is low, what next?", "Cost and data usage", and
  tracing-privacy sections added
• LLM-ism scrub: "small judges that emit", "the intuition is always",
  "the real recommended", "almost always", "small LLM judges", "emit a
  severity" all removed
• "The scripts" section compressed from 65 lines to 40 by removing
  overlap with the "Pick the right flow" table immediately above it
  (while keeping every load-bearing callout: schedule keeps running,
  redteam writes adversarial prompts to traces, etc.)
• One-line description of the demo agent added so readers don't
  need to open agent.yaml to figure out what it is

Verification

• All 12 new Python files AST-parse clean
• print_friendly_output() smoke-tested against Pydantic-like objects +
  single-turn / multi-turn / dataset-row shapes
• All 31 added "Related" link targets resolve to existing relative paths
• No stale references to removed CLI commands (azd ai agent deploy,
  builtin.jailbreak) anywhere in the diff
• Deploy flow matches the parent README's canonical pattern
  (mkdir foo && cd foo && azd ai agent init -m <manifest> && azd up)

Not yet verified

• End-to-end run against a live Foundry project (no Azure creds in the
  authoring environment). The preview API surfaces
  (evaluator_generation_jobs, data_generation_jobs, scheduled eval)
  are pinned to API version 2025-11-15-preview in eval_common.py
  as the single bump point.

File count

• 60 files in the first commit (8 scripts + helpers + manifest +
  Dockerfile + 4 seeds + 31 README updates)
• 3 files in the voice-pass commit (-25 net lines)

[github-actions]

👋 Thanks for your interest in contributing, @aprilk-ms!

This repository does not accept pull requests directly. If you'd like to report a bug, suggest an improvement, or propose a new sample, please open an issue instead.

See CONTRIBUTING.md for more details.