Amazon Bedrock AgentCore now runs AWS Lambda–based custom evaluators for market‑intelligence agents

A new technical walkthrough demonstrates how to add deterministic, code‑based evaluation into Amazon Bedrock AgentCore by running AWS Lambda functions inside the AgentCore Evaluations flow. The guide implements and registers four Lambda‑based evaluators for a financial market‑intelligence agent and shows how to execute them both on demand (for development and CI/CD gating) and online (to score live production traffic). This pattern gives builders a repeatable way to enforce deterministic checks and reduce foundation model token use for predictable validations.

The walkthrough defines a custom code‑based evaluator as an AWS Lambda function that acts as the evaluation engine and lets teams manage scoring logic directly in code. Evaluators can perform regex and structural validation, call external reference data, and embed arbitrary business rules. Because they are deterministic code, identical inputs yield identical results, enabling reuse without consuming foundation model tokens for each request.

Examples in the guide concentrate on finance‑specific checks that matter for market intelligence: verifying that quoted stock prices fall within a configurable live band; enforcing mandatory broker‑identification steps before profile access; ensuring tool outputs conform to strict JSON schemas; and detecting and withholding personally identifiable information. The authors note that numerical accuracy is highly sensitive in trading contexts — deviations as small as 0.1% can change decisions — so deterministic verification is preferred for those checks.

Operationally, Lambda evaluators are positioned to catch structural problems at the tool boundary, such as schema changes, parsing errors, or outages, before those issues propagate to downstream consumers. They can call reference systems to run numerical accuracy comparisons with defined tolerances, inspect sequences of tool calls across a session to validate workflow contract compliance, and trigger PII or secret‑scanning services to enforce hard content policies.

The guide also shows how to combine custom code evaluators with built‑in LLM‑as‑a‑Judge evaluators: deterministic code handles strict, repeatable validations while LLM judges assess softer qualities like clarity and usefulness. It outlines invoking other AWS services from Lambda for grounded fact checks, PII detection, and alerting, and demonstrates end‑to‑end registration and execution of each evaluator within AgentCore’s Evaluations flow.

For builders, the pattern enables the same Lambda evaluator to be applied consistently across development pipelines and production traffic, supporting CI/CD gating, real‑time monitoring, and uniform quality controls across agent frameworks. Reusing deterministic evaluators reduces foundation model token costs for routine checks and centralizes business logic for validation and alerting. The walkthrough credits contributors Stephanie Yuan, Lefan Zhang, Ritvika Pillai, Irene Wang, Carter Williams, T.J. Ariyawansa, Gitika Jha, Shoaib Javed, and product leadership from Vivek Singh.