Skip to content

feat(payments): add Pay for Data (Heurist) use case#1465

Closed
joshuamarksmith wants to merge 19 commits into
awslabs:mainfrom
joshuamarksmith:feat/pay-for-data-heurist
Closed

feat(payments): add Pay for Data (Heurist) use case#1465
joshuamarksmith wants to merge 19 commits into
awslabs:mainfrom
joshuamarksmith:feat/pay-for-data-heurist

Conversation

@joshuamarksmith
Copy link
Copy Markdown
Contributor

@joshuamarksmith joshuamarksmith commented May 9, 2026

A Strands-based finance research agent that calls paid Heurist x402 endpoints
for real-time prices, SEC filings, and macro indicators. The
AgentCorePaymentsPlugin intercepts HTTP 402 responses, asks the AgentCore
payment manager to generate a payment proof against the configured payment
instrument and payment session, and retries automatically — tool code stays
an ordinary http_request call. Data is analyzed in AgentCore Code Interpreter
and exported as charts and reports to S3.

Highlights

  • HTTP 402 payment processing via AgentCorePaymentsPlugin — no manual
    payment code in tools
  • Embedded wallet (Coinbase CDP) with USDC on Base mainnet
  • AgentCore Code Interpreter for pandas/matplotlib analysis and S3 artifact export
  • Deployed to AgentCore Runtime with full observability (OTel + CloudWatch GenAI)
  • Parallel paid tool calls supported — agent fans out independent data fetches
  • Public PyPI dependencies only (bedrock-agentcore[strands-agents]>=1.9.0)
  • Default model: Claude Sonnet 4.6 (us.anthropic.claude-sonnet-4-6)

Adds

  • 01-tutorials/13-AgentCore-payments/02-use-cases/pay-for-data/
    • pay-for-data.ipynb — full notebook walkthrough (deploy, invoke, observe)
    • agent/main.py — AgentCore Runtime entry point (BedrockAgentCoreApp)
    • agent/catalog.py — Heurist registry loader with input sanitization
    • agent/config.py — environment configuration
    • agent/sync_registry.py — CLI to refresh the catalog cache before deploy
    • agent/Dockerfile, agent/requirements.txt
    • README.md, .env.example, .gitignore

Updates

  • 01-tutorials/13-AgentCore-payments/README.md: add Use Cases table entry
  • 01-tutorials/13-AgentCore-payments/02-use-cases/README.md: add use case overview

Testing

End-to-end validated on Base mainnet with real USDC settlement:

  • Payment sessions created with budget caps
  • Paid Heurist endpoints called (YahooFinance, FredMacro) — 402 → ProcessPayment → retry
  • Code Interpreter analysis and S3 artifact export
  • Session spend tracked correctly
  • Parallel paid calls confirmed working (5/5, 8/8)

A Strands-based finance research agent that calls paid Heurist x402 endpoints
for real-time prices, SEC filings, and macro indicators. The
AgentCorePaymentsPlugin intercepts HTTP 402 responses, asks the AgentCore
payment manager to generate a payment proof against the configured payment
instrument and payment session, and retries automatically — tool code stays
an ordinary http_request call. Data is analyzed in AgentCore Code Interpreter
and exported as charts and reports.

Highlights:
- HTTP 402 payment processing via AgentCorePaymentsPlugin — no manual
  payment code in tools
- Embedded wallet (Coinbase CDP) with USDC as the settlement asset
- AgentCore Code Interpreter for pandas/matplotlib analysis and artifact
  export
- Public PyPI dependencies only (bedrock-agentcore==1.9.0) — no bundled or
  file-based SDK components
- Targets x402 on Base mainnet (Heurist endpoints settle on Base)

Adds:
- 01-tutorials/13-AgentCore-payments/02-use-cases/pay-for-data/
  - pay-for-data.ipynb (notebook walkthrough)
  - heurist_finance_agent/ (agent, catalog, config, artifact_export)
  - scripts/ (sync_registry, run CLI entry points)
  - requirements.txt, .env.example, README.md, .gitignore

Updates:
- 01-tutorials/13-AgentCore-payments/README.md: add Use Cases table entry
- 01-tutorials/13-AgentCore-payments/02-use-cases/README.md: add use case
  overview and highlights
@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@github-actions github-actions Bot added the 01-tutorials 01-tutorials label May 9, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 9, 2026

Latest scan for commit: 188d963 | Updated: 2026-05-21 14:38:21 UTC

Security Scan Results

Scan Metadata

  • Project: ASH
  • Scan executed: 2026-05-21T14:37:59+00:00
  • ASH version: 3.0.0

Summary

Scanner Results

The table below shows findings by scanner, with status based on severity thresholds and dependencies:

Column Explanations:

Severity Levels (S/C/H/M/L/I):

  • Suppressed (S): Security findings that have been explicitly suppressed/ignored and don't affect the scanner's pass/fail status
  • Critical (C): The most severe security vulnerabilities requiring immediate remediation (e.g., SQL injection, remote code execution)
  • High (H): Serious security vulnerabilities that should be addressed promptly (e.g., authentication bypasses, privilege escalation)
  • Medium (M): Moderate security risks that should be addressed in normal development cycles (e.g., weak encryption, input validation issues)
  • Low (L): Minor security concerns with limited impact (e.g., information disclosure, weak recommendations)
  • Info (I): Informational findings for awareness with minimal security risk (e.g., code quality suggestions, best practice recommendations)

Other Columns:

  • Time: Duration taken by each scanner to complete its analysis
  • Action: Total number of actionable findings at or above the configured severity threshold that require attention

Scanner Results:

  • PASSED: Scanner found no security issues at or above the configured severity threshold - code is clean for this scanner
  • FAILED: Scanner found security vulnerabilities at or above the threshold that require attention and remediation
  • MISSING: Scanner could not run because required dependencies/tools are not installed or available
  • SKIPPED: Scanner was intentionally disabled or excluded from this scan
  • ERROR: Scanner encountered an execution error and could not complete successfully

Severity Thresholds (Thresh Column):

  • CRITICAL: Only Critical severity findings cause scanner to fail
  • HIGH: High and Critical severity findings cause scanner to fail
  • MEDIUM (MED): Medium, High, and Critical severity findings cause scanner to fail
  • LOW: Low, Medium, High, and Critical severity findings cause scanner to fail
  • ALL: Any finding of any severity level causes scanner to fail

Threshold Source: Values in parentheses indicate where the threshold is configured:

  • (g) = global: Set in the global_settings section of ASH configuration
  • (c) = config: Set in the individual scanner configuration section
  • (s) = scanner: Default threshold built into the scanner itself

Statistics calculation:

  • All statistics are calculated from the final aggregated SARIF report
  • Suppressed findings are counted separately and do not contribute to actionable findings
  • Scanner status is determined by comparing actionable findings to the threshold
Scanner S C H M L I Time Action Result Thresh
bandit 0 0 0 0 0 0 848ms 0 PASSED MED (g)
cdk-nag 0 0 0 0 0 0 29.8s 0 PASSED MED (g)
cfn-nag 0 0 0 0 0 0 8ms 0 PASSED MED (g)
checkov 0 0 0 0 0 0 5.4s 0 PASSED MED (g)
detect-secrets 0 0 0 0 0 0 801ms 0 PASSED MED (g)
grype 0 0 0 0 0 0 45.2s 0 PASSED MED (g)
npm-audit 0 0 0 0 0 0 180ms 0 PASSED MED (g)
opengrep 0 0 0 0 0 0 <1ms 0 SKIPPED MED (g)
semgrep 0 0 0 0 0 0 <1ms 0 MISSING MED (g)
syft 0 0 0 0 0 0 1.9s 0 PASSED MED (g)

Joshua Smith added 4 commits May 9, 2026 14:57
1. Fix requirements.txt: bedrock-agentcore[strands-agents]==1.9.0 (extras
   install required for AgentCorePaymentsPlugin)
2. Change default prompt to non-crypto macroeconomic query (FredMacroAgent
   GDP/unemployment summary) in run.py and README
3. Remove 'no bundled/file-based SDK' line from README, 02-use-cases/README,
   and tutorial-level README table entry
4. Expand notebook: add architecture diagram (ASCII), AgentCore payments
   capabilities table (payment manager, instrument, session, processing,
   limits), step-by-step payment flow explanation, and restructure into
   numbered steps
5. Add cleanup section (local cache + artifacts) and shared responsibility
   model table to notebook
Comment thread 01-tutorials/13-AgentCore-payments/02-use-cases/pay-for-data/requirements.txt Outdated
- agent.py: remove redundant 'No manual payment logic is needed' from docstring
- artifact_export.py: replace multi-strategy JSON fallback parser with a
  single json.loads — the Code Interpreter runs our own script that always
  emits print(json.dumps(payload)), so the output is deterministic JSON
- catalog.py: warn (via logging.warning) when a requested agent ID is not
  found in the live catalog, so renames or removals surface immediately
  rather than silently loading zero tools
- requirements.txt: pin strands-agents==1.36.0, strands-agents-tools==0.5.0,
  boto3==1.43.1, botocore==1.43.3 to match PR awslabs#1459 versions; keep
  bedrock-agentcore[strands-agents]==1.9.0 (newer than awslabs#1459)
- catalog.py line 199 (f-string suggestion): left as-is — the lines.append
  + join pattern is idiomatic for multi-line table construction and already
  uses f-strings inside the loop
@mvangara10
Copy link
Copy Markdown
Contributor

Please update the use case to use AgentCore runtime and AgentCore observability

Copy link
Copy Markdown

@leewc leewc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM from my side. You might want to address Madhu's comments

Joshua Smith added 4 commits May 13, 2026 20:33
Deploy the Heurist finance agent to AgentCore Runtime with full feature
parity: AgentCore Code Interpreter, S3 artifact storage, and automatic
observability via CloudWatch GenAI Observability.

## What's new

### runtime_agent.py (new)
AgentCore Runtime entry point using BedrockAgentCoreApp. Key design:

- Stateless, payload-driven: all payment config (manager ARN, session ID,
  instrument ID) comes from the invocation payload. The container holds no
  credentials. The app backend (ManagementRole) creates payment sessions with
  spending limits; the execution role (ProcessPaymentRole) can only spend
  within those limits.

- AgentCore Code Interpreter: AgentCoreCodeInterpreter is a remote AWS API
  and works identically from a Runtime container. pandas/matplotlib analysis
  and chart generation are fully supported.

- S3 artifact storage: two @tool functions replace local disk writes —
  export_artifact_to_s3 (binary files from CI sandbox) and save_report_to_s3
  (text/CSV/markdown). Both upload to S3 and return presigned URLs in the
  invocation response:
    {"response": "...", "artifacts": [{"name": "chart.png", "url": "...", "expires_in": 3600}]}
  If CI_ARTIFACTS_BUCKET is not set the agent degrades gracefully: charts
  become markdown tables, text returns inline.

- Thread-local state (threading.local) for per-invocation CI session name
  and artifact list — correct for concurrent Runtime requests.

- Observability: agentcore deploy configures the container to run under
  opentelemetry-instrument, automatically emitting OTel traces and logs to
  CloudWatch GenAI Observability. No instrumentation code required.

### pay-for-data.ipynb (rewritten)
Runtime-only, 8-step notebook:
  1. Configure credentials
  2. Sync Heurist catalog (bundled in container image at build time)
  3. Create private S3 artifacts bucket
  4. Install agentcore CLI, scaffold project, copy files, deploy
  5. Add IAM permissions to execution role (payment + CI + S3)
  6. Invoke deployed agent, inspect response + session spend
  7. View CloudWatch GenAI Observability dashboard
  8. Cleanup (Runtime stack + S3 bucket)

### README.md (rewritten)
Runtime-only documentation:
- Architecture diagram reflects the deployed topology
- Quick Start maps to the 8-step notebook
- Runtime agent section covers: stateless design, Code Interpreter, S3
  artifact response shape, graceful degradation, IAM permissions table
- Prerequisites updated (Node.js 20+, Docker, CDK)

## IAM permissions added to execution role

  PaymentDataPlaneAccess:
    bedrock-agentcore: ProcessPayment, GetPaymentInstrument,
      GetPaymentInstrumentBalance, GetPaymentSession, GetResourcePaymentToken
    Resource: payment-manager/*

  CodeInterpreterAccess:
    bedrock-agentcore: StartCodeInterpreterSession,
      StopCodeInterpreterSession, InvokeCodeInterpreter
    Resource: code-interpreter/*

  S3ArtifactsAccess:
    s3: PutObject, GetObject
    Resource: <bucket>/heurist-finance-artifacts/*

The execution role does not receive CreateSession, CreateInstrument, or
any control-plane payment permissions.
Add aws-opentelemetry-distro>=0.10.0 to the Runtime container's pyproject.toml.
This enables botocore auto-instrumentation which injects W3C traceparent headers
into outgoing boto3 calls, stitching Code Interpreter and payment spans as child
spans in the unified Runtime trace.

AgentCore Observability docs explicitly list StartCodeInterpreterSession,
InvokeCodeInterpreter, and StopCodeInterpreterSession as supporting traceparent
for cross-service trace correlation:
https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/observability-configure.html

Update Step 7 observability description to accurately reflect what is now stitched:
- Strands agent spans (LLM calls, tool calls, agent turns)
- Code Interpreter calls as child spans via traceparent propagation
- Payment boto3 calls as child spans via botocore instrumentation

Update README Runtime Deployment section accordingly.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change the python query

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the tutorail dependency

@rajuans
Copy link
Copy Markdown

rajuans commented May 15, 2026

agent.py is undocumented — unclear when/why to use it

The README's Layout section omits agent.py entirely, and the Quick Start says "Open pay-for-data.ipynb" which only deploys runtime_agent.py. A user cloning this sample has no indication of when or why they would use agent.py.

Additionally, agent.py duplicates significant logic from runtime_agent.py (system prompt, payments plugin config, tool wiring, agent creation) — any future change must be made in two places.

Suggestion: Either:

  1. Add agent.py to the Layout section with a clear explanation (e.g., "local development CLI — run directly without deploying to AgentCore Runtime"), or
  2. Remove it if the notebook is the only supported entry point.

| | |
|---|---|
| AgentCore components | AgentCore payments, AgentCore Code Interpreter, AgentCore Runtime |
| Agent framework | [Strands Agents](https://strandsagents.com/) |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strands is Agent SDK. Its not framework.

Comment on lines +31 to +55
## How It Works

`AgentCorePaymentsPlugin` handles the entire x402 payment lifecycle:

```python
from bedrock_agentcore.payments.integrations.strands import (
AgentCorePaymentsPlugin,
AgentCorePaymentsPluginConfig,
)

payment_plugin = AgentCorePaymentsPlugin(
config=AgentCorePaymentsPluginConfig(
payment_manager_arn=PAYMENT_MANAGER_ARN,
user_id=USER_ID,
payment_instrument_id=PAYMENT_INSTRUMENT_ID,
payment_session_id=PAYMENT_SESSION_ID,
region="us-west-2",
)
)

agent = Agent(
model=BedrockModel(model_id=MODEL_ID),
tools=[http_request, code_interpreter, export_artifact_to_s3, ...],
plugins=[payment_plugin],
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to provide the code sample in readme? Can we add flow in How it work section?

@@ -0,0 +1,31 @@
#!/usr/bin/env python3
"""Fetch the live Heurist catalog and refresh the local cache.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also add how agent would be using this file?

Joshua Smith added 4 commits May 14, 2026 19:58
…ructure

Mirrors the working pay-for-content-browser-use sample structure exactly so
'agentcore deploy' just works. Replaces the previous heurist_finance_agent/
package + Dockerfile.runtime layout with a single flat agent/ directory.

Layout changes:
  Before:                            After:
    heurist_finance_agent/             agent/
      __init__.py                        main.py        (was runtime_agent.py)
      runtime_agent.py                   catalog.py     (flat imports)
      local_agent.py                     config.py      (flat imports)
      artifact_export.py                 sync_registry.py (was scripts/sync_registry.py)
      catalog.py                         catalog_live_cache.json
      config.py                          requirements.txt (was root)
      scripts/sync_registry.py           Dockerfile     (was Dockerfile.runtime)
    Dockerfile.runtime                   .env (gitignored)
    requirements.txt
    pyproject.toml

  - Drops local_agent.py and artifact_export.py — Runtime is the only mode.
  - Drops the pyproject.toml/uv pip path; container now uses requirements.txt
    + pip install -r requirements.txt (matches reference).
  - Catalog cache + container .env now live in agent/ and are gitignored;
    the notebook regenerates them every deploy.

Notebook updates:
  - Step 2: switch from 'from heurist_finance_agent.catalog import ...' to
    flat sys.path-based imports of catalog/config from agent/.
  - Step 7: rewrites the scaffold + stage flow. project name 'payfordata'
    (lowercase, mirrors reference); copies agent/* into payfordata/app/
    HeuristFinanceAgent/ on every run; refreshes the catalog cache via
    'python3 agent/sync_registry.py' before staging; writes the runtime .env
    with BYPASS_TOOL_CONSENT=true so http_request skips its TTY prompt
    (Runtime containers have no TTY); pins idleTimeout/maxLifetime.
  - Step 8: expands payment IAM resource scope to include the bare
    payment-manager/* ARN (the plugin's GetPaymentInstrument call hit
    AccessDeniedException without it); adds a separate Bedrock invoke
    statement that includes inference-profile/* and application-inference-
    profile/* (Claude Sonnet 4 in us-west-2 is fronted by a CRIS profile,
    so foundation-model/* alone is insufficient).
  - Renumbered the step table in the README to match (now 11 steps).

README:
  - Updated layout diagram, file references (agent/main.py), env-var table
    (split host vs. container), and the IAM permissions table.
  - Removed pyproject.toml mention.

Hygiene:
  - .gitignore now ignores payfordata/, agent/.env, agent/catalog_live_cache.json
    (matches reference's pattern of treating CLI scaffolds as build artifacts).
  - Added images/observability-dashboard.png reference asset.
  - Dropped 'production-grade' / 'production' phrasing from agent docstrings,
    notebook, and .env.example per AWS sample-content guidelines.
- README.md (use case): add a 'Mainnet sample' callout up top noting that the
  walkthrough targets Base mainnet against the live Heurist mesh x402 registry
  and settles real USDC on-chain. Embed images/obs-dashboard.png hero screenshot.
  Tighten Prerequisites: spell out the funded mainnet wallet, project-level
  delegated signing, and the per-wallet WalletHub grant.
- pay-for-data.ipynb: flip the NETWORK default to base-mainnet (matches what
  the rest of the notebook + README assume) and embed the same dashboard image
  in Step 10.
- .env.example: NETWORK default flipped to base-mainnet with a comment about
  the current Sepolia EIP-712 simulation issue on the Coinbase signer path.
- images/obs-dashboard.png: new screenshot of the AgentCore Observability
  dashboard with successful Heurist invocations. Replaces the old
  observability-dashboard.png placeholder.
- 13-AgentCore-payments/README.md + 02-use-cases/README.md: scope to this
  use case only — drop the cross-listing of the browser-use sample so each
  README is self-contained.
Joshua Smith added 4 commits May 16, 2026 12:23
…t costs

Two resilience fixes for multi-step research workflows hitting Strands' default
ceilings, plus README costs section so customers running this know what they're
paying for.

agent/main.py:
  - max_tokens=60000 (env-overridable via AGENT_MAX_TOKENS). The Bedrock Converse
    API default cap is ~4k output tokens; Strands raises MaxTokensReachedException
    when a single turn produces more than that. Multi-step research workflows
    (5+ paid Heurist calls + Code Interpreter executeCode + chart export +
    markdown report) routinely exceed it. Claude Sonnet 4 supports up to 64k
    output tokens, so 60k leaves headroom without surprising the SDK.
  - boto_client_config with read_timeout=900 / connect_timeout=15 / retries=1
    on the Bedrock client. The default 60s read timeout was tripping mid-stream
    on long single-turn outputs and Strands surfaced it as
    EventLoopException: Read timed out.
  - Hoisted 'from botocore.config import Config' to the top of the file so the
    BedrockModel instantiation isn't carrying a function-local import.

agent/.env (and the notebook's Step 7 .env writer):
  - Document AGENT_MAX_TOKENS as overridable, with a comment noting the cost
    implication so users running this for short Q&A can lower it.

README:
  - Add a Costs section consolidating the per-turn spend across Heurist x402,
    Bedrock model output, AgentCore Runtime container time, AgentCore Code
    Interpreter session time, and S3/CloudWatch. Worst-case ~$0.90 per turn at
    the 60k token cap; typical turns far less.
  - Add AGENT_MAX_TOKENS to the container .env table.
…er max_tokens to 32k

- Update default model from claude-sonnet-4 to claude-sonnet-4-6
  (us.anthropic.claude-sonnet-4-6)
- Add retry logic in notebook Step 9 invoke cell to handle Runtime
  cold-start timeouts gracefully (3 retries, 15s between)
- Lower AGENT_MAX_TOKENS from 60000 to 32000 — Sonnet 4.6 is more
  capable and concise, 32k is plenty for multi-step research workflows
  while halving worst-case Bedrock output cost to ~$0.48/turn
- Defer heavy imports in main.py to first request for faster container
  init under opentelemetry-instrument
- Update all README/notebook references to reflect new model and costs
Heurist's x402 facilitator now supports concurrent payments from the
same wallet. Update the agent system prompt to explicitly encourage
parallelizing independent data fetches across tool-use rounds.
@joshuamarksmith
Copy link
Copy Markdown
Contributor Author

Superseded by PR #1545 (moved to 06-workshops/ path after repo migration).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

01-tutorials 01-tutorials

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants