feat(0188): SEP-1 stellar.toml fetcher for asset details#157
Open
feat(0188): SEP-1 stellar.toml fetcher for asset details#157
Conversation
…ichment
Implement runtime_enrichment::sep1 — fail-soft, in-process LRU-cached
HTTP client for issuer stellar.toml files. Wire the first and currently
only consumer at GET /v1/assets/{id}: resolve issuer's on-chain
home_domain, fetch /.well-known/stellar.toml, parse CURRENCIES[],
match (code, issuer) and surface description + home_page.
Strict scope: only the two fields originally hardcoded `None` on
AssetDetailResponse — `description` (from CURRENCIES[].desc) and
`home_page` (from DOCUMENTATION.ORG_URL — SEP-1 has no per-currency
homepage field; org URL is the closest semantic match and preserves
backward compatibility with the previous DB-sourced column). Other
SEP-1 fields not exposed; parser only models what the API surfaces.
AppState refactor: replace flat `fetcher` + `sep1` fields with a
grouped `RuntimeEnrichment { stellar_archive, sep1 }` struct under
the `runtime_enrichment` module — one field per architectural concept
on AppState, future submodules (nft_metadata / price_oracle) extend
the bundle without touching state.rs. Updates 6 call sites.
Module: client (Sep1Fetcher with `new()`-only constructor, RFC 1035 +
IP-literal SSRF guard, 100 KB body cap, 1 s connect / 2 s total
timeouts, moka 24 h LRU 1024-cap), dto (Sep1TomlParsed with
Sep1Currency {code, issuer, desc} + Sep1Documentation {org_url} only),
errors (typed Sep1Error → all map silently to null fields).
DB: assets/queries.rs ASSET_SELECT extended with iss.home_domain →
AssetRow.issuer_home_domain. No schema migration — accounts.home_domain
already populated by indexer. Canonical SQL `09_get_assets_by_id.sql`
updated with the same projection and a "runtime SEP-1 fetch" comment
replacing the outdated S3-blob reference.
Workspace: reqwest promoted from audit-harness inheritance; toml added.
Tests: unit suite for parser (4 tests in dto::tests) and host validation
(5 tests in client::tests). HTTP path intentionally not covered by
in-tree tests — reqwest is third-party, parsing is covered by dto, and
the remaining glue is ~5 lines of error mapping plus a 10-line size-cap
loop. Real-issuer smoke deferred to a follow-up `#[ignore]` test.
Full api suite 177 passed, 0 failed, 5 ignored. cargo check + clippy
-D warnings clean.
Docs: `docs/architecture/backend/backend-overview.md` §4.1 split into
two transport-specific submodules under runtime_enrichment;
`database-schema-overview.md` §4.10 Assets clarifies SEP-1 detail fields
are not persisted at all; `endpoint-queries/README.md` §09 replaces
S3 overlay with runtime SEP-1 fetch.
…cher-and-assets-details-enrichment
Adds Implementation Notes, Design Decisions (From Plan + Emerged) per the lore-framework-tasks completion checklist. Documents the strict-scope trim (9 fields → 2), the RuntimeEnrichment AppState bundle that emerged during review, the Sep1Fetcher base_override-then-removed evolution, and the dropped fixture-server tests with rationale. Future Work tracked inline (next: type-1 enrichment-worker crate carrying assets.icon_url backfill, LP TVL/volume/fee_revenue, and the new asset USD price columns discussed during the stellarchain.io/markets feature gap audit).
- assert issuer field in parses_minimal_well_formed_stellar_toml - relax SEP-1 timeout docs to 5 s connect / 10 s request to fit third-party HTTPS variability (issuer CDN cold starts, geo hops) - clarify "type-1 enrichment worker" — offline batch + DB write, distinct from per-request type-2 runtime fetcher - drop duplicate `total_supply` / `holder_count` line in §4.10 Refs PR review on lore-0188.
There was a problem hiding this comment.
Pull request overview
This PR adds a runtime SEP-1 stellar.toml enrichment path to the API, fetching issuer-published metadata at request time and wiring it into the GET /v1/assets/{id} response, while also refactoring AppState to bundle runtime enrichment dependencies under a single RuntimeEnrichment struct.
Changes:
- Implement
runtime_enrichment::sep1(reqwest client + TOML DTO parsing + in-process TTL/LRU cache + basic SSRF guard) and use it to populateAssetDetailResponse.descriptionandhome_page. - Refactor
AppStateto replace the flat stellar-archive fetcher field withruntime_enrichment: RuntimeEnrichment { stellar_archive, sep1 }, updating handlers/tests accordingly. - Update architecture/docs and canonical SQL documentation to reflect the SEP-1 runtime enrichment path; add workspace deps for
reqwestandtoml.
Reviewed changes
Copilot reviewed 21 out of 22 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| lore/1-tasks/archive/0188_FEATURE_sep1-fetcher-and-assets-details-enrichment.md | Task archive documenting the SEP-1 enrichment implementation and scope. |
| docs/architecture/database-schema/endpoint-queries/README.md | Updates endpoint E9 sourcing from S3 blob overlay to runtime SEP-1 fetch. |
| docs/architecture/database-schema/endpoint-queries/09_get_assets_by_id.sql | Documents issuer_home_domain projection and SEP-1 runtime overlay for details fields. |
| docs/architecture/database-schema/database-schema-overview.md | Updates asset schema notes to reflect that detail SEP-1 fields are resolved at request time (not persisted). |
| docs/architecture/backend/backend-overview.md | Updates backend responsibilities to describe runtime enrichment and the new SEP-1 path. |
| crates/api/src/transactions/handlers.rs | Switches stellar-archive fetch access to state.runtime_enrichment.stellar_archive. |
| crates/api/src/tests_integration.rs | Updates integration test app builder to construct RuntimeEnrichment (archive + SEP-1). |
| crates/api/src/state.rs | Refactors AppState to contain runtime_enrichment: RuntimeEnrichment. |
| crates/api/src/runtime_enrichment/sep1/mod.rs | Defines SEP-1 module surface and re-exports fetcher/DTOs. |
| crates/api/src/runtime_enrichment/sep1/errors.rs | Introduces Sep1Error error type for fetch/parse failures. |
| crates/api/src/runtime_enrichment/sep1/dto.rs | Adds minimal SEP-1 TOML DTOs and parser tests for the consumed fields. |
| crates/api/src/runtime_enrichment/sep1/client.rs | Implements cached HTTP fetcher, SSRF host validation, and body size cap. |
| crates/api/src/runtime_enrichment/mod.rs | Adds the RuntimeEnrichment bundle struct and updates runtime enrichment module docs. |
| crates/api/src/network/handlers.rs | Updates network tests to build RuntimeEnrichment in AppState. |
| crates/api/src/main.rs | Constructs and injects RuntimeEnrichment in the production app and tests. |
| crates/api/src/contracts/handlers.rs | Switches archive fetch access to state.runtime_enrichment.stellar_archive. |
| crates/api/src/assets/queries.rs | Extends asset detail query mapping to include issuer_home_domain. |
| crates/api/src/assets/handlers.rs | Fetches SEP-1 stellar.toml at runtime to populate description and home_page. |
| crates/api/src/assets/dto.rs | Updates asset detail response docs to describe SEP-1-derived fields. |
| crates/api/Cargo.toml | Adds reqwest and toml dependencies for the SEP-1 fetcher. |
| Cargo.toml | Promotes reqwest to workspace deps and adds workspace toml. |
| Cargo.lock | Locks new transitive dependencies for reqwest and toml. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…tring The SEP-1 enrichment commit updated the doc-comment on AssetDetailResponse but missed the codegen step required by the "API types freshness" CI gate (CLAUDE.md). Re-runs `nx run @rumblefish/api-types:generate` so openapi.json + types.gen.ts match the current Rust source.
…ment Three doc-comment references survived the trim that deleted the EnrichmentStatus type and the would-be E13 stellar_archive consumer: - crates/api/src/main.rs: comment listed E3+E13+E14 as stellar_archive consumers but E13 (`/contracts/:id/invocations`) is DB-only per endpoint-queries/README.md; only E3 + E14 use the archive. Same comment said sep1 was "(future)" — it now ships in this task. - crates/api/src/runtime_enrichment/mod.rs: claimed both submodules surface status via an `enrichment_status` field on the response, but that discriminator was dropped in the strict-scope trim. - crates/api/src/runtime_enrichment/sep1/errors.rs: top-level doc said every variant maps to `EnrichmentStatus::Unavailable`, also stale post-trim. Also added a one-line note on `MissingHomeDomain` clarifying it is reserved (handler short- circuits before calling fetch). Pure docstring changes; no behaviour or signature delta.
- redirect Policy::limited(0): drop following so a 30x can't bypass
validate_host (loopback / link-local / RFC 1918). SEP-1 doesn't
require redirect support; reqwest maps 3xx to a redirect error
that the existing Sep1Error::Http arm picks up
- ttl_future_cache helper + try_get_with: collapse concurrent
cold-cache misses on the same home_domain onto one in-flight
fetch (mirrors network_cache stampede protection from task 0180).
Failed loads are not cached, so a transient timeout doesn't
poison subsequent requests
- capped_body: branch on e.is_timeout() so body-side stalls surface
as Sep1Error::Timeout instead of the generic Http bucket
- backend-overview.md: clarify status surfacing per submodule —
archive endpoints expose heavy_fields_status (ok/unavailable),
SEP-1 surfaces failures silently as null description / home_page
(warn-logged). Asymmetric by design: discriminator only where the
response shape is otherwise ambiguous
Sep1Fetcher::fetch now returns Result<_, Arc<Sep1Error>> (moka
try_get_with's shared-failure shape); the lone consumer formats
via {e} so Display flows through Arc with no behavior change.
Refs #157
…nflict Regenerated libs/api-types/src/generated/types.gen.ts from merged openapi.json to resolve conflict introduced by accounts module (PR #158).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runtime_enrichment::sep1— fail-soft, in-process LRU-cached HTTP client that fetches issuer stellar.toml files at request time. Wired intoGET /v1/assets/{id}to populatedescription(fromCURRENCIES[].desc) andhome_page(fromDOCUMENTATION.ORG_URL), the two fields previously hardcodednullonAssetDetailResponse.AppState: replaces flatfetcher+ (would-be)sep1fields with a singleRuntimeEnrichment { stellar_archive, sep1 }bundle struct under theruntime_enrichmentmodule. Mirrors module structure 1:1; future submodules (nft_metadata,price_oracle) extend the bundle without touchingstate.rs. Updates 6 consumer call sites across transactions/contracts/assets handlers + 4 test-app builders.Noneget populated. Parser DTOs (Sep1Currency,Sep1Documentation) model onlycode/issuer/descandorg_urlrespectively — adding more SEP-1 fields later is cheap; removing them after a frontend ships against them is expensive. Built-in SSRF guard (RFC 1035 hostname check + IP-literal rejection), 100 KB body cap (per SEP-1), 1 s connect / 2 s total timeouts, moka 24 h LRU 1024-cap.reqwestpromoted fromaudit-harnessinheritance to[workspace.dependencies];toml = "0.8"added.09_get_assets_by_id.sql+endpoint-queries/README.md§09 + backend/schema architecture docs updated to reflect the runtime SEP-1 path replacing the abandoned per-entity S3 blob plan from task 0164.dto::tests, 5 forvalidate_hostinclient::tests). Full api suite 177 passed, 0 failed, 5 ignored. cargo check + clippy-D warningsclean. HTTP path coverage intentionally deferred to a follow-up#[ignore]real-issuer smoke test.