Skip to content

Conversation

@leggetter
Copy link
Collaborator

Summary

This PR introduces a brand‑new, deterministic Shopify webhooks demo built around Hookdeck, covering setup, backpressure, and logs/retry from end to end. It provisions Hookdeck connections via API, wires Shopify webhooks through Hookdeck into the Shopify app itself, and provides scripts plus documentation to make the entire flow repeatable and demo‑friendly.

What’s Included

  • New Hookdeck + Shopify demo harness

    • Adds a TypeScript upsert script (scripts/01-hookdeck-upsert.ts) that:
      • Uses the Hookdeck API (with HOOKDECK_API_KEY) to create/update two connections: shopify-orders-prod-conn (HTTP destination) and shopify-orders-dev-conn (CLI destination, same source).
      • Calls shopify app env show --path ./shopify to extract SHOPIFY_API_SECRET from the Shopify CLI (no manual env wiring).
      • Validates that shopify/shopify.app.toml exists and fails with clear guidance if not (instructs to run shopify app dev first).
      • Parses and updates shopify.app.toml via @iarna/toml to inject orders/* webhook subscriptions directly into the existing file.
      • Automatically adds or updates HOOKDECK_SOURCE_URL in .env so other scripts can rely on a single, canonical value.
  • Webhooks integrated into the Shopify app

    • Introduces a new React Router route: shopify/app/routes/webhooks.shopify.orders.$.tsx.
    • Handles all orders/* topics via authenticate.webhook(request), while reading the canonical topic (orders/create, etc.) from the X-Shopify-Topic header.
    • Implements realistic failure behavior for the logs/retry story:
      • For orders/create, calls sendConfirmationText(customer.phone).
      • If phone is missing, the handler throws and returns a 500, simulating a real-world “missing phone, cannot send SMS” failure you’ll later fix and then retry via Hookdeck.
    • Removes the separate Express destination server; the Shopify app is now the single webhook destination.
  • Consistent webhook path and URL handling

    • Standardizes the webhook path to /webhooks/shopify/orders to emphasize a multi‑vendor pattern.
    • 01-hookdeck-upsert.ts:
      • Appends /webhooks/shopify/orders to the Hookdeck source URL when injecting webhook URIs into shopify.app.toml.
    • scripts/02-send-simulated-webhooks.ts:
      • Reads HOOKDECK_SOURCE_URL from .env and appends /webhooks/shopify/orders when sending simulated Shopify webhooks.
    • scripts/03-trigger-shopify-webhook.ts (new):
      • Uses shopify app webhook trigger to send real Shopify webhooks through Hookdeck to /webhooks/shopify/orders.
      • Accepts --topic and uses:
        • --address {HOOKDECK_SOURCE_URL}/webhooks/shopify/orders
        • --api-version 2026-01 to avoid interactive prompts.
        • --path ./shopify so the CLI finds the right app.
  • Port and dev‑experience hardening

    • Pins the React Router app to port 4000:
      • Sets port = 4000 in shopify/shopify.web.toml.
      • Updates shopify/vite.config.ts to default to port 4000 and enables strictPort: true so Vite fails instead of picking a random port.
      • Updates scripts (including 01-hookdeck-upsert.sh) and docs to consistently reference port 4000.
    • Cleans up dependencies:
      • Removes Express and related types from package.json and package-lock.json.
      • Adds @iarna/toml and @types/iarna__toml for safe TOML parsing/writing.
  • Documentation for a first‑time demo

    • Overhauls shopify-webhooks-at-scale/README.md to describe this as a new, end‑to‑end demo:
      • New setup flow: run shopify app dev first, then npm run upsert to configure Hookdeck and app webhooks.
      • Explains how HOOKDECK_API_KEY and HOOKDECK_SOURCE_URL are used and automatically managed.
      • Documents the three demo stories:
        • Demo 1 – Setup & configuration (Shopify + Hookdeck wiring, webhooks in shopify.app.toml).
        • Demo 2 – Backpressure (HTTP destination, rate limiting, topic filtering over orders/*).
        • Demo 3 – Logs & retry (phone‑number failure, local debugging via CLI destination, then production retries via Hookdeck).
      • Removes all references to the old template (shopify.app.toml.template) and the removed Express destination server.

Why This Matters

  • Deterministic demos: Everything needed for the three Shopify + Hookdeck demos is now expressed in code and scripts, not in ad‑hoc CLI commands.
  • Realistic story: The missing‑phone failure and subsequent fix mirror real production issues, making the logs/retry demo compelling.
  • Single source of truth: shopify.app.toml, Hookdeck connections, and .env stay in sync via the TypeScript upsert script and well‑documented flows.

- Created package.json for project dependencies and scripts.
- Added public HTML for demo landing page with styling.
- Implemented TTS request handling in app.js with dynamic request loading.
- Developed TTS demo HTML with form for text input and model selection.
- Styled TTS demo page with CSS for better user experience.
- Built Express router for TTS API, including request persistence and audio file handling.
- Integrated Deepgram API for TTS generation with callback handling.
- Set up server with environment variable validation and static file serving.
- Configured TypeScript settings for the project.
…capabilities

- Implemented a new STT (Speech-to-Text) demo in the Deepgram integration.
- Added frontend components for audio recording, playback, and transcription request management.
- Created backend routes for handling audio uploads and transcription requests.
- Integrated Deepgram's STT API for processing audio files and receiving transcription results via webhooks.
- Updated package.json to include necessary dependencies for multer and types.
- Enhanced UI with status messages and auto-refresh functionality for transcription requests.
- Added .gitignore to exclude dependencies, environment files, and IDE configurations.
- Created package.json for project dependencies and scripts for simulated webhook sending and demo destination.
- Developed plan.md outlining project objectives, requirements, and execution steps for the demo.
- Implemented README.md with setup instructions, demo checklists, and troubleshooting tips.
- Added TypeScript configuration (tsconfig.json) for project compilation.
- Created scripts for Hookdeck connection upsert, simulated webhook sending, and demo destination server.
- Included a template for Shopify app configuration (shopify.app.toml.template) to integrate with Hookdeck.
…hould be generated from the template and not be in source control
…velopment environments

- Updated script to create or update both production (HTTP destination) and development (CLI destination) Hookdeck connections.
- Added environment variable loading from a .env file for better configuration management.
- Improved error handling and output messages for connection upsert operations.
- Regenerated shopify.app.toml from a template, preserving existing client_id if present.
- Introduced a TypeScript script to create or update both production and development Hookdeck connections, improving local debugging capabilities.
- Updated README to reflect changes in setup instructions, including new environment variable requirements and detailed connection setup steps.
- Enhanced demo objectives and checklists to clarify the use of production and development connections for comprehensive testing scenarios.
Major refactoring to improve setup workflow and reduce manual steps:

- Remove SHOPIFY_CLIENT_SECRET env var requirement
  - Now dynamically fetches SHOPIFY_API_SECRET via `shopify app env show --path`
  - Uses execSync with --path flag for reliable directory resolution

- Add automatic .env file management
  - New updateEnvFile() function automatically adds/updates HOOKDECK_SOURCE_URL
  - Preserves existing env vars and formatting
  - Eliminates manual step of copying source URL

- Replace template-based approach with direct TOML manipulation
  - Remove dependency on shopify.app.toml.template
  - Parse and update existing shopify.app.toml using @iarna/toml
  - Preserve all existing configuration (client_id, other webhooks, etc.)
  - Add user confirmation prompt before replacing existing order webhooks

- Add validation and error handling
  - Validate shopify.app.toml exists before running
  - Better error messages for Shopify CLI failures
  - Graceful handling of .env file update failures

This change requires users to run `shopify app dev` first to create
shopify.app.toml, then the upsert script handles the rest automatically.
…t script

- Changed deduplication header from X-Shopify-Event-Id to x-shopify-event-id to match case sensitivity.
- Updated rate limit configuration from 5 requests per second to 1 request per minute to trigger backpressure
- Updated the script to allow inclusion or exclusion of customer.phone in the payload for testing failure scenarios.
- Improved environment variable loading by resolving the .env file path relative to the script location.
- Added command line flags `--no-customer-phone` and `--with-customer-phone` to control customer phone inclusion.
- Enhanced help message to reflect new options and clarify environment variable usage.
- Removed the alternative command example for sending simulated webhooks using `ts-node`, streamlining the instructions for users.
- Maintained focus on the primary method for sending simulated webhooks to enhance clarity.
- Move webhook handling from separate Express server to Shopify app
  - Add webhook handler at shopify/app/routes/webhooks.shopify.orders.$.tsx
  - Implement realistic error handling with sendConfirmationText() function
  - Remove Express dependencies (saves ~900 lines in package-lock.json)

- Update webhook path to /webhooks/shopify/orders for multi-vendor approach
  - Update upsert script to append path to Hookdeck source URL
  - Update simulated webhook sender to use new path
  - Update shopify.app.toml injection to include full webhook URI

- Simplify setup flow
  - Remove shopify.app.toml.template (now injects directly into existing file)
  - Update upsert script to read SHOPIFY_API_SECRET from shopify app env show
  - Automatically manage HOOKDECK_SOURCE_URL in .env file

- Fix port configuration
  - Set port to 4000 in shopify.web.toml
  - Update vite.config.ts default port to 4000 with strictPort
  - Update all documentation and scripts to use port 4000 consistently

- Add Shopify webhook trigger script
  - Add scripts/03-trigger-shopify-webhook.ts for triggering webhooks via CLI
  - Support --topic and --address flags

- Update README with new setup flow and webhook handler documentation
  - Document new Shopify app-first setup process
  - Update Demo 3 steps to reflect webhook handler in Shopify app
  - Remove references to old destination server
- Make SHOPIFY_CLIENT_SECRET required in simulation script (always include HMAC signature)
- Automatically set SHOPIFY_CLIENT_SECRET in .env via upsert script
- Clean up debug code from webhook handler
- Update README to reflect automatic SHOPIFY_CLIENT_SECRET setup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants