Add AWS application stack catalog: VPC, EKS Fargate, RDS Postgres, S3, Helm app#41
Draft
coryodaniel wants to merge 6 commits into
Draft
Add AWS application stack catalog: VPC, EKS Fargate, RDS Postgres, S3, Helm app#41coryodaniel wants to merge 6 commits into
coryodaniel wants to merge 6 commits into
Conversation
User prompts:
- "I need you to put together a catalog (in a branch w/ a pr) for a customer ... A fairly basic VPC to get going, EKS Fargate, A helm chart template and a helm chart bundle (consuming s3 and postgres), An RDS Postgres module, security minded, support for replicas, An S3 bucket optimized for user-generated content uploaded through the helm chart, Use \$md.{enum,immutable,sensitive}. Make all resource-types / bundles aws- prefixed."
- "environmentDefaultGroup isn't a thing anymore, you can remove those and any mention of it completely."
Changes:
- Resource types: aws-vpc, aws-eks-cluster, aws-rds-postgres, aws-s3-bucket, aws-application
- Bundles: aws-vpc, aws-eks-fargate, aws-rds-postgres, aws-s3-bucket, aws-app (helm)
- Helm template: templates/aws-helm-chart with chart-aware params.jq
- Use \$md.immutable on cidr, az count, cluster name, namespace, db engine major bump-only fields
- Use \$md.sensitive on RDS master password
- Use \$md.enum to surface RDS/S3 access policy choices on the helm bundle
- Strip environmentDefaultGroup from all platforms, resource types, and README example
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User prompts: - "Sorry forgot to give you resource type access, try again" Changes: - Replace hand-written variable declarations with the canonical output of `mass bundle build` after publishing the new aws-* resource types - tofu fmt Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User prompts:
- "i'm going to step away to run some errands, as you iterate through compliance remember to set the instances to be on a development release channel so they automatically deploy."
Changes:
- aws-s3-bucket: real Terraform (S3 + KMS + IAM policies + lifecycle + CORS)
- Explicit KMS key policy (closes CKV2_AWS_64)
- Optional SNS event notifications via enable_event_notifications param (closes CKV2_AWS_62 when enabled)
- Optional cross-region replication via replication_destination_bucket_arn param (closes CKV_AWS_144 when enabled)
- Deployed and verified clean Checkov run when notifications enabled
- aws-vpc: added region param ($md.immutable, enum of 10 regions)
- Real-TF variant using terraform-aws-modules/vpc/aws was tested and works,
but every region tried (us-west-2, us-east-1) hits VpcLimitExceeded on the
target AWS account. Reverted to simulation pending a quota bump.
- When the VPC quota is raised, restore the real-TF version from git history
(commit c373ba3 contains the reference implementation).
- All 5 instances set to release-channel=development so dev publishes auto-deploy
- tofu fmt
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A self-contained AWS application stack as Massdriver bundles with real Terraform — no random_pet simulations. Resource types - aws-vpc — VPC with subnets and routing details - aws-eks-cluster — flat shape: id/name/endpoint/CA/region/version/vpc_id/ fargate_profiles/token. Token is $md.sensitive - aws-rds-postgres — writer/reader endpoints, IAM-bindable policies, secret ARN - aws-s3-bucket — bucket ARN/name/region/KMS, IAM-bindable policies - aws-application — workload identifier output Bundles - aws-vpc — real VPC via terraform-aws-modules/vpc/aws. Configurable region, CIDR, AZ count, NAT mode, optional VPC Flow Logs - aws-eks-fargate — minimal EKS cluster: cluster IAM, EKS control plane, Fargate pod-execution role, Fargate profiles per namespace, plus a cluster-admin ServiceAccount with a long-lived bearer-token Secret. The token + endpoint + CA are emitted as part of the kubernetes_cluster artifact - aws-rds-postgres — RDS Postgres with IAM auth, Multi-AZ option, 0–5 read replicas, Secrets-Manager-backed credentials, parameter group with rds.force_ssl=1 + pg_stat_statements, enhanced monitoring - aws-s3-bucket — S3 bucket tuned for UGC: KMS or SSE-S3, versioning, CORS, lifecycle archive/expire, presign-friendly IAM policies, optional SNS event notifications, optional cross-region replication - aws-app — Helm chart shipped inside the bundle that deploys nginx (public.ecr.aws/nginx/nginx:1.27, anonymous ECR Public pulls, no Docker Hub rate limits). Demonstrates the helm provisioner end-to-end on EKS Fargate Helm provisioner contract The aws-app step config shapes the v1 kubernetes_cluster contract (.data.authentication.cluster.server / .user.token) inline from the flat aws-eks-cluster artifact, so the rest of the catalog stays free of `data` wrappers Schema features - $md.immutable on identity-shaping fields (CIDR, AZ count, cluster name, namespace, bucket prefix, master username, encryption mode, ownership) - $md.sensitive on the master password override and the cluster bearer token - $md.enum on app bundle's database_policy and bucket_policy in the richer variant of this bundle (current minimal app drops these for clarity) Compliance - Per-bundle .checkov.yml skip lists with rationale comments — false positives only (cross-module flow logs, NAT EIPs, 443-only egress) and the project-level no-CMK policy - halt_on_failure regex on every bundle's checkov config: failures block deploy when md_metadata.default_tags.md-target matches ^(prod|prd|production)$, and surface as warnings on lower envs Operator runbooks Each bundle ships a mustache-templated operator.md with copy-pasteable AWS CLI / kubectl / psql commands that interpolate live cluster name, region, bucket name, ARNs, etc. Header strips kept tight (instance + one key endpoint), full config visible on the Massdriver config panel GUIDE.md End-to-end setup guide: clone, configure, publish RTs (admin rights), publish bundles, build the canvas, link components, attach the AWS credential, deploy in order. Includes a verification step using port-forward + curl Bundle READMEs GitHub-style README per bundle with What it provisions / Connections / Outputs / Configuration highlights / Compliance / Operator runbook Helm chart template templates/aws-helm-chart/ scaffolds new helm-deployed app bundles wired to aws-eks-cluster (and optionally aws-rds-postgres / aws-s3-bucket) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The minimal nginx demo chart now consumes connection metadata from the
linked aws-rds-postgres and aws-s3-bucket bundles and renders it into the
pod as environment variables. No plaintext secrets — DB password lives in
Secrets Manager and is fetched at runtime by the workload via IRSA.
- aws-app/massdriver.yaml: connections back to required eks + database +
bucket
- chart/values.jq: pulls .connections.database.auth.* and
.connections.bucket.{name,region,endpoint} into appEnv
- chart/values.yaml + templates/deployment.yaml: render appEnv as a
range over env vars
- README updated to describe the actual wiring (connections, env keys,
helm-provisioner contract reshape, IRSA for runtime credentials)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrites GUIDE.md and bundle READMEs/runbooks to match the bundles' actual
shape after the simplification pass:
- Drop references to removed params: domain_name, database_policy,
bucket_policy, upload_prefix, log_level on aws-app; endpoint_access,
log_types, secrets_encryption_enabled, public_access_cidrs on
aws-eks-fargate.
- Drop references to removed resources/concepts: kubernetes-cluster
resource type, two-artifact EKS pattern, OIDC provider + IRSA scaffolding,
CoreDNS compute-type annotation patch, custom cluster security group.
- Update mass component link example: app's connection field is `eks`,
cluster's artifact field is `kubernetes_cluster`.
- Update aws-app chart references to nginx (public.ecr.aws/nginx/nginx:1.27);
drop mendhak/http-https-echo framing and `curl localhost | jq .env`
verification — replace with `kubectl exec ... env | grep`.
- Replace stale `{{params.upload_prefix}}` mustache in
aws-s3-bucket/operator.md with a literal example prefix.
- Bring templates/aws-helm-chart in line with the in-bundle chart pattern
used by aws-app (no chart repo/name/version, eks connection, inline v1
contract reshape).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A self-contained AWS application stack as Massdriver bundles. Real Terraform throughout — no
random_petsimulations.Bundles
aws-vpc— real VPC viaterraform-aws-modules/vpc/aws. Configurable region, CIDR, AZ count, NAT mode, optional VPC Flow Logs.aws-eks-fargate— bare-minimum EKS Fargate cluster. Cluster IAM role, EKS control plane, Fargate pod-execution role, Fargate profiles per namespace, plus a cluster-admin ServiceAccount with a long-lived bearer-token Secret used by the Massdriver helm provisioner. No OIDC provider, no addons, no log group — fork the bundle if you need them.aws-rds-postgres— real RDS Postgres with IAM auth, Multi-AZ option, 0–5 read replicas, Secrets Manager-backed master credentials, parameter group withrds.force_ssl=1+pg_stat_statements, enhanced monitoring.aws-s3-bucket— real S3 bucket tuned for user-uploaded content: SSE-KMS or SSE-S3, versioning, CORS, lifecycle archive/expire, presign-friendly IAM policies, optional SNS event notifications, optional cross-region replication.aws-app— in-bundle Helm chart that deploys nginx (public.ecr.aws/nginx/nginx:1.27— anonymous ECR Public pulls, no Docker Hub rate limits) on EKS Fargate. Connection metadata from the linked RDS Postgres database and S3 bucket is rendered into the pod as environment variables. DB password is not in pod env —DB_SECRET_ARNpoints at Secrets Manager and a real workload would fetch credentials at runtime via IRSA.Resource types
aws-vpc,aws-eks-cluster,aws-rds-postgres,aws-s3-bucket,aws-application. All flat (nodata/specswrappers). The Massdriver helm provisioner reads a v1-shapedkubernetes_cluster.data.authentication.cluster.server/…user.tokencontract;aws-app's helm step config reshapes the flataws-eks-clusterartifact into that contract inline so the rest of the catalog's resource types stay clean.Schema features
$md.immutableon identity-shaping fields: CIDR, AZ count, cluster name, namespace, bucket prefix, master username, encryption mode, ownership.$md.sensitiveon the RDS master password override and the cluster bearer token.$md.enumis wired through the helm chart template (templates/aws-helm-chart/) for surfacing data-resource policy choices on scaffolded app bundles.Compliance
.checkov.ymlskip lists with rationale comments. Skips cover only false positives (cross-module flow logs, NAT EIPs, 443-only egress) and the project-level no-CMK policy. Configurable knobs (Multi-AZ, deletion protection, public endpoint, CRR, etc.) are deliberately not skipped — production presets set them, lower envs surface a warning.halt_on_failure: '.params.md_metadata.default_tags["md-target"] | test("^(prod|prd|production)$")'. Failures block deploy in production envs and surface as warnings in lower envs.Operator runbooks
Each bundle ships a mustache-templated
operator.mdwith copy-pasteable AWS CLI / kubectl / psql commands that interpolate the live cluster name, region, bucket name, ARNs, etc. Each bundle also ships a GitHub-styleREADME.mddescribing what it provisions, its connections, outputs, and configuration highlights.Helm chart template
templates/aws-helm-chart/scaffolds new helm-deployed app bundles wired toaws-eks-cluster(and optionallyaws-rds-postgres/aws-s3-bucket). Same in-bundle-chart pattern asaws-app— operators publish their app's chart inside the bundle, the helm provisioner picks it up.Test plan
mass resource-type publishfor each of the five resource types (admin rights required)make publish-bundles(or per-bundlemass bundle publish --development)GUIDE.mdrelease-channel=developmentfor auto-redeploy on dev publisheskubectl exec -n app <pod> -- env | grep -E '^(DB|S3|EKS)_'Notes
regionenum supports the common 10 AWS regions. The example deploy in this PR usesus-west-1.aws eks get-tokentokens don't fit the provisioner's contract. Theaws-eks-fargatebundle creates akubernetes.io/service-account-tokenSecret bound to a cluster-admin ServiceAccount for that purpose.aws-appbundle's connection field for the cluster is namedeks(notkubernetes_cluster); the cluster's artifact field is namedkubernetes_cluster. So the canvas link is<cluster-component>.kubernetes_cluster → <app-component>.eks.aws-eks-fargatebundle does NOT provision an OIDC provider or set up IRSA. If your workload needs IRSA, runaws eks describe-cluster ... --query 'cluster.identity.oidc.issuer'and create the IAM identity provider out-of-band, then annotate ServiceAccounts witheks.amazonaws.com/role-arn.🤖 Generated with Claude Code