Skip to content

Pipeline fix #520

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dilip0515 merged 5 commits into from
Aug 1, 2025
Merged

Pipeline fix #520

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a299082
fix pipeline
dilip0515 Jul 31, 2025
0dd71c4
fix pipeline
dilip0515 Jul 31, 2025
56c2a3b
fix pipeline
dilip0515 Jul 31, 2025
17e1fd0
fix pipeline
dilip0515 Jul 31, 2025
74bbc1e
fix pipeline
dilip0515 Jul 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 57 additions & 42 deletions .github/workflows/pipeline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,6 @@ jobs:
- name: Configure AWS Namespace
env:
PR_NUMBER: ${{ github.event.number }}
# This is the branch name, or the git tag name
NS_BRANCH_OR_TAG: ${{ github.ref_name }}
run: |
echo "PR_NUMBER=${{ env.PR_NUMBER }}"
Expand All @@ -161,8 +160,6 @@ jobs:
test $(shasum -a 256 ./dce_linux_amd64.zip | awk '{print $1}') == "${expected_sha}"
unzip ./dce_linux_amd64.zip -d ./

# Lease a DCE account, to use for deploying our PR environment
# (deploy DCE in DCE)
- name: Lease DCE Account
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
Expand Down Expand Up @@ -201,18 +198,16 @@ jobs:
./dce --config=dce.yml leases login ${lease_id}
echo "${lease_id}" > ./lease_id.txt

# Install Terraform
- name: Install Terraform
uses: hashicorp/setup-terraform@v2
with:
terraform_version: ${{ inputs.terraform_version }}

# Configure the Terraform backend
- name: Configure Terraform Backend
run: |
lease_id=$(cat lease_id.txt)
./scripts/create-tf-backend.sh ${lease_id}
# terraform init

- name: Terraform Init/Apply
env:
NAMESPACE: ${{ env.namespace }}
Expand All @@ -223,7 +218,7 @@ jobs:
set -ex
cd modules
terraform init -input=false

export TF_VAR_global_tags='${{ vars.TF_VAR_GLOBAL_TAGS}}'
export TF_VAR_required_bucket_prefix="${TF_VAR_required_bucket_prefix}"

Expand All @@ -245,6 +240,41 @@ jobs:

- name: Build artifacts
run: ./scripts/build.sh

- name: Validate Build Artifacts
run: |
echo "Validating build artifacts..."
[ -f "./bin/build_artifacts.zip" ] && echo "build_artifacts.zip created" || (echo "build_artifacts.zip missing" && exit 1)
[ -f "./bin/terraform_artifacts.zip" ] && echo "terraform_artifacts.zip created" || (echo "terraform_artifacts.zip missing" && exit 1)
[ -f "./scripts/deploy.sh" ] && echo "deploy.sh exists" || (echo "deploy.sh missing" && exit 1)
[ -f "./scripts/restore_db.sh" ] && echo "restore_db.sh exists" || (echo "restore_db.sh missing" && exit 1)

# Validate file sizes
build_size=$(stat -c%s "./bin/build_artifacts.zip" 2>/dev/null || echo "0")
terraform_size=$(stat -c%s "./bin/terraform_artifacts.zip" 2>/dev/null || echo "0")

[ "$build_size" -gt 0 ] && echo "build_artifacts.zip has content ($build_size bytes)" || (echo "build_artifacts.zip is empty" && exit 1)
[ "$terraform_size" -gt 0 ] && echo "terraform_artifacts.zip has content ($terraform_size bytes)" || (echo "terraform_artifacts.zip is empty" && exit 1)

- name: Upload Build Artifacts for Release
uses: actions/upload-artifact@v4
with:
name: release_build_artifacts
path: ./bin/build_artifacts.zip

- name: Upload Terraform Artifacts for Release
uses: actions/upload-artifact@v4
with:
name: release_terraform_artifacts
path: ./bin/terraform_artifacts.zip

- name: Upload Deploy Scripts for Release
uses: actions/upload-artifact@v4
with:
name: release_deploy_scripts
path: |
./scripts/deploy.sh
./scripts/restore_db.sh

- name: DCE Lease Login
env:
Expand Down Expand Up @@ -313,26 +343,6 @@ jobs:
"${NAMESPACE}" \
"${ARTIFACTS_BUCKET}"

# Functional Tests --> TODO need to fix the test failures

# - name: Functional Tests
# run: |
# set -euxo pipefail
# mkdir -p junit-report

# go get github.com/jstemmer/go-junit-report
# go test -v ./tests/... -test.timeout 50m 2>&1 | tee >(go-junit-report > junit-report/functional.xml)

# Publish junit test results (for unit and functional tests) -- > TODO need to fix the test failures

# - name: Publish Test Results
# if: always()
# uses: actions/upload-artifact@v4
# with:
# name: Functional-Test-Results
# path: junit-report/functional.xml
# retention-days: 7

- name: Upload dce-cli Artifact
uses: actions/upload-artifact@v4
with:
Expand Down Expand Up @@ -397,7 +407,6 @@ jobs:
cp ${{ github.workspace }}/backend-tf/backend.tf ./modules/
chmod +x ./dce

# Cleanup the PR environment
- name: Terraform for Cleanup
uses: hashicorp/setup-terraform@v2
with:
Expand All @@ -419,7 +428,6 @@ jobs:
terraform init -input=false
terraform destroy -auto-approve

# End the DCE lease
- name: End DCE Lease
env:
AWS_DEFAULT_REGION: us-east-1
Expand Down Expand Up @@ -450,24 +458,31 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v3

- name: Download bin artifacts
- name: Download Build Artifacts
uses: actions/download-artifact@v4
with:
name: bin
name: release_build_artifacts
path: ./bin

- name: Download deploy_scripts artifacts
- name: Download Terraform Artifacts
uses: actions/download-artifact@v4
with:
name: deploy_scripts
path: ./deploy_scripts
name: release_terraform_artifacts
path: ./bin

- name: Download Deploy Scripts
uses: actions/download-artifact@v4
with:
name: release_deploy_scripts
path: ./scripts

- name: Verify Downloaded Artifacts
run: |
echo "Verifying downloaded artifacts..."
[ -f "./bin/build_artifacts.zip" ] && echo "build_artifacts.zip found" || echo "build_artifacts.zip missing"
[ -f "./bin/terraform_artifacts.zip" ] && echo "terraform_artifacts.zip found" || echo "terraform_artifacts.zip missing"
[ -f "./deploy_scripts/deploy.sh" ] && echo "deploy.sh found" || echo "deploy.sh missing"
[ -f "./deploy_scripts/restore_db.sh" ] && echo "restore_db.sh found" || echo "restore_db.sh missing"
[ -f "./scripts/deploy.sh" ] && echo "deploy.sh found" || echo "deploy.sh missing"
[ -f "./scripts/restore_db.sh" ] && echo "restore_db.sh found" || echo "restore_db.sh missing"

- name: Create GitHub Release
id: create_release
Expand All @@ -481,8 +496,8 @@ jobs:
- terraform_artifacts.zip
- deploy.sh
- restore_db.sh
draft: true
prerelease: true
draft: false
prerelease: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Expand All @@ -509,22 +524,22 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Upload Deploy Script
if: ${{ hashFiles('./deploy_scripts/deploy.sh') != '' }}
if: ${{ hashFiles('./scripts/deploy.sh') != '' }}
uses: actions/upload-release-asset@v1
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./deploy_scripts/deploy.sh
asset_path: ./scripts/deploy.sh
asset_name: deploy.sh
asset_content_type: text/x-shellscript
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Upload Restore DB Script
if: ${{ hashFiles('./deploy_scripts/restore_db.sh') != '' }}
if: ${{ hashFiles('./scripts/restore_db.sh') != '' }}
uses: actions/upload-release-asset@v1
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./deploy_scripts/restore_db.sh
asset_path: ./scripts/restore_db.sh
asset_name: restore_db.sh
asset_content_type: text/x-shellscript
env:
Expand Down