diff --git a/.github/actions/README.md b/.github/actions/README.md new file mode 100644 index 0000000..022b9dd --- /dev/null +++ b/.github/actions/README.md @@ -0,0 +1,42 @@ +# Composite actions + +Shared building blocks for the reusable workflows in +[`../workflows/`](../workflows/). + +Each action lives in its own folder and follows the same layout: + +``` +.github/actions// + action.yml # declares inputs, outputs, and the runs: using: composite block + run.ps1 # (optional) PowerShell logic invoked by the composite + run.sh # (optional) bash logic invoked by the composite +``` + +## Conventions + +- **Heavy logic lives in `.ps1` / `.sh` files**, not inline YAML. + Composite `run:` steps should be a single line that invokes the + script. This keeps the logic testable outside of GitHub Actions and + makes diffs readable. +- **No secrets in `with:`** — pass tokens via `env:` to avoid logging. +- **Inputs use `kebab-case`**, outputs use `kebab-case`. +- **Every input and output has a `description:`.** +- **Never interpolate `${{ inputs.* }}` or `${{ github.* }}` directly + inside a script body.** Pass them through `env:` and reference as + shell variables. +- **`shell:` is always explicit** on `run:` steps (`bash` or `pwsh`). + +## Referencing from a reusable workflow in this repo + +Composite actions are an implementation detail of the reusable +workflows. When a reusable workflow consumes one, reference it relative +to the repository root and pin to a full commit SHA so that callers who +pin the reusable workflow to a specific SHA get a fully reproducible +run: + +```yaml +- uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@ +``` + +The pins are rewritten on merge by the maintenance script (see plan). +Do not use `@main` for intra-repo composite references. diff --git a/.github/actions/apply-catalog-identifiers/action.yml b/.github/actions/apply-catalog-identifiers/action.yml new file mode 100644 index 0000000..535e3bd --- /dev/null +++ b/.github/actions/apply-catalog-identifiers/action.yml @@ -0,0 +1,25 @@ +name: Apply catalog identifiers +description: > + Rewrites the `id:` field in one or more `manifest.yml` files under + `CatalogInformation/` directories, using a list of + `=` mappings. Validates that each manifest + exists, that the identifier is a GUID, and that the manifest has an + active (non-commented) `id:` line. + +inputs: + mappings: + description: > + Newline-separated list of `=` entries. + When empty, the action is a no-op. + required: false + default: "" + +runs: + using: composite + steps: + - name: Update Catalog Identifiers + if: inputs.mappings != '' + shell: pwsh + env: + CATALOG_IDENTIFIERS: ${{ inputs.mappings }} + run: ${{ github.action_path }}/apply-catalog-identifiers.ps1 diff --git a/.github/actions/apply-catalog-identifiers/apply-catalog-identifiers.ps1 b/.github/actions/apply-catalog-identifiers/apply-catalog-identifiers.ps1 new file mode 100644 index 0000000..bdcbd66 --- /dev/null +++ b/.github/actions/apply-catalog-identifiers/apply-catalog-identifiers.ps1 @@ -0,0 +1,50 @@ +# Rewrites `id:` in manifest.yml files according to $CATALOG_IDENTIFIERS. +# Each mapping is "=", one per line. Comment lines (`# ...`) +# starting with '#' inside the manifest are preserved. +$ErrorActionPreference = 'Stop' + +$rawMappings = $env:CATALOG_IDENTIFIERS +$mappings = $rawMappings -split '[\r\n]+' | ForEach-Object { $_.Trim() } | Where-Object { $_ -ne '' } + +foreach ($mapping in $mappings) { + $splitIndex = $mapping.IndexOf('=') + if ($splitIndex -lt 1 -or $splitIndex -eq ($mapping.Length - 1)) { + Write-Error "Invalid entry '$mapping'. Expected format: =." + exit 1 + } + + $manifestPath = $mapping.Substring(0, $splitIndex).Trim() + $identifier = $mapping.Substring($splitIndex + 1).Trim() + + if ($identifier -notmatch '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$') { + Write-Error "Identifier '$identifier' is not a valid GUID." + exit 1 + } + + if ([System.IO.Path]::GetFileName($manifestPath) -ne 'manifest.yml') { + Write-Error "Manifest path '$manifestPath' must point to 'manifest.yml'." + exit 1 + } + + if (-not (Test-Path -Path $manifestPath -PathType Leaf)) { + Write-Error "Manifest file '$manifestPath' does not exist." + exit 1 + } + + $content = Get-Content -Path $manifestPath -Raw + $hasActiveIdLine = [regex]::IsMatch($content, '(?m)^(?!\s*#)\s*id:\s*.*$') + + if (-not $hasActiveIdLine) { + Write-Error "No active id line found in '$manifestPath'." + exit 1 + } + + $updatedContent = [regex]::Replace($content, '(?m)^(?!\s*#)\s*id:\s*.*$', "id: $identifier", 1) + + if ($updatedContent -eq $content) { + Write-Host "'$manifestPath' already has id: $identifier" + } else { + Set-Content -Path $manifestPath -Value $updatedContent -NoNewline + Write-Host "Updated '$manifestPath' to id: $identifier" + } +} diff --git a/.github/actions/apply-source-code-url/action.yml b/.github/actions/apply-source-code-url/action.yml new file mode 100644 index 0000000..ed2397b --- /dev/null +++ b/.github/actions/apply-source-code-url/action.yml @@ -0,0 +1,19 @@ +name: Apply source_code_url to manifest +description: > + For every `manifest.yml` under a `CatalogInformation/` directory, + fills in an empty `source_code_url:` field with the GitHub URL of + the current repository. + +inputs: + repository: + description: '`github.repository`. Inserted as `https://github.com/`.' + required: true + +runs: + using: composite + steps: + - name: Apply SourceCode Url To Manifest + shell: pwsh + env: + REPO: ${{ inputs.repository }} + run: ${{ github.action_path }}/apply-source-code-url.ps1 diff --git a/.github/actions/apply-source-code-url/apply-source-code-url.ps1 b/.github/actions/apply-source-code-url/apply-source-code-url.ps1 new file mode 100644 index 0000000..eff1a54 --- /dev/null +++ b/.github/actions/apply-source-code-url/apply-source-code-url.ps1 @@ -0,0 +1,25 @@ +# Fills in an empty `source_code_url:` field in every manifest.yml under a +# CatalogInformation directory with https://github.com/$REPO. +$ErrorActionPreference = 'Stop' + +$manifestFiles = Get-ChildItem -Recurse -Filter 'manifest.yml' | + Where-Object { $_.FullName -match '[\\/]CatalogInformation[\\/]' } + +foreach ($file in $manifestFiles) { + $lines = Get-Content -Path $file.FullName + $updated = $false + + for ($i = 0; $i -lt $lines.Count; $i++) { + if ($lines[$i] -match '^\s*source_code_url:\s*$') { + $indent = ($lines[$i] -match '^(\s*)source_code_url:')[1] + $lines[$i] = "$indent" + "source_code_url: 'https://github.com/$env:REPO'" + $updated = $true + break + } + } + + if ($updated) { + Write-Host "Updating: $($file.FullName) with 'source_code_url: https://github.com/$env:REPO'" + Set-Content -Path $file.FullName -Value $lines -Encoding UTF8 + } +} diff --git a/.github/actions/detect-test-runner/action.yml b/.github/actions/detect-test-runner/action.yml new file mode 100644 index 0000000..930375f --- /dev/null +++ b/.github/actions/detect-test-runner/action.yml @@ -0,0 +1,17 @@ +name: Detect test runner mode +description: > + Inspects `global.json` to determine whether the solution uses + Microsoft.Testing.Platform (MTP) or the default VSTest runner. + +outputs: + mode: + description: '`mtp` when MTP is configured in global.json, otherwise `vstest`.' + value: ${{ steps.detect.outputs.test-runner-mode }} + +runs: + using: composite + steps: + - name: Detect test runner mode + id: detect + shell: pwsh + run: ${{ github.action_path }}/detect-test-runner.ps1 diff --git a/.github/actions/detect-test-runner/detect-test-runner.ps1 b/.github/actions/detect-test-runner/detect-test-runner.ps1 new file mode 100644 index 0000000..08b3708 --- /dev/null +++ b/.github/actions/detect-test-runner/detect-test-runner.ps1 @@ -0,0 +1,18 @@ +# Detects the test runner configured in global.json. +$ErrorActionPreference = 'Stop' + +$globalJsonPath = Join-Path $env:GITHUB_WORKSPACE 'global.json' + +if (Test-Path $globalJsonPath) { + $jsonContent = Get-Content $globalJsonPath -Raw | ConvertFrom-Json + $runner = $jsonContent.test.runner + + if ($runner -eq 'Microsoft.Testing.Platform') { + Write-Host 'Detected Microsoft.Testing.Platform (MTP) test runner in global.json' + "test-runner-mode=mtp" | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append + return + } +} + +Write-Host 'Using default VSTest test runner' +"test-runner-mode=vstest" | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append diff --git a/.github/actions/guard-trigger/action.yml b/.github/actions/guard-trigger/action.yml new file mode 100644 index 0000000..5107a17 --- /dev/null +++ b/.github/actions/guard-trigger/action.yml @@ -0,0 +1,16 @@ +name: Guard trigger +description: > + Fails the workflow if it was invoked via the unsupported + `pull_request_target` trigger. Use as the first step of any + reusable workflow that must not run with elevated permissions on + untrusted pull request content. + +runs: + using: composite + steps: + - name: Check for unsupported trigger + if: github.event_name == 'pull_request_target' + shell: bash + run: | + echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." + exit 1 diff --git a/.github/actions/load-secrets/action.yml b/.github/actions/load-secrets/action.yml new file mode 100644 index 0000000..3f25e81 --- /dev/null +++ b/.github/actions/load-secrets/action.yml @@ -0,0 +1,50 @@ +name: Load secrets +description: > + Pulls a set of secrets from an Azure Key Vault, masks them, exports + them as environment variables in `$GITHUB_ENV`, and then applies any + caller-provided overrides (typically secrets passed into the + reusable workflow). Requires a prior `azure/login` step when + `use-oidc` is `true`. + +inputs: + vault-name: + description: Azure Key Vault name to pull secrets from. + required: false + default: kv-master-cicd-secrets + use-oidc: + description: When `true`, fetch from Key Vault. When `false`, skip the Key Vault pull and only apply overrides. + required: true + secret-names: + description: > + Newline-separated list of Key Vault secret names to pull. Each + name is uppercased and `-` becomes `_` to form the environment + variable name (`sonar-token` becomes `SONAR_TOKEN`). + required: false + default: "" + overrides: + description: > + Newline-separated list of `ENV_VAR=VALUE` pairs. Lines with an + empty value are skipped, so it is safe to pass caller secrets + directly even when they are not set. + required: false + default: "" + +runs: + using: composite + steps: + - name: Retrieve secrets from Azure Key Vault + if: inputs.use-oidc == 'true' && inputs.secret-names != '' + shell: bash + working-directory: ${{ github.action_path }} + env: + VAULT_NAME: ${{ inputs.vault-name }} + SECRET_NAMES: ${{ inputs.secret-names }} + run: bash ./load-from-keyvault.sh + + - name: Apply caller-provided overrides + if: inputs.overrides != '' + shell: bash + working-directory: ${{ github.action_path }} + env: + OVERRIDES: ${{ inputs.overrides }} + run: bash ./apply-overrides.sh diff --git a/.github/actions/load-secrets/apply-overrides.sh b/.github/actions/load-secrets/apply-overrides.sh new file mode 100644 index 0000000..63b2b51 --- /dev/null +++ b/.github/actions/load-secrets/apply-overrides.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Applies caller-provided ENV=VALUE overrides from $OVERRIDES. Empty values are +# skipped so callers can safely forward optional secrets unconditionally. +set -euo pipefail + +while IFS= read -r line; do + # Strip leading/trailing whitespace. + line="${line#"${line%%[![:space:]]*}"}" + line="${line%"${line##*[![:space:]]}"}" + [[ -z "$line" ]] && continue + + if [[ "$line" != *"="* ]]; then + echo "::warning::Ignoring malformed override line (no '='): $line" + continue + fi + + name="${line%%=*}" + value="${line#*=}" + + if [[ -z "$value" ]]; then + continue + fi + + echo "Using provided $name secret" + echo "::add-mask::$value" + echo "$name=$value" >> "$GITHUB_ENV" +done <<< "$OVERRIDES" diff --git a/.github/actions/load-secrets/load-from-keyvault.sh b/.github/actions/load-secrets/load-from-keyvault.sh new file mode 100644 index 0000000..54af5e7 --- /dev/null +++ b/.github/actions/load-secrets/load-from-keyvault.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# Pulls a newline-separated list of Key Vault secret names from $SECRET_NAMES, +# fetches each value from $VAULT_NAME with the Azure CLI, masks it, and exports +# it to $GITHUB_ENV. The env var name is the secret name uppercased with '-' +# replaced by '_'. +set -euo pipefail + +echo "Fetching secrets from Azure Key Vault '$VAULT_NAME'..." + +# Read newline-separated secret names into an array, dropping blank lines. +mapfile -t secret_names < <(printf '%s\n' "$SECRET_NAMES" | sed '/^[[:space:]]*$/d') + +for secret_name in "${secret_names[@]}"; do + secret_name="${secret_name//[[:space:]]/}" + [[ -z "$secret_name" ]] && continue + + env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') + + secret_value=$(az keyvault secret show --vault-name "$VAULT_NAME" --name "$secret_name" --query value -o tsv) + + echo "::add-mask::$secret_value" + echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" +done diff --git a/.github/actions/resolve-oidc/action.yml b/.github/actions/resolve-oidc/action.yml new file mode 100644 index 0000000..d2eba2d --- /dev/null +++ b/.github/actions/resolve-oidc/action.yml @@ -0,0 +1,73 @@ +name: Resolve OIDC parameters +description: > + Determines the Azure OIDC client id, tenant id, and subscription id + to use for `azure/login`. Returns the caller-provided values when + given, falls back to the SkylineCommunications defaults when running + inside that organization, and otherwise signals that OIDC must not + be attempted. + +inputs: + client-id: + description: Caller-provided Azure OIDC client id. Leave empty to fall back to defaults. + required: false + default: "" + tenant-id: + description: Caller-provided Azure OIDC tenant id. + required: false + default: "" + subscription-id: + description: Caller-provided Azure OIDC subscription id. + required: false + default: "" + repository-owner: + description: The repository owner (`github.repository_owner`). Used to enable Skyline defaults. + required: true + +outputs: + client-id: + description: Resolved Azure OIDC client id. + value: ${{ steps.set_oidc.outputs.client-id }} + tenant-id: + description: Resolved Azure OIDC tenant id. + value: ${{ steps.set_oidc.outputs.tenant-id }} + subscription-id: + description: Resolved Azure OIDC subscription id. + value: ${{ steps.set_oidc.outputs.subscription-id }} + use-oidc: + description: '`true` when OIDC parameters were resolved, `false` otherwise.' + value: ${{ steps.set_oidc.outputs.use-oidc }} + +runs: + using: composite + steps: + - name: Set Azure OIDC parameters + id: set_oidc + shell: bash + env: + OIDC_CLIENT_ID: ${{ inputs.client-id }} + OIDC_TENANT_ID: ${{ inputs.tenant-id }} + OIDC_SUBSCRIPTION_ID: ${{ inputs.subscription-id }} + REPO_OWNER: ${{ inputs.repository-owner }} + run: | + echo "Determining Azure OIDC parameters..." + + if [[ -n "$OIDC_CLIENT_ID" ]]; then + echo "Using provided OIDC parameters" + { + echo "client-id=$OIDC_CLIENT_ID" + echo "tenant-id=$OIDC_TENANT_ID" + echo "subscription-id=$OIDC_SUBSCRIPTION_ID" + echo "use-oidc=true" + } >> "$GITHUB_OUTPUT" + elif [[ "$REPO_OWNER" == "SkylineCommunications" ]]; then + echo "Using SkylineCommunications default OIDC parameters" + { + echo "client-id=c50da9cc-ba14-4138-8595-a62d97ab0e53" + echo "tenant-id=5f175691-8d1c-4932-b7c8-ce990839ac40" + echo "subscription-id=d6cbb8df-56eb-451d-9db7-67f49cba3220" + echo "use-oidc=true" + } >> "$GITHUB_OUTPUT" + else + echo "No OIDC parameters provided and owner does not match SkylineCommunications" + echo "use-oidc=false" >> "$GITHUB_OUTPUT" + fi diff --git a/.github/actions/setup-nuget-sources/action.yml b/.github/actions/setup-nuget-sources/action.yml new file mode 100644 index 0000000..43283ad --- /dev/null +++ b/.github/actions/setup-nuget-sources/action.yml @@ -0,0 +1,39 @@ +name: Setup NuGet sources +description: > + Registers the NuGet sources required by Skyline reusable workflows: + the per-owner GitHub Packages feed, and (when running inside + SkylineCommunications) the Skyline Cloud and Private Azure feeds. + Existing sources with the same name are updated in place so the + step is idempotent. + +inputs: + repository-owner: + description: The repository owner (`github.repository_owner`). + required: true + github-token: + description: Token used to authenticate against the GitHub Packages feed. + required: true + azure-token: + description: Personal access token for the Skyline Azure DevOps NuGet feeds. Optional; Skyline feeds are skipped when empty. + required: false + default: "" + include-skyline: + description: > + When `true`, register the Skyline Cloud + Private Azure NuGet + feeds in addition to GitHub Packages. Defaults to `auto`, which + enables them only when `repository-owner` is + `SkylineCommunications`. + required: false + default: auto + +runs: + using: composite + steps: + - name: Register NuGet sources + shell: pwsh + env: + REPO_OWNER: ${{ inputs.repository-owner }} + GH_TOKEN: ${{ inputs.github-token }} + AZURE_TOKEN: ${{ inputs.azure-token }} + INCLUDE_SKYLINE: ${{ inputs.include-skyline }} + run: ${{ github.action_path }}/setup-nuget-sources.ps1 diff --git a/.github/actions/setup-nuget-sources/setup-nuget-sources.ps1 b/.github/actions/setup-nuget-sources/setup-nuget-sources.ps1 new file mode 100644 index 0000000..819d7a8 --- /dev/null +++ b/.github/actions/setup-nuget-sources/setup-nuget-sources.ps1 @@ -0,0 +1,55 @@ +# Registers the NuGet sources required by Skyline reusable workflows. +# Idempotent: existing sources with the same name are updated in place. +# +# Inputs (via env vars): +# REPO_OWNER — the repository owner (used in the GitHub feed URL). +# GH_TOKEN — GitHub token for the GitHub Packages feed. +# AZURE_TOKEN — PAT for the Skyline Azure DevOps feeds. Optional. +# INCLUDE_SKYLINE — 'true' | 'false' | 'auto'. 'auto' enables Skyline feeds +# only when REPO_OWNER == 'SkylineCommunications'. +$ErrorActionPreference = 'Stop' + +function Register-NuGetSource { + param( + [Parameter(Mandatory)] [string] $Name, + [Parameter(Mandatory)] [string] $Url, + [Parameter(Mandatory)] [string] $Username, + [Parameter(Mandatory)] [string] $Password + ) + + if ([string]::IsNullOrEmpty($Password)) { + Write-Host "Skipping $Name because the password is not set." + return + } + + Write-Host "Checking source $Name..." + $existing = dotnet nuget list source | Select-String -Pattern $Name + + if ($existing) { + Write-Host "Updating existing source $Name." + dotnet nuget update source $Name --source $Url --username $Username --password $Password --store-password-in-clear-text + } else { + Write-Host "Adding new source $Name." + dotnet nuget add source $Url --name $Name --username $Username --password $Password --store-password-in-clear-text + } +} + +Register-NuGetSource -Name 'PrivateGitHubNugets' ` + -Url "https://nuget.pkg.github.com/$env:REPO_OWNER/index.json" ` + -Username 'USERNAME' -Password $env:GH_TOKEN + +$includeSkyline = switch ($env:INCLUDE_SKYLINE) { + 'true' { $true } + 'false' { $false } + default { $env:REPO_OWNER -eq 'SkylineCommunications' } +} + +if ($includeSkyline) { + Register-NuGetSource -Name 'CloudNuGets' ` + -Url 'https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json' ` + -Username 'az' -Password $env:AZURE_TOKEN + + Register-NuGetSource -Name 'PrivateAzureNuGets' ` + -Url 'https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json' ` + -Username 'az' -Password $env:AZURE_TOKEN +} diff --git a/.github/actions/sonarcloud-status/action.yml b/.github/actions/sonarcloud-status/action.yml new file mode 100644 index 0000000..af6a41e --- /dev/null +++ b/.github/actions/sonarcloud-status/action.yml @@ -0,0 +1,39 @@ +name: SonarCloud status +description: > + Queries the SonarCloud project status API and reports whether an + initial analysis is needed. Validates that the token is usable and + produces a clear error pointing at the right settings page when it + is not. + +inputs: + project-key: + description: SonarCloud project key. + required: true + branch: + description: Branch name to check status for. + required: true + token: + description: SonarCloud token. Passed as the basic-auth username. + required: true + repository: + description: '`github.repository` — used in the error message URL.' + required: true + +outputs: + needs-initial-analysis: + description: '`true` when SonarCloud reports no analysis yet, `false` otherwise.' + value: ${{ steps.status.outputs.needsInitialAnalysis }} + +runs: + using: composite + steps: + - name: Get SonarCloud Status + id: status + shell: bash + working-directory: ${{ github.action_path }} + env: + SONAR_PROJECT_NAME: ${{ inputs.project-key }} + SONAR_TOKEN: ${{ inputs.token }} + BRANCH_NAME: ${{ inputs.branch }} + REPO: ${{ inputs.repository }} + run: bash ./get-status.sh diff --git a/.github/actions/sonarcloud-status/get-status.sh b/.github/actions/sonarcloud-status/get-status.sh new file mode 100644 index 0000000..e8d1d92 --- /dev/null +++ b/.github/actions/sonarcloud-status/get-status.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# Queries the SonarCloud quality-gate status API for $SONAR_PROJECT_NAME on +# $BRANCH_NAME, validates the response, and writes `needsInitialAnalysis=true|false` +# to $GITHUB_OUTPUT. +set -euo pipefail + +sonarCloudProjectStatus=$(curl -s -u "$SONAR_TOKEN:" "https://sonarcloud.io/api/qualitygates/project_status?projectKey=$SONAR_PROJECT_NAME&branch=$BRANCH_NAME") + +if [ -z "$sonarCloudProjectStatus" ] || ! echo "$sonarCloudProjectStatus" | jq . > /dev/null 2>&1; then + echo "Error: The SONAR_TOKEN is invalid, expired, or the response is empty. Please check: https://sonarcloud.io/account/security and update your token: https://github.com/$REPO/settings/secrets/actions" >&2 + echo "Returned response: $sonarCloudProjectStatus" >&2 + exit 1 +fi + +if echo "$sonarCloudProjectStatus" | jq -e '.errors' > /dev/null 2>&1; then + echo "Error: SonarCloud API returned errors. Initial analysis needed." >&2 + echo "Returned response: $sonarCloudProjectStatus" >&2 + echo "needsInitialAnalysis=true" >> "$GITHUB_OUTPUT" + exit 0 +fi + +projectStatus=$(echo "$sonarCloudProjectStatus" | jq -r '.projectStatus.status // empty') +if [ "$projectStatus" = "NONE" ]; then + echo "Project status is NONE. Initial analysis needed." + echo "needsInitialAnalysis=true" >> "$GITHUB_OUTPUT" +else + echo "needsInitialAnalysis=false" >> "$GITHUB_OUTPUT" +fi + +echo "Returned response: $sonarCloudProjectStatus" diff --git a/.github/actions/update-global-json-sdks/action.yml b/.github/actions/update-global-json-sdks/action.yml new file mode 100644 index 0000000..dc90958 --- /dev/null +++ b/.github/actions/update-global-json-sdks/action.yml @@ -0,0 +1,114 @@ +name: Update global.json SDK versions +description: > + Updates `msbuild-sdks` entries in `global.json` to centrally-managed + versions. All managed SDKs are declared inside this action so that + version bumps happen in a single place. + + Two categories are supported: + + 1. The DataMiner SDK family: every key in `msbuild-sdks` matching one + of the configured DataMiner patterns (e.g. `Skyline.DataMiner.*`) + shares the same `DATAMINER_SDK_VERSION`. New sub-SDKs released in + that family are picked up automatically. + + 2. Other centrally-managed company SDKs that don't follow the + DataMiner naming and may live in different repositories. Each is + listed explicitly with its own version. + + Skips silently when `global.json` is absent or contains no + `msbuild-sdks` section. Keys not in any managed list are left + untouched. + +runs: + using: composite + steps: + - name: Update msbuild-sdks in global.json + shell: pwsh + run: | + # ------------------------------------------------------------ + # Central source of truth for all managed SDK versions. + # Update the values below to roll out new versions across the + # fleet via this reusable workflow. + # ------------------------------------------------------------ + + # Shared version for the entire Skyline.DataMiner SDK family. + $DATAMINER_SDK_VERSION = '2.5.2' + + # Regex patterns whose matching msbuild-sdks keys are bumped to + # $DATAMINER_SDK_VERSION. Add more patterns here when the + # DataMiner SDK family grows with differently-named members. + $dataMinerSdkPatterns = @( + '^Skyline\.DataMiner\..*' + ) + + # Other centrally-managed company SDKs (different naming, may + # live in different repositories, each with its own version). + # Exact-name matches here take precedence over the DataMiner + # pattern match below, so an SDK whose name happens to match + # the DataMiner family but is released independently (e.g. + # `Skyline.DataMiner.Sdk.WindowsInstaller`) can be pinned to + # its own version. Add new entries here as they appear. + $otherManagedSdks = @{ + # 'Skyline.DataMiner.Sdk.WindowsInstaller' = '1.0.0' + # 'Company.Other.Sdk' = '1.0.0' + } + + # ------------------------------------------------------------ + + $globalJsonPath = Join-Path $env:GITHUB_WORKSPACE "global.json" + + if (-Not (Test-Path $globalJsonPath)) { + Write-Host "No global.json found. Skipping update." + return + } + + $jsonContent = Get-Content $globalJsonPath -Raw | ConvertFrom-Json + + if (-not $jsonContent.'msbuild-sdks') { + Write-Host "global.json has no 'msbuild-sdks' section. Skipping update." + return + } + + $sdks = $jsonContent.'msbuild-sdks' + $changes = @() + + foreach ($prop in @($sdks.PSObject.Properties)) { + $name = $prop.Name + $oldValue = $prop.Value + $newValue = $null + + # 1. Explicit exact-name override wins. + if ($otherManagedSdks.ContainsKey($name)) { + $newValue = $otherManagedSdks[$name] + } + + # 2. Otherwise, fall back to the DataMiner SDK family pattern + # -> shared DATAMINER_SDK_VERSION. + if (-not $newValue) { + foreach ($pattern in $dataMinerSdkPatterns) { + if ($name -match $pattern) { + $newValue = $DATAMINER_SDK_VERSION + break + } + } + } + + if ($null -ne $newValue -and $oldValue -ne $newValue) { + $sdks.$name = $newValue + $changes += " ${name}: $oldValue -> $newValue" + } + } + + if ($changes.Count -eq 0) { + Write-Host "No msbuild-sdks entries needed updating." + return + } + + $updatedJson = $jsonContent | ConvertTo-Json -Depth 10 + $updatedJson | Set-Content $globalJsonPath -Encoding UTF8 + + Write-Host "Updated msbuild-sdks in global.json:" + $changes | ForEach-Object { Write-Host $_ } + Write-Host "" + Write-Host "New global.json:" + Write-Host $updatedJson diff --git a/.github/actions/validate-inputs/action.yml b/.github/actions/validate-inputs/action.yml new file mode 100644 index 0000000..036edb2 --- /dev/null +++ b/.github/actions/validate-inputs/action.yml @@ -0,0 +1,72 @@ +name: Validate inputs +description: > + Validates that required SonarCloud and DataMiner inputs / secrets are + present, and produces clear error messages with clickable URLs + pointing the developer at the right settings page. Intended to be + used early in a job, before any expensive build steps. + +inputs: + sonarcloud-project-name: + description: Value of the `sonarcloud-project-name` workflow input. Required when `check-sonar` is `true`. + required: false + default: "" + sonarcloud-token: + description: SonarCloud token. Required when `check-sonar` is `true`. + required: false + default: "" + dataminer-token: + description: DataMiner catalog token. Required when `check-dataminer` is `true`. + required: false + default: "" + repository: + description: '`github.repository`. Used in the error message URLs.' + required: true + run-number: + description: '`github.run_number`. Used in the publish-step example.' + required: true + has-dataminer-projects: + description: Whether the solution contains DataMiner projects. Controls the extra publish hint. + required: false + default: "false" + check-sonar: + description: When `true`, validates `sonarcloud-project-name` and `sonarcloud-token`. + required: false + default: "true" + check-dataminer: + description: When `true`, validates `dataminer-token` (typically only on tag pushes for DataMiner solutions). + required: false + default: "false" + +runs: + using: composite + steps: + - name: Validate SonarCloud Project Name + if: inputs.check-sonar == 'true' + shell: bash + working-directory: ${{ github.action_path }} + env: + SONAR_PROJECT_NAME: ${{ inputs.sonarcloud-project-name }} + HAS_DM_PROJECTS: ${{ inputs.has-dataminer-projects }} + REPO: ${{ inputs.repository }} + RUN_NUMBER: ${{ inputs.run-number }} + run: bash ./validate-sonar-name.sh + + - name: Validate SonarCloud Secret Token + if: inputs.check-sonar == 'true' + shell: bash + working-directory: ${{ github.action_path }} + env: + SONAR_TOKEN: ${{ inputs.sonarcloud-token }} + HAS_DM_PROJECTS: ${{ inputs.has-dataminer-projects }} + REPO: ${{ inputs.repository }} + RUN_NUMBER: ${{ inputs.run-number }} + run: bash ./validate-sonar-token.sh + + - name: Validate DataMiner Secret Token + if: inputs.check-dataminer == 'true' + shell: bash + working-directory: ${{ github.action_path }} + env: + DATAMINER_TOKEN: ${{ inputs.dataminer-token }} + REPO: ${{ inputs.repository }} + run: bash ./validate-dataminer-token.sh diff --git a/.github/actions/validate-inputs/validate-dataminer-token.sh b/.github/actions/validate-inputs/validate-dataminer-token.sh new file mode 100644 index 0000000..7fbe89b --- /dev/null +++ b/.github/actions/validate-inputs/validate-dataminer-token.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# Validates that $DATAMINER_TOKEN is set. +set -euo pipefail + +if [[ -n "$DATAMINER_TOKEN" ]]; then + exit 0 +fi + +echo "Error: dataminerToken is not set. Release not possible!" +echo "Please create or re-use an admin.dataminer.services token by visiting: https://admin.dataminer.services/." +echo "Navigate to the right organization, then go to Keys and create or find a key with the permissions Register catalog items, Download catalog versions, and Read catalog items." +echo "Copy the value of the token." +repo_url="https://github.com/$REPO/settings/secrets/actions" +echo "Then set a DATAMINER_TOKEN secret in your repository settings: $repo_url" +exit 1 diff --git a/.github/actions/validate-inputs/validate-sonar-name.sh b/.github/actions/validate-inputs/validate-sonar-name.sh new file mode 100644 index 0000000..b528212 --- /dev/null +++ b/.github/actions/validate-inputs/validate-sonar-name.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# Validates that $SONAR_PROJECT_NAME is set. Prints a developer-friendly +# error pointing at the right settings page when it is not. +set -euo pipefail + +if [[ -n "$SONAR_PROJECT_NAME" ]]; then + exit 0 +fi + +echo "Error: sonarcloud-project-name is not set." +echo "Please create a SonarCloud project by visiting: https://sonarcloud.io/projects/create and copy the id of the project as mentioned in the sonarcloud project url." +repo_url="https://github.com/$REPO/settings/variables/actions" +echo "Then set a SONAR_NAME variable in your repository settings: $repo_url" + +if [[ "$HAS_DM_PROJECTS" == "true" ]]; then + echo "Alternatively, if you do not wish to use the Skyline Quality Gate but intend to publish your results to the catalog, you may create your workflow to include the equivalent of a dotnet publish step as shown below:" + echo " - name: Publish" + echo " env:" + echo " api-key: \${{ secrets.DATAMINER_TOKEN }}" + echo " run: dotnet publish -p:Version=\"0.0.$RUN_NUMBER\" -p:VersionComment=\"Iterative Development\" -p:CatalogPublishKeyName=api-key" +fi + +exit 1 diff --git a/.github/actions/validate-inputs/validate-sonar-token.sh b/.github/actions/validate-inputs/validate-sonar-token.sh new file mode 100644 index 0000000..f496615 --- /dev/null +++ b/.github/actions/validate-inputs/validate-sonar-token.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +# Validates that $SONAR_TOKEN is set. +set -euo pipefail + +if [[ -n "$SONAR_TOKEN" ]]; then + exit 0 +fi + +echo "Error: sonarCloudToken is not set." +echo "Please create a SonarCloud token by visiting: https://sonarcloud.io/account/security and copy the value of the created token." +repo_url="https://github.com/$REPO/settings/secrets/actions" +echo "Then set a SONAR_TOKEN secret in your repository settings: $repo_url" + +if [[ "$HAS_DM_PROJECTS" == "true" ]]; then + echo "Alternatively, if you do not wish to use the Skyline Quality Gate but intend to publish your results to the catalog, you may create your workflow to include the equivalent of a dotnet publish step as shown below:" + echo " - name: Publish" + echo " env:" + echo " api-key: \${{ secrets.DATAMINER_TOKEN }}" + echo " run: dotnet publish -p:Version=\"0.0.$RUN_NUMBER\" -p:VersionComment=\"Iterative Development\" -p:CatalogPublishKeyName=api-key" +fi + +exit 1 diff --git a/.github/instructions/reusable-workflows.instructions.md b/.github/instructions/reusable-workflows.instructions.md new file mode 100644 index 0000000..98d6a3c --- /dev/null +++ b/.github/instructions/reusable-workflows.instructions.md @@ -0,0 +1,58 @@ +--- +description: "Conventions for editing reusable workflows and composite actions inside SkylineCommunications/_ReusableWorkflows." +applyTo: ".github/workflows/**,.github/actions/**" +--- + +# `_ReusableWorkflows` authoring conventions + +This file applies when editing workflows under `.github/workflows/` or composite actions under `.github/actions/` **inside the `_ReusableWorkflows` repo**. For broader org-wide guidance on consuming these workflows from caller repos, the `skyline-workflow-author` Copilot agent (org-level) is the source of truth. + +## Source-of-truth references + +- Master workflow catalog and deprecation rules: [README.md](../../README.md) +- Composite action conventions: [.github/actions/README.md](../actions/README.md) + +## Editing reusable workflows (`.github/workflows/*.yml`) + +- **Do not add another wrapper workflow.** New input scenarios go into [`Master Workflow.yml`](../workflows/Master%20Workflow.yml). The three deprecated wrappers (`NuGet Solution Master Workflow.yml`, `Internal NuGet Solution Master Workflow.yml`, `DataMiner App Packages Master Workflow.yml`) are kept only as redirects; do not extend them. +- **First step is `guard-trigger`** (or its SHA-pinned external form). It rejects `pull_request_target`. +- **OIDC parameters flow top-down.** Resolve once via `resolve-oidc` and pass `oidc-client-id` / `oidc-tenant-id` / `oidc-subscription-id` / `use-oidc` through to any sub-workflow that needs Key Vault access. +- **Secrets travel via `env:`, never `with:`.** Never log secrets. +- **Job-scoped `permissions:`** — start from the least set the job needs (`contents: read` minimum) and add only what is required. + +## Referencing composite actions from inside this repo + +```yaml +- uses: SkylineCommunications/_ReusableWorkflows/.github/actions/@ +``` + +- Pin to a **full commit SHA**. Never `@main` for intra-repo composite references; pins are rewritten on merge by the maintenance script. +- The relative form `uses: ./.github/actions/` is acceptable for the `guard-trigger` first step only, where SHA pinning would chicken-and-egg. + +## Adding or editing composite actions (`.github/actions//`) + +Mirrors [.github/actions/README.md](../actions/README.md): + +1. **Folder layout**: `action.yml` + `run.ps1` and/or `run.sh`. Heavy logic lives in the scripts. The composite `run:` step should be a single line invoking the script. +2. **Naming**: kebab-case folder, kebab-case inputs and outputs. `name:` and `description:` are required on the action and on every input/output. +3. **Always set explicit `shell:`** on `run:` steps (`bash` or `pwsh`). +4. **Pass `${{ inputs.* }}` and `${{ github.* }}` through `env:`** — never interpolate them inside a script body. Reference them as shell variables (`$env:FOO` in pwsh, `$FOO` in bash). +5. **No secrets in `with:`** — pass tokens via `env:` to avoid logging. +6. **Outputs are surfaced through a step `id:`**, for example `value: ${{ steps.detect.outputs.test-runner-mode }}`. +7. **Idempotency** when the action mutates external state (NuGet sources, manifest files). Re-running the same step twice must not break. + +## Migration workflows are part of the design + +When touching anything that interacts with the deprecated redirecting wrappers, remember that a migration workflow exists and will run automatically: + +- [`Wrapper Migration Workflow.yml`](../workflows/Wrapper%20Migration%20Workflow.yml) — opens a PR rewriting callers off the deprecated NuGet / Internal NuGet / App Packages wrappers. + +Do not duplicate this migration logic in other workflows; extend the existing migration workflow instead. + +## Forbidden patterns + +- `pull_request_target` triggers (the `guard-trigger` action fails the run). +- `@main` for intra-repo composite action references. +- Interpolating `${{ inputs.* }}` or `${{ github.* }}` inside `.ps1` / `.sh` scripts invoked by composite actions. +- Passing secrets through `with:`. +- Echoing secrets to stdout. diff --git a/.github/workflows/Automation Master Legacy Workflow.yml b/.github/workflows/Automation Master Legacy Workflow.yml index 4e75242..e19ef03 100644 --- a/.github/workflows/Automation Master Legacy Workflow.yml +++ b/.github/workflows/Automation Master Legacy Workflow.yml @@ -43,11 +43,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI validate_skyline_quality_gate: name: Legacy Skyline Quality Gate @@ -66,41 +62,16 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token" "sonar-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi - - if [[ -n "${{ secrets.sonarCloudToken }}" ]]; then - echo "Using provided sonarCloudToken secret" - echo "SONAR_TOKEN=${{ secrets.sonarCloudToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + sonar-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} + SONAR_TOKEN=${{ secrets.sonarCloudToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -177,39 +148,15 @@ jobs: - name: Get SonarCloud Status id: get-sonarcloud-status - run: | - sonarCloudProjectStatus=$(curl -s -u "${{ env.SONAR_TOKEN }}:" "https://sonarcloud.io/api/qualitygates/project_status?projectKey=${{ inputs.sonarCloudProjectName }}&branch=${{ github.ref_name }}") - - # Check if the response is empty or not valid JSON - if [ -z "$sonarCloudProjectStatus" ] || ! echo "$sonarCloudProjectStatus" | jq . > /dev/null 2>&1; then - echo "Error: The SONAR_TOKEN is invalid, expired, or the response is empty. Please check: https://sonarcloud.io/account/security and update your token: https://github.com/${{ github.repository }}/settings/secrets/actions" >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - exit 1 - fi - - # Check if the response contains errors - if echo "$sonarCloudProjectStatus" | jq -e '.errors' > /dev/null 2>&1; then - echo "Error: SonarCloud API returned errors. Initial analysis needed." >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - exit 0 - fi - - # Check if project status is NONE (needs initial analysis) - projectStatus=$(echo "$sonarCloudProjectStatus" | jq -r '.projectStatus.status // empty') - if [ "$projectStatus" = "NONE" ]; then - echo "Project status is NONE. Initial analysis needed." - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - else - echo "needsInitialAnalysis=false" >> $GITHUB_OUTPUT - fi - - # Output the JSON response if valid - echo "Returned response: $sonarCloudProjectStatus" - shell: bash + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/sonarcloud-status@AI + with: + project-key: ${{ inputs.sonarCloudProjectName }} + branch: ${{ github.ref_name }} + token: ${{ env.SONAR_TOKEN }} + repository: ${{ github.repository }} - name: Trigger Initial Analysis - if: steps.get-sonarcloud-status.outputs.needsInitialAnalysis == 'true' + if: steps.get-sonarcloud-status.outputs.needs-initial-analysis == 'true' env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any run: | @@ -360,36 +307,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -483,36 +408,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -615,36 +518,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("dataminer-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.api-key }}" ]]; then - echo "Using provided api-key secret" - echo "DATAMINER_TOKEN=${{ secrets.api-key }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + dataminer-token + overrides: | + DATAMINER_TOKEN=${{ secrets.api-key }} - name: Enable long paths run: git config --global core.longpaths true diff --git a/.github/workflows/Automation Master SDK Workflow.yml b/.github/workflows/Automation Master SDK Workflow.yml index 9abbd56..1d9b6dd 100644 --- a/.github/workflows/Automation Master SDK Workflow.yml +++ b/.github/workflows/Automation Master SDK Workflow.yml @@ -43,11 +43,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI validate_skyline_quality_gate: name: SDK Skyline Quality Gate @@ -66,41 +62,16 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token" "sonar-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi - - if [[ -n "${{ secrets.sonarCloudToken }}" ]]; then - echo "Using provided sonarCloudToken secret" - echo "SONAR_TOKEN=${{ secrets.sonarCloudToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + sonar-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} + SONAR_TOKEN=${{ secrets.sonarCloudToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -121,31 +92,12 @@ jobs: echo solutionFilePath=$(find . -type f -name '*.sln' -o -name '*.slnx') >> $GITHUB_OUTPUT shell: bash - - name: Enable Skyline NuGet Registries - if: github.repository_owner == 'SkylineCommunications' - run: | - $sources = @( - @{ Name = "PrivateGitHubNugets"; URL = "https://nuget.pkg.github.com/SkylineCommunications/index.json"; Username = "USERNAME"; Password = "${{ secrets.GITHUB_TOKEN }}" }, - @{ Name = "CloudNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" }, - @{ Name = "PrivateAzureNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" } - ) - - foreach ($source in $sources) { - if ($source.Password -ne "") { - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - } else { - Write-Host "Skipping $($source.Name) because the password is not set." - } - } - shell: pwsh + - name: Setup NuGet sources + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/setup-nuget-sources@AI + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + azure-token: ${{ env.AZURE_TOKEN }} - name: Install .NET Tools run: | @@ -163,39 +115,15 @@ jobs: - name: Get SonarCloud Status id: get-sonarcloud-status - run: | - sonarCloudProjectStatus=$(curl -s -u "${{ env.SONAR_TOKEN }}:" "https://sonarcloud.io/api/qualitygates/project_status?projectKey=${{ inputs.sonarCloudProjectName }}&branch=${{ github.ref_name }}") - - # Check if the response is empty or not valid JSON - if [ -z "$sonarCloudProjectStatus" ] || ! echo "$sonarCloudProjectStatus" | jq . > /dev/null 2>&1; then - echo "Error: The SONAR_TOKEN is invalid, expired, or the response is empty. Please check: https://sonarcloud.io/account/security and update your token: https://github.com/${{ github.repository }}/settings/secrets/actions" >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - exit 1 - fi - - # Check if the response contains errors - if echo "$sonarCloudProjectStatus" | jq -e '.errors' > /dev/null 2>&1; then - echo "Error: SonarCloud API returned errors. Initial analysis needed." >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - exit 0 - fi - - # Check if project status is NONE (needs initial analysis) - projectStatus=$(echo "$sonarCloudProjectStatus" | jq -r '.projectStatus.status // empty') - if [ "$projectStatus" = "NONE" ]; then - echo "Project status is NONE. Initial analysis needed." - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - else - echo "needsInitialAnalysis=false" >> $GITHUB_OUTPUT - fi - - # Output the JSON response if valid - echo "Returned response: $sonarCloudProjectStatus" - shell: bash + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/sonarcloud-status@AI + with: + project-key: ${{ inputs.sonarCloudProjectName }} + branch: ${{ github.ref_name }} + token: ${{ env.SONAR_TOKEN }} + repository: ${{ github.repository }} - name: Trigger Initial Analysis - if: steps.get-sonarcloud-status.outputs.needsInitialAnalysis == 'true' + if: steps.get-sonarcloud-status.outputs.needs-initial-analysis == 'true' env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any run: | @@ -315,36 +243,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -357,31 +263,12 @@ jobs: echo solutionFilePath=$(find . -type f -name '*.sln' -o -name '*.slnx') >> $GITHUB_OUTPUT shell: bash - - name: Enable Skyline NuGet Registries - if: github.repository_owner == 'SkylineCommunications' - run: | - $sources = @( - @{ Name = "PrivateGitHubNugets"; URL = "https://nuget.pkg.github.com/SkylineCommunications/index.json"; Username = "USERNAME"; Password = "${{ secrets.GITHUB_TOKEN }}" }, - @{ Name = "CloudNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" }, - @{ Name = "PrivateAzureNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" } - ) - - foreach ($source in $sources) { - if ($source.Password -ne "") { - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - } else { - Write-Host "Skipping $($source.Name) because the password is not set." - } - } - shell: pwsh + - name: Setup NuGet sources + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/setup-nuget-sources@AI + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + azure-token: ${{ env.AZURE_TOKEN }} - name: NuGet restore solution run: dotnet restore "${{ steps.findSlnFile.outputs.solutionFilePath }}" @@ -435,67 +322,26 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true - uses: actions/checkout@v6 - - name: Enable Skyline NuGet Registries - if: github.repository_owner == 'SkylineCommunications' - run: | - $sources = @( - @{ Name = "PrivateGitHubNugets"; URL = "https://nuget.pkg.github.com/SkylineCommunications/index.json"; Username = "USERNAME"; Password = "${{ secrets.GITHUB_TOKEN }}" }, - @{ Name = "CloudNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" }, - @{ Name = "PrivateAzureNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" } - ) - - foreach ($source in $sources) { - if ($source.Password -ne "") { - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - } else { - Write-Host "Skipping $($source.Name) because the password is not set." - } - } - shell: pwsh + - name: Setup NuGet sources + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/setup-nuget-sources@AI + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + azure-token: ${{ env.AZURE_TOKEN }} - name: Install .NET Tools run: | @@ -567,36 +413,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("dataminer-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.api-key }}" ]]; then - echo "Using provided api-key secret" - echo "DATAMINER_TOKEN=${{ secrets.api-key }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + dataminer-token + overrides: | + DATAMINER_TOKEN=${{ secrets.api-key }} - name: Enable long paths run: git config --global core.longpaths true diff --git a/.github/workflows/Automation Master Workflow.yml b/.github/workflows/Automation Master Workflow.yml index 6d56a73..2883283 100644 --- a/.github/workflows/Automation Master Workflow.yml +++ b/.github/workflows/Automation Master Workflow.yml @@ -60,11 +60,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI check_deprecated_item: name: Check deprecated items @@ -111,36 +107,18 @@ jobs: name: Check OIDC runs-on: ubuntu-latest outputs: - client-id: ${{ steps.set_oidc.outputs.client-id }} - tenant-id: ${{ steps.set_oidc.outputs.tenant-id }} - subscription-id: ${{ steps.set_oidc.outputs.subscription-id }} - use-oidc: ${{ steps.set_oidc.outputs.use-oidc }} + client-id: ${{ steps.resolve.outputs.client-id }} + tenant-id: ${{ steps.resolve.outputs.tenant-id }} + subscription-id: ${{ steps.resolve.outputs.subscription-id }} + use-oidc: ${{ steps.resolve.outputs.use-oidc }} steps: - - name: Set Azure OIDC parameters - id: set_oidc - run: | - echo "Determining Azure OIDC parameters..." - - if [[ -n "${{ inputs.oidc-client-id }}" ]]; then - echo "Using provided OIDC parameters" - { - echo "client-id=${{ inputs.oidc-client-id }}" - echo "tenant-id=${{ inputs.oidc-tenant-id }}" - echo "subscription-id=${{ inputs.oidc-subscription-id }}" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - elif [[ "${{ github.repository_owner }}" == "SkylineCommunications" ]]; then - echo "Using SkylineCommunications default OIDC parameters" - { - echo "client-id=c50da9cc-ba14-4138-8595-a62d97ab0e53" - echo "tenant-id=5f175691-8d1c-4932-b7c8-ce990839ac40" - echo "subscription-id=d6cbb8df-56eb-451d-9db7-67f49cba3220" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - else - echo "No OIDC parameters provided and owner does not match SkylineCommunications" - echo "use-oidc=false" >> "$GITHUB_OUTPUT" - fi + - id: resolve + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/resolve-oidc@AI + with: + client-id: ${{ inputs.oidc-client-id }} + tenant-id: ${{ inputs.oidc-tenant-id }} + subscription-id: ${{ inputs.oidc-subscription-id }} + repository-owner: ${{ github.repository_owner }} check_solution_type: name: Check Project Types diff --git a/.github/workflows/Connector Master Legacy Workflow.yml b/.github/workflows/Connector Master Legacy Workflow.yml index 253566f..d645a69 100644 --- a/.github/workflows/Connector Master Legacy Workflow.yml +++ b/.github/workflows/Connector Master Legacy Workflow.yml @@ -43,11 +43,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI validate_skyline_quality_gate: name: Legacy Skyline Quality Gate @@ -66,41 +62,16 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token" "sonar-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi - - if [[ -n "${{ secrets.sonarCloudToken }}" ]]; then - echo "Using provided sonarCloudToken secret" - echo "SONAR_TOKEN=${{ secrets.sonarCloudToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + sonar-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} + SONAR_TOKEN=${{ secrets.sonarCloudToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -177,39 +148,15 @@ jobs: - name: Get SonarCloud Status id: get-sonarcloud-status - run: | - sonarCloudProjectStatus=$(curl -s -u "${{ env.SONAR_TOKEN }}:" "https://sonarcloud.io/api/qualitygates/project_status?projectKey=${{ inputs.sonarCloudProjectName }}&branch=${{ github.ref_name }}") - - # Check if the response is empty or not valid JSON - if [ -z "$sonarCloudProjectStatus" ] || ! echo "$sonarCloudProjectStatus" | jq . > /dev/null 2>&1; then - echo "Error: The SONAR_TOKEN is invalid, expired, or the response is empty. Please check: https://sonarcloud.io/account/security and update your token: https://github.com/${{ github.repository }}/settings/secrets/actions" >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - exit 1 - fi - - # Check if the response contains errors - if echo "$sonarCloudProjectStatus" | jq -e '.errors' > /dev/null 2>&1; then - echo "Error: SonarCloud API returned errors. Initial analysis needed." >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - exit 0 - fi - - # Check if project status is NONE (needs initial analysis) - projectStatus=$(echo "$sonarCloudProjectStatus" | jq -r '.projectStatus.status // empty') - if [ "$projectStatus" = "NONE" ]; then - echo "Project status is NONE. Initial analysis needed." - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - else - echo "needsInitialAnalysis=false" >> $GITHUB_OUTPUT - fi - - # Output the JSON response if valid - echo "Returned response: $sonarCloudProjectStatus" - shell: bash + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/sonarcloud-status@AI + with: + project-key: ${{ inputs.sonarCloudProjectName }} + branch: ${{ github.ref_name }} + token: ${{ env.SONAR_TOKEN }} + repository: ${{ github.repository }} - name: Trigger Initial Analysis - if: steps.get-sonarcloud-status.outputs.needsInitialAnalysis == 'true' + if: steps.get-sonarcloud-status.outputs.needs-initial-analysis == 'true' env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any run: | @@ -360,36 +307,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -492,36 +417,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -629,36 +532,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("dataminer-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.api-key }}" ]]; then - echo "Using provided api-key secret" - echo "DATAMINER_TOKEN=${{ secrets.api-key }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + dataminer-token + overrides: | + DATAMINER_TOKEN=${{ secrets.api-key }} - name: Enable long paths run: git config --global core.longpaths true diff --git a/.github/workflows/Connector Master SDK Workflow.yml b/.github/workflows/Connector Master SDK Workflow.yml index 9704f1a..24c7199 100644 --- a/.github/workflows/Connector Master SDK Workflow.yml +++ b/.github/workflows/Connector Master SDK Workflow.yml @@ -43,11 +43,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI validate_skyline_quality_gate: name: SDK Skyline Quality Gate @@ -64,46 +60,18 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token" "sonar-token" "dataminer-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi - - if [[ -n "${{ secrets.sonarCloudToken }}" ]]; then - echo "Using provided sonarCloudToken secret" - echo "SONAR_TOKEN=${{ secrets.sonarCloudToken }}" >> "$GITHUB_ENV" - fi - - if [[ -n "${{ secrets.api-key }}" ]]; then - echo "Using provided api-key secret" - echo "DATAMINER_TOKEN=${{ secrets.api-key }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + sonar-token + dataminer-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} + SONAR_TOKEN=${{ secrets.sonarCloudToken }} + DATAMINER_TOKEN=${{ secrets.api-key }} - name: Enable long paths run: git config --global core.longpaths true @@ -130,31 +98,12 @@ jobs: dotnet tool install -g Skyline.DataMiner.CICD.Tools.Validator --version 3.* dotnet tool install -g Skyline.DataMiner.CICD.Tools.Sbom --version 1.* - - name: Enable Skyline NuGet Registries - if: github.repository_owner == 'SkylineCommunications' - run: | - $sources = @( - @{ Name = "PrivateGitHubNugets"; URL = "https://nuget.pkg.github.com/SkylineCommunications/index.json"; Username = "USERNAME"; Password = "${{ secrets.GITHUB_TOKEN }}" }, - @{ Name = "CloudNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" }, - @{ Name = "PrivateAzureNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" } - ) - - foreach ($source in $sources) { - if ($source.Password -ne "") { - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - } else { - Write-Host "Skipping $($source.Name) because the password is not set." - } - } - shell: pwsh + - name: Setup NuGet sources + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/setup-nuget-sources@AI + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + azure-token: ${{ env.AZURE_TOKEN }} - name: Prepare SonarCloud Variables id: prepSonarCloudVar @@ -167,39 +116,15 @@ jobs: - name: Get SonarCloud Status id: get-sonarcloud-status - run: | - sonarCloudProjectStatus=$(curl -s -u "${{ env.SONAR_TOKEN }}:" "https://sonarcloud.io/api/qualitygates/project_status?projectKey=${{ inputs.sonarCloudProjectName }}&branch=${{ github.ref_name }}") - - # Check if the response is empty or not valid JSON - if [ -z "$sonarCloudProjectStatus" ] || ! echo "$sonarCloudProjectStatus" | jq . > /dev/null 2>&1; then - echo "Error: The SONAR_TOKEN is invalid, expired, or the response is empty. Please check: https://sonarcloud.io/account/security and update your token: https://github.com/${{ github.repository }}/settings/secrets/actions" >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - exit 1 - fi - - # Check if the response contains errors - if echo "$sonarCloudProjectStatus" | jq -e '.errors' > /dev/null 2>&1; then - echo "Error: SonarCloud API returned errors. Initial analysis needed." >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - exit 0 - fi - - # Check if project status is NONE (needs initial analysis) - projectStatus=$(echo "$sonarCloudProjectStatus" | jq -r '.projectStatus.status // empty') - if [ "$projectStatus" = "NONE" ]; then - echo "Project status is NONE. Initial analysis needed." - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - else - echo "needsInitialAnalysis=false" >> $GITHUB_OUTPUT - fi - - # Output the JSON response if valid - echo "Returned response: $sonarCloudProjectStatus" - shell: bash + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/sonarcloud-status@AI + with: + project-key: ${{ inputs.sonarCloudProjectName }} + branch: ${{ github.ref_name }} + token: ${{ env.SONAR_TOKEN }} + repository: ${{ github.repository }} - name: Trigger Initial Analysis - if: steps.get-sonarcloud-status.outputs.needsInitialAnalysis == 'true' + if: steps.get-sonarcloud-status.outputs.needs-initial-analysis == 'true' env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any run: | @@ -379,67 +304,26 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true - uses: actions/checkout@v6 - - name: Enable Skyline NuGet Registries - if: github.repository_owner == 'SkylineCommunications' - run: | - $sources = @( - @{ Name = "PrivateGitHubNugets"; URL = "https://nuget.pkg.github.com/SkylineCommunications/index.json"; Username = "USERNAME"; Password = "${{ secrets.GITHUB_TOKEN }}" }, - @{ Name = "CloudNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" }, - @{ Name = "PrivateAzureNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" } - ) - - foreach ($source in $sources) { - if ($source.Password -ne "") { - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - } else { - Write-Host "Skipping $($source.Name) because the password is not set." - } - } - shell: pwsh + - name: Setup NuGet sources + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/setup-nuget-sources@AI + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + azure-token: ${{ env.AZURE_TOKEN }} - name: Install .NET Tools run: | @@ -509,36 +393,14 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.azureToken }}" ]]; then - echo "Using provided azureToken secret" - echo "AZURE_TOKEN=${{ secrets.azureToken }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + azure-token + overrides: | + AZURE_TOKEN=${{ secrets.azureToken }} - name: Enable long paths run: git config --global core.longpaths true @@ -555,31 +417,12 @@ jobs: echo solutionFilePath=$(find . -type f -name '*.sln' -o -name '*.slnx') >> $GITHUB_OUTPUT shell: bash - - name: Enable Skyline NuGet Registries - if: github.repository_owner == 'SkylineCommunications' - run: | - $sources = @( - @{ Name = "PrivateGitHubNugets"; URL = "https://nuget.pkg.github.com/SkylineCommunications/index.json"; Username = "USERNAME"; Password = "${{ secrets.GITHUB_TOKEN }}" }, - @{ Name = "CloudNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" }, - @{ Name = "PrivateAzureNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json"; Username = "az"; Password = "${{ env.AZURE_TOKEN }}" } - ) - - foreach ($source in $sources) { - if ($source.Password -ne "") { - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - } else { - Write-Host "Skipping $($source.Name) because the password is not set." - } - } - shell: pwsh + - name: Setup NuGet sources + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/setup-nuget-sources@AI + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + azure-token: ${{ env.AZURE_TOKEN }} - name: NuGet restore solution run: dotnet restore "${{ steps.findSlnFile.outputs.solutionFilePath }}" @@ -647,36 +490,21 @@ jobs: tenant-id: ${{ inputs.oidc-tenant-id }} subscription-id: ${{ inputs.oidc-subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: inputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("dataminer-token" "signing-client-id" "signing-client-secret" "signing-tenant-id" "signing-key-vault-certificate" "signing-key-vault-url" "protocol-signing-username" "protocol-signing-password") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.api-key }}" ]]; then - echo "Using provided api-key secret" - echo "DATAMINER_TOKEN=${{ secrets.api-key }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ inputs.use-oidc }} + secret-names: | + dataminer-token + signing-client-id + signing-client-secret + signing-tenant-id + signing-key-vault-certificate + signing-key-vault-url + protocol-signing-username + protocol-signing-password + overrides: | + DATAMINER_TOKEN=${{ secrets.api-key }} - name: Enable long paths run: git config --global core.longpaths true diff --git a/.github/workflows/Connector Master Workflow.yml b/.github/workflows/Connector Master Workflow.yml index ed8bf69..5e9f7c4 100644 --- a/.github/workflows/Connector Master Workflow.yml +++ b/.github/workflows/Connector Master Workflow.yml @@ -61,11 +61,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI check_deprecated_item: name: Check deprecated items @@ -112,36 +108,18 @@ jobs: name: Check OIDC runs-on: ubuntu-latest outputs: - client-id: ${{ steps.set_oidc.outputs.client-id }} - tenant-id: ${{ steps.set_oidc.outputs.tenant-id }} - subscription-id: ${{ steps.set_oidc.outputs.subscription-id }} - use-oidc: ${{ steps.set_oidc.outputs.use-oidc }} + client-id: ${{ steps.resolve.outputs.client-id }} + tenant-id: ${{ steps.resolve.outputs.tenant-id }} + subscription-id: ${{ steps.resolve.outputs.subscription-id }} + use-oidc: ${{ steps.resolve.outputs.use-oidc }} steps: - - name: Set Azure OIDC parameters - id: set_oidc - run: | - echo "Determining Azure OIDC parameters..." - - if [[ -n "${{ inputs.oidc-client-id }}" ]]; then - echo "Using provided OIDC parameters" - { - echo "client-id=${{ inputs.oidc-client-id }}" - echo "tenant-id=${{ inputs.oidc-tenant-id }}" - echo "subscription-id=${{ inputs.oidc-subscription-id }}" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - elif [[ "${{ github.repository_owner }}" == "SkylineCommunications" ]]; then - echo "Using SkylineCommunications default OIDC parameters" - { - echo "client-id=c50da9cc-ba14-4138-8595-a62d97ab0e53" - echo "tenant-id=5f175691-8d1c-4932-b7c8-ce990839ac40" - echo "subscription-id=d6cbb8df-56eb-451d-9db7-67f49cba3220" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - else - echo "No OIDC parameters provided and owner does not match SkylineCommunications" - echo "use-oidc=false" >> "$GITHUB_OUTPUT" - fi + - id: resolve + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/resolve-oidc@AI + with: + client-id: ${{ inputs.oidc-client-id }} + tenant-id: ${{ inputs.oidc-tenant-id }} + subscription-id: ${{ inputs.oidc-subscription-id }} + repository-owner: ${{ github.repository_owner }} check_solution_type: name: Check Project Types diff --git a/.github/workflows/DataMiner App Packages Master Workflow.yml b/.github/workflows/DataMiner App Packages Master Workflow.yml index fdaa2b7..290ecac 100644 --- a/.github/workflows/DataMiner App Packages Master Workflow.yml +++ b/.github/workflows/DataMiner App Packages Master Workflow.yml @@ -72,11 +72,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI check_deprecated_item: name: Check deprecated items @@ -137,4 +133,21 @@ jobs: SONAR_TOKEN: ${{ secrets.sonarCloudToken }} AZURE_TOKEN: ${{ secrets.azureToken }} DATAMINER_TOKEN: ${{ secrets.dataminerToken }} - OVERRIDE_CATALOG_DOWNLOAD_TOKEN: ${{ secrets.overrideCatalogDownloadToken }} \ No newline at end of file + OVERRIDE_CATALOG_DOWNLOAD_TOKEN: ${{ secrets.overrideCatalogDownloadToken }} + + request_wrapper_migration: + # The body of this workflow is a thin redirect to Master Workflow.yml. + # We ask the caller repo to migrate off this wrapper and call the + # master workflow directly. Idempotent and gated by the migration + # workflow itself - safe to run on every invocation. + if: github.event_name != 'pull_request' + name: Request wrapper migration + uses: ./.github/workflows/Wrapper Migration Workflow.yml + with: + wrapper-kind: app-packages + oidc-client-id: ${{ inputs.oidc-client-id }} + oidc-tenant-id: ${{ inputs.oidc-tenant-id }} + oidc-subscription-id: ${{ inputs.oidc-subscription-id }} + use-oidc: ${{ inputs.oidc-client-id != '' && 'true' || 'false' }} + debug: ${{ inputs.debug }} + secrets: inherit \ No newline at end of file diff --git a/.github/workflows/Internal NuGet Solution Master Workflow.yml b/.github/workflows/Internal NuGet Solution Master Workflow.yml index 84f53e4..5955d67 100644 --- a/.github/workflows/Internal NuGet Solution Master Workflow.yml +++ b/.github/workflows/Internal NuGet Solution Master Workflow.yml @@ -56,11 +56,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI check_deprecated_item: name: Check deprecated items @@ -117,4 +113,20 @@ jobs: secrets: NUGET_API_KEY: ${{ secrets.nugetApiKey }} SONAR_TOKEN: ${{ secrets.sonarCloudToken }} - AZURE_TOKEN: ${{ secrets.azureToken }} \ No newline at end of file + AZURE_TOKEN: ${{ secrets.azureToken }} + + request_wrapper_migration: + # The body of this workflow is a thin redirect to Master Workflow.yml. + # We ask the caller repo to migrate off this wrapper and call the + # master workflow directly. Idempotent and gated by the migration + # workflow itself - safe to run on every invocation. + if: github.event_name != 'pull_request' + name: Request wrapper migration + uses: ./.github/workflows/Wrapper Migration Workflow.yml + with: + wrapper-kind: internal-nuget + oidc-client-id: ${{ inputs.oidc-client-id }} + oidc-tenant-id: ${{ inputs.oidc-tenant-id }} + oidc-subscription-id: ${{ inputs.oidc-subscription-id }} + use-oidc: ${{ inputs.oidc-client-id != '' && 'true' || 'false' }} + secrets: inherit \ No newline at end of file diff --git a/.github/workflows/Master Workflow.yml b/.github/workflows/Master Workflow.yml index 1935483..ed5feb4 100644 --- a/.github/workflows/Master Workflow.yml +++ b/.github/workflows/Master Workflow.yml @@ -68,7 +68,6 @@ on: required: false env: - VERSION_SDK: '2.5.2' VERSION_APPPACKAGEINSTALLER: '4.0.0' jobs: @@ -77,52 +76,25 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI check_oidc: name: Check OIDC needs: [validate_trigger] runs-on: ubuntu-latest outputs: - client-id: ${{ steps.set_oidc.outputs.client-id }} - tenant-id: ${{ steps.set_oidc.outputs.tenant-id }} - subscription-id: ${{ steps.set_oidc.outputs.subscription-id }} - use-oidc: ${{ steps.set_oidc.outputs.use-oidc }} + client-id: ${{ steps.resolve.outputs.client-id }} + tenant-id: ${{ steps.resolve.outputs.tenant-id }} + subscription-id: ${{ steps.resolve.outputs.subscription-id }} + use-oidc: ${{ steps.resolve.outputs.use-oidc }} steps: - - name: Set Azure OIDC parameters - id: set_oidc - env: - OIDC_CLIENT_ID: ${{ inputs.oidc-client-id }} - OIDC_TENANT_ID: ${{ inputs.oidc-tenant-id }} - OIDC_SUBSCRIPTION_ID: ${{ inputs.oidc-subscription-id }} - REPO_OWNER: ${{ github.repository_owner }} - run: | - echo "Determining Azure OIDC parameters..." - - if [[ -n "$OIDC_CLIENT_ID" ]]; then - echo "Using provided OIDC parameters" - { - echo "client-id=$OIDC_CLIENT_ID" - echo "tenant-id=$OIDC_TENANT_ID" - echo "subscription-id=$OIDC_SUBSCRIPTION_ID" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - elif [[ "$REPO_OWNER" == "SkylineCommunications" ]]; then - echo "Using SkylineCommunications default OIDC parameters" - { - echo "client-id=c50da9cc-ba14-4138-8595-a62d97ab0e53" - echo "tenant-id=5f175691-8d1c-4932-b7c8-ce990839ac40" - echo "subscription-id=d6cbb8df-56eb-451d-9db7-67f49cba3220" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - else - echo "No OIDC parameters provided and owner does not match SkylineCommunications" - echo "use-oidc=false" >> "$GITHUB_OUTPUT" - fi + - id: resolve + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/resolve-oidc@AI + with: + client-id: ${{ inputs.oidc-client-id }} + tenant-id: ${{ inputs.oidc-tenant-id }} + subscription-id: ${{ inputs.oidc-subscription-id }} + repository-owner: ${{ github.repository_owner }} discover_projects: name: Discover Project Types @@ -198,117 +170,30 @@ jobs: tenant-id: ${{ needs.check_oidc.outputs.tenant-id }} subscription-id: ${{ needs.check_oidc.outputs.subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: needs.check_oidc.outputs.use-oidc == 'true' - shell: bash - env: - HAS_DM_PROJECTS: ${{ needs.discover_projects.outputs.has-dataminer-projects }} - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("azure-token" "sonar-token") - - if [[ "$HAS_DM_PROJECTS" == "true" ]]; then - secret_names+=("dataminer-token") - fi - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - env: - USER_AZURE_TOKEN: ${{ secrets.AZURE_TOKEN }} - USER_SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} - USER_DATAMINER_TOKEN: ${{ secrets.DATAMINER_TOKEN }} - run: | - if [[ -n "$USER_AZURE_TOKEN" ]]; then - echo "Using provided AZURE_TOKEN secret" - echo "AZURE_TOKEN=$USER_AZURE_TOKEN" >> "$GITHUB_ENV" - fi - - if [[ -n "$USER_SONAR_TOKEN" ]]; then - echo "Using provided SONAR_TOKEN secret" - echo "SONAR_TOKEN=$USER_SONAR_TOKEN" >> "$GITHUB_ENV" - fi - - if [[ -n "$USER_DATAMINER_TOKEN" ]]; then - echo "Using provided DATAMINER_TOKEN secret" - echo "DATAMINER_TOKEN=$USER_DATAMINER_TOKEN" >> "$GITHUB_ENV" - fi - - - name: Validate SonarCloud Project Name - id: validate-sonar-name - if: github.actor != 'dependabot[bot]' - env: - SONAR_PROJECT_NAME: ${{ inputs.sonarcloud-project-name }} - HAS_DM_PROJECTS: ${{ needs.discover_projects.outputs.has-dataminer-projects }} - REPO: ${{ github.repository }} - run: | - if [[ -z "$SONAR_PROJECT_NAME" ]]; then - echo "Error: sonarcloud-project-name is not set." - echo "Please create a SonarCloud project by visiting: https://sonarcloud.io/projects/create and copy the id of the project as mentioned in the sonarcloud project url." - repo_url="https://github.com/$REPO/settings/variables/actions" - echo "Then set a SONAR_NAME variable in your repository settings: $repo_url" - if [[ "$HAS_DM_PROJECTS" == "true" ]]; then - echo "Alternatively, if you do not wish to use the Skyline Quality Gate but intend to publish your results to the catalog, you may create your workflow to include the equivalent of a dotnet publish step as shown below (remove the \\):" - echo " - name: Publish" - echo " env:" - echo " api-key: $\{{ secrets.DATAMINER_TOKEN }}" - echo " run: dotnet publish -p:Version=\"0.0.$\{{ github.run_number }}\" -p:VersionComment=\"Iterative Development\" -p:CatalogPublishKeyName=api-key" - fi - exit 1 - fi - - - name: Validate SonarCloud Secret Token - id: validate-sonar-token - if: github.actor != 'dependabot[bot]' - env: - HAS_DM_PROJECTS: ${{ needs.discover_projects.outputs.has-dataminer-projects }} - REPO: ${{ github.repository }} - run: | - if [[ -z "$SONAR_TOKEN" ]]; then - echo "Error: sonarCloudToken is not set." - echo "Please create a SonarCloud token by visiting: https://sonarcloud.io/account/security and copy the value of the created token." - repo_url="https://github.com/$REPO/settings/secrets/actions" - echo "Then set a SONAR_TOKEN secret in your repository settings: $repo_url" - if [[ "$HAS_DM_PROJECTS" == "true" ]]; then - echo "Alternatively, if you do not wish to use the Skyline Quality Gate but intend to publish your results to the catalog, you may create your workflow to include the equivalent of a dotnet publish step as shown below (remove the \\):" - echo " - name: Publish" - echo " env:" - echo " api-key: $\{{ secrets.DATAMINER_TOKEN }}" - echo " run: dotnet publish -p:Version=\"0.0.$\{{ github.run_number }}\" -p:VersionComment=\"Iterative Development\" -p:CatalogPublishKeyName=api-key" - fi - exit 1 - fi - - - name: Validate DataMiner Secret Token - id: validate-dataminer-token - if: github.ref_type == 'tag' && needs.discover_projects.outputs.has-dataminer-projects == 'true' - env: - REPO: ${{ github.repository }} - run: | - if [[ -z "$DATAMINER_TOKEN" ]]; then - echo "Error: dataminerToken is not set. Release not possible!" - echo "Please create or re-use an admin.dataminer.services token by visiting: https://admin.dataminer.services/." - echo "Navigate to the right organization, then go to Keys and create or find a key with the permissions Register catalog items, Download catalog versions, and Read catalog items." - echo "Copy the value of the token." - repo_url="https://github.com/$REPO/settings/secrets/actions" - echo "Then set a DATAMINER_TOKEN secret in your repository settings: $repo_url" - exit 1 - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ needs.check_oidc.outputs.use-oidc }} + secret-names: | + azure-token + sonar-token + ${{ needs.discover_projects.outputs.has-dataminer-projects == 'true' && 'dataminer-token' || '' }} + overrides: | + AZURE_TOKEN=${{ secrets.AZURE_TOKEN }} + SONAR_TOKEN=${{ secrets.SONAR_TOKEN }} + DATAMINER_TOKEN=${{ secrets.DATAMINER_TOKEN }} + + - name: Validate inputs + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/validate-inputs@AI + with: + sonarcloud-project-name: ${{ inputs.sonarcloud-project-name }} + sonarcloud-token: ${{ env.SONAR_TOKEN }} + dataminer-token: ${{ env.DATAMINER_TOKEN }} + repository: ${{ github.repository }} + run-number: ${{ github.run_number }} + has-dataminer-projects: ${{ needs.discover_projects.outputs.has-dataminer-projects }} + check-sonar: ${{ github.actor != 'dependabot[bot]' }} + check-dataminer: ${{ github.ref_type == 'tag' && needs.discover_projects.outputs.has-dataminer-projects == 'true' }} - name: Enable long paths run: git config --global core.longpaths true @@ -323,48 +208,12 @@ jobs: with: packages: mono-complete - - name: Enable GitHub NuGet Registry - env: - REPO_OWNER: ${{ github.repository_owner }} - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - $source = @{ Name = "PrivateGitHubNugets"; URL = "https://nuget.pkg.github.com/$env:REPO_OWNER/index.json"; Username = "USERNAME"; Password = "$env:GH_TOKEN" } - - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - shell: pwsh - - - name: Enable Skyline NuGet Registries - if: github.repository_owner == 'SkylineCommunications' - run: | - $sources = @( - @{ Name = "CloudNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json"; Username = "az"; Password = "$env:AZURE_TOKEN" }, - @{ Name = "PrivateAzureNuGets"; URL = "https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json"; Username = "az"; Password = "$env:AZURE_TOKEN" } - ) - - foreach ($source in $sources) { - if ($source.Password -ne "") { - Write-Host "Checking source $($source.Name)..." - - if (dotnet nuget list source | Select-String -Pattern $source.Name) { - Write-Host "Updating existing source $($source.Name)." - dotnet nuget update source $source.Name --source $source.URL --username $source.Username --password $source.Password --store-password-in-clear-text - } else { - Write-Host "Adding new source $($source.Name)." - dotnet nuget add source $source.URL --name $source.Name --username $source.Username --password $source.Password --store-password-in-clear-text - } - } else { - Write-Host "Skipping $($source.Name) because the password is not set." - } - } - shell: pwsh + - name: Setup NuGet sources + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/setup-nuget-sources@AI + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + azure-token: ${{ env.AZURE_TOKEN }} - name: Install .NET tools env: @@ -386,152 +235,44 @@ jobs: fi shell: bash - - name: Validate NuGet projects - if: github.actor != 'dependabot[bot]' - env: - SOLUTION_PATH: ${{ needs.discover_projects.outputs.solution-path }} - REPO_OWNER: ${{ github.repository_owner }} - run: | - $minimal = "$env:REPO_OWNER" -ne "SkylineCommunications" - NuGetValidateSkylineSpecifications --solution-filepath "$env:SOLUTION_PATH" --minimal $minimal - shell: pwsh - # ────────────────────────────────────────────────────────────── # A. DataMiner-specific setup # ────────────────────────────────────────────────────────────── + - name: Update msbuild-sdks in global.json + if: needs.discover_projects.outputs.has-dataminer-projects == 'true' + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/update-global-json-sdks@AI + - name: Update Skyline.DataMiner.Core.AppPackageInstaller if: needs.discover_projects.outputs.has-dataminer-projects == 'true' env: SOLUTION_PATH: ${{ needs.discover_projects.outputs.solution-path }} run: NuGetChangeVersion --solution-filepath "$SOLUTION_PATH" --nuget-name Skyline.DataMiner.Core.AppPackageInstaller --nuget-version $VERSION_APPPACKAGEINSTALLER - - name: Update Skyline.DataMiner.Sdk version in global.json (if present) - if: needs.discover_projects.outputs.has-dataminer-projects == 'true' - shell: pwsh - run: | - $globalJsonPath = Join-Path $env:GITHUB_WORKSPACE "global.json" - - if (-Not (Test-Path $globalJsonPath)) { - Write-Host "No global.json found. Skipping update." - return - } - - $jsonContent = Get-Content $globalJsonPath -Raw | ConvertFrom-Json - - if (-not $jsonContent.'msbuild-sdks' -or -not $jsonContent.'msbuild-sdks'.PSObject.Properties['Skyline.DataMiner.Sdk']) { - Write-Host "Skyline.DataMiner.Sdk not found in global.json. Skipping update." - return - } - - $jsonContent.'msbuild-sdks'.'Skyline.DataMiner.Sdk' = $env:VERSION_SDK - - $updatedJson = $jsonContent | ConvertTo-Json -Depth 10 - - $updatedJson | Set-Content $globalJsonPath -Encoding UTF8 - - Write-Host "Updated global.json:" - Write-Host $updatedJson - - - name: Update Catalog Identifiers - if: inputs.override-catalog-identifiers != '' + - name: Validate NuGet projects + if: github.actor != 'dependabot[bot]' env: - CATALOG_IDENTIFIERS: ${{ inputs.override-catalog-identifiers }} + SOLUTION_PATH: ${{ needs.discover_projects.outputs.solution-path }} + REPO_OWNER: ${{ github.repository_owner }} run: | - $rawMappings = $env:CATALOG_IDENTIFIERS - $mappings = $rawMappings -split '[\r\n]+' | ForEach-Object { $_.Trim() } | Where-Object { $_ -ne '' } - - foreach ($mapping in $mappings) { - $splitIndex = $mapping.IndexOf('=') - if ($splitIndex -lt 1 -or $splitIndex -eq ($mapping.Length - 1)) { - Write-Error "Invalid entry '$mapping'. Expected format: =." - exit 1 - } - - $manifestPath = $mapping.Substring(0, $splitIndex).Trim() - $identifier = $mapping.Substring($splitIndex + 1).Trim() - - if ($identifier -notmatch '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$') { - Write-Error "Identifier '$identifier' is not a valid GUID." - exit 1 - } - - if ([System.IO.Path]::GetFileName($manifestPath) -ne 'manifest.yml') { - Write-Error "Manifest path '$manifestPath' must point to 'manifest.yml'." - exit 1 - } - - if (-not (Test-Path -Path $manifestPath -PathType Leaf)) { - Write-Error "Manifest file '$manifestPath' does not exist." - exit 1 - } - - $content = Get-Content -Path $manifestPath -Raw - $hasActiveIdLine = [regex]::IsMatch($content, '(?m)^(?!\s*#)\s*id:\s*.*$') - - if (-not $hasActiveIdLine) { - Write-Error "No active id line found in '$manifestPath'." - exit 1 - } - - $updatedContent = [regex]::Replace($content, '(?m)^(?!\s*#)\s*id:\s*.*$', "id: $identifier", 1) - - if ($updatedContent -eq $content) { - Write-Host "'$manifestPath' already has id: $identifier" - } else { - Set-Content -Path $manifestPath -Value $updatedContent -NoNewline - Write-Host "Updated '$manifestPath' to id: $identifier" - } - } + $minimal = "$env:REPO_OWNER" -ne "SkylineCommunications" + NuGetValidateSkylineSpecifications --solution-filepath "$env:SOLUTION_PATH" --minimal $minimal shell: pwsh - - name: Apply SourceCode Url To Manifest - if: needs.discover_projects.outputs.has-dataminer-projects == 'true' - env: - REPO: ${{ github.repository }} - run: | - $manifestFiles = Get-ChildItem -Recurse -Filter 'manifest.yml' | - Where-Object { $_.FullName -match '[\\/]CatalogInformation[\\/]' } - - foreach ($file in $manifestFiles) { - $lines = Get-Content -Path $file.FullName - $updated = $false - - for ($i = 0; $i -lt $lines.Count; $i++) { - if ($lines[$i] -match '^\s*source_code_url:\s*$') { - $indent = ($lines[$i] -match '^(\s*)source_code_url:')[1] - $lines[$i] = "$indent" + "source_code_url: 'https://github.com/$env:REPO'" - $updated = $true - break - } - } + - name: Apply catalog identifiers + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/apply-catalog-identifiers@AI + with: + mappings: ${{ inputs.override-catalog-identifiers }} - if ($updated) { - Write-Host "Updating: $($file.FullName) with 'source_code_url: https://github.com/$env:REPO'" - Set-Content -Path $file.FullName -Value $lines -Encoding UTF8 - } - } - shell: pwsh + - name: Apply source_code_url to manifest + if: needs.discover_projects.outputs.has-dataminer-projects == 'true' + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/apply-source-code-url@AI + with: + repository: ${{ github.repository }} - name: Detect test runner mode id: detect-test-mode - shell: pwsh - run: | - $globalJsonPath = Join-Path $env:GITHUB_WORKSPACE "global.json" - - if (Test-Path $globalJsonPath) { - $jsonContent = Get-Content $globalJsonPath -Raw | ConvertFrom-Json - $runner = $jsonContent.test.runner - - if ($runner -eq "Microsoft.Testing.Platform") { - Write-Host "Detected Microsoft.Testing.Platform (MTP) test runner in global.json" - echo "test-runner-mode=mtp" >> $env:GITHUB_OUTPUT - return - } - } - - Write-Host "Using default VSTest test runner" - echo "test-runner-mode=vstest" >> $env:GITHUB_OUTPUT + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/detect-test-runner@AI # ────────────────────────────────────────────────────────────── # B. SonarCloud analysis (skipped for Dependabot) @@ -547,43 +288,15 @@ jobs: - name: Get SonarCloud Status if: github.actor != 'dependabot[bot]' id: get-sonarcloud-status - env: - SONAR_PROJECT_NAME: ${{ inputs.sonarcloud-project-name }} - BRANCH_NAME: ${{ github.ref_name }} - REPO: ${{ github.repository }} - run: | - sonarCloudProjectStatus=$(curl -s -u "$SONAR_TOKEN:" "https://sonarcloud.io/api/qualitygates/project_status?projectKey=$SONAR_PROJECT_NAME&branch=$BRANCH_NAME") - - # Check if the response is empty or not valid JSON - if [ -z "$sonarCloudProjectStatus" ] || ! echo "$sonarCloudProjectStatus" | jq . > /dev/null 2>&1; then - echo "Error: The SONAR_TOKEN is invalid, expired, or the response is empty. Please check: https://sonarcloud.io/account/security and update your token: https://github.com/$REPO/settings/secrets/actions" >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - exit 1 - fi - - # Check if the response contains errors - if echo "$sonarCloudProjectStatus" | jq -e '.errors' > /dev/null 2>&1; then - echo "Error: SonarCloud API returned errors. Initial analysis needed." >&2 - echo "Returned response: $sonarCloudProjectStatus" >&2 - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - exit 0 - fi - - # Check if project status is NONE (needs initial analysis) - projectStatus=$(echo "$sonarCloudProjectStatus" | jq -r '.projectStatus.status // empty') - if [ "$projectStatus" = "NONE" ]; then - echo "Project status is NONE. Initial analysis needed." - echo "needsInitialAnalysis=true" >> $GITHUB_OUTPUT - else - echo "needsInitialAnalysis=false" >> $GITHUB_OUTPUT - fi - - # Output the JSON response if valid - echo "Returned response: $sonarCloudProjectStatus" - shell: bash + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/sonarcloud-status@AI + with: + project-key: ${{ inputs.sonarcloud-project-name }} + branch: ${{ github.ref_name }} + token: ${{ env.SONAR_TOKEN }} + repository: ${{ github.repository }} - name: Trigger Initial Analysis - if: github.actor != 'dependabot[bot]' && steps.get-sonarcloud-status.outputs.needsInitialAnalysis == 'true' + if: github.actor != 'dependabot[bot]' && steps.get-sonarcloud-status.outputs.needs-initial-analysis == 'true' env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} SONAR_PROJECT_NAME: ${{ inputs.sonarcloud-project-name }} @@ -869,20 +582,16 @@ jobs: tenant-id: ${{ needs.check_oidc.outputs.tenant-id }} subscription-id: ${{ needs.check_oidc.outputs.subscription-id }} - - name: Retrieve signing secrets from Azure Key Vault - if: needs.check_oidc.outputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - secret_names=("signing-client-id" "signing-client-secret" "signing-tenant-id" "signing-key-vault-certificate" "signing-key-vault-url") - - for secret_name in "${secret_names[@]}"; do - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - echo "::add-mask::$secret_value" - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done + - name: Load signing secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ needs.check_oidc.outputs.use-oidc }} + secret-names: | + signing-client-id + signing-client-secret + signing-tenant-id + signing-key-vault-certificate + signing-key-vault-url - name: Download Unsigned NuGet uses: actions/download-artifact@v8 @@ -938,30 +647,14 @@ jobs: tenant-id: ${{ needs.check_oidc.outputs.tenant-id }} subscription-id: ${{ needs.check_oidc.outputs.subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: needs.check_oidc.outputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - secret_names=("nuget-api-key-github") - - for secret_name in "${secret_names[@]}"; do - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - echo "::add-mask::$secret_value" - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - env: - USER_NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }} - run: | - if [[ -n "$USER_NUGET_API_KEY" ]]; then - echo "Using provided NUGET_API_KEY secret" - echo "NUGET_API_KEY_GITHUB=$USER_NUGET_API_KEY" >> "$GITHUB_ENV" - fi + - name: Load NuGet push secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ needs.check_oidc.outputs.use-oidc }} + secret-names: | + nuget-api-key-github + overrides: | + NUGET_API_KEY_GITHUB=${{ secrets.NUGET_API_KEY }} - name: Download Signed NuGet uses: actions/download-artifact@v8 @@ -1024,30 +717,19 @@ jobs: tenant-id: ${{ needs.check_oidc.outputs.tenant-id }} subscription-id: ${{ needs.check_oidc.outputs.subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: needs.check_oidc.outputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - secret_names=("dataminer-token" "signing-client-id" "signing-client-secret" "signing-tenant-id" "signing-key-vault-certificate" "signing-key-vault-url") - - for secret_name in "${secret_names[@]}"; do - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - echo "::add-mask::$secret_value" - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - env: - USER_DATAMINER_TOKEN: ${{ secrets.DATAMINER_TOKEN }} - run: | - if [[ -n "$USER_DATAMINER_TOKEN" ]]; then - echo "Using provided DATAMINER_TOKEN secret" - echo "DATAMINER_TOKEN=$USER_DATAMINER_TOKEN" >> "$GITHUB_ENV" - fi + - name: Load catalog upload secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ needs.check_oidc.outputs.use-oidc }} + secret-names: | + dataminer-token + signing-client-id + signing-client-secret + signing-tenant-id + signing-key-vault-certificate + signing-key-vault-url + overrides: | + DATAMINER_TOKEN=${{ secrets.DATAMINER_TOKEN }} - name: Enable long paths run: git config --global core.longpaths true @@ -1143,31 +825,8 @@ jobs: fi shell: bash - - name: Update Skyline.DataMiner.Sdk version in global.json (if present) - shell: pwsh - run: | - $globalJsonPath = Join-Path $env:GITHUB_WORKSPACE "global.json" - - if (-Not (Test-Path $globalJsonPath)) { - Write-Host "No global.json found. Skipping update." - return - } - - $jsonContent = Get-Content $globalJsonPath -Raw | ConvertFrom-Json - - if (-not $jsonContent.'msbuild-sdks' -or -not $jsonContent.'msbuild-sdks'.PSObject.Properties['Skyline.DataMiner.Sdk']) { - Write-Host "Skyline.DataMiner.Sdk not found in global.json. Skipping update." - return - } - - $jsonContent.'msbuild-sdks'.'Skyline.DataMiner.Sdk' = $env:VERSION_SDK - - $updatedJson = $jsonContent | ConvertTo-Json -Depth 10 - - $updatedJson | Set-Content $globalJsonPath -Encoding UTF8 - - Write-Host "Updated global.json:" - Write-Host $updatedJson + - name: Update msbuild-sdks in global.json + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/update-global-json-sdks@AI - name: Publish To Catalog shell: pwsh diff --git a/.github/workflows/NuGet Solution Master Workflow.yml b/.github/workflows/NuGet Solution Master Workflow.yml index 4a3fa9d..828cdaf 100644 --- a/.github/workflows/NuGet Solution Master Workflow.yml +++ b/.github/workflows/NuGet Solution Master Workflow.yml @@ -60,11 +60,7 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI check_deprecated_item: name: Check deprecated items @@ -126,4 +122,20 @@ jobs: secrets: NUGET_API_KEY: ${{ secrets.nugetApiKey }} SONAR_TOKEN: ${{ secrets.sonarCloudToken }} - AZURE_TOKEN: ${{ secrets.azureToken }} \ No newline at end of file + AZURE_TOKEN: ${{ secrets.azureToken }} + + request_wrapper_migration: + # The body of this workflow is a thin redirect to Master Workflow.yml. + # We ask the caller repo to migrate off this wrapper and call the + # master workflow directly. Idempotent and gated by the migration + # workflow itself - safe to run on every invocation. + if: github.event_name != 'pull_request' + name: Request wrapper migration + uses: ./.github/workflows/Wrapper Migration Workflow.yml + with: + wrapper-kind: nuget + oidc-client-id: ${{ inputs.oidc-client-id }} + oidc-tenant-id: ${{ inputs.oidc-tenant-id }} + oidc-subscription-id: ${{ inputs.oidc-subscription-id }} + use-oidc: ${{ inputs.oidc-client-id != '' && 'true' || 'false' }} + secrets: inherit \ No newline at end of file diff --git a/.github/workflows/SRM Function Master Workflow.yml b/.github/workflows/SRM Function Master Workflow.yml deleted file mode 100644 index d247a28..0000000 --- a/.github/workflows/SRM Function Master Workflow.yml +++ /dev/null @@ -1,217 +0,0 @@ -name: SRM Function Master Workflow - -# Controls when the workflow will run -on: - # Allows you to run this workflow from another workflow - workflow_call: - outputs: - quality_gate: - description: "Results from Skyline Quality Gate." - value: ${{ jobs.validate_skyline_quality_gate.outputs.quality}} - inputs: - referenceName: - required: true - type: string - runNumber: - required: true - type: string - referenceType: - required: true - type: string - repository: - required: true - type: string - owner: - required: true - type: string - sonarCloudProjectName: - required: true - type: string - secrets: - api-key: - required: true - sonarCloudToken: - required: true - azureToken: - required: false - -# A workflow run is made up of one or more jobs that can run sequentially or in parallel -jobs: - - validate_trigger: - name: Validate Trigger - runs-on: ubuntu-latest - steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 - - validate_skyline_quality_gate: - name: Skyline Quality Gate - needs: [validate_trigger] - runs-on: windows-latest - env: - detected-unit-tests: none - outputs: - quality: ${{ steps.quality-step.outputs.results}} - steps: - - uses: actions/checkout@v6 - - name: Initialize - run: | - echo "workspace" ${{ github.workspace }} - echo "ref name" ${{ inputs.referenceName }} - echo "run number" ${{ inputs.runNumber }} - echo "ref type" ${{ inputs.referenceType }} - echo "repository" ${{ inputs.repository }} - - name: Set up JDK 17 - uses: actions/setup-java@v5 - with: - java-version: 17 - distribution: 'zulu' - - name: Cache SonarCloud packages - uses: actions/cache@v5 - with: - path: ~\sonar\cache - key: ${{ runner.os }}-sonar - restore-keys: ${{ runner.os }}-sonar - - name: Cache SonarCloud scanner - id: cache-sonar-scanner - uses: actions/cache@v5 - with: - path: .\.sonar\scanner - key: ${{ runner.os }}-sonar-scanner - restore-keys: ${{ runner.os }}-sonar-scanner - - name: Install SonarCloud scanner - if: steps.cache-sonar-scanner.outputs.cache-hit != 'true' - run: | - dotnet tool install dotnet-sonarscanner --global - - name: Prepare SonarCloud Variables - id: prepSonarCloudVar - run: | - import os - env_file = os.getenv('GITHUB_ENV') - with open(env_file, "a") as myfile: - myfile.write("lowerCaseOwner=" + str.lower("${{ inputs.owner }}")) - shell: python - - # TODO: Refactor this in the future to a single stage with a loop that adds all the sources you specify. - - name: Enable Skyline GitHub NuGet Registry - if: inputs.owner == 'SkylineCommunications' - run: | - $SOURCE_NAME="PrivateGitHubNugets" - $SOURCE_URL="https://nuget.pkg.github.com/SkylineCommunications/index.json" - - # Check if the source exists. If it does, update it. - if (dotnet nuget list source | Select-String -Pattern $SOURCE_NAME) { - Write-Host "Updating existing source $SOURCE_NAME." - dotnet nuget update source $SOURCE_NAME --source $SOURCE_URL --username USERNAME --password ${{ secrets.GITHUB_TOKEN }} --store-password-in-clear-text - } else { - Write-Host "Adding new source $SOURCE_NAME." - dotnet nuget add source $SOURCE_URL --name $SOURCE_NAME --username USERNAME --password ${{ secrets.GITHUB_TOKEN }} --store-password-in-clear-text - } - shell: pwsh - - - name: Enable Skyline Azure Cloud NuGet Registry - env: - AZURE_TOKEN_EXISTS: ${{ secrets.azureToken }} - if: env.AZURE_TOKEN_EXISTS != null && inputs.owner == 'SkylineCommunications' - run: | - $SOURCE_NAME="CloudNuGets" - $SOURCE_URL="https://pkgs.dev.azure.com/skyline-cloud/Cloud_NuGets/_packaging/CloudNuGet/nuget/v3/index.json" - - # Check if the source exists. If it does, update it. - if (dotnet nuget list source | Select-String -Pattern $SOURCE_NAME) { - Write-Host "Updating existing source $SOURCE_NAME." - dotnet nuget update source $SOURCE_NAME --source $SOURCE_URL --username az --password ${{ secrets.azureToken }} --store-password-in-clear-text - } else { - Write-Host "Adding new source $SOURCE_NAME." - dotnet nuget add source $SOURCE_URL --name $SOURCE_NAME --username az --password ${{ secrets.azureToken }} --store-password-in-clear-text - } - - - name: Enable Skyline Azure Private NuGet Registry - env: - AZURE_TOKEN_EXISTS: ${{ secrets.azureToken }} - if: env.AZURE_TOKEN_EXISTS != null && inputs.owner == 'SkylineCommunications' - run: | - $SOURCE_NAME="PrivateAzureNuGets" - $SOURCE_URL="https://pkgs.dev.azure.com/skyline-cloud/_packaging/skyline-private-nugets/nuget/v3/index.json" - - # Check if the source exists. If it does, update it. - if (dotnet nuget list source | Select-String -Pattern $SOURCE_NAME) { - Write-Host "Updating existing source $SOURCE_NAME." - dotnet nuget update source $SOURCE_NAME --source $SOURCE_URL --username az --password ${{ secrets.azureToken }} --store-password-in-clear-text - } else { - Write-Host "Adding new source $SOURCE_NAME." - dotnet nuget add source $SOURCE_URL --name $SOURCE_NAME --username az --password ${{ secrets.azureToken }} --store-password-in-clear-text - } - - - name: Building - run: dotnet build -p:DefineConstants="DCFv1%3BDBInfo%3BALARM_SQUASHING" --configuration Release -nodeReuse:false - - name: Unit Tests - id: unit-tests - run: dotnet test --filter TestCategory!=IntegrationTest --logger \"trx;logfilename=unitTestResults.trx\" --collect \"XPlat Code Coverage\" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover - continue-on-error: true - - name: Get SonarCloud Status - id: get-sonarcloud-status - run: | - echo "sonarCloudProjectStatus=$(curl https://${{ secrets.sonarCloudToken }}@sonarcloud.io/api/qualitygates/project_status?projectKey=${{ inputs.sonarCloudProjectName }})" >> $env:GITHUB_OUTPUT - continue-on-error: true - - name: Trigger Initial Analysis - if: fromJson(steps.get-sonarcloud-status.outputs.sonarCloudProjectStatus).projectStatus.status == 'NONE' - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any - SONAR_TOKEN: ${{ secrets.sonarCloudToken }} - run: | - dotnet sonarscanner begin /k:"${{ inputs.sonarCloudProjectName }}" /o:"${{ env.lowerCaseOwner }}" /d:sonar.token="${{ secrets.sonarCloudToken }}" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.cs.opencover.reportsPaths="**/TestResults/**/coverage.opencover.xml" /d:sonar.cs.vstest.reportsPaths="**/TestResults/**.trx" - dotnet build -p:DefineConstants="DCFv1%3BDBInfo%3BALARM_SQUASHING" --configuration Release -nodeReuse:false - dotnet sonarscanner end /d:sonar.token="${{ secrets.sonarCloudToken }}" - continue-on-error: true - - name: Analyze - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any - SONAR_TOKEN: ${{ secrets.sonarCloudToken }} - run: | - dotnet sonarscanner begin /k:"${{ inputs.sonarCloudProjectName }}" /o:"${{ env.lowerCaseOwner }}" /d:sonar.token="${{ secrets.sonarCloudToken }}" /d:sonar.host.url="https://sonarcloud.io" /d:sonar.cs.opencover.reportsPaths="**/TestResults/**/coverage.opencover.xml" /d:sonar.cs.vstest.reportsPaths="**/TestResults/**.trx" - dotnet build -p:DefineConstants="DCFv1%3BDBInfo%3BALARM_SQUASHING" --configuration Release -nodeReuse:false - dotnet sonarscanner end /d:sonar.token="${{ secrets.sonarCloudToken }}" - continue-on-error: true - - name: SonarQube Quality Gate check - id: sonarqube-quality-gate-check - uses: sonarsource/sonarqube-quality-gate-action@master - with: - scanMetadataReportFile: .sonarqube/out/.sonar/report-task.txt - continue-on-error: true - # Force to fail step after specific time. - timeout-minutes: 5 - env: - SONAR_TOKEN: ${{ secrets.sonarCloudToken }} - - - name: Upload Sonar report-task.txt artifact - continue-on-error: true - uses: actions/upload-artifact@v7 - with: - name: sonar-report-task - path: .sonarqube/out/.sonar/report-task.txt - if-no-files-found: ignore - retention-days: 14 - - - name: Quality Gate - id: quality-step - run: | - if "${{ steps.unit-tests.outcome }}" == "failure" or "${{ steps.sonarqube-quality-gate-check.outputs.quality-gate-status }}" == "FAILED": - print("Quality gate failed due to:") - if "${{ steps.unit-tests.outcome }}" == "failure": - print("- Test failures") - if "${{ steps.sonarcloud-quality-gate-check.outcome }}" == "failure" and "${{ steps.sonarcloud-quality-gate-check.outputs.quality-gate-status }}" != "FAILED": - print("- Could not retrieve SonarCloud quality gate status, potentially due to reaching License max LoC. Ignoring SonarCloud quality gate status in this case.") - if "${{ steps.sonarqube-quality-gate-check.outputs.quality-gate-status }}" == "FAILED": - print("- Code analysis quality gate failed") - if "${{ steps.unit-tests.outcome }}" == "failure" or "${{ steps.sonarqube-quality-gate-check.outputs.quality-gate-status }}" == "FAILED": - exit(1) - shell: python - - name: Post Actions - run: echo "todo" - -#TODO: add Pack, Upload when this gets supported. \ No newline at end of file diff --git a/.github/workflows/Test Downstream.yml b/.github/workflows/Test Downstream.yml index ba25fe6..8784a7c 100644 --- a/.github/workflows/Test Downstream.yml +++ b/.github/workflows/Test Downstream.yml @@ -171,10 +171,45 @@ jobs: with: fetch-depth: 0 - - name: Force-push test-downstream tag + - name: Build self-consistent test commit and force-push tag if: steps.targets.outputs.has_targets == 'true' + env: + HEAD_SHA: ${{ steps.pr-details.outputs.head_sha }} run: | - git tag -f test-downstream ${{ steps.pr-details.outputs.head_sha }} + set -euo pipefail + git config user.email "github-actions[bot]@users.noreply.github.com" + git config user.name "github-actions[bot]" + + # Start from the PR head so the downstream sees the PR's + # version of every reusable workflow and composite action. + git checkout -B test-downstream-tmp "$HEAD_SHA" + + # Rewrite every cross-repo composite-action ref so that it + # resolves to the same tagged commit we are about to push. + # Without this, the master workflow files from the PR would + # still load composite actions from @main / @AI, hiding any + # regression introduced by the PR. + python - <<'PY' + import re, pathlib + pat = re.compile( + r'(SkylineCommunications/_ReusableWorkflows/\.github/actions/[A-Za-z0-9._-]+)@[A-Za-z0-9._/-]+' + ) + targets = list(pathlib.Path('.github/workflows').glob('*.yml')) \ + + list(pathlib.Path('.github/actions').glob('*/action.yml')) + for p in targets: + s = p.read_text(encoding='utf-8') + new = pat.sub(r'\1@test-downstream', s) + if new != s: + p.write_text(new, encoding='utf-8') + print(f'rewrote {p}') + PY + + if ! git diff --quiet; then + git add .github/workflows .github/actions + git commit -m "test-downstream: pin composite actions to @test-downstream" + fi + + git tag -f test-downstream git push -f origin test-downstream - name: Dispatch to target repos diff --git a/.github/workflows/Test composite actions.yml b/.github/workflows/Test composite actions.yml new file mode 100644 index 0000000..c610e9e --- /dev/null +++ b/.github/workflows/Test composite actions.yml @@ -0,0 +1,357 @@ +name: Test composite actions + +# Self-contained smoke tests for the composite actions in .github/actions/. +# Runs every composite that can be exercised without external secrets and +# asserts on its outputs / side effects. Composites that require live Azure +# Key Vault or live SonarCloud are covered by the downstream integration +# workflows, not here. + +on: + push: + paths: + - '.github/actions/**' + - '.github/workflows/Test composite actions.yml' + pull_request: + paths: + - '.github/actions/**' + - '.github/workflows/Test composite actions.yml' + workflow_dispatch: + +permissions: + contents: read + +jobs: + + guard-trigger: + name: guard-trigger + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - name: Invoke composite + uses: ./.github/actions/guard-trigger + - name: Confirm pass + run: echo "guard-trigger passed on event '${{ github.event_name }}'" + + resolve-oidc: + name: resolve-oidc + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Explicit inputs + id: explicit + uses: ./.github/actions/resolve-oidc + with: + client-id: 11111111-1111-1111-1111-111111111111 + tenant-id: 22222222-2222-2222-2222-222222222222 + subscription-id: 33333333-3333-3333-3333-333333333333 + repository-owner: NotSkyline + + - name: Assert explicit inputs were honored + env: + USE_OIDC: ${{ steps.explicit.outputs.use-oidc }} + CLIENT_ID: ${{ steps.explicit.outputs.client-id }} + run: | + test "$USE_OIDC" = "true" + test "$CLIENT_ID" = "11111111-1111-1111-1111-111111111111" + + - name: Skyline defaults + id: skyline + uses: ./.github/actions/resolve-oidc + with: + repository-owner: SkylineCommunications + + - name: Assert Skyline defaults + env: + USE_OIDC: ${{ steps.skyline.outputs.use-oidc }} + CLIENT_ID: ${{ steps.skyline.outputs.client-id }} + run: | + test "$USE_OIDC" = "true" + test "$CLIENT_ID" = "c50da9cc-ba14-4138-8595-a62d97ab0e53" + + - name: External owner, no inputs + id: external + uses: ./.github/actions/resolve-oidc + with: + repository-owner: SomeOtherOrg + + - name: Assert OIDC disabled + env: + USE_OIDC: ${{ steps.external.outputs.use-oidc }} + run: test "$USE_OIDC" = "false" + + load-secrets-overrides: + name: load-secrets (overrides only) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Invoke composite with overrides only + uses: ./.github/actions/load-secrets + with: + use-oidc: 'false' + overrides: | + TEST_FOO=hello + TEST_BAR= + TEST_BAZ=world + + - name: Assert overrides + run: | + test "$TEST_FOO" = "hello" + test "$TEST_BAZ" = "world" + # Empty override must be skipped. + test -z "${TEST_BAR:-}" + + setup-nuget-sources: + name: setup-nuget-sources + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Invoke composite (no Skyline feeds, no azure token) + uses: ./.github/actions/setup-nuget-sources + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + include-skyline: 'false' + + - name: Assert GitHub source registered + run: dotnet nuget list source | grep -q PrivateGitHubNugets + + - name: Re-invoke (idempotency check) + uses: ./.github/actions/setup-nuget-sources + with: + repository-owner: ${{ github.repository_owner }} + github-token: ${{ secrets.GITHUB_TOKEN }} + include-skyline: 'false' + + - name: Assert still exactly one entry + run: | + count=$(dotnet nuget list source | grep -c PrivateGitHubNugets) + test "$count" -eq 1 + + validate-inputs: + name: validate-inputs + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: All inputs set (should pass) + uses: ./.github/actions/validate-inputs + with: + sonarcloud-project-name: my-project + sonarcloud-token: sk_fake_token + dataminer-token: dm_fake_token + repository: ${{ github.repository }} + run-number: ${{ github.run_number }} + check-sonar: 'true' + check-dataminer: 'true' + + - name: Missing sonar name (should fail) — captured + id: missing-name + continue-on-error: true + uses: ./.github/actions/validate-inputs + with: + sonarcloud-token: sk_fake_token + repository: ${{ github.repository }} + run-number: ${{ github.run_number }} + check-sonar: 'true' + + - name: Assert failure was detected + run: | + if [[ "${{ steps.missing-name.outcome }}" != "failure" ]]; then + echo "Expected validate-inputs to fail when sonar name is missing" + exit 1 + fi + + apply-catalog-identifiers: + name: apply-catalog-identifiers + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Stage fake manifest + run: | + mkdir -p Test/CatalogInformation + cat > Test/CatalogInformation/manifest.yml <<'EOF' + id: 00000000-0000-0000-0000-000000000000 + name: Test + EOF + + - name: Invoke composite + uses: ./.github/actions/apply-catalog-identifiers + with: + mappings: Test/CatalogInformation/manifest.yml=12345678-1234-1234-1234-123456789abc + + - name: Assert id rewritten + run: | + grep -q '^id: 12345678-1234-1234-1234-123456789abc$' Test/CatalogInformation/manifest.yml + + - name: Invoke with empty mappings (no-op) + uses: ./.github/actions/apply-catalog-identifiers + with: + mappings: '' + + apply-source-code-url: + name: apply-source-code-url + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Stage fake manifest with empty source_code_url + run: | + mkdir -p TestSrc/CatalogInformation + cat > TestSrc/CatalogInformation/manifest.yml <<'EOF' + id: 00000000-0000-0000-0000-000000000000 + source_code_url: + name: Test + EOF + + - name: Invoke composite + uses: ./.github/actions/apply-source-code-url + with: + repository: ${{ github.repository }} + + - name: Assert source_code_url filled in + env: + REPO: ${{ github.repository }} + run: | + grep -q "source_code_url: 'https://github.com/$REPO'" TestSrc/CatalogInformation/manifest.yml + + detect-test-runner: + name: detect-test-runner + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: No global.json (default vstest) + id: default + uses: ./.github/actions/detect-test-runner + + - name: Assert vstest + env: + MODE: ${{ steps.default.outputs.mode }} + run: test "$MODE" = "vstest" + + - name: Stage global.json with MTP + run: | + cat > global.json <<'EOF' + { + "test": { "runner": "Microsoft.Testing.Platform" } + } + EOF + + - name: Detect MTP + id: mtp + uses: ./.github/actions/detect-test-runner + + - name: Assert mtp + env: + MODE: ${{ steps.mtp.outputs.mode }} + run: test "$MODE" = "mtp" + + update-global-json-sdks: + name: update-global-json-sdks + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + # The composite hard-codes the centrally-managed DataMiner SDK + # version. Keep this in sync with $DATAMINER_SDK_VERSION in + # .github/actions/update-global-json-sdks/action.yml. + - name: Expected DataMiner SDK version + id: expected + shell: bash + run: echo "version=2.5.2" >> "$GITHUB_OUTPUT" + + - name: No global.json (skip silently) + uses: ./.github/actions/update-global-json-sdks + + - name: Stage global.json without msbuild-sdks + shell: bash + run: | + cat > global.json <<'EOF' + { + "sdk": { "version": "8.0.100" } + } + EOF + + - name: Invoke composite (no msbuild-sdks, skip silently) + uses: ./.github/actions/update-global-json-sdks + + - name: Assert global.json untouched + shell: bash + run: | + grep -q '"version": "8.0.100"' global.json + if grep -q 'msbuild-sdks' global.json; then + echo "Expected no msbuild-sdks section" + exit 1 + fi + + - name: Stage global.json with managed and unmanaged SDKs + shell: bash + run: | + cat > global.json <<'EOF' + { + "sdk": { "version": "8.0.100" }, + "msbuild-sdks": { + "Skyline.DataMiner.Sdk": "0.0.1", + "Skyline.DataMiner.SomethingElse": "0.0.1", + "Microsoft.Build.NoTargets": "3.7.0" + } + } + EOF + + - name: Invoke composite (should bump DataMiner family) + uses: ./.github/actions/update-global-json-sdks + + - name: Assert managed keys bumped, unmanaged untouched + shell: bash + env: + EXPECTED: ${{ steps.expected.outputs.version }} + run: | + jq -e --arg v "$EXPECTED" '."msbuild-sdks"."Skyline.DataMiner.Sdk" == $v' global.json > /dev/null + jq -e --arg v "$EXPECTED" '."msbuild-sdks"."Skyline.DataMiner.SomethingElse" == $v' global.json > /dev/null + jq -e '."msbuild-sdks"."Microsoft.Build.NoTargets" == "3.7.0"' global.json > /dev/null + + - name: Re-invoke (idempotency check) + uses: ./.github/actions/update-global-json-sdks + + - name: Assert still in expected state + shell: bash + env: + EXPECTED: ${{ steps.expected.outputs.version }} + run: | + jq -e --arg v "$EXPECTED" '."msbuild-sdks"."Skyline.DataMiner.Sdk" == $v' global.json > /dev/null + jq -e '."msbuild-sdks"."Microsoft.Build.NoTargets" == "3.7.0"' global.json > /dev/null + + sonarcloud-status: + # Excluded from automatic runs: requires a live SonarCloud token. Run + # manually via workflow_dispatch when SONAR_TOKEN + SONAR_NAME are set. + name: sonarcloud-status (manual only) + if: github.event_name == 'workflow_dispatch' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Check prerequisites + id: prereq + env: + SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }} + SONAR_NAME: ${{ vars.SONAR_NAME }} + run: | + if [[ -z "$SONAR_TOKEN" || -z "$SONAR_NAME" ]]; then + echo "Skipping: SONAR_TOKEN secret or SONAR_NAME variable is not configured." + echo "skip=true" >> "$GITHUB_OUTPUT" + else + echo "skip=false" >> "$GITHUB_OUTPUT" + fi + + - name: Invoke composite + if: steps.prereq.outputs.skip != 'true' + uses: ./.github/actions/sonarcloud-status + with: + project-key: ${{ vars.SONAR_NAME }} + branch: ${{ github.ref_name }} + token: ${{ secrets.SONAR_TOKEN }} + repository: ${{ github.repository }} diff --git a/.github/workflows/Update Catalog Details Workflow.yml b/.github/workflows/Update Catalog Details Workflow.yml index 7ff2ffd..0ab0c21 100644 --- a/.github/workflows/Update Catalog Details Workflow.yml +++ b/.github/workflows/Update Catalog Details Workflow.yml @@ -31,47 +31,25 @@ jobs: name: Validate Trigger runs-on: ubuntu-latest steps: - - name: Check for unsupported trigger - if: github.event_name == 'pull_request_target' - run: | - echo "::error::This workflow does not support the 'pull_request_target' trigger. Use 'pull_request' instead to avoid security risks." - exit 1 + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI check_oidc: name: Check OIDC needs: [validate_trigger] runs-on: ubuntu-latest outputs: - client-id: ${{ steps.set_oidc.outputs.client-id }} - tenant-id: ${{ steps.set_oidc.outputs.tenant-id }} - subscription-id: ${{ steps.set_oidc.outputs.subscription-id }} - use-oidc: ${{ steps.set_oidc.outputs.use-oidc }} + client-id: ${{ steps.resolve.outputs.client-id }} + tenant-id: ${{ steps.resolve.outputs.tenant-id }} + subscription-id: ${{ steps.resolve.outputs.subscription-id }} + use-oidc: ${{ steps.resolve.outputs.use-oidc }} steps: - - name: Set Azure OIDC parameters - id: set_oidc - run: | - echo "Determining Azure OIDC parameters..." - - if [[ -n "${{ inputs.oidc-client-id }}" ]]; then - echo "Using provided OIDC parameters" - { - echo "client-id=${{ inputs.oidc-client-id }}" - echo "tenant-id=${{ inputs.oidc-tenant-id }}" - echo "subscription-id=${{ inputs.oidc-subscription-id }}" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - elif [[ "${{ github.repository_owner }}" == "SkylineCommunications" ]]; then - echo "Using SkylineCommunications default OIDC parameters" - { - echo "client-id=c50da9cc-ba14-4138-8595-a62d97ab0e53" - echo "tenant-id=5f175691-8d1c-4932-b7c8-ce990839ac40" - echo "subscription-id=d6cbb8df-56eb-451d-9db7-67f49cba3220" - echo "use-oidc=true" - } >> "$GITHUB_OUTPUT" - else - echo "No OIDC parameters provided and owner does not match SkylineCommunications" - echo "use-oidc=false" >> "$GITHUB_OUTPUT" - fi + - id: resolve + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/resolve-oidc@AI + with: + client-id: ${{ inputs.oidc-client-id }} + tenant-id: ${{ inputs.oidc-tenant-id }} + subscription-id: ${{ inputs.oidc-subscription-id }} + repository-owner: ${{ github.repository_owner }} update_catalog_details: name: Update Catalog Details @@ -86,36 +64,14 @@ jobs: tenant-id: ${{ needs.check_oidc.outputs.tenant-id }} subscription-id: ${{ needs.check_oidc.outputs.subscription-id }} - - name: Retrieve needed secrets from Azure Key Vault - if: needs.check_oidc.outputs.use-oidc == 'true' - shell: bash - run: | - echo "Fetching secrets from Azure Key Vault..." - - # List of secret names needed for this job - secret_names=("dataminer-token") - - for secret_name in "${secret_names[@]}"; do - # Convert to uppercase and replace hyphens with underscores - env_var_name=$(echo "$secret_name" | tr '[:lower:]' '[:upper:]' | tr '-' '_') - - # Retrieve the secret value - secret_value=$(az keyvault secret show --vault-name kv-master-cicd-secrets --name "$secret_name" --query value -o tsv) - - # Mask the secret value - echo "::add-mask::$secret_value" - - # Export as environment variable - echo "$env_var_name=$secret_value" >> "$GITHUB_ENV" - done - - - name: Overwrite default secrets with user-defined secrets - shell: bash - run: | - if [[ -n "${{ secrets.api-key }}" ]]; then - echo "Using provided api-key secret" - echo "DATAMINER_TOKEN=${{ secrets.api-key }}" >> "$GITHUB_ENV" - fi + - name: Load secrets + uses: SkylineCommunications/_ReusableWorkflows/.github/actions/load-secrets@AI + with: + use-oidc: ${{ needs.check_oidc.outputs.use-oidc }} + secret-names: | + dataminer-token + overrides: | + DATAMINER_TOKEN=${{ secrets.api-key }} - name: Enable long paths run: git config --global core.longpaths true diff --git a/.github/workflows/Wrapper Migration Workflow.yml b/.github/workflows/Wrapper Migration Workflow.yml new file mode 100644 index 0000000..90a6de0 --- /dev/null +++ b/.github/workflows/Wrapper Migration Workflow.yml @@ -0,0 +1,794 @@ +name: Wrapper Migration Workflow + +# Reusable workflow that rewrites a caller repo's wrapper workflow +# file(s) from one of the legacy "redirecting" master workflows +# (NuGet Solution, Internal NuGet Solution, DataMiner App Packages) +# to call `Master Workflow.yml` directly, and opens a PR with the +# change. +# +# It is invoked by each legacy wrapper master workflow so that any +# repo still using the old wrapper organically discovers a migration +# PR the next time CI runs. The rewrite is purely mechanical +# (input/secret renames) so the workflow performs it directly. +# +# Idempotency: skip if an open PR with the `wrapper-migration` label +# or a branch named `chore/wrapper-migration-` already exists. +# +# Token caveat: the default `GITHUB_TOKEN` cannot push changes that +# touch files under `.github/workflows/*` - GitHub refuses unless the +# token has the `workflows` scope. The workflow therefore needs a +# user-owned token. Retrieves via OIDC from Azure Key Vault. +# When the token is not available the job emits a warning and exits without +# pushing - the rewrite is logged so maintainers can apply it manually. + +on: + workflow_call: + inputs: + # Which legacy wrapper is calling us. Drives the rename map. + wrapper-kind: + required: true + type: string # nuget | internal-nuget | app-packages + + dry_run: + required: false + type: boolean + default: false + + debug: + required: false + type: boolean + default: false + + # OIDC parameters (typically forwarded from the caller's + # `check_oidc` job). When `use-oidc == 'true'` the workflow logs + # in to Azure and pulls the migration token from Key Vault. + oidc-client-id: + required: false + type: string + oidc-tenant-id: + required: false + type: string + oidc-subscription-id: + required: false + type: string + use-oidc: + required: false + type: string + default: 'false' + +permissions: {} + +jobs: + + validate_trigger: + name: Validate Trigger + runs-on: ubuntu-latest + steps: + - uses: SkylineCommunications/_ReusableWorkflows/.github/actions/guard-trigger@AI + + resolve_oidc: + name: Resolve OIDC + needs: [validate_trigger] + runs-on: ubuntu-latest + outputs: + client-id: ${{ steps.set.outputs.client-id }} + tenant-id: ${{ steps.set.outputs.tenant-id }} + subscription-id: ${{ steps.set.outputs.subscription-id }} + use-oidc: ${{ steps.set.outputs.use-oidc }} + steps: + - name: Set Azure OIDC parameters + id: set + env: + IN_CLIENT: ${{ inputs.oidc-client-id }} + IN_TENANT: ${{ inputs.oidc-tenant-id }} + IN_SUB: ${{ inputs.oidc-subscription-id }} + IN_USE: ${{ inputs.use-oidc }} + OWNER: ${{ github.repository_owner }} + run: | + # Same defaults Master Workflow.yml uses, so internal Skyline + # callers don't have to wire `check_oidc` into the three + # legacy wrapper workflows themselves. + if [[ "$IN_USE" == "true" && -n "$IN_CLIENT" ]]; then + { + echo "client-id=$IN_CLIENT" + echo "tenant-id=$IN_TENANT" + echo "subscription-id=$IN_SUB" + echo "use-oidc=true" + } >> "$GITHUB_OUTPUT" + elif [[ "$OWNER" == "SkylineCommunications" ]]; then + { + echo "client-id=c50da9cc-ba14-4138-8595-a62d97ab0e53" + echo "tenant-id=5f175691-8d1c-4932-b7c8-ce990839ac40" + echo "subscription-id=d6cbb8df-56eb-451d-9db7-67f49cba3220" + echo "use-oidc=true" + } >> "$GITHUB_OUTPUT" + else + echo "use-oidc=false" >> "$GITHUB_OUTPUT" + fi + + migrate: + name: Migrate wrapper to Master Workflow + needs: [validate_trigger, resolve_oidc] + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + id-token: write # required for OIDC login to Azure + issues: write + steps: + - name: Validate input + env: + KIND: ${{ inputs.wrapper-kind }} + run: | + case "$KIND" in + nuget|internal-nuget|app-packages) ;; + *) + echo "::error::Unknown wrapper-kind '$KIND'. Expected one of: nuget, internal-nuget, app-packages." + exit 1 + ;; + esac + + - name: Checkout caller repository + uses: actions/checkout@v6 + with: + path: caller + + - name: Write rewrite script + # The script is inlined here (rather than fetched from a side + # checkout of _ReusableWorkflows) because resolving the "right" + # ref of this repo from inside a reusable-workflow call is + # unreliable. + run: | + mkdir -p "$RUNNER_TEMP/wrapper-migration" + cat > "$RUNNER_TEMP/wrapper-migration/rewrite.py" <<'PYEOF' + """Rewrite a caller repo's wrapper workflow YAML files so that jobs + which currently call one of the legacy *redirecting* master workflows + (NuGet Solution, Internal NuGet Solution, DataMiner App Packages) + call ``Master Workflow.yml`` directly, with the input / secret names + that the master workflow actually expects. + + Implementation note: this is a *text-surgical* rewrite. A previous + version used ``ruamel.yaml`` round-trip mode but, even with + ``preserve_quotes`` enabled, ruamel reformats list indentation, + quote styles, blank lines, multi-line string folding, etc., which + polluted the generated PR with hundreds of whitespace-only changes. + The rewrite here is purely mechanical (rename a few keys under + ``with:`` / ``secrets:``, rewrite one ``uses:`` line, optionally + insert ``nuget-push-source``), so we operate on the raw lines and + touch nothing else. Comments, quoting, indentation width and line + endings of every other line are preserved byte-for-byte. + """ + + from __future__ import annotations + + import argparse + import re + import sys + from pathlib import Path + from urllib.parse import unquote + + KINDS = { + "nuget": { + "legacy_file": "NuGet Solution Master Workflow.yml", + "with_renames": { + "sonarCloudProjectName": "sonarcloud-project-name", + "solutionName": "solution-filter-name", + }, + "with_drops": [ + "referenceName", "runNumber", "referenceType", + "repository", "owner", + ], + "with_extra": { + "nuget-push-source": "https://api.nuget.org/v3/index.json", + }, + "secret_renames": { + "nugetApiKey": "NUGET_API_KEY", + "sonarCloudToken": "SONAR_TOKEN", + "azureToken": "AZURE_TOKEN", + }, + "secret_drops": ["pfx", "pfxPassword"], + }, + "internal-nuget": { + "legacy_file": "Internal NuGet Solution Master Workflow.yml", + "with_renames": { + "sonarCloudProjectName": "sonarcloud-project-name", + "solutionFilterName": "solution-filter-name", + }, + "with_drops": [ + "referenceName", "runNumber", "referenceType", + "repository", "owner", + ], + "with_extra": {}, + "secret_renames": { + "nugetApiKey": "NUGET_API_KEY", + "sonarCloudToken": "SONAR_TOKEN", + "azureToken": "AZURE_TOKEN", + }, + "secret_drops": [], + }, + "app-packages": { + "legacy_file": "DataMiner App Packages Master Workflow.yml", + "with_renames": { + "sonarCloudProjectName": "sonarcloud-project-name", + "solutionFilterName": "solution-filter-name", + "overrideCatalogIdentifiers": "override-catalog-identifiers", + }, + "with_drops": [ + "referenceName", "runNumber", "referenceType", + "repository", "owner", + ], + "with_extra": {}, + "secret_renames": { + "sonarCloudToken": "SONAR_TOKEN", + "azureToken": "AZURE_TOKEN", + "dataminerToken": "DATAMINER_TOKEN", + "overrideCatalogDownloadToken": "OVERRIDE_CATALOG_DOWNLOAD_TOKEN", + }, + "secret_drops": [], + }, + } + + + def uses_matches_legacy(uses_value, legacy_file): + if not isinstance(uses_value, str): + return False + decoded = unquote(uses_value) + return ( + "_ReusableWorkflows/.github/workflows/" in decoded + and legacy_file in decoded + ) + + + def rewrite_ref_in_uses(uses_value): + decoded = unquote(uses_value) + m = re.match( + r"^(?P[^/]+/_ReusableWorkflows)/\.github/workflows/" + r".+?(?P@[^@]+)?$", + decoded, + ) + if not m: + return decoded + ref = m.group("ref") or "@main" + return f"{m.group('prefix')}/.github/workflows/Master Workflow.yml{ref}" + + + def _indent_of(line): + return len(line) - len(line.lstrip(" ")) + + + def _is_blank_or_comment(line): + s = line.lstrip(" ") + return s == "" or s == "\n" or s == "\r\n" or s.startswith("#") + + + def _eol(line): + if line.endswith("\r\n"): + return "\r\n" + if line.endswith("\n"): + return "\n" + return "" + + + def _find_block_end(lines, start, body_indent): + """Exclusive end index of a block whose body lines are indented >= + ``body_indent``. Blank/comment lines are considered part of the + block.""" + i = start + last_content = start + while i < len(lines): + line = lines[i] + if _is_blank_or_comment(line): + i += 1 + continue + if _indent_of(line) < body_indent: + return last_content + 1 if last_content >= start else start + last_content = i + i += 1 + return len(lines) + + + # A key entry inside a `with:` / `secrets:` mapping. We require the + # exact body indent so we don't accidentally match nested keys of a + # multi-line value. + def _key_re(body_indent): + return re.compile( + rf"^( {{{body_indent}}})([A-Za-z_][\w-]*):(\s*)(.*?)(\r?\n?)$" + ) + + + def _format_scalar(value): + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, (int, float)): + return str(value) + s = str(value) + # Quote anything that could be misinterpreted by YAML; URLs in + # particular start with a scheme that contains ``:``. + specials = set(':#{}[],&*!|>\'"%@`') + if any(c in specials for c in s) or s.strip() != s or s == "": + return '"' + s.replace("\\", "\\\\").replace('"', '\\"') + '"' + return s + + + def _process_named_block( + lines, body_start, body_end, block_name, + block_indent, body_indent, + renames, drops, extras, default_eol, + ): + """Rename / drop keys inside the named block, and optionally add + missing extras. Returns the (possibly shifted) job body end index + and whether any change was made.""" + block_header_re = re.compile( + rf"^( {{{block_indent}}}){re.escape(block_name)}:\s*(\r?\n)?$" + ) + block_idx = None + for j in range(body_start, body_end): + if block_header_re.match(lines[j]): + block_idx = j + break + + changed = False + + if block_idx is None: + if not extras: + return body_end, False + # Create the block right before the next top-level job key, i.e. + # at the end of the current job's body. + eol = default_eol + new_lines = [" " * block_indent + f"{block_name}:{eol}"] + for k, v in extras.items(): + new_lines.append(" " * body_indent + f"{k}: {_format_scalar(v)}{eol}") + lines[body_end:body_end] = new_lines + return body_end + len(new_lines), True + + # Walk the block body and apply renames / drops to entries whose + # key sits at exactly ``body_indent``. + key_re = _key_re(body_indent) + j = block_idx + 1 + block_body_end = _find_block_end(lines, j, body_indent) + while j < block_body_end: + line = lines[j] + if _is_blank_or_comment(line): + j += 1 + continue + m = key_re.match(line) + if not m: + j += 1 + continue + key = m.group(2) + # Determine entry span: this line plus any following lines + # that are blank or indented deeper than ``body_indent``. + entry_end = j + 1 + while entry_end < block_body_end: + l2 = lines[entry_end] + if _is_blank_or_comment(l2): + entry_end += 1 + continue + if _indent_of(l2) <= body_indent: + break + entry_end += 1 + + if key in drops: + del lines[j:entry_end] + removed = entry_end - j + block_body_end -= removed + body_end -= removed + changed = True + continue + + if key in renames: + new_key = renames[key] + indent_part, _, sep, rest, line_eol = m.groups() + # Rebuild only the key portion; keep separator + value + # + EOL byte-for-byte. + lines[j] = f"{indent_part}{new_key}:{sep}{rest}{line_eol}" + changed = True + + j = entry_end + + # Add any missing extras at the end of the block. + if extras: + existing = set() + for j in range(block_idx + 1, block_body_end): + m = key_re.match(lines[j]) + if m: + existing.add(m.group(2)) + to_add = [(k, v) for k, v in extras.items() if k not in existing] + if to_add: + eol = _eol(lines[block_idx]) or default_eol + # Insert after the last non-blank line of the block. + insert_at = block_body_end + while insert_at > block_idx + 1 and _is_blank_or_comment(lines[insert_at - 1]): + insert_at -= 1 + new_lines = [ + " " * body_indent + f"{k}: {_format_scalar(v)}{eol}" + for k, v in to_add + ] + lines[insert_at:insert_at] = new_lines + block_body_end += len(new_lines) + body_end += len(new_lines) + changed = True + + # If we drained the block dry, remove the header (and trailing + # blank lines that belonged to it). + has_any = False + for j in range(block_idx + 1, block_body_end): + if key_re.match(lines[j]): + has_any = True + break + if not has_any: + # Remove from block_idx up to block_body_end, but keep blank + # separators outside the block intact. + remove_to = block_body_end + while remove_to > block_idx + 1 and _is_blank_or_comment(lines[remove_to - 1]): + remove_to -= 1 + # Also swallow the block header itself. + del lines[block_idx:remove_to] + body_end -= (remove_to - block_idx) + changed = True + + return body_end, changed + + + def _process_job(lines, body_start, body_end, job_indent, cfg, + drop_skyline_secrets, default_eol, debug, job_name): + body_indent = job_indent + 2 # `uses:`, `with:`, `secrets:` + child_indent = body_indent + 2 # entries inside with/secrets + + uses_re = re.compile( + rf"^( {{{body_indent}}})uses:(\s+)(.*?)(\r?\n?)$" + ) + uses_line_idx = None + uses_match = None + for j in range(body_start, body_end): + m = uses_re.match(lines[j]) + if m: + raw = m.group(3).strip() + # Strip surrounding quotes for the legacy-match check + # only; the line itself is rewritten preserving the + # original quote style. + bare = raw + if (bare.startswith('"') and bare.endswith('"')) or ( + bare.startswith("'") and bare.endswith("'") + ): + bare = bare[1:-1] + if uses_matches_legacy(bare, cfg["legacy_file"]): + uses_line_idx = j + uses_match = m + break # only one `uses:` per job + + if uses_line_idx is None: + return body_end, False + + indent_part, sep, raw_val, line_eol = uses_match.groups() + quote = "" + val = raw_val.strip() + if val.startswith('"') and val.endswith('"'): + quote = '"' + val = val[1:-1] + elif val.startswith("'") and val.endswith("'"): + quote = "'" + val = val[1:-1] + new_val = rewrite_ref_in_uses(val) + lines[uses_line_idx] = ( + f"{indent_part}uses:{sep}{quote}{new_val}{quote}{line_eol}" + ) + + # `with:` block + body_end, _ = _process_named_block( + lines, body_start, body_end, "with", + body_indent, child_indent, + cfg["with_renames"], cfg["with_drops"], cfg["with_extra"], + default_eol, + ) + + # `secrets:` block. Drop SkylineCommunications-managed secrets + # *after* renames are applied, because those drops target the + # already-renamed (SHOUTING_CASE) keys. + secret_drops = list(cfg["secret_drops"]) + body_end, _ = _process_named_block( + lines, body_start, body_end, "secrets", + body_indent, child_indent, + cfg["secret_renames"], secret_drops, {}, + default_eol, + ) + if drop_skyline_secrets: + body_end, _ = _process_named_block( + lines, body_start, body_end, "secrets", + body_indent, child_indent, + {}, ["SONAR_TOKEN", "AZURE_TOKEN"], {}, + default_eol, + ) + + if debug: + print(f" -> job '{job_name}': rewrote uses to {new_val}", + file=sys.stderr) + return body_end, True + + + def process_file(path, cfg, *, debug=False, drop_skyline_secrets=False): + text = path.read_text(encoding="utf-8") + if "_ReusableWorkflows" not in text: + return False + if cfg["legacy_file"] not in unquote(text): + return False + + lines = text.splitlines(keepends=True) + # Detect dominant EOL for any inserted lines. + crlf = sum(1 for l in lines if l.endswith("\r\n")) + lf = sum(1 for l in lines if l.endswith("\n")) - crlf + default_eol = "\r\n" if crlf > lf else "\n" + + # Locate `jobs:` at column 0. + jobs_re = re.compile(r"^jobs:\s*(\r?\n)?$") + jobs_start = None + for i, l in enumerate(lines): + if jobs_re.match(l): + jobs_start = i + 1 + break + if jobs_start is None: + return False + + # Each job header is ` :` at indent 2. Job body lines are + # indented >= 4. + job_header_re = re.compile(r"^( {2})([^\s#][^\s:]*):\s*(\r?\n)?$") + + changed_any = False + i = jobs_start + while i < len(lines): + line = lines[i] + if _is_blank_or_comment(line): + i += 1 + continue + if _indent_of(line) == 0: + break # next top-level key (e.g. another root mapping) + m = job_header_re.match(line) + if not m: + i += 1 + continue + job_name = m.group(2) + body_start = i + 1 + body_end = _find_block_end(lines, body_start, 4) + new_end, changed = _process_job( + lines, body_start, body_end, 2, cfg, + drop_skyline_secrets, default_eol, debug, job_name, + ) + if changed: + changed_any = True + i = new_end + + if changed_any: + with path.open("w", encoding="utf-8", newline="") as f: + f.writelines(lines) + return changed_any + + + def main(argv=None): + ap = argparse.ArgumentParser() + ap.add_argument("--kind", required=True, choices=sorted(KINDS)) + ap.add_argument("--workflows-dir", default=".github/workflows") + ap.add_argument("--debug", action="store_true") + ap.add_argument("--drop-skyline-secrets", action="store_true") + args = ap.parse_args(argv) + + cfg = KINDS[args.kind] + wf_dir = Path(args.workflows_dir) + if not wf_dir.is_dir(): + print(f"No {wf_dir} directory - nothing to do.") + return 0 + + touched = [] + for p in sorted([*wf_dir.glob("*.yml"), *wf_dir.glob("*.yaml")]): + try: + if process_file(p, cfg, debug=args.debug, + drop_skyline_secrets=args.drop_skyline_secrets): + touched.append(str(p)) + except Exception as e: + print(f"::warning::Could not process {p}: {e}") + + if touched: + print("Migrated workflow files:") + for p in touched: + print(f" - {p}") + else: + print(f"No workflow file references '{cfg['legacy_file']}'.") + return 0 + + + if __name__ == "__main__": + sys.exit(main()) + PYEOF + + - name: Check for existing migration PR + id: existing + working-directory: caller + env: + GH_TOKEN: ${{ github.token }} + KIND: ${{ inputs.wrapper-kind }} + run: | + set -euo pipefail + branch="chore/wrapper-migration-${KIND}" + existing_pr=$(gh pr list \ + --state open \ + --label "wrapper-migration" \ + --json number \ + --jq 'length') + branch_pr=$(gh pr list \ + --state open \ + --head "$branch" \ + --json number \ + --jq 'length') + if [ "$existing_pr" != "0" ] || [ "$branch_pr" != "0" ]; then + echo "An open wrapper-migration PR already exists - skipping." + echo "skip=true" >> "$GITHUB_OUTPUT" + else + echo "skip=false" >> "$GITHUB_OUTPUT" + fi + echo "branch=$branch" >> "$GITHUB_OUTPUT" + + - name: Ensure wrapper-migration label exists + if: steps.existing.outputs.skip != 'true' && inputs.dry_run != true + working-directory: caller + env: + GH_TOKEN: ${{ github.token }} + run: | + gh label create "wrapper-migration" \ + --description "Migrate caller from legacy redirecting master workflow to Master Workflow.yml" \ + --color "1f6feb" \ + 2>/dev/null || true + + - name: Run rewrite script + if: steps.existing.outputs.skip != 'true' + env: + KIND: ${{ inputs.wrapper-kind }} + DEBUG: ${{ inputs.debug }} + OWNER: ${{ github.repository_owner }} + run: | + set -euo pipefail + args=( --kind "$KIND" --workflows-dir caller/.github/workflows ) + if [ "$DEBUG" = "true" ]; then + args+=( --debug ) + fi + # Master Workflow already pulls SONAR_TOKEN and AZURE_TOKEN + # from Azure Key Vault for SkylineCommunications-owned repos, + # so dropping them from the caller's `secrets:` block keeps + # the migrated wrapper tidy. + if [ "$OWNER" = "SkylineCommunications" ]; then + args+=( --drop-skyline-secrets ) + fi + python3 "$RUNNER_TEMP/wrapper-migration/rewrite.py" "${args[@]}" + + - name: Detect changes + if: steps.existing.outputs.skip != 'true' + id: diff + working-directory: caller + env: + KIND: ${{ inputs.wrapper-kind }} + run: | + set -euo pipefail + if git diff --quiet -- .github/workflows; then + echo "No wrapper workflow file referenced the legacy '$KIND' master workflow - nothing to migrate." + echo "changed=false" >> "$GITHUB_OUTPUT" + else + echo "changed=true" >> "$GITHUB_OUTPUT" + echo "----- diff -----" + git --no-pager diff -- .github/workflows + echo "----------------" + fi + + - name: Azure Login (for Key Vault token retrieval) + if: steps.existing.outputs.skip != 'true' && steps.diff.outputs.changed == 'true' && needs.resolve_oidc.outputs.use-oidc == 'true' && inputs.dry_run != true + uses: azure/login@v3 + with: + client-id: ${{ needs.resolve_oidc.outputs.client-id }} + tenant-id: ${{ needs.resolve_oidc.outputs.tenant-id }} + subscription-id: ${{ needs.resolve_oidc.outputs.subscription-id }} + + - name: Retrieve migration token from Key Vault + if: steps.existing.outputs.skip != 'true' && steps.diff.outputs.changed == 'true' && needs.resolve_oidc.outputs.use-oidc == 'true' && inputs.dry_run != true + shell: bash + run: | + set -euo pipefail + # The PAT must carry `contents: write`, `pull-requests: write`, + # and the `workflows` scope - without `workflows`, git push + # refuses any edit under `.github/workflows/` and peter-evans + # surfaces a misleading 403. + echo "Fetching reusable-workflows-token from Azure Key Vault..." + secret_value=$(az keyvault secret show \ + --vault-name kv-master-cicd-secrets \ + --name reusable-workflows-token \ + --query value -o tsv 2>/dev/null || true) + if [ -z "$secret_value" ]; then + echo "::warning::No 'reusable-workflows-token' secret in Azure Key Vault 'kv-master-cicd-secrets'. Add a user-owned PAT (or fine-grained token) with 'contents: write', 'pull-requests: write', and 'workflows' scope on target repos to enable automated wrapper-migration PRs. The rewrite diff is in the job log so it can be applied manually." + else + echo "::add-mask::$secret_value" + echo "WRAPPER_MIGRATION_TOKEN=$secret_value" >> "$GITHUB_ENV" + fi + + - name: Create migration PR (dry run) + if: steps.existing.outputs.skip != 'true' && steps.diff.outputs.changed == 'true' && inputs.dry_run == true + run: | + echo "[dry-run] Would open PR on branch '${{ steps.existing.outputs.branch }}' with the diff above." + + - name: Create migration PR + if: steps.existing.outputs.skip != 'true' && steps.diff.outputs.changed == 'true' && inputs.dry_run != true && env.WRAPPER_MIGRATION_TOKEN != '' + uses: peter-evans/create-pull-request@v7 + with: + path: caller + token: ${{ env.WRAPPER_MIGRATION_TOKEN }} + branch: ${{ steps.existing.outputs.branch }} + base: ${{ github.event.repository.default_branch }} + commit-message: | + chore: migrate wrapper workflow to call Master Workflow.yml directly + + The '${{ inputs.wrapper-kind }}' legacy master workflow is a + thin wrapper around Master Workflow.yml. Removing it shortens + the call chain by one job, exposes the full set of + Master Workflow inputs (e.g. runs-on), and lets us evolve + inputs without breaking callers. + + Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> + title: "chore: migrate wrapper to call Master Workflow.yml directly" + labels: wrapper-migration + delete-branch: true + body: | + This PR was opened automatically by the + **Wrapper Migration Workflow** in + `SkylineCommunications/_ReusableWorkflows`. + + ### What changed + + The wrapper workflow file(s) in `.github/workflows/` were + updated to call + `SkylineCommunications/_ReusableWorkflows/.github/workflows/Master Workflow.yml` + directly instead of the legacy + **${{ inputs.wrapper-kind }}** master wrapper. Input and + secret names were renamed to the kebab-case / SHOUTING_CASE + names that `Master Workflow.yml` expects, and any obsolete + passthrough inputs (`referenceName`, `runNumber`, + `referenceType`, `repository`, `owner`, …) were dropped. + + ### Why + + The legacy `NuGet Solution`, `Internal NuGet Solution` and + `DataMiner App Packages` master workflows are thin wrappers + that internally call `Master Workflow.yml`. Removing the + indirection: + + - shortens the CI call chain by one job; + - exposes the full set of `Master Workflow.yml` inputs + (e.g. `runs-on`) without us having to forward each one + through every wrapper; + - lets us evolve the master inputs without breaking + callers. + + ### Review checklist + + - [ ] The rewrite preserves all `with:` values that were + previously passed through (only the keys were + renamed). + - [ ] All `secrets:` mappings are intact. + - [ ] If your wrapper job had extra steps or conditions + around the legacy `uses:` call, verify they still + make sense pointing at the master workflow. + - [ ] CI on this PR is green. + + *Idempotent: re-running the legacy wrapper while this PR + is open will not create another one.* + + - name: Summary + if: always() + env: + KIND: ${{ inputs.wrapper-kind }} + SKIP: ${{ steps.existing.outputs.skip }} + CHANGED: ${{ steps.diff.outputs.changed }} + HAVE_TOKEN: ${{ env.WRAPPER_MIGRATION_TOKEN != '' }} + run: | + { + echo "## Wrapper Migration result" + echo + echo "- kind: \`$KIND\`" + echo "- skipped (existing PR): \`${SKIP:-false}\`" + echo "- rewrite produced changes: \`${CHANGED:-false}\`" + echo "- migration token available: \`${HAVE_TOKEN}\`" + } >> "$GITHUB_STEP_SUMMARY" diff --git a/README.md b/README.md new file mode 100644 index 0000000..bdc1b99 --- /dev/null +++ b/README.md @@ -0,0 +1,142 @@ +# \_ReusableWorkflows + +Centralized GitHub Actions **reusable workflows** used by 2000+ Skyline +repositories. Caller repos reference one of the *master* workflows here so +that build, validation, packaging and publishing logic stays consistent +across the fleet and can be evolved in a single place. + +## Master workflows + +| Workflow | Purpose | +| --------------------------------------------------------- | ----------------------------------------------------------------------------- | +| `Master Workflow.yml` | Core CI/CD engine — build, validate, package, publish. | +| `Connector Master Workflow.yml` | CI/CD for DataMiner connector solutions (SDK and Legacy). | +| `Connector Master SDK Workflow.yml` | SDK-style connector pipeline (validator, Sonar, packaging). | +| `Connector Master Legacy Workflow.yml` | Legacy connector pipeline. | +| `Automation Master Workflow.yml` | CI/CD for Automation scripts (SDK and Legacy). | +| `Automation Master SDK Workflow.yml` / `Legacy` | SDK / Legacy automation pipelines. | +| `NuGet Solution Master Workflow.yml` | *(deprecated)* Thin redirect to `Master Workflow.yml` for public NuGet. | +| `Internal NuGet Solution Master Workflow.yml` | *(deprecated)* Thin redirect to `Master Workflow.yml` for internal NuGet. | +| `DataMiner App Packages Master Workflow.yml` | *(deprecated)* Thin redirect to `Master Workflow.yml` for app packages. | +| `Update Catalog Details Workflow.yml` | Update Catalog metadata on release. | +| `Test Downstream.yml` | Verifies downstream repos still build against changes here. | +| `Wrapper Migration Workflow.yml` | Opens a PR migrating callers off the deprecated redirecting wrappers. | + +--- + +## Wrapper Migration — automatic on legacy redirecting wrappers + +`Wrapper Migration Workflow.yml` is a reusable workflow that rewrites a +caller repo's CI wrapper file(s) from one of the **legacy redirecting** +master workflows to call `Master Workflow.yml` directly, and opens a PR +with the change. + +The three legacy wrappers are thin redirects that internally just call +`Master Workflow.yml` with renamed inputs / secrets: + +- `NuGet Solution Master Workflow.yml` +- `Internal NuGet Solution Master Workflow.yml` +- `DataMiner App Packages Master Workflow.yml` + +Each of those wrappers now also calls `Wrapper Migration Workflow.yml` +on non-PR invocations (branch/tag push, `workflow_dispatch`, schedule) +so that any repo still using the wrapper organically discovers a +migration PR the next time CI runs on a push. The rewrite is mechanical +(rename `with:` / `secrets:` keys, swap the `uses:` reference, drop +obsolete passthrough inputs like `referenceName`/`runNumber`/...), so +the workflow performs it directly instead of delegating to the Copilot +coding agent. + +### Trigger gating + +The `request_wrapper_migration` job in each legacy wrapper only fires +when: + +```yaml +if: github.event_name != 'pull_request' +``` + +This avoids opening duplicate migration PRs on every PR run and keeps +the migration limited to branch/tag pushes and manual dispatches. The +migration workflow is also idempotent as a second line of defense. + +### What it does + +1. The legacy wrapper workflow runs `master_workflow:` as before. +2. In parallel, `request_wrapper_migration:` calls + `Wrapper Migration Workflow.yml` with the appropriate + `wrapper-kind` (`nuget` / `internal-nuget` / `app-packages`). +3. The migration workflow checks for an open PR with the + `wrapper-migration` label or a `chore/wrapper-migration-` + branch. If one exists, it skips. +4. Otherwise it checks out the caller, rewriting any job whose `uses:` points at + the legacy wrapper. URL-encoded `uses:` values and pinned git refs + (`@main`, `@1.2.3`, `@`) are preserved. +5. If the rewrite produced changes, it opens a PR titled + `chore: migrate wrapper to call Master Workflow.yml directly` on + branch `chore/wrapper-migration-` with label + `wrapper-migration`. + +For repos in the `SkylineCommunications` organization the rewriter +also **drops `SONAR_TOKEN` and `AZURE_TOKEN`** from the migrated +`secrets:` block, because `Master Workflow.yml` already fetches those +from Azure Key Vault via OIDC for Skyline repos. Other secrets +(`NUGET_API_KEY`, `DATAMINER_TOKEN`, +`OVERRIDE_CATALOG_DOWNLOAD_TOKEN`) are kept. + +### Guarantees + +- **No direct writes to `main`.** Always opens a PR. +- **Idempotent.** Skips if an open PR with the `wrapper-migration` + label, or a PR from `chore/wrapper-migration-`, already exists. +- **Scoped per-job.** Only rewrites jobs whose `uses:` references the + legacy file for the requested kind, so files mixing multiple + wrappers stay correct. +- **Round-trip YAML.** Uses `ruamel.yaml` so comments and (most) + formatting in the caller's workflow are preserved. + +### Required setup — migration token + +The default `GITHUB_TOKEN` **cannot** push commits that modify files +under `.github/workflows/`; GitHub rejects the push unless the token +carries the `workflows` scope. The migration workflow therefore needs a +user-owned token. + +The token is retrieved from **Azure Key Vault via OIDC**. The workflow +logs in to Azure and reads `reusable-workflows-token` from +`kv-master-cicd-secrets`. The secret must be a user-owned PAT (or +fine-grained token) with `contents: write`, `pull-requests: write`, +`issues: write`, and **`workflows`** scope on target repos. + +For repos in the `SkylineCommunications` organization the OIDC +parameters are auto-defaulted, so no extra setup is required from +caller repos beyond `secrets: inherit`. External callers must +configure their own OIDC + Key Vault setup to provide the token. + +If no token is available, the rewrite is computed and the diff is +printed in the job log but no PR is opened (a warning is emitted). +Maintainers can apply the change manually. + +### Standalone use (optional) + +The workflow can also be called directly (e.g. one-shot sweep across +the fleet on a schedule): + +```yaml +jobs: + Migration: + uses: SkylineCommunications/_ReusableWorkflows/.github/workflows/Wrapper Migration Workflow.yml@main + with: + wrapper-kind: nuget # or internal-nuget, or app-packages + dry_run: true # recommended for the initial pilot +``` + +### Inputs + +| Input | Type | Default | Description | +| ------------------------- | ------- | -------- | --------------------------------------------------------------------------------- | +| `wrapper-kind` | string | — | One of `nuget`, `internal-nuget`, `app-packages`. Drives the rename map. | +| `dry_run` | boolean | `false` | Log the planned diff without opening a PR. | +| `debug` | boolean | `false` | Verbose logging in the rewrite script. | +| `use-oidc` | string | `false` | When `'true'`, log in to Azure and pull the migration token from Key Vault. | +| `oidc-client-id` / `oidc-tenant-id` / `oidc-subscription-id` | string | — | Azure OIDC parameters (auto-defaulted for `SkylineCommunications`). |