Skip to content

fix: Revise e2e-test-review + e2e-test-staging step names#1200

Merged
andygout merged 1 commit intomainfrom
revise-e2e-test-review-e2e-test-staging-step-names
Mar 17, 2026
Merged

fix: Revise e2e-test-review + e2e-test-staging step names#1200
andygout merged 1 commit intomainfrom
revise-e2e-test-review-e2e-test-staging-step-names

Conversation

@andygout
Copy link
Contributor

Description

The tool-kit/e2e-test-review and tool-kit/e2e-test-staging jobs include a step that is intended to run the end-to-end tests.

However, the name of that step its:

  • "Smoke tests" (in the tool-kit/e2e-test-review job)
  • "Run smoke tests on staging" (in the tool-kit/e2e-test-staging job)

The explanation for this is given by @apaleslimghost in this Slack post from Wed 07 Jan 2026 at 10:50:

those jobs aren't just for smoke tests, it's just that by default, n-test runs in those jobs (fwiw, n-test is capable of more than smoke tests, it can do end-to-end-test-like things). for example, the cypress plugin will also run in those jobs (the jobs run the test:review/test:staging commands).

you shouldn't need custom jobs for your own e2e tests; if you configure them to run on the test:review/test:staging commands they'll run at the same point in the workflow.

However, in those instances where we do run end-to-end tests in the tool-kit/e2e-test-review and tool-kit/e2e-test-staging jobs, they appear in a step whose name misleadingly suggests that smoke tests are being run, e.g. https://app.circleci.com/pipelines/github/Financial-Times/next-appetite/25318/workflows/47493522-5069-4669-a459-0a522303d5cf/jobs/52244 (the "Smoke tests" step actually runs Cypress end-to-end tests).

The ideal would be to be able to provide a custom name for the step. However, as that would likely entail additional work, the next best solution would seem to be to rename the step at source to something that corresponds more closely with the job name (which is presumably considered to be a bit more abstract in that it is intended to run a mixture of both end-to-end and smoke tests).

That change of the step name is proposed by this PR.

Additional notes

@apaleslimghost talked @ytcleon through the options for adding custom post-steps to an existing job (see below yml snippet). The obstacle we would face here would be that adding end-to-end tests in a post-step at the end of the tool-kit/e2e-test-review or tool-kit/e2e-test-staging job would be that it comes after the "Uploading artifacts" step.

We would require end-to-end tests to occur prior to the "Uploading artifacts" step as they typically produce artifacts.

However, the smoke tests provide by n-test do not produce artifacts, and so they could be added in a post-step for which we could provide a custom name when defining it.

The final wrinkle would then be the apps in our estate which have smoke tests but no end-to-end tests (e.g. next-barrier-page, user-onboarding) where we'd want to have a post-step for smoke tests but nothing running in the spot of end-to-end tests, but that's something we can worry about later.

hooks:
    - CircleCi:
        workflows:
          - name: "tool-kit"
            jobs:
              - name: "e2e-test-review"
                custom:
                  post-steps:
                    - run:
                        name: "Post tool-kit/e2e-test-review step"
                        command: |
                          npm run e2e-test-review-post-steps

Checklist:

  • My branch has been rebased onto the latest commit on main (don't merge main into your branch)
  • My commit messages are conventional commits, for example: feat(circleci): add support for nightly workflows, fix: set Heroku app name for staging apps too

@andygout andygout requested a review from a team as a code owner March 13, 2026 13:23
Copy link
Member

@apaleslimghost apaleslimghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

chore: to be releaseable, this commit would have to be a fix: or a feat:. i think fix: is closest.

@andygout andygout force-pushed the revise-e2e-test-review-e2e-test-staging-step-names branch from f6f0ad5 to 2ad8b61 Compare March 13, 2026 15:08
@andygout
Copy link
Contributor Author

Fab — I've amended the commit name to fix: now 👍

@andygout andygout changed the title chore: Revise e2e-test-review + e2e-test-staging step names fix: Revise e2e-test-review + e2e-test-staging step names Mar 13, 2026
@andygout andygout merged commit 21884b1 into main Mar 17, 2026
14 checks passed
@andygout andygout deleted the revise-e2e-test-review-e2e-test-staging-step-names branch March 17, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants