Skip to content

Integration test tutorial: Your First Integration Test#649

Open
SlamBamActionman wants to merge 5 commits into
space-wizards:masterfrom
SlamBamActionman:your-first-integration-test
Open

Integration test tutorial: Your First Integration Test#649
SlamBamActionman wants to merge 5 commits into
space-wizards:masterfrom
SlamBamActionman:your-first-integration-test

Conversation

@SlamBamActionman
Copy link
Copy Markdown
Member

This PR adds a tutorial for creating a basic integration test.

I am myself a beginner to making tests so I took the opportunity to write as I learned. That of course means I am not fully certain of the accuracy of the text, so do point out inconsistencies. I'd also love to see some additions made to the "Extra credit" section to further explain the core parts of how tests function.

Copy link
Copy Markdown
Contributor

@mqole mqole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very very cool guide, tysm for your work on it slam! in addition to the notes i left (knowing the bare minimum about tests), some other stuff id like to see covered if time permits:

  • when should someone consider making a new integration test? is there any consideration for how much time/memory a new test takes to run / if it would be better to edit an existing test / etc?
  • other example keywords that can be used to affect tests (glossary maybe?) - ive seen NonParallelizable, Explicit, etc
  • what happens when a test is written poorly and uses up a lot of memory? are there any common pitfalls that should be avoided?

@@ -0,0 +1,170 @@
# Your First Integration Test

In this guide you will learn about integration testing and how to create an integration test.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe a small note clarifying the differences between integration and unit testing, and if theres ever a situation where it is more appropriate to create a unit test?

could also use [...]about SS14 integration testing and[...] to note that integration tests are a universal concept in software development. (i did not know this until recently as ss14 was my introduction to software dev :p )

Comment on lines +7 to +9
**Integration testing** is a useful tool to ensure that changes to one part of the game doesn't unexpectedly cause another part of the game to change too.
It can catch unintended behavior, bugs and even rare game-crashing errors when used properly!
This is achieved through **integration tests**, which basically run short simulations of the game and makes sure ingame values match what the test expects.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

term integration test is used 2x, small rephrase suggestion

Suggested change
**Integration testing** is a useful tool to ensure that changes to one part of the game doesn't unexpectedly cause another part of the game to change too.
It can catch unintended behavior, bugs and even rare game-crashing errors when used properly!
This is achieved through **integration tests**, which basically run short simulations of the game and makes sure ingame values match what the test expects.
The process of **integration testing** ensures that changes to one part of the game don't unexpectedly cause another part of the game to change too.
It can catch unintended behavior, bugs and even rare game-crashing errors when used properly!
Integration testing is performed through the use of **integration tests**, which basically run short simulations of the game and make sure the end results of those simulations match what the test expects.

This is achieved through **integration tests**, which basically run short simulations of the game and makes sure ingame values match what the test expects.

An example would be changing a Cargo order to cost less.
If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop!
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop!
If you have an integration test that compares order costs to sell values and throws an error if a Cargo order would sell for more money than it costs to buy, you'll be able to automatically catch if this change would result in an infinite money loop!

An example would be changing a Cargo order to cost less.
If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop!

Integration tests are ran on all pull requests submitted to the SS14 repository and all tests must pass for a PR to be mergeable.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Integration tests are ran on all pull requests submitted to the SS14 repository and all tests must pass for a PR to be mergeable.
Integration tests are automatically run on all pull requests submitted to the SS14 repository, and all tests must pass for a PR to be mergeable.

If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop!

Integration tests are ran on all pull requests submitted to the SS14 repository and all tests must pass for a PR to be mergeable.
You can also run tests locally in your IDE (useful if you fail a specific test when submitting a PR).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add a link to src/en/general-development/tips/debugging-tools.md after #629 is merged to clarify how this is done locally

### Simulation & Checking

`InteractionTest` has many helper methods used for simulating interactions.
With our testcase being simply clicking on the huggable entity, we can use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method!
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
With our testcase being simply clicking on the huggable entity, we can use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method!
With our testcase being simply clicking on the huggable entity, we can make use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method!

`InteractionTest` has many helper methods used for simulating interactions.
With our testcase being simply clicking on the huggable entity, we can use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method!

Since the player entity spawns with one free hand, we should expect a basic interaction to result in the `InteractionPopupSystem.InteractHandEvent` event subscription triggering, and therefore `LastInteractTime` should be updated to the current time. We assert that the previous `LastInteractTime` should not be equal to the new `LastInteractTime`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was initially going to note that you might be able to talk here about how you could go above and beyond and check that they actually spawn with a hand, but then i realised that LastInteractTime wouldnt update if they didnt have a hand lol. might be worth rearticulating that this example test will "check whether a hug is performed" in addition to all the specific technical stuff! (or not, but i imagine someone else might read this and think its a good idea to add a hand test :p )

It can be good to understand this process since a lot can be modified and extended, and there are several helper methods that can save time and make your tests much better.

`PoolManager` is a static core class that manages server-client simulation relationships, and is used for tests, benchmarks and map rendering.
For tests specifically it allows for client-servers to be reused for multiple tests and for tests to be run in parallel, instead of constantly starting and shutting down such systems.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add that this makes tests run fast (good thing (because they usually take a long time))

Comment on lines +166 to +167
It's unlikely you will access `PoolManager` yourself, but a key property that all integration tests make use of is the `TestPair` class.
`TestPair` gives access to the Client and Server instances and therefore the ability to set CVars, resolve manager/system dependencies and map management.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'gives access' could be a little clearer. 'gives the test access'?

`PoolManager` is a static core class that manages server-client simulation relationships, and is used for tests, benchmarks and map rendering.
For tests specifically it allows for client-servers to be reused for multiple tests and for tests to be run in parallel, instead of constantly starting and shutting down such systems.

It's unlikely you will access `PoolManager` yourself, but a key property that all integration tests make use of is the `TestPair` class.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe an example of an integration test which uses PoolManager?

@riccardi48
Copy link
Copy Markdown

From the perspective of someone that doesn't know much about intergration test writing.
I agree that a keyword section would be good to include. A table of the diffrent syntax used in intergration tests would be useful. Like the changes in space-wizards/space-station-14#43875.
An explanation on how the Server and Client are used in intergration tests. On when and how you would test interactions between server and client. I personally don't know the answer to this which is why I think it could be helpful to include.
A section on the thought processes on what to make an intergration test on. A basic checklist of when it is good to make one maybe. The Cargo example you give is good but it is very clear it should be a test, the less obvious times for intergration tests would be good to get across.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants