Integration test tutorial: Your First Integration Test#649
Integration test tutorial: Your First Integration Test#649SlamBamActionman wants to merge 5 commits into
Conversation
mqole
left a comment
There was a problem hiding this comment.
very very cool guide, tysm for your work on it slam! in addition to the notes i left (knowing the bare minimum about tests), some other stuff id like to see covered if time permits:
- when should someone consider making a new integration test? is there any consideration for how much time/memory a new test takes to run / if it would be better to edit an existing test / etc?
- other example keywords that can be used to affect tests (glossary maybe?) - ive seen
NonParallelizable,Explicit, etc - what happens when a test is written poorly and uses up a lot of memory? are there any common pitfalls that should be avoided?
| @@ -0,0 +1,170 @@ | |||
| # Your First Integration Test | |||
|
|
|||
| In this guide you will learn about integration testing and how to create an integration test. | |||
There was a problem hiding this comment.
maybe a small note clarifying the differences between integration and unit testing, and if theres ever a situation where it is more appropriate to create a unit test?
could also use [...]about SS14 integration testing and[...] to note that integration tests are a universal concept in software development. (i did not know this until recently as ss14 was my introduction to software dev :p )
| **Integration testing** is a useful tool to ensure that changes to one part of the game doesn't unexpectedly cause another part of the game to change too. | ||
| It can catch unintended behavior, bugs and even rare game-crashing errors when used properly! | ||
| This is achieved through **integration tests**, which basically run short simulations of the game and makes sure ingame values match what the test expects. |
There was a problem hiding this comment.
term integration test is used 2x, small rephrase suggestion
| **Integration testing** is a useful tool to ensure that changes to one part of the game doesn't unexpectedly cause another part of the game to change too. | |
| It can catch unintended behavior, bugs and even rare game-crashing errors when used properly! | |
| This is achieved through **integration tests**, which basically run short simulations of the game and makes sure ingame values match what the test expects. | |
| The process of **integration testing** ensures that changes to one part of the game don't unexpectedly cause another part of the game to change too. | |
| It can catch unintended behavior, bugs and even rare game-crashing errors when used properly! | |
| Integration testing is performed through the use of **integration tests**, which basically run short simulations of the game and make sure the end results of those simulations match what the test expects. |
| This is achieved through **integration tests**, which basically run short simulations of the game and makes sure ingame values match what the test expects. | ||
|
|
||
| An example would be changing a Cargo order to cost less. | ||
| If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop! |
There was a problem hiding this comment.
| If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop! | |
| If you have an integration test that compares order costs to sell values and throws an error if a Cargo order would sell for more money than it costs to buy, you'll be able to automatically catch if this change would result in an infinite money loop! |
| An example would be changing a Cargo order to cost less. | ||
| If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop! | ||
|
|
||
| Integration tests are ran on all pull requests submitted to the SS14 repository and all tests must pass for a PR to be mergeable. |
There was a problem hiding this comment.
| Integration tests are ran on all pull requests submitted to the SS14 repository and all tests must pass for a PR to be mergeable. | |
| Integration tests are automatically run on all pull requests submitted to the SS14 repository, and all tests must pass for a PR to be mergeable. |
| If you have an integration test that compares order costs to sell values, you'll be able to automatically catch if this change would result in an infinite money loop! | ||
|
|
||
| Integration tests are ran on all pull requests submitted to the SS14 repository and all tests must pass for a PR to be mergeable. | ||
| You can also run tests locally in your IDE (useful if you fail a specific test when submitting a PR). |
There was a problem hiding this comment.
maybe add a link to src/en/general-development/tips/debugging-tools.md after #629 is merged to clarify how this is done locally
| ### Simulation & Checking | ||
|
|
||
| `InteractionTest` has many helper methods used for simulating interactions. | ||
| With our testcase being simply clicking on the huggable entity, we can use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method! |
There was a problem hiding this comment.
| With our testcase being simply clicking on the huggable entity, we can use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method! | |
| With our testcase being simply clicking on the huggable entity, we can make use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method! |
| `InteractionTest` has many helper methods used for simulating interactions. | ||
| With our testcase being simply clicking on the huggable entity, we can use of the basic `await Interact();` method to simulate hugging. Since we spawned the `MobHuman` with `SpawnTarget` earlier, all we have to do is run the method! | ||
|
|
||
| Since the player entity spawns with one free hand, we should expect a basic interaction to result in the `InteractionPopupSystem.InteractHandEvent` event subscription triggering, and therefore `LastInteractTime` should be updated to the current time. We assert that the previous `LastInteractTime` should not be equal to the new `LastInteractTime`. |
There was a problem hiding this comment.
i was initially going to note that you might be able to talk here about how you could go above and beyond and check that they actually spawn with a hand, but then i realised that LastInteractTime wouldnt update if they didnt have a hand lol. might be worth rearticulating that this example test will "check whether a hug is performed" in addition to all the specific technical stuff! (or not, but i imagine someone else might read this and think its a good idea to add a hand test :p )
| It can be good to understand this process since a lot can be modified and extended, and there are several helper methods that can save time and make your tests much better. | ||
|
|
||
| `PoolManager` is a static core class that manages server-client simulation relationships, and is used for tests, benchmarks and map rendering. | ||
| For tests specifically it allows for client-servers to be reused for multiple tests and for tests to be run in parallel, instead of constantly starting and shutting down such systems. |
There was a problem hiding this comment.
add that this makes tests run fast (good thing (because they usually take a long time))
| It's unlikely you will access `PoolManager` yourself, but a key property that all integration tests make use of is the `TestPair` class. | ||
| `TestPair` gives access to the Client and Server instances and therefore the ability to set CVars, resolve manager/system dependencies and map management. |
There was a problem hiding this comment.
'gives access' could be a little clearer. 'gives the test access'?
| `PoolManager` is a static core class that manages server-client simulation relationships, and is used for tests, benchmarks and map rendering. | ||
| For tests specifically it allows for client-servers to be reused for multiple tests and for tests to be run in parallel, instead of constantly starting and shutting down such systems. | ||
|
|
||
| It's unlikely you will access `PoolManager` yourself, but a key property that all integration tests make use of is the `TestPair` class. |
There was a problem hiding this comment.
maybe an example of an integration test which uses PoolManager?
|
From the perspective of someone that doesn't know much about intergration test writing. |
This PR adds a tutorial for creating a basic integration test.
I am myself a beginner to making tests so I took the opportunity to write as I learned. That of course means I am not fully certain of the accuracy of the text, so do point out inconsistencies. I'd also love to see some additions made to the "Extra credit" section to further explain the core parts of how tests function.