“A test strategy is an outline that describes the testing approach of the software development cycle.”
I was recently afforded the opertunity for a deep dive into software testing strategies - it was a amazing! I’ve realized that different teams have a different understanding and naming of tests. As long as the team agree and document the stratagy all will be well in the world. Its also not always best practice to do every kind of test possible, as a team you need to agree on your working/social contracts and then add this to your DoD (Definition of done).
These define the smallest unit of work we can have, normally a well written method in a class that takes advantage of dependency inversion.
- This will have a very narrow scope & test logic would be given
xthe behaviour is
y, this doesnt guarantee feature works
- Mock out all dependencies
- Pick a Unit Testing Framework
- Decide on a pattern and stick to it, AAA is the only one I really know
See Unit Tests Overview for more details.
U_I_E signatures are
Unit of work,
inital condition and
Most of the names Ive see would however end up as
UnitOfWork_GivenX_ShouldY (which is pretty much the below)
If you need to use
And then consider what you are actually testing, potentially the test is too board and you can narrow scope (not always possible though!) Example:
UnitOfWork_GivenXAndY_ShouldZ. You could sneak in a with and have
UnitOfWork_GivenXWithY_ShouldZ … LOL.
Another signature could be
G_W_T which is Given-When-Then: This pattern is often used in behavior-driven development (BDD). It consists of three parts:
- Given: The preconditions that must be met before the test can be run.
- When: The action that is being tested.
- Then: The expected result of the action.
This can be described as
Given: A database with a table called “users”
When: I insert a new user into the table
Then: The user should be visible in the table
Most of the names I’ve see would however end up as
* The understanding of
Integration tests are often switched around, this means what one team calls
Component testing is
Integration testing to another and vice versa. It doesnt matter as long as you and your team (current pod and adjacent) have a simliar understanding. This understanding needs to be shared as part of your onboarding into the team.
Component Test will test a deployable boundary and generally dependencies outside the boundary are mocked.
- Makes sense to run this on PR and Master builds
- Wiremocks are useful here to mock external dependencies (anything the component doesnt own)
- You could test the integration of the service and its db
- Genrally tests the integration with a single external component at a time (ie: the database)
The goal is to give higher confidence that components work together.
This will cross boundaries and the real implementation of resources are used, this is normally done in a UAT environment. Depending on data constraints this can also be done in Production but thats not always possible or sensible if it impacts what your clients will see.
- This can be run after Master is deployed to UAT
- Ensures that the external services that were mocked out in
Component Testsare right
- Tests the boundaries between components (focus on the contract). Test interactions with 3rd party components (the actual contract)
- Usually doesn’t involve UI (but it can)
See Integration Tests for more details.
These are slow, expensive and generally pretty flakey! A great testing tool for this is cypress
- Only do these tests for the most common flows
- Normally tests the happy path
- Driven by the UI (User Interface) and can test down the stack to the database
- Should cover what you would manually test after deployment
Its often thought that E2E is only for the front end but this is not true, you can have E2E tests that tests the end to end functionality of a worker process or API.
These are manual tests done by a user, should be the engineer and QA doing these to ensure they fifull the stories acceptance criteria and mandatory requirements.
- Tests the UI from a user perspective and is is focused on user behaviour
“A penetration test, colloquially known as a pen test, pentest or ethical hacking, is an authorized simulated cyberattack on a computer system, performed to evaluate the security of the system;”
This is things like DDOS (Distributed Denial of Service) or known variabilities on the host or code base.
I briefly touched on this at a previouse employer, we used NVDA screen reader to look for accessibility tags and help make our HTML more semantic. Things like ALT tags in images, linking text in A HREF anchors that made sense, tabindex for keyboard control ect.
“Semantic HTML is the use of HTML markup to reinforce the semantics, or meaning, of the information in webpages and web applications rather than merely to define its presentation or look.”
- System Testing
- Feature Testing
- Functional Testing