Testing Strategies

“A test strategy is an outline that describes the testing approach of the software development cycle. - https://en.wikipedia.org/wiki/Test_strategy"

I was recently afforded the opertunity for a deep dive into software testing strategies - it was a amazing! I’ve realized that different teams have a different understanding and naming of tests. As long as the team agree and document the stratagy all will be well in the world. Its also not always best practice to do every kind of test possible, as a team you need to agree on your working/social contracts and then add this to your DoD (Definition of done).

Unit Test

These define the smallest unit of work we can have, normally a well written method in a class that takes advantage of dependency inversion.

Key points

  • This will have a very narrow scope & test logic would be given x the behaviour is y, this doesnt guarantee feature works
  • Mock out all dependencies
  • Pick a Unit Testing Framework
  • Decide on a pattern and stick to it, AAA is the only one I really know

See Unit Tests Overview for more details.

Unit Test Test Naming

U_I_E

U_I_E signatures are Unit of work, inital condition and expected result.

Most of the names Ive see would however end up as UnitOfWork_GivenX_ShouldY (which is pretty much the below)

If you need to use And then consider what you are actually testing, potentially the test is too board and you can narrow scope (not always possible though!) Example: UnitOfWork_GivenXAndY_ShouldZ. You could sneak in a with and have UnitOfWork_GivenXWithY_ShouldZ … LOL.

G_W_T

Another signature could be G_W_T which is Given-When-Then: This pattern is often used in behavior-driven development (BDD). It consists of three parts:

  • Given: The preconditions that must be met before the test can be run.
  • When: The action that is being tested.
  • Then: The expected result of the action.

This can be described as

Given: A database with a table called “users”
When: I insert a new user into the table
Then: The user should be visible in the table

Most of the names I’ve see would however end up as UnitOfWork_GivenX_ShouldY or UnitOfWork_WhenX_ShouldY

Component Test *

* The understanding of Component and Integration tests are often switched around, this means what one team calls Component testing is Integration testing to another and vice versa. It doesnt matter as long as you and your team (current pod and adjacent) have a simliar understanding. This understanding needs to be shared as part of your onboarding into the team.

Component Test will test a deployable boundary and generally dependencies outside the boundary are mocked.

Key points

  • Makes sense to run this on PR and Master builds
  • Wiremocks are useful here to mock external dependencies (anything the component doesnt own)
  • You could test the integration of the service and its db
  • Genrally tests the integration with a single external component at a time (ie: the database)

Integration Test *

The goal is to give higher confidence that components work together.

This will cross boundaries and the real implementation of resources are used, this is normally done in a UAT environment. Depending on data constraints this can also be done in Production but thats not always possible or sensible if it impacts what your clients will see.

Key points

  • This can be run after Master is deployed to UAT
  • Ensures that the external services that were mocked out in Component Tests are right
  • Tests the boundaries between components (focus on the contract). Test interactions with 3rd party components (the actual contract)
  • Usually doesn’t involve UI (but it can)

See Integration Tests for more details.

End to end testing (E2E Test)

E2E will test the flow as a whole where integration tests normally test a feature made up of the applications components.

Front End

These are slow, expensive and generally pretty flakey! A great testing tool for this is cypress

Key points

  • Only do these tests for the most common flows
  • Normally tests the happy path
  • Driven by the UI (User Interface) and can test down the stack to the database
  • Should cover what you would manually test after deployment

Back End

Its often thought that E2E is only for the front end but this is not true, you can have E2E tests that tests the end to end functionality of a worker process or API.

Acceptance Test

These are manual tests done by a user, should be the engineer and QA doing these to ensure they fifull the stories acceptance criteria and mandatory requirements.

Key points

  • Tests the UI from a user perspective and is is focused on user behaviour

Load Test / Performance Testing

Penetration Test

“A penetration test, colloquially known as a pen test, pentest or ethical hacking, is an authorized simulated cyberattack on a computer system, performed to evaluate the security of the system;”

This is things like DDOS (Distributed Denial of Service) or known variabilities on the host or code base.

Accessibility Test

I briefly touched on this at a previouse employer, we used NVDA screen reader to look for accessibility tags and help make our HTML more semantic. Things like ALT tags in images, linking text in A HREF anchors that made sense, tabindex for keyboard control ect.

“Semantic HTML is the use of HTML markup to reinforce the semantics, or meaning, of the information in webpages and web applications rather than merely to define its presentation or look.”

Other tests

  • System Testing
  • Feature Testing
  • Functional Testing