Front-end Testing Strategy
Defines the different layers of testing and gives an overview of the current situation to propose an effective strategy.
Before starting is important to mention that this article is :
- Focused on front-end testing specifically (i.e. React applications and their interactions with components).
- Highly based on testing principles defined by Kent C. Dodds, who is a famous developer within the React community. He has released multiple open-source libraries (https://testing-library.com/) as well as many qualitative courses and articles (https://kentcdodds.com/).
I) Introduction
1. Why do we write tests?
Testing could help to speed up your workflow and improve your code quality, but the biggest and most important reason to write tests is CONFIDENCE. You want to be confident that the code you are writing won’t break the app while being future-proof. So whatever you do, you want to make sure that the kinds of tests you write bring you the most confidence possible, and you need to be conscious of the trade-offs you are making when testing.
2. Types of test
You can throw paint against the wall and eventually you might get most of the wall, but until you go up to the wall with a brush, you’ll never get the corners. 🖌️
This metaphor applies perfectly to testing because it basically says that choosing the right testing strategy is the same type of choice you would make when choosing a brush to paint a wall. There are different types of tests (A/B testing, Performance testing, Smoke testing, Regression testing, …) for different use cases, each with its own tradeoffs.
Manual testing is one layer. Then when we talk about automated testing, the following are the most common ones: Unit, Integration, and End to End (E2E).
Finally, in JavaScript, because it’s untying by default (unlike other languages such as Java for example), it’s common to include a Static type tool (Typescript, Flow, ESLint, …).
Static Test
Static tests are performed without requiring the code to be executed. These helpers are easy and fast to set up and allow you to catch typos and type errors continuously while developing the application.
TypeScript (Typing programming language) and ESLint(Linters) are common tools to perform that type of test.
Unit test
Unit tests verify that small, isolated pieces (Or atoms) of your software work as expected. They generally test units with no dependencies (Collaborators) or those mocked for the test.
Jest is the common tool to perform that type of test.
Integration test
Integration tests verify that several units (Functions, Components, Classes, …) work together in harmony. They test their behavior as a whole and try to mock as little as possible. Integration test covers one system (Ex: Front-end) isolated from the others.
Jest is the common tool to perform that type of test.
The notion of “what is a unit? VS “what is an integration?” depends of the point of view from which you are looking at the code.
If we take the example of an application, using a Dropdown belonging to component library:
- From a component library perspective; an internal Dropdown’s function is a unit. And the Dropdown as a whole is an integration.
- From an application perspective; The Dropdown becomes a Unit. And the page’s Form is an Integration.
E2E test
E2E test (Sometimes called “functional testing”) covers all the systems (Front-end, Back-end, …) put together. They automate the browser and try to reproduce a typical user flow around the application (Load the application, go through the login flow, interact with the page, …). E2E helps you cover an enormous amount of testing of your software.
Cypress is the common tool to perform that type of test.
Manual test
Manual testing suggests tests performed without using any automated tools. A person will try to reproduce exactly what the end-users are going to do :
- Sit down at the computer
- Open the application on the browser
- Go through the login
- Click around
Their goal is to catch bugs that weren’t anticipated by the developers or detected by the automated tools.
3. What tests give the most confidence?
With testing, you are making sure that your software will work as expected when releasing it to users. To do so, the best way will be to write your tests with the end-user’s perception of your software in mind.
Following the simple principle, Manual testing looks like the best option because it is closest to how the end-user will interact with the application. However, this is not a scalable solution for testing your software.
But for any changes you make in your software, you want to ensure that nothing gets broken before the deployment to Production. If you go with manual testing, that will take a very long time. In addition, Manual testing is highly subject to human error.
This is why we favor automated tests instead because they are faster, more scalable, and free of human error (if they were well written).
Resources:
II) Let’s talk about Trade-offs
1. Code coverage
Mandating 100% code coverage for applications is not a good idea. You get diminishing returns on your tests when the coverage increases beyond a certain number (Let’s say 70% for example). When you strive for 100% all the time, you find yourselves spending time testing things that really don’t need to be tested:
- Things that really have no logic in them at all: So any bugs could be caught by ESLint and Typescript.
- Implementations details: Testing implementation details don’t give you very much confidence that your application is working, and it slows you down when refactoring (You should very rarely have to change tests when you refactor code because the behavior remains the same).
It can lead to:
- False negatives: Can break when you refactor application code.
- False positives: May not fail when you break application code.
How to determine an implementation detail?
If your test does something that the consumer of your code doesn’t, then it’s testing implementation details. (Exposing a private function by example).
Commons misunderstanding about code coverage
What code coverage is telling you:
- This line of code was run when these tests were run.
What code coverage is NOT telling you:
- This part of the code will work according to the business requirements.
- This part of code works well with all the other parts of code in the application.
- This part of code is more important to test than other parts.
Code Coverage < Use Case Coverage
You should concern less about the code you are testing and more about the use cases that code supports because :
- Code changes more often than use cases do
- The code can be “working” when the use case is broken
2. How to know what to test?
Since you won’t be able to test everything, you will have to decide where you want to focus your effort.
Testing pyramid
You may already know the Testing pyramid, which is a popular way to talk about the different layers of testing. The following one is a combination of Martin Fowler’s blog and one from the Google Testing blog :
The size of these forms of testing on the pyramid is relative to the amount of focus you should give them when testing your applications.
Both arrows indicate the trade-off you are making when you move through the pyramid. As you move up the pyramid, the tests get slower to write/run and more expensive (in terms of time and resources) to run/maintain. When you go up, your tests tend to become also more finicky and there are more points to failure.
Following only those two aspects, It’s meant to indicate that you should spend more of your time on unit tests because they are at the same time cheap and fast.
But there are a few problems here :
1 — This pyramid was created in 2012 and based on this assumption (provides from the bottom notes of Martin Fowler’s blog) :
Which is less true now. The testing tools are much better in comparison to what they were before (Better in terms of performance, but also to be able to tell what part of the test has caused the failure as well as describe the problem).
2 — The pyramid was created at a time when the front-end testing ecosystem wasn’t as great as it is now. A decade ago, our tools were not well-equipped for reproducing the front-end end-user.
Most of the tests, tools, and the pyramid were back-end focused. And the reason is that the back-end is “easier” to test in comparison to testing the front-end. They build software which talks directly to other software (Which can be considered as their users). So it’s natural for them to write tests that would look like the code they want to test.
For the front-end, the users could be the end-users, who have fingers, hands and will click around, type some stuff on the keyboard, potentially use some screen readers, etc. It’s a lot more complicated to reproduce their behavior. Fortunately, a lot of things have changed in these recent years, and we have now better tools (Ex: Testing Library) to be able to test our software in the same way that the user would do.
3 — The pyramid does not address Static testing. It’s because it was defined for developers who are using static type languages (Ex: Java, C). But with JavaScript, you have to add external tools (Typescript, Eslint) to get such feedback, so in your reality, it’s one of the layers of testing.
4 — There is one aspect that is left out in this pyramid: As you move up the pyramid, the confidence coefficient of each form of testing increases.
The confidence coefficient means, the higher you go up the testing pyramid, the closer you are getting to your tests resembling the way that your software is used. And so the confidence that you get from those also goes up.
For example, if you have a checkout feature, testing the entire process in the same way that a real user would interact with, allows you to cover and ensure that all the sub-elements used work together.
So while E2E tests may be slower and more expensive than unit tests, they bring you much more confidence that your application is working as intended.
On the opposite, Units tests are faster and cheaper, but by testing them you might find yourselves:
- Testing pieces in isolation (which doesn’t give you confidence that they work properly together)
- Testing implementations details
- Mocking a lot
which are many factors that take you away from the principle of “Tests your application in the exact same way that your user would interact with” and so trade away confidence.
Testing trophy
With the testing trophy, we keep the same testing stack: E2E, Integrations, and Unit; but you changed the ratio between the different shapes. we also add the Static layer to represent the reality of JavaScript.
E2E tests are the type of automated testing that gives the most confidence that your software works well as intended. Maybe in the future our E2E tools will just get so much better that you will just focus 100% on it, but today they can still be expensive to write and run. So you are going to focus only on some high-level critical tests which can require the interactions of multiple systems (Front-end, back-ends, …) and represent production data. In E2E you are generally going to assume the happy path and especially avoid over-testing.
Integration tests are the widest layer of the trophy because it’s where you should focus most of your effort. They strike a great balance on the trade-offs between confidence and speed/expense to write and run your tests. They test some happy paths and unhappy paths. By testing that several parts of your software work well together as intended, you will be able to cover a lot of use cases in the same way that the end-user would interact with it; And you’ll find that you often don’t need to bother testing each smaller piece in isolation.
Unit tests are the smallest section of the trophy. It’s all about focusing your effort on testing little edge cases, or typically pure functions which have a lot of complex logic. Here you are not testing the business cases (the cases that the end-user will have). The users of your units are typically the other developers which would call your function from a developer’s perspective.
Static tests should be applied everywhere in the application because they give you a lot of information without even needing to run your application.
The ratio between each shapes is not 100% accurate, most of the time you would like to have more E2E than Unit tests.
III) Conclusion
This tweet from Guillermo Rauch (CEO and founder of Vercel) is a perfect summary of that testing philosophy :
- Write tests: They give you CONFIDENCE that your software work as intended
- Not too many: Do not mandate 100% code coverage. You should think less about the code you are testing and more about the use cases that the code supports.
- Mostly integration: They are the best balance on the trade-offs between confidence and speed/expense to write and run your tests.
It is also important to mention that Manual testing will surely still have a place for decades among the testing strategy. There will always be some use cases that weren’t anticipated and tested by the developers, and a real human eye can help on that part. But as you write automated tests, confidence increases and your need for manual testing will decrease. There are lots of benefits with manual testing but many problems such as the amount of time and resources needed.
It’s again all about trade-offs, what is critical for the end-user and so where you want to put your effort. You want to be confident that when you deploy your changes, your code satisfies the business requirements, and you provide an awesome experience to your Users. And you should use a mix of the different testing strategies to accomplish that goal.
Thank you for reading.
