Cross Browser Testing

CROSS BROWSER TESTING

Some Recent Posts

RECENT POSTS

We all know that test automation can work wonders for your business. It can help to reduce cost, save time and effort, get quicker feedback and thereby directly contributing to increased productivity. But in order to get good results from automation testing, you need to avoid mistakes because they can result in wastage of automation efforts.

To make your test automation strategy successful, you have to avoid some common mistakes & failures. With that in mind, here’re some of the common problems with automation testing that you need to avoid to build a long-term successful test automation strategy.

#### Going for 100% test coverage with automation.

It’s necessary to understand the importance that not everything can can be automated. One of the main benefits of test automation is to free up time for testing that must be done manually, i.e. regression or exploratory testing. In other words, it’s about applying testers’ skills to where they provide the most value. 

Automation testing is not there to replace manual testing but rather to help manual testers to free up their time. Sometimes, automating those tests brings minimal returns, or human intervention is required to execute those tests. 

As far as test automation is concerned, there are plenty of ways to go about setting up your test automation suite. You could use a test frame work for testing for cross browser testing & testing functionalities involving any browser interaction. If your team is not skilled with test frameworks like TestCafe, Cypress or Selenium, you can even opt for Play & Record testing to achieve some automation level.

Test automation takes time, and going big from the beginning is not recommended. It would help if you asked the following questions to yourself and your team before moving to test automation:

*   What pain points encountered in manual testing does test automation solve?

*   What kind of tests should be automated (Regression, Sanity, Functional, etc.)?

*   What is the test automation team’s skill-set, and is there any training required before jumping on the test automation bandwagon?

*   Which type of tests should be part of the CI/CD pipeline?

Just automating all manual test cases is one of the major reasons for automation testing failures since that task is ideally not achievable!

Plan for gradually automating your test suite. I recommend starting out by focusing on the flows that are easiest to automate. In most cases, you will find that it’s the relatively simple and very repetitive flows that, by far, take up most of your testing time.

#### Testing for multiple things in one test case.

A test case that is built to verify more than one aspect can fail in multiple ways, for that very reason. This means that if the test case fails, it must be manually inspected to figure out which aspect failed. Sometimes businesses wants to check multiple assertions or validations in one test in order to expand coverage or save time and effort, but the longer a test is running, it increases the chances for timeouts.

Build test cases so that they logically only test one aspect. This way, there is no doubt about what goes wrong when a test case fails. Instead of bundling up multiple tests in one test case, it is best practice to build reusable components with your test. This way, it is easy to reuse the logic contained in other test cases, and the time required to create a new test case is minimised.

#### Using a tool that is not a good technical fit

When looking for a test automation platform, make sure it fits your technical requirements.

A common mistake is to choose a tool based only on your current challenges and automation objectives. This comes with the risk of being stuck with a tool that is not a good technical fit for your business in the long run. I have experienced where management want to use automation frameworks because it’s popular or because other big corporate companies are using them. This a where the advice of an experienced automation engineer is very valuable.

Solution: Implementing test automation is a long term strategic choice and should be handled as such. When evaluating automation tools, look across your business and identify all the applications and technologies that could be potential targets for automation. Identify the scenarios where test cases need to move between technologies, e.g. both web and desktop applications, and select an automation platform that has matching capabilities. And don’t forget cross browsing compatibility. 

#### Inadequate Infrastructure

This is one of the major factors for automation testing failure. Whether you are developing web products for the end-consumers, it would involve interaction with the web browsers. During the early stages of product testing, your team might be using a manual testing approach, which has many limitations. You cannot attain good test coverage since the number of tests may be limited or may not challenge the test infrastructure.

Take the case of cross browser testing; your team can verify your product against limited combinations of web browsers, operating systems, and devices. You can test on browsers installed on your system or machines connected to the network, but you can still not cover the entire spectrum of web browsers (and their different versions). The scenario will change when you shift the focus to automation testing or automated cross browser testing.

When you shift to automation testing, the number of tests being executed will increase significantly. To get the most out of automation, the test infrastructure on which the tests are executed should be responsive but having a responsive test infrastructure comes at a cost. The cost will rise as the product development & test phase progresses; not investing in the infrastructure will hamper the throughput of testing, leading to automation testing failure. 

Your management team should take a conscious decision on whether to have a local test infrastructure or shift to a cloud-based testing where you can leverage the advantages of parallel testing for improved test efficiency.

#### Test Automation In Silos

It is a wrong notion that testing is a separate task. Testing and development should be done hand-in-hand. It should be a part of the development cycle and continuous integration process, i.e., Continuous Integration/Continuous Delivery pipeline. One of the major reasons for an automation testing failure is that the test teams work in silos and have little or no interaction with the other teams in the project. For more on Continuous Integration, read my blog post on the Importance of Continuous Integration For Testing.

Wouldn’t it be a disaster if significant changes happen in the product’s UX when the test team is simultaneously testing that particular product flow?  It will eventually end up in duplication of work and cause an immense amount of frustration within the test team.

Test automation should be a part of the build-test-deploy pipeline. Whether your project uses the traditional water-flow or kanban model or CI/CD for product development & testing, automation testing should be considered a part of the development. 

Automation tests should be integrated into the continuous delivery pipeline, and tests should be executed regularly. Another cause for an automation testing failure is that automation tests are considered necessary during the later stage of product development. 

Doing so would lead to the discovery of critical bugs (functional, non-functional, performance, etc.) at a later point in time. This can delay the product release, which can eventually hurt the business of your business. 

Automation testing has to be part of the planing and should be included into the SDLC. The more you test, the beter chances of generating the best results out of test automation.

#### Flawed Planning And Execution

Shifting gears to test automation involves careful planning and sound execution. You need to have a test plan or strategy in place. The test plan should highlight what you plan to achieve through test automation and expected test coverage. There might be other projects in your business where test automation tools were used. While it is important to understand & document those learnings, you cannot follow a one size fits all approach.

There are many open-source tools, and you might be tempted to use those tools since there is no cost factor involved. Before choosing any open-source tool, you need to look at the tool development history, feedback from existing users, and the current state of development. There is no point in using a deprecated tool since the tool will become unusable one day. 

Since web browsers are an integral part of any website/web application, you should choose a cloud-based cross browser testing platform like TestCafe. Choosing the right automation tool is the key automation plans. You do not want to regret choosing the wrong or inefficient tool during later stages of testing. Below are some of the things that need to be kept in mind while selecting the automation tool.

*   What is the kind of throughput when multiple tests are executed in parallel?

*   Does the automation tool support popular code tracking & bug tracking tools like, Jira, GitHub, Slack, GitLab.

*   How simple is it to integrate the tool in the CI/CD pipeline?

*   What frameworks and programming languages are supported by the tool?

*   What are reporting mechanisms supported by the automation tool and the usefulness of the report’s information?

*   What sort of tests need to be automated?

*   What kind of web browsers, operating systems, and devices would be crucial for test automation?

*   What is the responsibility of every engineer involved in test automation, and what is the kind of skill set required for test suite development?

*   What programming language and test framework will be used mainly for enabling test automation?

*   What is the key focus of test automation, i.e., GUI testing, functional testing, Stress testing, etc.?

*   What kind of CI tools are used for the execution of the builds?

*   What are the target test coverage and some blueprints/plan to attain the test coverage requirements?

There are many other points that need to be considered, but these will depend on your project’s nature & scale. Once you have the answer to the questions above, you might be able to solve 90% of the problems with automated testing.


#### Not Doing Maintenance.

If automation testing is implemented correctly, it will allow companies to ship new features or new products to clients much more frequently and with more confidence.

However, most of those benefits rapidly decrease or even goes away if you don’t take the time to maintain your tests.

Maintenance has to be part of your automation suite, unless your codebase never changes you will have to keep on working and maintaining your automation tests, in order for it to keep up with the latest code as you applications evolves. Work that is been done during the software development process, will effect your existing tests in one way or another. Automation engineers might have to adjust existing ones.


#### Conclusion

Test automation is not rocket science, and the success will largely depend on the test planning & execution. It would be best to avoid automation testing failures discussed above, overlooking these points can result in the loss of the entire test automation task. 

Avoiding an automation testing failure is more of the right mindset needed for project planning, development, and testing – the key pillars of any software product.


Why Your Automation Tests Are Failing | Oswin Losper

WHY AUTOMATION TESTS ARE FAILING

In our ever changing world we live in today, when you take on a new project, a new hobby or just even when you enjoy something, you would want to get the most out of it. It’s the same with automation testing. If you do have an automation suite in your project and you don't have continuous integration (CI) on it, then you not getting the most value out of your automation testing.

#### Learn why it is important for you and your business to implement CI in your project.

A lot of businesses these days are embracing automation testing, and with good reason, I might add. Automation testing help developers and testers to be more effective during the software development process. With automation, teams don’t have to spend their days going through testcases manually, this can lead to quicker turn around times and also help catching issues quickly before the project reach the customers.

However, some companies does have access to an automated test suite, but don’t run them as frequently as they should. They only use the automation test suite on occasions, then on the other occasions they still execute tests manually. These companies are missing out on a big opportunity to get the most value out of their automation test suite. 

Automation can be a very valuable tool that can validate the state of your application at any given time. But if these tests don’t run continuously, their value diminishes rapidly, especially at companies where automation engineers needs to proof the value of automation.

In order to get the most value our of your automation test suite, you will need run them regularly. Ideally, you want to use your automated test suite to ensure that your application still behaves as expected when changes were made to it. And like mentioned above, in the ever changing world we live in today, applications changes occurs frequently. To get the most value, your team can get this automation up and running with a continuous integration system in place.

### What Is Continuous Integration?

Continuous Integration is a development practice to help manage and automate workflows when changes occur in a project. Typical uses for continuous integration environments is building software application, deploying new change and of course running automated tests. 

Continuous integration system plays an important part of a healthy software development pipeline, saving teams a lot of time and effort through the development cycle. Like most things today, there  are plenty of CI services available, from environments you can host on your own to cloud systems.

Most of them are easy to start up and do not require a full time devops team. Most testing tools today can integrate fairly easy with most CI systems, helping you go from local testing to continuous integration testing seamlessly. 

If for some reason your company are still unsure about continuous integration, here are some reasons why may want to consider implementing a CI pipeline in your project.

##### You Will get Faster Feedback Loops

Doesn't matter the size of your project, you will inevitable get bugs. As a project gets bigger, you will add more complexity to it, ever changing requirements and many other issues that increases the probability of errors. It's impossible to avoid mistakes all the time. As good software developers and testers know that the key isn't preventing errors but the time you take to recover from them quickly and efficiently as possible.

When bugs are discovered, it's cheaper to fix to them right away rather than waiting to fix them months down the line. No matter where you are in a project, fixing bugs quickly makes a huge difference in time and money wasted.

The reason being why it's cheaper to fix bug straight away, is that the more time passes between code changes and discovering a bug, the more difficult it is for the developer to go back and find the root cause of it. The work wont be fresh in their mind, so they will have to spend time remembering what they did before.

Also, adding to this, other developers might have worked on the same section in the codebase at a later stage, so on top of the above, the developer will also have to figure out additional changes to the code in an attempt to fix the bug. 

This might seem like a small thing, but I have seen companies ignore bugs until a later stage than months down the line, business justifies the bugs in the project.

By using continuous integration with a well-built automated test suite, this will eliminate most of these issues, because the team will quickly see how a recent change in the project has cause a defect and investigate to fix it quickly. 

If an issue has been found using an automated test suite with continuous integration, an alert can be send to the developer and they can take much more quicker to fix the bug rather than the bug being discovered later down the line. Early detection can help reduce how much effort your developer and tester spend on issues.

##### You'll Increase Accountability With Your Team

Let's be honest, I'm sure all automated testers would love to have an automation test suite that just works, that doesn’t have failing tests, but realistically at some point you will get tests that fail. The failure can be due to recent changes in the code base or it can be a intermittent failure that no one can track down, regardless of the reason, its good to notify the team immediately and in a prominent place, if you don't have reporting in place so that the whole team can see it.

Unfortunately, a lot testers prefer to take the easy way out and push failing tests one side to silent the failure alerts. They either mark the failing tests as pending, delete the result or even silence the notifications. It begins with what seems to be a justifiable moment, like we need to get this build out to the client or management want to see visibility on tests. These decisions usually goes with *'I will fix that next week when I have time’*. Then the inevitable happens, you never get time to fix it and before you know your whole automation test suite losses value.

I hear a lot and have experienced where the health of a project is the sole responsibility of the QA team, in order to have a successfull project all stakeholder within the team has a responsibility to the health of a project. It might be easier for developers and testers to check the state of an application because they are hands on with the project, but for designers and project managers, it might be more difficult for them. If you don't have a way to make the projects health visible to these team members you missing out on a good opportunity to have other contribute with a fresh set of eyes.

You can avoid this problem my using continuous integration to run your test automatically for you. Most continuous integration systems allows you to send alerts when an issue arises. Your continuous integration can send a messages to Slack when a test fails. You can also add additional service to generate reports on a test run, this will help the non technical people in the team to understand the status of the project. This can allow everybody in the team to contribute to the project.

##### You'll Give Testers More Time for High-Value Tasks.

For many testers, they're consist of doing a lot of repetitive tasks daily. Everyday they open up the application they are responsible to test, doing the same steps, clicking buttons, filling in the same forms and checking validations, the same did the day before.

This can become a dull routine. Automated tests can free them up form this cycle, letting continuous integration take care of the repetition. 

The main benefit of having automated tests and setting up a CI service isn't to liberate testers from tedious work, its to free them up to spend more of there work day to do more higher value testing tasks, like more exploratory testing, which can pick up bugs that automated testing wont find and also documentation of testing procedures, test cases and test plans.

Automated tests are very good tool to handle regression testing, ensuring new changes don't break the application, but this will only be able to cover the areas you have planned or wrote scripts for. If there is new features, that was done by the developer and no automation tests for the new feature there will be a gap that is not checked. Continuous integration wont cover for those scenarios, therefor testers time is best spend on the new features that has not automation tests and not the old features. No continuous integration system can go beyond what's defined in the test suite.

A common mistake a lot of companies make, is *‘they want to automate everything’* . They switch all they focus onto automation and ignore all types of other testing. Automation testing wont solve all testing problems teams have. As a project gets bigger and bigger, automation coverage wont cover everything. Freeing up the team to go beyond what continuous integration does will undoubtedly improve the health of your project.

##### You Can Automate More the Just Tests

Yes, this article is mainly focussed on implementing continuous integration to handle the repetitive testing work that most companies are facing with todays software development. 

The most common use case for a continuous integration service is to set up automation testing. However, with that said, if you only using continuous integration for your automation tests, you are only touching the surface.

In modern software development projects, teams have plenty of workflows that can take advantage of continuous integration. These are a few tasks that continuous integration can handle automatically for teams:

*   Building binaries for mobile applications on different platforms like iOS and Android and send it to beta testers.

*   Package web application to docker images, ready to distribute internally or in a public container registry.

*   Run a complete set of tests and, on success, automatically deploy the latest code changes to production.

Continuous integration can help your company perform repetitive tasks that computers can handle better and quicker.

#### Conclusion

A well set up continuous integration environment plays an important part of the modern testing teams. It can help eliminate the repetitive aspects of manual testing and ensure you get the most value out of your automation testing.

Continuous integration can also help the whole teams including the business to get better insight of the project health status and valuable metrics, like graphs and statistics. (We all know business people like graphs).

Developers can also get much more quicker feedback on their new commits and thus reducing tons of effort and time in the development process. Tester will have more time to work on more high value testing scenarios. 

With a continuous integration process in place your entire team will benefit. This will go onto and benefit your customers as well with faster updates, fewer bugs. 

At the end of the day continuous integration wil result in a beter product for everyone.


Importance Of Continuous Integration

THE IMPORTANCE OF CI IN YOUR PROJECT

I think its safe to say that in todays world, client and stakeholders expect software application they use everyday to provide them with rich experiences. The days of simple applications with a few form fields and store basic data are gone. We live in a world where we have real-time applications with nice designs and multiple services on the cloud.

Because of this applications can become very complex with a lot of moving parts. With this, testing has also become much more complex, with multiple pieces of functionality working  differently and stores information in different locations.

Developers and tester needs to make sure that all of the pieces of the puzzle of their applications works well together. Usually these complicated applications are being tested manually, but let's be honest we are humans after all and we do get tired and this is prone to mistakes. One way to get over this mistake is to implement automated end-to-end testing. Along side other automated tests like unit and API testing, end-to-end tests are essential part of the software development lifecycle.

#### What is end-to-end testing

End-to-end testing, or E2E testing, is to make sure the  whole work flow on an entire application works from start to finish. Unlike other kinds of testing, end-to-end testing goes through the whole functionality, not just a certain part. Applications that interact with different services or require complex actions, this type of testing simulates real world scenarios and validates what your customers see.

#### Benefits of end-to-end testing and Automation

End-to-end tests provide a few benefits over other forms of testing. These tests will simulate a real word scenario that your customers will also experience under normal circumstances. Other tests do not simulate a real world scenario but only a section of the functionality that a user experiences. This kind of coverage helps to ensure that third-party services that aren't in your control work as intended. And thanks to the rise of low-code or code-less testing tools, the barrier to entry for creating end-to-end testing is much lower.

Some teams perform these complex tests manually, going through each step to cover their acceptance criteria. As mentioned earlier, this process take time and this can lead to tiredness and mistakes set to creep in and can even rush to tick all the boxes on the acceptance criteria and just forget to do their work, however more and more businesses are now leveraging the power of automation testing to free them from doing the same stuff over and over again.

Because automated tests does simulate real world scenarios quicker then a human would perform these steps, the entire team can get quicker feedback if something went wrong in there application.

This is just a few benefits of end-to-end automation. For more on this, few my blog post on [benefits of automation testing](https://www.oswinlosper.co.za/blog/benefits-of-automation-testing/). 

#### Challenges of end-to-end testing

As saying the goes *'in the perfect world'*, I think we all want that, but we do live in the real world. With that everything that sounds good also does have its flaws, and it's no different for automation testing either, as much as we want it to solve all our testing problems, it doesn’t come with its own sets of challenges. Its important that, when you do start your automation journey that you understand the challenges and have plans in place to fix them. 

I have worked with companies that told me, ’they want to automate everything’ without thinking of the challenges that lays ahead and if these challenges are not planned for, your automation dream wil soon turn into a nightmare. 

Below are challenges most testers face with end-to-end testing and also a few tips on how to overcome each to make you turn into an E2E testing hero.

##### Challenge #1: Flaky Tests

End-to-end testing has an earned reputation for unstable, flaky tests that fail irregularly. It’s very frustrating, if you run your test suite and everything passes, only to see a test fail for no reason the next day but they passed yesterday and there were no code changes. 

End-to-end testing does consist of a lot of moving parts and needs to validate different components through a test run, it is difficult to isolate whats causing the issue. This inconsistency leads to a lot of lost time trying to figure out the root cause of the flakiness since it’s nearly impossible to replicate the failing scenario. And worse of all, flaky tests leads to teams accepting them, ignoring potential issues and decreasing the value of automation tests.

The main challenge of tackling an unstable test suite is to find a solution. You can take a few steps to minimise inconsistent tests. Keep in mind there is no *‘one-size-fits all’* quick fix for this.

**Keep track of test failures:**  On most automated test suites with flaky tests, you’ll often see the same test cases fail. The tests will give you ideas or clues that can help you iron out the instabilities in the application or the test suite.

**Make the most of your testing tools:** Most end-to-end tools has build in features that can re-run failing tests. For instance, if you run your test against a live API, you will most probably get time-outs, and you don't want these to be marked as failed. For example, TestCafe has a build in feature called ***‘quarantine mode’,*** if a test fails it will run the test again and if it passes on the second attempt it will mark it pass or unstable.

**Don’t ignore flaky tests:** Ignoring a flaky test will not make the problem go away, it will lead to acceptance of flaky tests and this can lead to actual problems being ignored as well. We all would like to have more time in the day, but set time aside for investigate flaky tests before the problem gets worse.

##### Challenge #2: Slow Tests

Besides flaky tests, another top complaint is that end-to-end tests are slow and will slow down building times. As mentioned before end-to-end test goes through the entire stack and test every component, both internal and third party. Its just plain unfair to compare end-to-end testing with the speedy unit tests that only test a partial section of your application.

Even with saying that it doesn't make the end-to-end tests go faster, because a slow test suite can drastically slow down the effectiveness of software development and testing teams. For instance, a lot teams now rely on successful test runs in the continuous integration system before merging code or deploying new features. Having to wait for these build can grind your work to a stand still.

Speeding up your tests can rely on different factors, your architecture of your application, any dependencies or third party services and network connectivity to mention a few. Your strategy will vary according to the application. The following are some steps you can follow to speed up your tests.

**Mocking and stubbing functionality:** Most applications these days connect to multiple external services or have to perform complex calculations. You can avoid the overhead of these time consuming functions by using mocks to simulate the behaviour of the functionality.

**Set up the correct data as needed:** Your tests can spend too much time setting up the applications initial state and cleaning things up after a test run. Take time to review the best way to ensure your test has what it needs - nothing more and nothing less.

**Build your test to run concurrently:** Most end-to-end testing tools like TestCafe, allow you your tests in parallel with one another or use multiple browsers. Your test scenario have to run independently from one another.

##### Challenge #3: Long-Term Maintenance

One very important aspect of end-to-end testing, is that businesses under estimate the amount of maintenance goes into end-to-end testing. Lot of businesses think, that once you are done with an application feature, the end-to-end testing doesn’t need maintenance, very easy mistake to make but its wrong.

As you add more testcases to your test suite, over time your team may will begin to struggle to keep the test under control. It will become very difficult to add new test cases without effecting existing ones.

A mistake many testing teams make is to automate everything and not thinking about the management of the tests down the road. The focus to automate everything, this mindset leads them down a road where the tests become more detrimental than helpful. To avoid falling into this trap, here a few thing to consider before you begin your end-to-end tests.

**Define your test suite structure from the start:** At the start of your end-to-end journey take the time to plan test suite structure, for example: you directory structure, helper files, page object models and so on. The time you spend in setting these up will pay of in the long run.

**Reorganise as soon as the opportunity presents itself:** When you feel some of your files or code could be organised in a better way, do it as soon as you can to keep your test suite tidy at all times. It's quicker to refactor and rearrange your tests while you're working on them.

**Know your tools:** Knowing what your testing tools can and can't do, this will help shape your decisions on how to write your tests in the best way possible for long-term use.

##### Challenge #4: Demonstrating Value

Like mentioned above, companies under estimate the amount of time and effort needed to implement end-to-end automation testing. Yes, we do have advanced automated testing tools these days. Each project has its own set of needs, so it's not as simple as deciding to build an automated test suite. Teams need to plan their strategies, choose how to allocate resources and find time to start.

Sadly, many organisations are unaware of how automated testing can help the long term health of their projects. These companies look primarily for a return on their investment. 

Since it's difficult to immediately see value of end-to-end testing at the start of a project, it can lead to the organisation abandoning its efforts too quickly without anything to show for it. Also the primary focus is making management loose focus on the long term of their project that they don’t take the time to understand what end-to-end testing is. This leads to management question end-to-end testing value.

With some early preparation work you can begin to show the benefits of what you creating. If you're in a situation where you feel like you need to produce results and demonstrate the value of end-to-end testing, you can take a few measures to show that your work is helping everyone involved in the project:

**Make your results visible to the teams:** Show the entire team that your end-to-end test suite is working by displaying test run results using tools like Slack, emails, dashboards or any other place the team can see them quickly and effortlessly.

**Make your results visible to the teams:** Almost all testing tools can generate a well organised report that show how your tests performs.

**Keep track of how your tests improve your project:** Come up with a list of valuable metrics you can measure for the project, like defects found before a release, development velocity and so on. Once you have a list of metrics, take a baseline measurement and see how they improve over time as your end-to-end test suite expands.

#### Conclusion  

End-to-end testing is a vital part of an application. It complements other forms of testing, such as unit testing or functional testing, by providing additional coverage and exercising the system as a whole. An end-to-end test can uncover issues that other isolated tests might not expose and works best if you have an application with different services interacting with each other

However, the total amount of coverage provided by end-to-end tests also means increased complexity in managing them. The tests can be unstable and slow, requiring extra attention to keep them running smoothly. It's also challenging to demonstrate immediate value since they take longer to put into action. By following the tips listed in this article, you'll reduce these issues and provide a boost in quality across the entire project. 

Automating end-to-end tests can be tricky and hard to show your organisation that they're working. You'll need more time to plan, implement and execute. Once running, you'll also have to ensure they're working for your needs instead of slowing everyone down. But when done right, these tests will save your team countless hours of manual testing and make your application and team better for it.


Challenges testers face with end-to-end testing and tips on how to overcome them.

Challenges Of Automation Testing | Oswin Losper

CHALLENGES OF AUTOMATION TESTING

By now most companies (hopefully) must have automation testing or is in the progress to implement automation testing to their project. The benefits of automation testing is clear, refer to my article about, [benefits of automation testing](https://oswinlosper.co.za/posts/benefits-of-automation-testing/). 

Automation testing will help your organisation build better software quickly and efficiently. If automation testing is implemented correctly, it will allow companies to ship new features or new products to clients much more frequently and with more confidence.

However, most of those benefits rapidly decrease or even goes away if you don’t take the time to maintain your tests.

Maintenance has to be part of your automation suite, unless your codebase never changes you will have to keep on working and maintaining your automation tests, in order for it to keep up with the latest code as you applications evolves. Work that is been done during the software development process, will effect your existing tests in one way or another. Automation engineers might have to adjust existing ones

#### **WHY MAINTAINING YOUR AUTOMATED TESTS IS IMPORTANT**

If you do maintain your automation tests, it will reduce the risks and difficulties for everyone involve on the project. Tests that are not kept up to date increase the possibility of false positives, flakiness and slow test runs. These problems will slowly compound until it eventually reaches a point that its so much and no one either has the time to fix them or no one wants to deal with them. This is where a lot of companies lose trust in automation testing and test become a lot less valuable and is more of burden.

Flipping the switch to a well-maintained test suite. It can help developers and testers to deal with inevitable test automation issues that pop up. Tests will be easier to follow, update and to expand. If tests does fail due to a false positive or because of flakiness, it will take much less time to fix the problem. Well maintained test suites can also have additional benefits across a company.

Tests can become a good source of documentation about the project self. New developers and testers can see how the project is supposed to run and behave, help with onboarding quicker.

#### **EASY WAYS TO MAINTAIN YOUR AUTOMATED TESTS**

Maintaining your automated test suite doesn’t have to be an endless and exhausting task.

The best way to keep a high level of quality in your automation strategy is to have a mindset of maintainability while working on the tests. You might get away with building your test suite fast and deal with maintenance later, but it’s always more manageable while you’re in the middle of the work. 

Creating an organised test suite from the start will save countless hours and plenty of frustration in the future. The following tips will help you keep your testing work organised and tidy.

#### **ONLY TEST WHAT IS NEEDED**

The most common mistake made by companies is that they try to replace manual testers and therefor try to automate every possible thing. As good of an idea it might sound like, its not. It doesn't mean because you can automate most of your testcases, you should.  Some test cases don’t provide enough value to invest the time needed to write and maintain.

Every test that is added to a test suite, requires additional time and resources and the more tests you add the more additional risk and maintenance are required. When new test are added it will also add to total execution time of test runs.

The more tests your have, the more you and your team will need to maintain your test suite. Best is to keep your automate test suite lean to get the most out of each test run

If you’re not clear about what your team should focus on testing, be on the lookout for these places in the application under test:

**Features or parts of an application that often break for users:** Focus on parts of your application that have a high defect rate, this can increase your applications quality.

**Journeys that are difficult to test manually:** Test cases sometimes can have very long journey and this can be 'boring' for manual testers and things might slip through. Automate long journeys to avoid human error.

**Features that is critical for business rules:** It’s very important for areas that contain high value business logic and values to work reliable, to automate these and set up alerts or reports to notify if something goes wrong.

**Split your test journeys in to small test cases:** Testing like integration or end-to-end testing, allows you to automate long test journeys with steps from start to finish, and we all know we want to automate everything or as much as we can. 

It is a good idea to cover as much as you can, but also keep in mind, to many long test will make your test suite unreliable and difficult to fix.

In automation all steps you take to run a test has the potential to fail, whether is because of a timeout (if you run against a live api), a change in the application or a unexpected problem. The more steps in a single test makes it more likely to fail. You might regret writing a test that has to many checks at certain spots and figuring out what went wrong becomes a time-consuming process.

Instead of trying to long tests, focus on keeping your test small and specific to what you want to test. If you have shorter tests, it will be easier to debug and fix if needed.

#### **DON'T REPEAT YOURSELF**

A very important skill for test automation engineers is to to minimise and eliminate any redundancy throughout the codebase

Most developers are familiar with the Don't Repeat Yourself concept, that states, duplication in logic should be eliminated. I am a believer that this concept should be applied to automation testing as well. However a lot of companies are not taking up this approach because automation testing code is not treated as production code because its not client-facing.

> ***To avoid the headaches and hassles caused by code duplication, you should build your test suite keeping the DRY Principle in mind.***

If you have the same set of code twice in your suite, it will slow your team a lot, especially if changes are required on all of those lines. This is a also one of the main reasons that tests can be unreliable and that your test suite is not scalable. So the more re-usbale code you have - you can make a change in one spot and it will fix the issue through out your suite.

#### **REMOVE TESTS THAT DOESN'T BRING ANY VALUE**

Software applications and test suites are living, breathing system. The codebase changes frequently and many reasons, like new feature getting implemented and older feature are being removed, to mention a few.

Changes in an application wont live forever, and it’s the same for test automation. Test can eventually outlive their usefulness and weigh down the rest of the tests, often showing up in the form of a sluggish and broken test suite.

Many automation testers dislike deleting tests from their codebase, especially tests that they’ve written and business also plays a part in this. Sometimes they focus more on test coverage rather than the value of the tests.

Test automation engineers feels a certain closeness to the thing they created and don't want to let go, even when its clear a test doesn't bring value anymore. Instead of removing the test they prefer to comment it out or mark it as pending, hoping the test will somehow make a return and bring value to the test suite in future again.

**Below are a few good test that qualify for removal from your test suite**

**Test that is covering redundant scenarios:** If you have test cases that does have long flows to complete, you may not need dedicated test cases for them. For example, if you user journeys included authentication with a login page, there is no need for specific login test case

**Tests that cover soon-to-be deprecated functionality:** If there are plans to remove certain functionality within a system, it's a good idea to begin removing test cases that is related to that functionality.

**Low-value functionality that can be tested manually:** Many applications have sections that, while necessary to verify occasionally, aren't as essential to ensure they work at all times. Some examples are submitting "Contact Us" forms, ensuring static pages load properly, and checking content for localisation.

Deleting old tests doesn’t mean that you work was not helpful to a project. It means that the project has evolved (which is good) and that the test suite should also evolve with the project. If you notice at anytime that automated test is no longer helpful or doesn’t bring any value, delete it and do not look back, it will improve the performance of your test suite and keep your test suite is good spot and maintenance will also be much more easier.

#### **FIX BROKEN TESTS, DONT DELAY**

Im sure this goes for every daily activity out in the world, delaying until you get time to fix something, however, in the real world, you will never get time.

Ignoring broken test and to continue adding new test cases on top of them is one of the biggest mistakes teams make with their automation efforts.

When deadlines are coming into the discussion, companies prefer to stay on course to meet the deadline to complete their milestones rather than taking the extra time to fix the broken tests. While the team might think they are speeding up their workflow, they are actually doing the opposite.

Poor code begins to slide into the code base and then the quality starting to slip, slowing down their future efforts for automation more than if they had taken the extra time to fixes the issues promptly. 

One of the important keys to a highly maintainable test suite is to fix any problems quick instead of dragging the issue out. Fixing a flaky test quickly that someone pushed into the code base is much more easier than fixing a flaky test six months down. 

You will most probably forget about details of that test case and spend a few hours just trying research what the test case was about, then you have to switch context back to your current work flow. It can leads to laziness once a team decide to live with flaky tests in their test suite and it almost impossible to change that state of mind once you start.

If your team is in the habit of ignoring issues in your test suite while adding new functionality, even if there is flaky tests, put a stop to it immediately and change the mentality as soon as possible. Try to establish a work flow cycle where fixing current flaky tests is a priority over adding new ones just for the sake to increase visibility.

This doesn’t mean that if a test breaks, the whole team needs to switch focus and fix the broken tests, make sure someone is taking responsibility to get the the build back to a stable state quickly and not ignoring or delaying the problem.

#### **CONCLUSION**

A successful automation suite, starts with long term maintenance in mind. No matter how much time you invest in automation testing, you and your team needs to plans proper maintenance for your test suite. And as in life doing the small stuff like organising and refactor or cleaning up tests feels like a waste of time but will pay off in the long run. 

Maintaining you automated test suite doesn’t have to be a time-consuming process. You can keep your test suite in good condition by planing maintenance and establishing good practices as you go forward with your automation strategy. 

Focus on testing only what’s essential and removing tests when they’re no longer valuable. Organise your tests into logical groups and avoid unnecessary duplication with patterns and techniques like the Page Object Model. If your test suite breaks, fix it before it gets worse.

Keeping your tests lean and tidy will give you the speed and flexibility to tackle problems and improve your test suite with little to no friction. No one is immune to getting test failures due to always-changing applications. But with a bit of care from the start, a well-maintained test suite can help your organisation develop and deliver high-quality applications with ease for years to come.


If you do maintain your automation tests, it will reduce the risks and difficulties for everyone involve on the project.

BLOG POSTS

CROSS BROWSER TESTING

WHY AUTOMATION TESTS ARE FAILING

THE IMPORTANCE OF CI IN YOUR PROJECT

CHALLENGES OF AUTOMATION TESTING

MAINTAINING AUTOMATION TESTS