A global overview revealed that more than half of the world’s web traffic now comes from mobile devices and by 2018, more than 50 percent of users will go to a tablet or smartphone first for all online activities. So, it is not surprising that today’s software development is based on a mobile-first, even if it is not mobile-only, imperative. But for all these dramatic changes, developers still struggle when it comes to mobile testing.
There are many factors that can make or break the success of a mobile app. For example, consider device fragmentation. At present, users use smartphones (Android and iOS) widely but there are hundreds of millions of other devices running older firmware. When your app needs to work across most of these devices, it can become a serious burden for testers.
Another factor that contributes to this complexity is the device itself: varying screen and device sizes, resolutions, and orientations. Next, we have multiple app types, such as web, hybrid, and native. These multitudes of devices operate differently on various device and OS combinations. Finally, you have users all over the world in different regions that must be tested for translations, time zones, and targets. These factors make testing with mobile a challenge.
Even though mobile testing is complex, it can be done successfully with the correct strategy. A sound mobile test automation strategy must include test automation frameworks, real devices, and emulators and simulators.
When building a mobile testing strategy, there are three key areas of focus:
- Real device testing
- Emulator and simulator testing
- Test automation frameworks
Read our blog on Simulators, Emulators, or Real Devices: Which is the Best for Mobile Testing?
By focusing on these three areas, organizations will thrive in the fast-paced world of mobile app development. Scale continuous testing with emulators and simulators For several years, the use of emulators and simulators to test mobile applications has been met with some resistance. This is based on the perception that if you’re not testing on a real device, you’re not testing at all. For continuous testing and continuous delivery of mobile apps, both emulators/simulators and real devices are needed.
An emulator, as the term suggests, emulates the device software and hardware on a desktop PC, or as part of a cloud testing platform. The Android (SDK) emulator is one example. On the other hand, a simulator delivers a replica of a phone’s user interface. It does not run the real device OS; rather, it’s a partial reimplementation of the operating system written in a high-level language. The iOS simulator for Apple devices is one such example.
Emulators give teams the ability to implement parallel testing and test automation via external frameworks like Appium or Espresso. Selenium revolutionized the world of web app testing by pioneering browser-based test automation. Today, Appium is its counterpart for mobile app testing and uses the same WebDriver API that powers Selenium. This enables the automation of native, hybrid, and mobile web apps and also increases the speed of tests for organizations. Emulators enable parallel testing in a way that can’t be achieved with devices in a lab. Because tests on emulators are software-defined, multiple tests can be run on tens of emulators at the click of a button without having to manually prepare each emulator for the tests. Further, automation is easier with emulators as the tests can be executed without manual intervention, and be controlled remotely.
DevOps, continuous testing, and continuous delivery
The speed of release and change demands of mobile development is agile and requires continuous testing as a key component of the overall regression testing strategy. In order to properly build out a mobile regression testing strategy, the dev/test teams should be well-equipped with the following:
- A comprehensive web and mobile testing platform: Since building a test lab from the ground up is timely and expensive, it is advisable to use a cloud-based, device lab solution with an extensive choice of real devices, as well as emulators and simulators. Parallel testing should also be included so that test execution can be done in a shorter amount of time.
- Highly scalable, highly available solution: Developers and testers need to ensure that the infrastructure for mobile testing allows the team to expand coverage as the test suite grows. The goal is to decrease test execution time while providing fast feedback and to ensure that the team spends less time chasing false failures and more time on actual testing and defect remediation.
- CI/CD optimization: The regression testing process must have tight integration with the development workflow and mobile app delivery pipeline to keep up with the fast pace of continuous delivery. This will eliminate any unnecessary manual steps and encourages automation testing throughout the testing process. The use of mobile regression testing reduces the risks of flaky software.
Mobile test automation frameworks
Test automation frameworks are an integral part of mobile test automation. There are many test automation frameworks freely available, but the top mobile testing frameworks are Appium, Espresso, and XCUITest. Sauce Labs recently conducted a survey of teams using test automation frameworks for mobile app testing The survey found that 63 % of the participants use the Appium framework, while 31 % didn’t use any test automation framework at all. Since Appium is based on Selenium (the popular open-source functional testing framework), it wasn’t a huge surprise that most of the users surveyed were using Appium. But the main question is why is it that over 30 % of users were not using some sort of testing framework? The answer lies in the implementation or lack of a mobile testing strategy.
When implementing a mobile testing strategy, it is very important to understand
- the skill set of the test automation team and
- the framework that best fits the organization’s preferred development methodology.
Tester automation skillset
When evaluating frameworks, it’s important to understand the technical background of your team. Some frameworks take a “black box” test approach, which is typically best for traditional test automation engineers. While other frameworks take a “white box” approach suitable for developers.
