Mobile testing is becoming a fast paced industry that is driven by constant evolution. The increasing adoption of mobile devices and changing technologies have brought a whole new set of challenges for the testing world. End users of apps aren’t forgiving and even the most minor issue will lead to customers switching to a different app immediately. Consequently, every mobile website and app placed in the market needs to be “rock solid” in terms of looks and performance. Smartphones have become the primary tool of communication for both individuals and businesses, across the globe.
According to Cap Gemini’s Quality Report on Mobile Testing, about 18% companies feel that they don’t have the time to test their mobile apps and further, about 65% say that they don’t have adequate tools for testing. Every week, new devices are being launched into the market and newer applications are designed to deliver complex set of functionalities over simple platforms. This diversity and increasing expectations present unique challenges for mobile testers.
1. A multitude of devices
Since Google launched the Android 1.0, more than 500m devices have been shipped. For Apple, the numbers close in to more than 220m. Smart devices capable of running applications span across smartphones, tablets, iPad, watches and each of these categories further include thousands of devices, each with different screen sizes, operating systems, and other capabilities. It is a real challenge to be able to create an app that would run smoothly across devices and platforms as you never know which device your target user is on.
2. Device fragmentation / different OS platforms
Device fragmentation is probably the most challenging aspect of mobile testing. The iOS device matrix has been growing but when it comes to Android devices (which again have a larger share in the market), fragmentation can be a major issue. In July 2013, OpenSignal, an analytical firm reported more than 12,000 Android devices as opposed to just 4,000 in 2012. It is not just the changing hardware but also the constant upgrades and different mobile operating systems that add to the challenge.
3. Types of mobile apps
Mobile apps can be categorized as being native, hybrid or web apps. Both the implementation and testing process of each app category is different than the other.
4. Varied test interfaces
Mobile simulators and emulators are crucial testing tools as they help verify regular regression tests and general app functionalities. However, the advantages are limited in scope and would never be a substitute for real world scenarios.
5. Varied testing tools
If the complexity of automation wasn’t enough, testing labs are also challenged by the variety of testing tools spanning across free/paid, native/web, android/iOS, etc.
The solutions
- Mobile test labs: Cloud testing labs like SauceLabs, MobileLabs and Xamarine Test Cloud have been able to create effective sampling and cover a large segment of devices. They also cater well to the changing OS, screen, memory, fragmentation and device types.
- The perfect testing solution: Solutions like Appium, Robotium and Calabash are open source platforms for app testing. Commercial alternatives include perfecto, eggplant, etc.
However, it finally depends upon the app strategy, the target market and the quality of app to be able to create a successful campaign.
