How to design Android application test cases

One of the biggest challenges faced by ordinary Android developers in their daily work is that the range of terminal devices and [url=] operating system [/url] versions is too wide. According to a research conducted by OpenSignal, there were more than 1 1, 828 different Android terminal devices on the market in July of 20 13, and these devices were different in type/size/screen resolution and specific configuration. Considering that only 3,997 different devices were recorded in the previous year's survey, this is indeed a growing challenge.

From the perspective of mobile application development, terminal equipment is defined as having four basic characteristics:

1. operating system: the version of Android operating system defined by "API indicator" (1. 1 ~ 4.3).

2. Display: The screen is mainly defined by screen resolution (in pixels), screen pixels per inch (in DPI) and/or screen size (in inches).

3.CPU: This "application binary interface" (ABI) defines the instruction set of CPU. The main difference here is ARM and Intel-based CPU.

4. Memory: the device includes internal memory (RAM) and predefined heap memory of Dalvik virtual memory (VM heap).

These are the first two features, the operating system and the monitor, which need special attention, because they are directly and obviously felt by the end users and should be continuously and strictly covered by the test. As for the version of Android, in July of 20 13, eight different versions were running at the same time, which caused inevitable fragmentation. In July, nearly 90% of these devices, 34. 1% were running gingerbread version (2.3.3-2.3.7), 32.3% were running jelly bean (4. 1.x), and 23.3% were running ice cream sandwiches (.

Considering the device display screen, a research conducted by TechCrunch in April 2065438+03 showed that the vast majority (79.9%) of effective devices used "ordinary" screens with sizes of 3 inches and 4.5 inches. The screen densities of these devices vary between "mdpi" (~160 dpi), "hdpi”(~ 240 dpi) and" xhdpi”(~ 320 dpi). There are exceptions. Only 9.5% of the devices have low screen density "HDPI" (~120 dpi) and small screen.

If this diversity is ignored in the quality assurance process, it is absolutely foreseeable that bugs will sneak into the application, and then there will be a storm of bug reports, and finally there will be negative user comments in the Google Play store. Therefore, the current question is: How do you use a reasonable level of testing work to effectively solve this challenge? Defining test cases and the accompanying test process is an effective weapon to meet this challenge.

Use cases-where to test, what to test, how to test and when to test?

"Where to test"

In order to save your expensive testing time, we suggest reducing the 32 combinations of Android versions mentioned above and the 5- 10 version of the display screen representing the screen of leading devices in use in the market. When selecting reference devices, make sure to cover a wide enough range of versions and screen types. As a reference, you can use the survey of OpenSignal or the infographic detected by your mobile phone to help you choose the most widely used devices.

In order to satisfy curiosity, the size and resolution of the screen can be mapped from the Android file [5] to the density ("ldpi", "mdpi", etc.). ) and resolution ("small", "standard", etc. ) the above data.

With the help of 20 13 mobile phone detection research, it is easy to find a series of representative devices. There is an interesting little thing: the device resolution of 30% Android users in India is very low, only 240×320 pixels. As can be seen from the above list, Samsung Galaxy Y S5360 is also among them. In addition, the most commonly used pixel is 480×800 resolution (visible in the Samsung Galaxy S II in the above table).

"Test what"

Mobile applications must provide the best user experience and display correctly on various smartphones and tablets with different sizes and resolutions (UI testing) (keyword "responsive design"). At the same time, the application must be functional and compatible (compatibility test), with as many device specifications as possible (memory, CPU, sensor, etc. ). Coupled with the previous "direct" fragmentation problem (about the version of Android and the characteristics of the screen), "environment-related" fragmentation plays a decisive role. This role includes various situations or environments in which the user uses the terminal equipment in his own environment. For example, if the network connection is unstable, the incoming call is interrupted and the screen is locked, you should carefully consider the stress test [4] and exploratory test to ensure perfection.

You need to prepare all possible test scenarios covering the most commonly used functions of the app in advance. Early bug detection and simple source code modification can only be achieved through continuous testing.

"How to test"

A practical way to consider this wide diversity is the Android emulator, which provides an adjustable tool to simulate the end-user device of Android on a standard PC. In short, Android simulator is an ideal tool for continuous regression testing (user interface, unit and integration testing) of various device configurations (compatibility testing) in QA process. In exploratory testing, the simulator can be configured in various scenarios. For example, the simulator can simulate the change of connection speed or quality. However, QA on real devices is indispensable. In practice, virtual devices used as reference can still be different in some small (but very important for some applications) aspects, such as the Android operating system does not provide program-specific adjustments or does not support headphones and Bluetooth. The performance of real hardware plays a very important role in the evaluation process. Considering the support of touch hardware and the physical form of the device, it should also be tested (usability test) on all possible terminal devices.

"when to test"

Now that we have defined where to test (reference device), what to test (test scenario) and how to test (Android emulator and real device), it is very important to simply describe a process and determine when to execute which test scenario. Therefore, we recommend the following two-level process:

1. virtual device regression test.

This includes continuous automated regression testing on virtual reference equipment to identify basic errors at an early stage. The idea here is to identify bugs quickly and economically.

2. Use real equipment for acceptance test.

This includes intensive testing (mainly manual testing) on real devices during the "planning and promotion" before release to Google Play stores (such as alpha and beta test groups in Google Play [5]).

In the first stage, test automation is very helpful to realize this strategy in an economical way. At this stage, only those test cases that are easy to be automated (that is, executed every day) can be included.

In the process of continuous development of an app, this kind of automated testing provides a safety net for developers and testers. The daily test operation ensures the normal work of the core functions, and the overall stability and quality of the app are transparently reflected through the test data, and the certification regression can be easily associated with recent changes. This kind of test can be easily designed and recorded from the tester's computer using SaaS solutions (such as the UI mobile application test of TestObject in the cloud).

If and only if this stage has been successfully implemented, the process will continue to conduct labor-intensive testing in the second stage. The idea here is that if the core function passes the automatic test, only test resources will be put into it, so that testers can focus on advanced scenarios. This stage may include test cases, such as performance test, usability test or compatibility test. The combination of these two methods has produced a powerful quality assurance strategy for mobile applications [7].

Conclusion-Do the right test

Used correctly, testing can be a powerful tool against scattered Android. The key to an effective test strategy is to define the customized test cases for the application at hand and define the workflow or process to simplify the test. Testing a mobile application is a big challenge, but through a structured method, the right tool set and professional knowledge can effectively solve this problem.