Mobile Application Testing

Introduction:

Handheld devices are evolving and becoming increasingly complex with the continuous addition of features and functionalities. Testing is challenging in the handheld, wireless world because problems are new, or they show up in new ways. This paper is aimed to highlight certain crucial areas the tester needs to concentrate on while testing mobile applications.

The 4 main areas to consider:

• Understanding the behavior of the device
• UI & Usability Testing.
• External Constraints
• Stress Testing

Understanding the behavior of the device

If you are new to a device the first thing you should do is to get familiar with how the common device functionalities work – such as its Phone, Camera, Contacts, Calendar, Program, etc.

Things to note while exploring inbuilt applications:

• Overall color scheme/theme of the device.
• Style and color of icons
• Progress indicators when pages are loading.
• Menus – How they are invoked and typical items they contain.
• Overall responsiveness of applications on the device.

UI & Usability Testing:

The unique features of mobile devices pose a number of significant challenges for examining the usability of mobile applications, including screen orientation, multi-modality, small screen size, different display resolutions, soft keyboards, and touch screens.

Screen Resolution

If your application is supported on various devices that have different screen resolutions, make sure you test with the device that has the smallest screen and the application still looks good on larger screen sizes as well.

Screen orientation (Landscape/Portrait modes)

• If your device supports screen orientation changes be sure to include lots of testing where you rotate the device from portrait to landscape display, and vice versa, on all of the pages within your application.
• It is also important to test input reactions when the screen orientation is changed. Try using the soft keyboard while changing the orientation repeatedly. Attempt this repeatedly and quickly to see if the rapid changes in orientation have a negative effect on the application.

Touch Screens:

Make sure that the application supports multi-touch (eg: pinch, two-finger tap, two-finger scroll, spread, two-hand spread, etc), single touch – eg: tap, double tap, scroll, etc, touch based on the requirement. The application should be tested for long touch and soft touch behavior.

Soft keyboards – Points to consider

1. Does the soft keyboard appears automatically
2. Does the first layer of the soft keyboard include shortcuts related to highlights?
3. Does a long touch on a soft character key bring up several different character choices?
4. Can the soft keyboard be dismissed and re-displayed easily
5. Can the soft and hard keyboards be used interchangeably (if the device has both)
6. Do soft keyboard characters entered in password fields only show up as ****

Multi-modality:

Multi-modality combines voice and touch (via a keypad or stylus) as input with relevant spoken output (e.g., users are able to hear synthesized, prerecorded streaming or live instructions, sounds, and music on their mobile devices) and onscreen visual displays in order to enhance the mobile user experience and expand network operator service offerings. Make sure that the application supports the functionality based on the requirement.

External Factors Affecting Mobile Application Testing

Network Connections:

App going to be used on devices in various locations with various network connection speeds, it is important to plan testing coverage for the following scenarios:

1. Only Wi-Fi connection
2. Only a 3G/2G connection
3. With no SIM card in the device
4. In Airplane mode (or all connections disabled)
5. Using the network through a USB connection to a PC
6. Test intermittent network scenarios that a user might encounter in the real world:

Phone calls:

The tester has to check the application behavior during incoming and outgoing calls. Make sure that the application works fine during the following phone calls.

1. The application is interrupted by an incoming call, originator hangs up the call
2. The application is interrupted by an incoming call, terminator hangs up the call
3. The application is interrupted by placing an outgoing call, originator hangs up the call
4. The application is interrupted by placing an outgoing call, terminator hangs up the call.

Other Interruptions:

The tester has to consider the below interrupts that could have an impact on the functionality or overall responsiveness of your application.

1. Text messages
2. Voicemail notifications
3. Calendar events
4. Social media notifications (Facebook, Twitter, etc)
5. Alarm clocks
6. Low battery notifications

Device Settings

Explore your device’s options, and change settings such as the following to see how they affect your application:

1. Sound profiles – Does your application respect the device’s sound settings?
2. Device password/unlock pattern – Does your application still install correctly when prompted for a password/unlock pattern?
3. Font – How does choosing a different font family, size, or style affect the appearance and usability of your application?
4. Screen time out/Auto on, off- Is your application subject to screen dimming or automatically turning off even when it is actually busy?
5. Screen orientation – Does your application respect this setting?
6. Connections – How does enabling/disabling Bluetooth or other connection types affect your application’s behavior?

Stress Testing

Certain mobile applications consume more memory and CPU than desktop applications. Stress testing is a must to identify exceptions, situations with the application hang, and deadlocks that may go unnoticed during functional and user interface testing.

Note the behavior of the application while testing with the following scenarios:

1. Load your application with as much data as possible in an attempt to reach its breaking point.
2. Perform the same operations over and over again, particularly those that load large amounts of data repeatedly.
3. Perform the repeated operations at varying speeds – very quickly or very slowly.
4. Leave your application running for a long period of time, both interacting with the device and just letting it sit idle, or perform some automatic task that takes a long time.
5. Test multiple applications running on your device so you can switch between your application and other applications.
6. After testing several functionality switched off and switch on the device.

Have questions? Contact the software testing experts at InApp to learn more.