Tag: mobile application testing

Mobile App Testing: The Basics Part 3 (Mobile App Testing Tools)

Younes

Mobile App Testing Tools

We previously discussed the nature of mobile app testing, common problems in the testing process, and testing strategies. Now, we’ll take a look at mobile app testing tools that can streamline the testing process. And since one of the best strategies you can use over time is to “automate whenever it is possible”, it is important that you don’t miss in automation tool selection.

 

Here we list five mobile app testing tools you can consider before you dive into testing your apps. Note that the basis for the inclusion of a tool in the list is the recommendations from other sources and that we don’t favor one over the other. And though not extensive, we hope that this list will be helpful for you.

 

Appium (Open Source, Cross-Platform)

Appium is a test automation framework that works for native, web-based, and hybrid apps. It also drives cross-platform apps through the WebDriver protocol. This means that you can create test cases with the use any WebDriver-compatible development languages and tools. Appium’s philosophy centers on the idea that app testing automation frameworks should let app testers use their preferred tools, test frameworks, and test practices.

 

BitBar Testdroid (Cloud-based)

This testing tool is available in either public or private cloud. It is a “mobile device farm” with an inventory of thousands of real mobile devices (Android and iOS) for manual and automated mobile app testing. These devices are accessible round the clock through the monitoring of distributed teams. That, and other services like quick delivery of test data, logs, screenshots, and videos are some of the reasons why this is the testing tool of choice of many payment providers.

 

TestFairy (Beta Testing Platform)

Testfairy removes one major problem with crowd testing – the guessing on what really went wrong. Testfairy’s solution is to provide a video of the exact test aside from the logs and crash reports. It also works on any development platform, whether native, web-based or hybrid. Another selling point is its compatibility with bug trackers like Bugzilla and Jira.

 

SIGOS (Cloud-based)

This is Keynote’s mobile testing solution. SIGOS boasts of the combination of powerful mobile app test automation features and total control on manual device tests. Testers can run exploratory tests from anywhere through the DeviceAnywhere cloud service. You can even test on newly-released devices using real mobile carrier and Wi-Fi networks.

 

Ranorex

Finally, Ranorex Studio offers to test for desktop, web, and mobile apps. It is a service marketed to both beginners and experts alike. At the forefront are easy to use automation tools that support many third party tools and frameworks. Features that stand out are the GUI object recognition, easy step by step setup and recording of daily tests, reusable code modules, and early bug detection.

Mobile App Testing Strategy

Mobile App Testing: The Basics Part 2

Younes

In the previous post, we discussed the basic definition and types of mobile app testing. Today’s post will focus on common problems and strategies in mobile app testing. As earlier stated, mobile app testing is a necessary process but it can also be tedious and tricky especially for those just starting out in app development. Some even skip this process entirely.

 

So aside from the issue of cost and time, there are other issues that an app developer can face in mobile app testing.

 

Common Problems in Mobile App Testing

  • Different mobile device types and sizes

    About hundreds of millions of mobile devices were sold in the past few years. Apple alone launched more or less 51 mobile devices. Meanwhile, there are hundreds of brands under Android’s wing. Each of these brands already launched from 1-250 devices of varying sizes. According to DeviceAtlas’ Q4 2016 data, device fragmentation is still a major problem, with different countries favoring different brands and screen sizes.

  • Mobile operating platforms/systems

    A lot of devices from the same brand run differing mobile OS. Sometimes, these devices run on different versions of the same mobile OS.

  • Mobile network operators

    Depending on the source of the data, there are about 400-800 mobile network operators around the world. Arguably, the number of the network operators is not the main problem but the network technologies and infrastructure they are using. For example, some countries only have limited LTE coverage or use obscure networking standards. Another main issue is the location of the network. In order to fully test a specific operator network, you should be in an area where the network has service.

  • App types (native, web, or hybrid)

    Different app types have different installation processes and functionality features to test, therefore, these apps are tested in different ways. Native apps are considered quite complex to test. A successful testing on one device doesn’t necessarily mean that the app will download, run, and successfully execute tasks on other devices. Web-based and hybrid apps are also not free of any testing problems.

  • Test method (emulator or real device)

    You have three options: test on real devices, emulators, or use both. All options have their own advantages, disadvantages, and limitations. For instance, testing on real devices can be costly, time-consuming, and somehow chaotic. Emulators, on the other hand, can’t really emulate all the characteristics of a real device, especially when it comes to real-time behaviors.

  • Testing tools

    It’s actually not difficult to look for mobile app testing automation tools. The problem is that there’s a lot of them. You need to consider if a particular tool is suited for the tests you want to conduct and if it is compatible with the device and OS you are testing for. These tools also use different test interfaces that are far from the real environment of the app.

 

Mobile App Testing Strategies

  • Emulator selection

    Do not discount the use of emulators. Emulators are actually quite useful and can save you a lot of time and money. You can select from three types of mobile app emulators:

    • Browser Emulators – emulates mobile browsers
    • Device Emulators – some device manufacturers provide their own emulators.
    • Mobile OS Emulators – the Android SDK has a device emulator. Apple and Microsoft also offer emulators for their respective devices.

  • Mobile device selection

    It’s almost impossible to test for all devices available, especially with Android devices. You need to find a middle ground and this can be achieved by closely analyzing your target market. Then Select the most widely used device in the region and start from there.

  • Automation approach

    Automate whenever it is possible or practical. This is to achieve consistent results. You can select from two types of automation tools: object-based testing tools and image-based testing tools. Object-based testing tools map the elements on a device’s screen identify them as objects. Image-based testing tools, on the other hand, use the device screen coordinates of elements to create automation scripts.

  • Implementation of beta testing

    Beta testing allows you to cheaply test an app in real-time using real users, different devices, and local mobile network operators.

  • Cloud testing as mobile app testing solution

    Cloud testing, especially device cloud can give you flexibility in performing real-time tests. You can conduct these tests remotely, no matter how complex and dissimilar the app you are testing.

  • Testing on supported mobile networks

    Lastly, you can use tools that mimic mobile network infrastructure in order to test in the app’s supported network. These tools can simulate varying network speeds, bandwidths, and other connectivity conditions.

 

Next week will be the last installment of the “Mobile App Testing” series. We will discuss various automation tools.

 

 

 

Mobile App Testing Basics

Mobile App Testing: The Basics Part 1

Younes

What is mobile app testing and why is it necessary?

Mobile app testing is a process that involves the evaluation of a mobile app’s usability, functionality, and consistency. There are different types of these tests that work for different mobile platforms, operating systems (and their different versions), devices (types and sizes), and networks (types and operators).

The results of these tests help mobile app developers quantify the app’s performance level. It also ensures that the app’s quality. A simple, overlooked mistake can cause not only monetary loss but also damage to a brand’s image.

 

Types of Mobile App Testing

 

Functional Testing

This is the test for the app’s functionality. The test mainly centers on the app’s UI and screen flow.

 

Usability Testing

This test aims to assess the user-friendliness and UI effectiveness. Usability Testing ensures the uniformity of user experience in all devices

 

Laboratory Testing

Usually, only network carriers perform this type of mobile app testing. The test’s main goal is to evaluate the app’s online functionality.

 

Performance Testing

This tests the situational performance of an app from both the side of the app’s server and the side of the client. The possible scenarios for testing include heavy server traffic, low device battery, low device memory, etc. The app’s behavior during these scenarios determines its performance level.

 

Load Testing

This test is related to performance but focusses on one issue: the app’s load time. If an app server can’t handle heavy traffic, the load time may slow down or the app can crash.

 

Memory Leakage Testing

Memory leakage happens when an app fails to “return” the memory it temporarily obtained from the mobile device. This can cause the slowdown of not only of the app but also of the system. If the app opens constantly or runs continuously, even the smallest memory leak can cause the app to crash.

A bug is more often than not the cause of memory leaks. That’s why it is important to test for these leaks while still in the development phase of the app.

 

Interrupt Testing

This is a test to determine if an app can handle common interruptions while it is running. These interruptions may include common device functions like voice calls, messaging, notifications, etc. There are also interruptions that can cause the app or even the device to terminate like battery removal, data transfer interruptions, system warnings, etc. Upon reopening, the app should be able to resume to the last state it was in before the interruption occurred.

 

Installation Testing

The app is installed and then uninstalled on different devices in order to ensure that the installation process goes without a hitch. This test also includes the process of updating the app.

 

Security Testing

Any data and network security vulnerabilities surface during this test. The app needs to pass the industry set security and user privacy standards.

 

Outdated Software Testing

This test aims to check if an app works on previous versions of the mobile operating system it is developed for.

 

How are these tests conducted?

You can conduct a mobile app test manually or with the use of automation tools. With manual testing, the tester acts as an end-user. The app is installed on different devices and the tester will conduct any desired tests in real time. But soon enough, device fragmentation becomes a problem. It’s impossible and impractical to test on all myriad of devices carrying different versions of different mobile operating systems. To make things easier, you can use emulators or simulators (we’ll discuss in the next posts).

But there are times where automated testing is more practical. Scenarios where automation is advisable includes:

  • Routine test cases. These are cases you often perform and has predictable results.
  • Test cases that are too tedious for manual testing.
  • The app is consistently growing
  • Outdated Software tests and other OS compatibility test cases

On the next post, we’ll be discussing common problems in mobile app testing and mobile app testing strategies.