Testing is a topic we talk a lot about. It’s also, unsurprisingly, a topic we care a lot about. When done correctly, testing can save time, money, and even a brand’s reputation. This is especially true in the fast-paced world of mobile app development, where small glitches, performance issues, and integration struggles can lead to high levels of app abandonment. Testing is always a chance to guarantee a good experience, but with mobile apps, it can mean the difference between an app that succeeds and one that fails.
Mobile apps, like their wearable cousins, exist in a landscape that is growing and evolving rapidly. And like wearables, mobile app testing requires two things: First, a foundation of the essential components to be tested, the must-checks that will likely always be a factor no matter how much the landscape evolves. And second, an eye to the trends in mobile development (and user expectations), to understand how to test for the ways the app will be used now and in the future.
It can be a lot to try and keep track of, so we created this checklist for mobile app testing to help give us all a solid footing when we go into testing for mobile. Read, share, and comment, but most of all, get testing!
*Note: Native apps, web apps… they’re slightly different beasts, we know. But there is a great deal of overlap between the two and with that in mind, we created one big list for mobile testing. This means not every item will be relevant to every app scenario. We trust you’ll sort out what’s best for your needs on your own!
Integration & API Testing (e.g. Google Maps, Facebook Sign-in, Paypal)
This one shouldn’t be too much of a surprise to those of us working in app development already, but it also can’t be stressed enough. These could be make or break issues for your apps, so making sure the integrations are tight and functional should be a top priority. Be sure to test for things like invalid credentials so you can be certain all levels of the integration are functioning well.
IoT connectivity (e.g. connected cars, wearables)
As the Internet of Things evolves, so too will the scope of testing for it. Mobile apps, in particular, are fertile ground for IoT connections. Testing for IoT will include things like the distance to a home device and bluetooth compatibility, obviously, but there will be other, more specific functions you’ll need to test. And testing for IoT compatibility and functionality will be somewhat manual-labour heavy. Things like turning lights on and off or locking doors will be much easier and faster with two or even three testers working in tandem. Thinking through how this will impact your testing time-frame ahead of time will make things much easier.
Personalization (e.g. Every. Single. App.)
There’s no point in dancing around it: personalization is already huge, and it’s poised to be even more important in the next few years. Every app will be integrating more data and harnessing whatever machine learning/AI-based development possible to make apps — both web and native — better and more useful for the user. But even without AI & vast amounts of data, a certain amount of personal profile building and preference-setting is standard at this point, which means there are a lot of things you’ll need to test for even if big data and AI are a long way off for your app.
Whether it’s data based on location, browser history, stated preferences, or some other facet of personalization, it’s important that industry-wide we get comfortable with this layer of testing — and fast. Set a bunch of different preferences and see how they impact the app and the device you’re testing on. Employ emulators, so you can change location, weather, time of day (in essence, virtually travel around the world) to be sure you’re getting information tailored to the region, time of day, or weather of the place you’re currently in. It’s going to be a big chunk of testing time, but it will be worth it.
Location-based services (e.g. Uber, Airbnb, most retail & restaurants, parking)
Most apps can have region-specific components, and many are entirely based around a user’s locale, so testing the location-based abilities of the app is a big slice of the testing pie. A lot of this testing is similar to personalization — often directly tied to it, even — and will require you to rely on location emulators and their ilk, then push data for that location to see if you, as a user in that location, are seeing the correct data (e.g. recommended events or appropriate store locations). The scope of this kind of testing will depend on what services are available through the app, so be sure you’ve thought about just how vast the location-based needs of your app are in advance of testing to ensure you’ve allotted enough time to properly test.
Checkout flow (e.g. e-commerce apps)
Increasingly, people turn to mobile when making their purchases even when desktop is an available option, which means you need to be sure you’re testing all aspects of the checkout flow: Is the credit card data being handled properly? Is it secure end-to-end? Is it being validated before the user can check out the items? Does it integrate with other pay options (e.g. Apple Pay, Paypal, etc)?
Push notifications are becoming as standard as apps themselves. Come up with a list of notifications your app has and be forewarned — it could be very, very long. From beacons that alter location-based notifications (alerting a user to a sale when they’re in store, for example), to alerts about friends who have liked something, leave no notification off the list. Because equally as important to knowing the notifications work is giving your users the ability to opt out of the them entirely. There is no better way to alienate users and have them turn away from your app than to constantly pop-up on their screen, interrupting their day with information they don’t find useful. This testing will needs to be thorough.
Permissions testing is an important one. Hardware components (e.g. a camera or earphones for videoconferencing) can be finicky, but they’re also crucial parts of the app experience. It’s important to make sure they work and, just like with push notifications, it’s equally important to be sure users can disable a permission. In particular watch out for location-based permissions — invading a user’s privacy is no way to win them over.
In the age of mobile, real-time testing has become much more important. Testing real-time data includes things like ensuring your app connects to the various sensors (such as the hardware, gps, etc) that make it run and, even trickier, making sure that it displays the right information at the right time. Think about a fitness tracking or route planning app: if a user has a specific time or distance goal and the app can’t serve that information up in real-time, the entire functionality goes out the window. Equally useless is a videoconferencing feature that can’t enable a seamless experience. Guaranteed your app has real-time functions that will turn users off if they don’t work properly.
Adoption may not be there (yet), but voice-assistive technology is on almost every phone now and needs to be well integrated. First and foremost, it’s a necessity for achieving a level of accessibility that should be built into your app from the get-go. But it’s also going to be a standard operating mode, as systems like Amazon’s Alexa ramp up. Even if you’re not currently seeing users make use of the voice-assist technology they have on their devices, future-proofing takes a lot less time and money than fixing what’s broken and trying to win users back after you’ve lost their trust. Testing plans need to include voice-assistant integrations to be sure they work and work well.
It seems almost redundant to talk about, since it probably feels like second nature for most app development teams, but make sure your testing plan includes any social integrations. It’s all standard stuff — that the data is being sent correctly, safely, and shared as expected — but just because it’s standard doesn’t mean it’s old hat. Do this right and it’ll make a big difference to your app’s success. Guaranteed.
One of the trickier things to test for is localization. Again, you’ll probably rely on emulators to change the default area so you can be sure that the info in the app shows up correctly and for the correct areas. Being great in one language or one country is wonderful, but making the experience seamless and pleasant for everyone requires an extra level of attention. Is the time-zone changing when you set the country to the other side of the world? Does the default language reflect the expectation of the region you set? Pay attention to the details (and get a good translator!).
Regression testing & backward compatibility
Regression testing & backward compatibility are not new concepts, and the myriad ways in which overlooking these tests can impact your app should be well known by now, but our list wouldn’t be complete without them. So, just once for posterity: Every time you send out a new update, you need to re-test everything that could be impacted. No ifs, ands, or buts.
The essentials: It’s a big category, filled with lots of small, pivotal things. Like battery power, for instance. Does your app consume a lot of battery power, draining the device and leaving users stranded? And if the battery does die while the app is in progress, what happens to the data being generated? Other considerations, like screen visibility, matter a lot too. Is your app likely to be used outside? If so, how well can the colours you’ve chosen be seen in different lighting conditions? Or across different devices?
These are the things that apply to every app and concern every user type. Does your app consume a lot of data or memory (like music and video players)? How will you ensure this impacts your users as little as possible? And most importantly, what are the possible ways the security of your users could be compromised by your app, and how are you ensuring you’ve done everything possible to make sure it won’t happen?
The final category for testing isn’t so much a test itself, but it is as crucial to the testing as the rest. Your launch time will have an enormous impact on your ability to test if you don’t factor it in as early as you can. Companies often make a big deal of announcing the deployment of new, cool features or devices, which means you’ve got timelines that must be strictly adhered to, whether you’re ready or not.
And remember, you’ve got a secondary layer to factor into your timing, which is that the app needs to pass through the Apple or Google Play store’s own testing before it can go to the public. You’d better be sure you thoroughly tested everything prior to submission, because their own, quite stringent testing will unearth any flaws, and depending on how severe they consider them, your app could get rejected, delaying your timeline significantly. The best thing you can do is to build in a buffer of a week or two so you can loop back on issues if need be, prior to the official launch date.
As with everything in the world of tech — and especially the world of mobile development — things are constantly in flux. This list isn’t comprehensive, even now (though it should serve as a solid foundation with ample edge case scenarios to give fairly good coverage), and as things like VR/AR and further extensions of IoT continue to emerge and evolve, the landscape for testing will alter significantly. Staying on top of these important areas of testing will become even more important as the ground continues to move beneath your feet. So good luck and happy testing!
Interested in mobile app development? Have a yen for testing? Apply to work with us here.