KINTO Tech Blog
QA

Key Aspects of QA Testing for Native Apps

Cover Image for Key Aspects of QA Testing for Native Apps

Introduction

I am Okapi, from the Quality Assurance Group.
In the previous article, titled "Increased Awareness of QA ," I described how the QA Group participates in projects and performs QA tasks.
In this article, I would like to write about the key characteristics of QA testing for native apps, drawing from my recent experience working on several native app projects.

About QA Testing for Native Apps

In basic QA design, a test scenario is created that clarifies the scope of the testing (what is to be tested or excluded), and test cases (preconditions, procedures, and expectations) are then created from that perspective. However, in addition to the above, a native app design requires consideration of the following points:

・ iOS and Android
・ OS versions and smartphone devices to be tested
・ Native app functionality

As a result, even native apps with less complex functions exhibit characteristics that tend to increase the scale of testing.

Item Characteristics
iOS and Android ・ iOS is an operating system developed by Apple
・ Android is an operating system developed by Google
Due to the differences in the development environments, the amount of testing simply doubles.
OS versions tested and smartphone devices to be tested There are many types of smartphone devices available to the public.
Testing all of these and OS version combinations is practically impossible.
Native app functionality Due to the unique functions and specifications of mobile devices affecting app functionality,
it is necessary to verify the operation of installed native apps based on these characteristics.
・ Push Notification:
 The function that enables apps to send notifications to the user
・ Touchscreen:
 The feature that allows users to interact with the app by touching areas on the screen
・ Permission:
 The feature that enables a pop-up dialogue requesting the user’s consent before the app attempts to access certain functionalities of the device.

iOS and Android

As the operating systems of iOS and Android are different, the amount of testing is doubled, so we want to reduce the man-hours. However, the development team is also divided into iOS, Android and BFF.

  • Even if there are no bugs on the iOS version, bugs may be found on the Android version.
  • Same may be true on reverse: no bugs on Android, but issues on the iOS version.
  • Issues with the API may require testing on both Android and iOS.

These patterns may occur with each function. Therefore, if we proceed in such a way as to omit testing on Android because it has been confirmed on iOS, or vice versa, there is a risk that 'bugs affecting quality' may persist. So we are proceeding with a basic policy of implementing the system for both operating systems.

OS Versions and Smartphone Devices to be Tested

Since testing all combinations of OS and devices is impossible, we always confirm the recommended environment for the target app (e.g., iOS 15.0 or higher / Android 9 or higher), taking into account the market share of each OS. And then assign:

  • Main OS: All tests conducted
  • Non-main OS: Display check and locations where OS-related bugs occurred
    As device-dependent bugs are rare, and even if they occur, they are unlikely to be fatal, testing is basically conducted on devices that comply with the above-mentioned OS.
    However, there is an exception: Functional tests involving camera activation will be conducted on all types of devices available to us. The reason for this is that during the initial QA testing of a native app, there was a relatively high frequency of issues where the app would crash when the camera was launched on certain devices.

Native App Functionality

In the process of QA of native apps, it is necessary to verify whether the app operates in accordance with specifications, considering the functionality of the mobile device. However, the specification sheet typically outlines only the app specifications and often does not describe the functions of the mobile device. Therefore, bugs can occur when device functions affect the behavior of the app. Test designs based on an understanding of device functions can effectively detect bugs from a usability perspective.

Item App Specifications Device Functions Common Bugs from a Usability Perspective
Push Notifications ・ Push notifications'
 - timing
 - content
 - transition destination when tapped
・ In the app notification badge's
 - display timing
 - deletion timing
・ Notification badge of app icon
 - display when multiple accumulations
 - display timing
 - deletion timing
・ Specification by change of condition
 - login/logout
 - App foreground/background/lock screen
・ The number of app icons does not match the number of notifications when receiving multiple notifications
・ The app notification badge
 - is not displayed
 - display timing is slow
・ The app notification badge
 - does not disappear
 - deletion timing is slow
・ When logging out, the target notification
 - cannot be received
 - error occurs when notification is pressed
・ No notifications when the app is in the foreground
Touchscreen Behavior when a button or link is tapped ・ Double-tap
 - tap twice quickly
・ Flick
 - swipe left and right with a fingertip
・ Swipe
 - slowly slide left and right
・ Pinch in
 - touch the screen with two fingers simultaneously and move them closer together to zoom out
・ Pinch out
 - touch the screen with two fingers simultaneously and move them apart to zoom in
・ Double-tap
 - to open the same screen twice
 - registration details are registered twice
・ Display error occurs when performing flick/swipe/pinch in/pinch out
Permissions ・ Allow/disallow push notifications
・ Allow/disallow location information
・ Allow/disallow camera
・ Push notification permission setting is not linked to settings
・ Crash occurs with location permissions and camera permissions

Next Challenges

In QA testing scenarios for native apps, we check for specifications according to system requirements and app-specific functions. As I became more fully involved in QA for app development, I learned that the iOS screening process is very strict.

For example,
・ Rejected unless it is in "non-login accessible" format
・ Rejected if it contains a link to GooglePlay
and so on.

I believe that by adding these insights into our knowledge, we can incorporate them into further ad-hoc testing scenarios as one perspective and lead to further quality improvements.

Conclusion

In this article, I introduced the characteristics of QA testing for native apps that have a tendency to increase the scale of testing. However, in native apps, QA reports numerous bugs from various perspectives, including iOS, Android, and API. This places a substantial burden on the development side to address modifications, resulting in higher costs for both sides compared to web apps, making the current situation less efficient.

The latest app development languages I worked with include Kotlin for Android and Swift for iOS, respectively. Native development for each platform has the advantage of individual optimization that allows for fine-tuning.

Although there are cross-platform technologies such as Kotlin Multiplatform Mobile (KMM) and Flutter that can be developed using the same source code, there are disadvantages, such as the need to learn separately the parts that depend on individual OS, such as device control.[1]

In that regard, native languages are directly supported by the corresponding OS, providing a high degree of freedom in leveraging device functions.
In the future, I expect and hope that the most suitable development method will be chosen based on the preferences of development engineers, as well as the purpose of app development. As a QA engineer, I also would like to exert effort in flexibly improving the efficiency of QA testing in alignment with the development method.

脚注
  1. Camera, GPS, sensor system, etc. ↩︎

Facebook

関連記事 | Related Posts

Cover Image for Exploring the Why Behind QA

Exploring the Why Behind QA

mmm
mmm
Cover Image for Key Points on Communication from Test Design

Key Points on Communication from Test Design

Cover Image for QA業務の認知度向上

QA業務の認知度向上

Cover Image for Initial Challenges in the Website Restructuring Project

Initial Challenges in the Website Restructuring Project

Cover Image for Reviewing QA Work From the Perspective of Verification and Validation

Reviewing QA Work From the Perspective of Verification and Validation

Cover Image for QA作業風景の紹介

QA作業風景の紹介

We are hiring!

【iOS/Androidエンジニア】モバイルアプリ開発G/東京・大阪

モバイルアプリ開発GについてKINTOテクノロジーズにおける、モバイルアプリ開発のスペシャリストが集まっているグループです。KINTOやmy routeなどのサービスを開発・運用しているグループと協調しながら品質の高いモバイルアプリを開発し、サービスの発展に貢献する事を目標としています。

【QAエンジニア】QAG/東京・大阪

QAグループについてQAグループでは、自社サービスである『KINTO』サービスサイトをはじめ、提供する各種サービスにおいて、リリース前の品質保証、およびサービス品質の向上に向けたQA業務を行なっております。QAグループはまだ成⾧途中の組織ですが、テスト管理ツールの導入や自動化の一部導入など、QAプロセスの最適化に向けて、積極的な取り組みを行っています。