iOS automation testing: the new baseline skill for ensuring quality

More and more, I’m noticing that automation testing is a key skill set for the modern Quality Engineer. More companies are looking to automation as a way to allow a few people to be incredibly productive. Although people often think of automation testing for front end web applications, there is also a lot of recent attention on adopting the same techniques for iOS native apps.

Here is a very high level list of tools and skills one should adopt when building out a great automation practice.

Tools

• iOS UI Automation framework. I have reviewed a number of test automation products, and found that the UI Automation toolkit from Apple provides great flexibility, with ease of test authoring. Also, because its the Apple recommended solution, I believe it will serve you well as the iOS platform evolves (i.e. it will likely improve). The only drawback to this solution is that it only supports iOS. Android tests would need to use an alternate toolkit, and thus an alternate workflow.
• XCode and Instruments. It’s easy to grab this from the Mac App Store, but getting to know these tools will take a little more effort.
• Jenkins. The automation really comes from the automated running of the tests you’ve written. Without employing something like Jenkins, your tests will be nothing more than fancy manually executed tests. Ultimately, you want your tests to run automatically, either overnight, or perhaps when developers checkin new code. Whatever your strategy, having a dedicated continuous integration environment will improve the value delivered by the tests you’ve written.
• Overview of UI Automation Testing. Take a look at some of the great detailed resources for learning how to perform iOS automation testing.
• Tuneup.js. This is a useful helper library which will take some of the pain out of writing your Javascript automation tests. Less code, higher productivity.

Skills (priority order)

• Javascript development. Ultimately, most of the time you spend creating automation tests will be with writing the tests in Javascript. The UI Automation framework, and tools like tuneup.js, will reduce the amount of code you write, but this is still where you’ll put in 95% of there automation effort. There are tons of great places top learn Javascript. For example, Code School.
• An eye for identifying those tests worth automating. This is where you’ll want to consider carefully those areas of the application that provide the most value from automation testing. I’d recommend against trying to automate everything. Instead, be realistic. I’d shoot for the areas of the app that are part of a smoke test, those things fundamental to the app’s function. Perhaps picking on areas that break often, or take a long time to manually test. At the same time, you might want to avoid areas of the UI that undergo a lot of developer change. These areas are expensive to automate, because you’re constantly having to tweak the test scripts.
• A tiny bit of Objective-C. The tests will require a little bit of massaging in the Objective-C code within the app you’re testing. Usually, this means adding Accessibility labels to the UI controls that you want to test. These labels are used in the test framework to identify the individual UI components so that they can be used in the test. The labels could be applied by the development team, but it’s even cooler if the Quality Engineer has the skill to add the labels they need, as it might not be a priority for the dev team to address these requirements.

Getting Started. I think the most important thing to do is start small. Write a manual test script that approximates what you might do for a smoke test. Your first project might be to turn this manual test into something automated. After you have an automated script created in Javascript, it’s important to get it hooked up to some continuous integration system, like Jenkins, so that it’s actually running automatically. Without this, your test will likely rot in that you might forget about it and it might start to break over time without anyone noticing. Once those steps are complete, then you can safely say that you have a small amount of test coverage! Now keep going and write more tests; you’re likely hooked at this point.