September 8, 2018

Modifying Your Mobile App Tests for iOS10

By admin Mobile App Development 0 Comments

Part 1 – The Challenges facing Appium testers when switching from iOS9 to iOS10.

Apple iOS10 was met with much of the usual fanfare upon release in September of 2016. For developers and testers that use open source software like Appium testing, the excitement soon turned to frustration as they discovered that their mobile app tests that worked in iOS9 with UI Automation did not function with the new XCUITest framework in iOS10.

It then became a mad dash to develop solutions and workarounds in order to try to solve the problems that had arisen as a result of the upgrade.

An additional wrench was thrown into the process when it became clear that another issue was the lack of backward compatibility. This meant that if you tried to take your new tests that you were creating in iOS10 they would not run smoothly on devices running iOS9.

So let’s take a closer look at the changes in iOS and Appium functionality, and how we were able to solve the problem and in in the process create an entirely new product, Enterprise Appium by Experitest.

Appium Testing Architecture for iOS version <=9

In order to address the problem with failing tests and a lack of backward compatibility, we have to dig deeper and take a look at the Appium Architecture for iOS9. This is in reference to Appium version 1.5 and lower.

The way that it operated is that the WebDriver testing layer script was in testcode/logic. It communicated with the Appium server using HTTP/JSON. One of the great benefits of this was that you could write your test using any language (Java, C#, Python, Ruby etc.).

The WebDriver script would communicate with the Appium server. The Appium testing server, in turn, would accept the requests from your WebDriver scripts. Then it would process the scripts using the Instruments command client to communicate with your devices.

The Appium server was written in Node.js, a JavaScript, that used the Instruments command client that is a part of an older version of Xcode.

The Appium server used the Instruments command client to run the instrument endless test, bootstrap.js. This ran continuously and waited for the user to send commands.

Let me give you an example. Apple introduced UI Automation as a framework for their automation testing. If you would ask Apple how to run your test on JavaScript or Objective-C they would tell you to execute your automation testing on the device using Xcode.

Appium leveraged those capabilities and created something that you could execute from any language and work on Android testing as well as iOS testing. Using the WebDriver script to translate any script into JavaScript.

In order to use this capability, there was a sort of a hack, because as it happens using UI Automation in this manner was not how the software was originally intended to be used.

When you started a test an endless loop would be executed on the device. The endless loop waited for commands from the user. The infrastructure used on the device is the UI Automation and the code for this endless test is JavaScript.

Simply put, your test would communicate with the Appium testing server and then would be translated by the Instruments command client to be able to communicate with the device or simulator. At least that is how it worked in iOS9.

What happens in iOS10

Much has not changed in iOS10 and higher. Tests continue to run in any language that can communicate with JSON or HTTP Protocols. As in the past, we have the Appium WebDriver aka Selenium that has served all of these requests.

What happened in iOS10 is as follows. Apple stopped supporting UI Automation in favor of an alternative application called XCUITest. The interaction with this new framework is done using an Xcode builder in the test flag. The Xcode builder then communicates with the test itself.

This is similar to how it works in iOS9 in that the test is executed in an endless loop, but the mechanism has changed. What happens is that the Xcode builder first communicates with the debug server in order to launch a new application called WebDriverAgent. This means that a new application is being installed, rather than simply communicating with the old built in UI Automation software. The new tool, called XCUITest, is installed when you use this new framework.

A lot of the challenges we are discussing stem from this change. It comes from the “Sign” of the XCUITest application, and the understanding that there is not only your application but an additional one which is the tool. This new application communicates with the test manager process that is then responsible for communicating with the device. The application is written in Objective-C so there can easily be some communication difficulties between the test, application and the test framework. Where we used the built-in test manager in the past it was not actually meant to be used this way. The new XCUITest application automatically mimics Xcode so the communication between the WebDriver and the device will be disrupted.

XCUITest Advantages

The advantages to XCUITest start with better code generation. Testers that use the Xcode environment, will see their tests executed much faster.

You can write your tests in Objective-C and Swift. This gives testers additional capabilities that were not available in the old UI Automation JavaScript programming language. For example, testers now have new ways to access the device memory and processes, for more complex testing. XCUITest also handles better when it is waiting for elements so working with elements that are not yet ready is much easier.

The XCUITest framework is specifically designed to enable users to understand what is happening to the onscreen elements. If you have an example of UI Automation that is written in JavaScript you will have a UI target and a local target from the session that you are working with. After you make your requests and query for all of the buttons you will see that the test scripts from Objective-C and XCUITest look quite similar.

Get Updates on User Quality at our Page

XCUITest Challenges

Element Identification Overview Challenges

The biggest challenges that you will face when moving to Appium 1.6 with iOS10 is changes in the page source. Elements are identified in a new way, and the changes in the hierarchy are what will cause the lion’s share of tests to fail. We have provided a list below. You can see the comparison of elements as they were named in iOS9 versus the new iOS10 version.

Page Source Challenges

Another major challenge in switching to XCUITest is that there are many differences in the page source. We used a command in Appium testing to get the page source. We then use it to compare the source from iOS9 to iOS10.

There are differences in the node name of an element, which reflects the class name, has been changed.

Also missing are the application class names. Those are wrappers and this is a UI Automation wrapper. What you will be able to see is the XCUITest wrapper object. The reason that a lot of your existing tests will fail is due to this class change. This node does not exist in iOS9 so if you have used a full path to identify an object the identification will fail.

One important lesson to learn here is how to handle and build a very good object identification strategy. Do not use a full path to identify elements. Use the element id instead. When we refer to full path identification it means to identify objects one by one via the element hierarchy, until you get to the actual object that you want to identify.

One more change that we would like to highlight here is that that the path to the object has been changed. If you use indexes to identify the elements in your application, in some cases this will cause your identification to fail.

Appium 1.6 Functionality Drop Challenges

There are many drops in functionality between iOS9 and iOS10. Even though your scripts and API have stayed the same, the engine that is running on top of it all has changed.

This introduced a new challenge to Appium developers. Part of the functionality is not compatible with this new engine and the new XCUITest is not available. This led to many different compatibility issues.

Landscape/Portrait Mode

The first compatibility issue that has been fixed in version 1.6.2 is working in landscape mode. Second is interacting with the devices when they are in landscape mode. One of the dropped and currently unavailable features is performing a device rotation. You cannot take a device that is currently in portrait mode and move it to landscape mode, and vice versa.

GEO Location Testing

Testing GEO location is not supported nor is the ability to simulate different locations in Appium 6.

Multi-Gestures

Gesture simulations like device shake, screen lock, performing pinch gestures are not supported.

Autoaccept Alert

Another feature that is not supported is the autoaccept alert. What this alert does is allows you to interact with device alerts and dismiss them yourself.

Shortcut Drops

The functionality also drops around shortcuts that were made between the script and UI Automation. Commands like (findelementbyiosuiautomation) that enabled you to add your JavaScript and execute it directly on your console is not supported because all of the infrastructure has changed. It is no longer a JavaScript UI commission that is running. Part of the functionality of (executescript) has been eliminated for the same reason.

We are confident that some of these capabilities will return in later versions. We cannot guarantee which capabilities we will see again and which others will be gone forever.

Enterprise Appium Solution Setup

This is what prompted us at Experitest to create Enterprise Appium.

What we provide is a combination backward compatibility and enterprise level scalability for your mobile testing.

We saw that when switching from iOS9 to 10 there were a lot of backward compatibility issues. There were other drawbacks to open source Appium as well. For example, offshore testing is practically impossible, as is out of app testing. There is also no company roadmap either. Basically, Experitest mobile app testing tools came out with an Enterprise version of Appium testing. This new product allows you to take you Appium tests from iOS9 and run them in iOS10. You will also enjoy Experitest’s mobile cloud testing tools that allow users to access real mobile devices remotely, from a secure centralized environment. It also provides testers with the ability to test with enhanced capabilities and scale any project.

In our next post, we will be expanding on the architecture and enhanced functionality of Enterprise Appium by Experitest. For more information and demos take a look at our webinar recording on this topic.

Dana Natan – Head of Marketing

Source

https://experitest.com/blog/seetestautomation/appium-testing-ios9-10-blog/

Vinova – Vibrant Innovators

Modifying Your Mobile App Tests for iOS10

Part 1 – The Challenges facing Appium testers when switching from iOS9 to iOS10.