Xamarin Offers Test Cloud Service For Mobile Developers

Mobile application development tools specialist Xamarin, has announced the Xamarin Test Cloud, which is an automated user interface testing service that enables mobile developers to easily test their apps on hundreds of mobile devices.

Xamarin, which boasts some 300,000 developers, made the announcement at its Xamarin Evolve 2013 event in Austin, Texas, 16 April. Xamarin Evolve 2013, with more than 600 mobile developers attending, is the company’s first worldwide developer conference.

App Testing

Nat Friedman, CEO and co-founder of Xamarin, told eWEEK that after helping developers with cross-platform development and enabling them to build fully native mobile apps for iOS and Android, “the next big challenge was how to help them test their apps. Mobile app development is quite different in some key ways. Quality is much more important than on the desktop. For instance, app sessions are really short and if an app crashes, users may not go back to it.”

Indeed, Friedman said a highly competitive app marketplace and the consumerisation of IT have put tremendous pressure on developers to deliver high-quality mobile user experiences for both consumers and employees. A small bug or crash can lead to permanent app abandonment or poor reviews. Device fragmentation, with hundreds of devices on the market for iOS and Android alone, multiplies testing efforts resulting in a time-consuming and costly development process. This is further complicated by faster release cycles for mobile, necessitating more stringent and efficient regression testing.

However, with Xamarin Test Cloud, developers can overcome device fragmentation by testing their apps automatically on hundreds of real, non-jailbroken mobile devices, Xamarin officials said. The service simulates real user interactions by testing apps through the UI directly. Now developers can deliver high-quality apps on multiple device platforms more quickly, reliably and cost effectively than with traditional manual testing methods, Friedman said. Xamarin Test Cloud will be available to developers building apps in Objective-C, Java and other frameworks, as well as tightly integrated into Xamarin’s development platform.

In a recent Xamarin survey, only 8 percent of developers reported using existing automated UI testing tools. These tools require significant investment to get started and rely on fragile methods of defining correct UI behaviour that can break test cases with small UI changes. In addition, many do not provide cross-platform support and do not integrate well into developer’s tools and workflow.

Automated UI testing is the way to go here, Friedman said. Yet Xamarin Test Cloud goes beyond other automated UI testing solutions and enables developers to conquer device fragmentation to ensure apps work properly on actual physical devices with hundreds of combinations of operating systems, screens and resolutions. Devices are not jailbroken and can be configured to run in a variety of languages and other environments.

Testing Options

The new Xamarin Test Cloud’s App Explorer will automatically navigate through the apps, visiting every screen and exercising buttons, entries, and other UI controls. This gives developers an instant way to view their app on hundreds of devices and to find bugs without having to write a single test script.

Xamarin also provides reports including detailed test results, browseable screenshots of apps running on real physical devices, performance monitoring, and detailed device logs and stack traces to help developers find and fix bugs quickly.

The Xamarin Test Cloud also supports continuous integration and object based UI testing. Xamarin Test Cloud includes plug-ins for popular continuous integration systems, including Jenkins, TFS and TeamCity. A command-line interface and API make it possible to run tests and get results from any custom build system and to integrate with existing tools. And regarding object-based UI testing, UI elements are identified by object IDs, not using image recognition or gesture recording, so that tests continue working even if changes are made to the app user interface, Xamarin said.

“At every step of the way, our mission is to make building mobile apps fast, easy and fun for developers. Mobile testing is a natural extension for Xamarin,” Friedman said in a statement. “This is by far the biggest pain point for our entire developer community. We’re excited to solve this problem for all mobile developers.”

Xamarin Test Cloud is based on Calabash, a widely used cross-platform mobile test automation framework. As part of the new Xamarin Test Cloud announcement, Xamarin also announced its acquisition of LessPainful, the company that created Calabash. Some of LessPainful’s current customers include the BBC, Axel Springer and eBay company Marktplaats.nl.

“Calabash and LessPainful have enabled us to continuously execute our automated acceptance tests for mobile projects, obtaining higher quality, higher test coverage and reducing manual regression testing time from several days to a couple of hours,” said Marcel Stekelenburg, QA Manager, Marktplaats.nl, an eBay company, in a statement. “With this we are also able to do cross-platform testing on Android and iPhone.”

“Xamarin has a truly amazing cross-platform development technology that delivers a high degree of code reuse while supporting beautiful native user-interfaces customised on each platform,” said Jonas Maturana Larsen, co-founder, LessPainful, in a statement. “Calabash has the same approach to automated testing: allow for a high degree of test code-reuse while providing powerful automation libraries specialised to the platforms. This is a fantastic opportunity to combine forces, and help developers and enterprise bring the best apps to market with Xamarin Test Cloud.”

Xamarin Test Cloud will be generally available in Q3, 2013. For more information and to sign up for the beta go here.

Think you know all about Android? Try our quiz!

Originally published on eWeek.