Xamarin, a company which provides tools for cross-platform development in C#, has announced its acquisition of LessPainful and the creation of cloud-based testing for mobile apps based on LessPainful’s technology and the Calabash scripting language it created.
The Test Cloud will perform automated user-interface tests on real devices, hosted by Xamarin, will provide detailed reports in the event of test failures, and will support Continuous Integration so that bugs are caught as early as possible.
“After you’ve conquered the cross-platform mobile development problem, testing is the next large pain point” says Xamarin CEO Nat Friedman. “You can’t just get by with manual testing. There’s a need for the same level of tools and processes in mobile testing that you have in desktop and web testing.”
“Quality is actually more important on mobile than in other places. Mobile sessions are very short. People are really intolerant of low quality on mobile. The release cycles are shorter too. People are revving more frequently, and testing is a bigger challenge.”
Another issue with mobile testing is the number of devices out there, especially if you throw cross-platform into the mix. “You have on Android all these manufacturers who customise the OS in different ways, you have multiple different versions that are in use, and you have multiple different form factors and device capabilities. The testing permutation matrix is huge.”
“Automated UI testing is the only kind of testing that can ensure that the app does what it is supposed to do.”
Friedman says that the Xamarin UI tests are more robust than competing UI test frameworks because they do not depend on UI image recognition. “The right answer is object-based, you identify user interface elements on the screen by object IDs”.”
How does testing on real devices work? If you have 50 developers testing on 27 devices in Xamarin’s cloud, will there be racks and racks of devices to support them? “That’s what it looks like at our end, racks and racks of devices,” confirmed Friedman. “The service is going to be built based on device/hour usage. We’ll be able to scale up to match what people need.
“We talk to developers who spend $8,000 a month just to get new devices. That’s not counting the labour and everything else they need to do, to set up their own testing infrastructure. It’s a giant pain point.”
Xamarin’s Test Cloud will offer plug-ins for Jenkins, TeamCity, and Microsoft’s Team Foundation Server, to support Continuous Integration.
The scripting language for the Test Cloud is either Calabash , or C# scripting which is under development by Xamarin.
The Test Cloud is not just for applications developed using Xamarin’s C# framework, but also supports other frameworks including those written in native iOS Objective C. However, only iOS and Android are supported.
Availability is set for the third quarter of 2013.
Xamarin’s Evolve conference is currently under way in Austin, Texas, with around 600 developers in attendance. Friedman says the company is growing fast. 1000 developers a day download the tools, there are over 15,000 paid developers, and the company now has 65 employees.
More information on the Xamarin Test Cloud is here.