How is the performance of Surface RT? Tolerable but rather slow would be my quick summary. Surface RT has a quad-core NVIDIA Tegra 3 chipset.
In order to investigate further, I built a simple app to count primes, using essentially the code here. I ran the code both in JavaScript and in C#. This is a single ad-hoc and naive test that only covers one aspect of performance, but I have still found it an interesting indicator. I then ran the app both on my Intel Core i5 Samsung Slate and on Surface RT (yes it was interesting figuring out how to debug on Surface RT in a hotel room).
Here are the results:
On Intel Core i5:
- Count primes up to 1,000,000: 1.05 secs (in browser)
- Count primes up to 1,000,000: 0.88 secs (in embedded webview)
- Count primes up to 1,000,000: 0.32 secs (C#)
- Count primes up to 10,000,000: 6.31 secs (C#)
On Surface RT:
- Count primes up to 1,000,000: 3.49 secs (in browser) Slower by 332%
- Count primes up to 1,000,000: 3.53 secs (in embedded webview) Slower by 401%
- Count primes up to 1,000,000: 1.81 secs (C#) Slower by 565%
- Count primes up to 10,000,000: 49.03 secs (C#) Slower by 777%
The Core i5 is running at 1.6 Ghz. Surface RT has an NVidia Tegra 3 1.3 Ghz chipset.
You could count primes more quickly in both cases by using parallel processing; this is a single-threaded test.
What is notable here?
- First, Surface RT is slower than I would expect and I hope Microsoft improves performance with future updates.
- Second, C# is substantially faster than JavaScript in this test, around twice as fast, which makes me question the advice I have heard from some spokespersons that HTML and JavaScript is the preferred approach for apps.
- Third, the factor by which C# is slower is greater than the factor by which JavaScript is slower. This surprised me; I had expected the reverse for some reason.
Of course, other performance tests will yield different results. When I tried the Sunspider JavaScript test Surface RT was about 5 times slower.
One piece of good news: there was little difference between performance in the embedded or non-embedded browser.
Update: I also tried this test in C++. There was barely any difference. The Core i5 counted primes up to 10,000,000 in 6 seconds. The Surface RT performed the same feat in 46 seconds.
Update 2: A Google Nexus 7, which also has a Tegra 3 chipset, can only manage 76 seconds for the JavaScript test (primes up to 10,000,000).
Particularily odd that a higher number of primes cause further slowdown. Memory cache contention?
(for C#, that is)
Great post!
The only reason why HTML is preceived to be the prefered language of choice is that there are lots of HTML script kiddies out there that don’t know how to program in real programming languages so it was perceived that most apps would be ported from HTML 5.
I think that with monotouch the best approach is .NET native and then GUIs for each platform. Even if you use HTML 5 you’re going to have to have native GUIs if you want in the app stores. And productivity in C# is WAY better than in HTML + javascript. It isn’t even close even for expert HTML 5 programmers that have even marginal C# abilities.
Here’s hoping that Monotouch gets native Windows 8 support with everything and eventually gets the ability to design all GUIs in VS.net 2012 so that you can maintain one shared project and send it off to compile. This would be the absolute KILLER feature that would result in unified development in a real programming language instead of javascript.
I’m surprised that C# is only faster by a factor of two. On my i7 920 system with Windows 7, the factor is 4.8 for n = 10 million: IE9 takes 18.5 s whereas C# 4.0 only takes 3.85 s, using the 32-bit runtime in each case. I guess IE10’s JavaScript engine is improved over IE9, whereas the ARM CLR’s optimizer is still worse than the Intel CLR’s.
@Lars – Not sure what you mean, counting primes with that algorithm is not a linear process.
@Tim – I am not surprised the ARM platform is slower than Intel, it is a feature of ARM. Similar Ghz will not yield similar speeds.
A more interesting comparison would be to run it on a Nexus 7.
Tested on core i5 here:
Chrome 22: 0.368 secs
IE9: 0.651 secs
Which matches my experiences that Chrome V8 is roughly comparable to C# in terms of performance.
@Tim: your C# figures look surprising, the ratio especially, wasn’t there some CPU throttling at play for the longer runs?
F*ck Javascript!
It would be interesting to run this test on an Android tablet of similar power such as the Nexus 7 using JavaScript on Chrome. It has the same CPU but I don’t know how the clocks compare. If I was setup for programming on one I would run it and report back. Anyone willing to try?
I tried the JavaScript test, and Sunspider, in Chrome on a Nexus 7; it was significantly slower than the IE10 on the Surface.
The real question is: does it matter? In many cases the real compute grunt happens back on the cloud so slow HTML5+Javascript really doesn’t matter except for display of the data. Even on the cloud, ARM may make more sense as a server if multiple CPUs can be used at a lower total watt usage than a single Xeon to gain the same performance.
For running office apps, provided the spreadsheet calculations happen on the cloud, it really doesn’t matter if the Javascript display is a little slow. Running a Photoshop app with local photo storage and filter operations is another matter, but I doubt that this will be how it’s done. It all depends on how the app is put together. If the photos are on the cloud and are processed there, then speed on the result display tablet is not always needed, and would normally be optimized in the browser engine as necessary to display the downloaded proxy image.
Thanks Tim, that’s nice to know that MS appears to have a better product. I do large C++ machine vision projects so won’t be running on a tablet any day soon, but I do like to explore new technologies for information display and other purposes. Looks like the only way I would use a Surface would be for display and control, but it might be powerful enough to do 3D display and video with some things I do.