Tag Archives: open source

google, internet, open source, software, software development

How to brew better software: The Monki Gras in London

I attended The Monki Gras in London yesterday, a distinctive developer event arranged by the analyst firm RedMonk.

This was not only a developer event, with the likes of Andre Charland and Dave Johnson from the PhoneGap team at Adobe, Mike Milinkovich the executive director of the Eclipse Foundation, and Jason Hoffman with Bryan Cantrill from cloud services (and Node.js sponsors) Joyent. It was also a serious beer event, complete with a range of craft beers, a beer tasting competition with nine brews to try, and a talk plus a free book from  beer expert Melissa Cole. An unusual blend of flavours.

image

In charge of the proceedings was RedMonk co-founder and all round impressario James Governor. I am a big fan of RedMonk and its developer-focused approach; it has been a fresh and heady brew in the dry world of IT analysts.

image

The Monki Gras did seem like an attempt by a regular IT conference sufferer to fix problems often encountered. The Wi-Fi worked, the food was fresh, unusual and delicious, the coffee was superb; though brewing good coffee takes time so the queues were long. Not everything scales. Fortunately this was a small event, and a rare treat for the couple of hundred or so who attended.

That said, there were frustrations. The sessions were short, which in general is a good thing, but left me wanting more depth and more details in some cases; we did not learn much about PhoneGap other than a brief overview, for example.

Nevertheless there was serious content. Redmonk’s Stephen O’Grady made the point succinctly: IT decision makers are ignorant about what developers actually use and what they want to use, which is one reason why there is so much dysfunction in this industry. Part of the answer is to pay more attention, and several sessions covered different aspects of analytics: Matt LeMay from bitly on what users click on the Web; Matt Biddulph (ex BBC, Dopplr, Nokia) gave a mind-stretching talk on social network analysis which, contrary to what some think, was not invented by Facebook but predates the Internet; and O’Grady shared some insights from developer analytics at RedMonk.

I had not noticed before that github now gets nearly double the number of commits than does Google Code. That is partly because developers like git, but may also say something about Google’s loss of kudos in the open source developer community.

Kohsuke Kawaguchi, lead for Jenkins Continuous Integration and an architect at CloudBees, spoke on building a developer community. His context was how Jenkins attracted developers, but his main point has almost limitless application:  “Make everything easy, relentlessly.”

Something I see frequently is how big companies (the bigger the worse) place obstacles in front of developers or users who have an interest in their products or services. Examples are enforced registration, multiple clicks through several complex pages to get to the download you want, complex installs, and confusing information. It all adds friction. If the target is sufficiently compelling, like apps on Apple’s app store, developers will get there anyway; but it all adds friction, and if you are not Apple that can be fatal.

The Joyent guys did not speak about Node.js, sadly, but rather on the distinction between a VP of engineering and a Chief Technology Officer. Sounds dry and abstruse? I thought so too, but the delivery was so energetic that they were soon forgiven. Hoffman and Cantrill moved on to talk about management antipatterns in the software industry, prompting many wry nods of recognition from the audience. “It is very hard for middle management to add value,” said Cantrill.

Milinkovich made the point that the most valued open source projects generally make their way to a software foundation; PhoneGap to Apache is a recent example. He then gave the talk he really wanted to give, noting that as new software stacks emerge they have a tendency to re-implement CORBA, a middleware specification from the Nineties that tackled problems including remote objects, language independence, and transactions across the Internet. CORBA is remembered for drowning in complexity, but Milinkovich’s point is that the creators of exciting new stacks like Node.js should at least research and learn from past experience.

Milinkovich also found time to proclaim that “Flash is dead, Silverlight is dead, browser plugins are dead.” Perhaps premature; but I did not hear many dissenting voices.

I tweeted the conference extensively yesterday (losing at least one follower but gaining several more). Look out also for a couple of follow-up posts on topics of particular importance.

development, linux, open source, php, software development

Top languages on Github: JavaScript reigns, Ruby and Python next

I cloned a github repository today, and while browsing the site noticed the language stats:

image

Git was originally developed for the Linux kernel and is mainly for the open source community. I was interested to see JavaScript, the language of HTML 5, riding so high. PHP, C and C++ are lower than I would have guessed, Ruby and Python higher.

Here are some figures for the venerable Sourceforge:

Java (7,163) 19%
C++ (6,449) 17%
C (4,752) 13%
PHP (3,521) 10%
Python (2,694) 7%
C# (2,481) 7%
JavaScript (2,011) 5%
Perl (1,138) 3%
Shell (757) 2%
Visual Basic NET (688) 2%
Delphi/Kylix (581) 2%

This comes with a health warning. I have taken the figures from the what you get if you browse the directory and drop down Programming Languages; but the total is only about 37,000, whereas Sourceforge hosts around 324,000 projects. I am not sure what accounts for the discrepancy; it could be that language is not specified for the other projects, or they are dormant, or some other reason. But I hope the proportions indicate something of value.

Github is madly trendy, and Sourceforge ancient, so this tells us something about how open source activity has shifted towards JavaScript, Ruby and Python, and away from Java, C/C++ and C#.

Of course the overall picture of programming language usage is quite different. For example, you can get some kind of clue about commercial activity from a job board like indeed.com, which currently has 77,457 US vacancies for Java, 22,413 for JavaScript, and only 5030 for Ruby.

Nevertheless, interesting to see what languages developers on Github are choosing to work with, and perhaps an indicator of what may be most in demand on the job boards a few years from now.

Finally, looking at these figures I cannot help thinking how short-sighted Microsoft was in abandoning IronPython and IronRuby back in 2010.

adobe, professional, software development

Adobe: no new features for open source BlazeDS data services

Adobe’s Damon Cooper, who runs the BlazeDS and Data Services team at Adobe, has posted about BlazeDS vs the paid-for Data Services.

It is a curious post, in that he simultaneously highlights new features coming in Data Services 4.6 while also giving a number of reasons not to use BlazeDS.

BlazeDS is the free and open source version of Data Services, for publish/subscribe messaging and remote object invocation of Java objects in a Flash or AIR application.

He points out that the LGPL licence may be problematic; he emphasises that BlazeDS is unsupported; he says that while it is open source there are no non-Adobe committers; and as the knock-out punch adds:

Additionally, while we will absolutely be making sure we keep BlazeDS fresh and the bug fixes flowing, we don’t currently have any major new features planned for BlazeDS. That could change, but we’re currently full-out on delivering innovation to our customers have asked for in Data Services and we are full steam ahead there. 

It does sound like a retreat to me; and while I do not think Adobe is under any moral obligation to continue developing BlazeDS it does make me wonder what has changed between the moment in 2007 when Adobe decided it was a good idea to open source part of its LiveCycle Data Services, and today.

At Adobe MAX last week Adobe announced the acquisition of Nitobi and with it the open source PhoneGap project. PhoneGap is heading to the Apache Foundation – probably a good thing considering that Adobe sometimes seems to struggle when it comes to managing open source software.

delphi, embarcadero, professional, software development

Delphi and RAD Studio XE2 gets its first update as Embarcadero confesses copyright issue

Embarcadero has posted its first update for Delphi XE2 and C++Builder XE2. Whether this shows commendable responsiveness, or that that the original release was buggy and premature, is a matter for debate.

Either way, the list of fixed bugs is extensive. There is also a copyright issue, since Embarcadero says – note use of that mightily abused word “may”:

We were recently made aware that some code in the 3D support in FireMonkey may be similar to code in GLScene, an MPL open source project. We worked with Eric Grange, a key contributor to the GLScene project to remedy the issue and replace the code in question.

Unfortunately applying the patch means a full uninstall and reinstall, though Embarcardero says that future patches will be based on this new build which I presume means this surgery will not be required again.

.net, apple, linux, microsoft, mobile, windows

Miguel de Icaza talks about Windows 8 and the failure of Linux on the desktop

29 Comments

At Microsoft BUILD earlier this month I arrived early to hear Anders Hejlsberg talk about the future of C#, and found myself next to Miguel de Icaza, co-creator of the GNOME desktop and of Mono, the open source implementation of Microsoft .NET. I took the opportunity to ask a few questions, which I have his permission to post.

I recall that when .NET was first announced in 2000, it was not long before de Icaza announced Mono. I was interested therefore to know his reaction to Windows 8 and the new Window Runtime which powers “Metro-style” apps. Will we get an open source implementation of Metro-style on Linux?

I don’t think so. To be honest, with Linux on the desktop, the benefits of open source have really played against Linux on the desktop in that we keep breaking things. It is not only incompatibilities between Red Hat, Unbuntu, Suse, but even between the same distribution.  Ubuntu from this week is incompatible with the one nine months ago. And then there are multiple editions, the KDE version, the Gnome edition, the one that is the new launching system.

When you count how many great desktop apps there are on Linux, you can probably name 10. You work really hard, you can probably name 20. We’ve managed to piss off developers every step of the way, breaking APIs all the time.

I’m heartbroken, that’s the bottom line.

What about compiling your Metro app for iOS or Android?

I think that Linux has a tough time on the desktop. And the desktop is starting to not matter any more. On the other hand, building WinRT is going to be a significant amount of work. A large chunk probably could be reused from Moonlight. But it is a lot of work, to be able to reuse existing Windows apps, and in the case of iOS they already have their own stack, and Mac has its own, Cocoa is really nice and we have .NET bindings for it.

So I think we’ll learn interesting lessons from Metro. There is stuff that will be useful on other platforms like the JSON reader. But I’m not going to spend any time on WinRT for other systems.

And we can speculate about how well Metro will work in the market …

They are Microsoft, it’s going to succeed. In three years they are going to have this thing on half a billion computers, so it will be out there.

It seems like they are going to use their muscle for two things. It’s going to be a tempting space [for developers], but if you want to go into the right distribution channel for that half a billion computers, you need to abide by the Metro guidelines. They are not going to give you full API access, they are going to give you the sandboxed version. Which is good, because it can finally fix the security problems on Windows. They are going to use their muscle to reset the rules for Windows.

Especially on ARM

Right, and it is needed, they definitely need to fix this mess, a lot of malware, spyware, and the fact that everybody is sysadmin, and has to reinstall their machine every so often.

I’ve heard the word “safe” a number of times.

Right, and think of an iPad, you don’t need to be a sysadmin.

Now, you could argue that by WPF not being available to everybody and being bound to .NET they limited the effect WPF would have had, whereas Metro gives this to C++ developers, but they’re saying, hey, you can’t call Win32, there is all the Win32 stuff you can’t call. You have to use Metro. So they might be repeating that [mistake], but maybe it’s eclipsed by the fact that there’s going to be a rush to the app store. It seems like there is a big enough carrot now.

How are you getting on with the Windows 8 tablet?

I have to say, I actually like Windows 8. I am not a Windows user. It’s probably the first time that I would use a Windows machine.

Miguel de Icaza is now at Xamarin, providing cross-platform tools for using C# and .NET to build apps for Apple iOS and Google Android.

apple, development, google, mobile, nokia, open source, professional, software development, symbian, Web

Android only 23% open says report; Linux, Eclipse win praise

Vision Mobile has published a report on what it calls the Open Governance Index. The theory is that if you want to measure the extent to which an open source project is really open, you should look at its governance, rather than focusing on the license under which code is released:

The governance model used by an open source project encapsulates all the hard questions about a project. Who decides on the project roadmap? How transparent are the decision-making processes? Can anyone follow the discussions and meetings taking place in the community? Can anyone create derivatives based on the project? What compliance requirements are there for creating derivative handsets or applications, and how are these requirements enforced? Governance determines who has influence and control over the project or platform – beyond what is legally required in the open source license.

The 45-page report is free to download, and part-funded by the European Union Seventh Framework Program. It is a good read, covering 8 open source projects, including the now-abandoned Symbian Foundation. Here is the result:

Open Governance Index (%open)
Eclipse 84%
Linux 71%
WebKit 68%
Mozilla 65%
MeeGo 61%
Symbian 58%
Qt 58%
Android 23%

The percentages are derived by analysing four aspects of each project.

  • Access covers availability of source code and transparency of decisions.
  • Development refers to the transparency of contributions and acceptance processes.
  • Derivatives covers constraints on use of the project, such as trademarks and distribution channels.
  • Community structure looks at project membership and its hierarchy.

What is wrong with Android? I am not sure how the researchers get to 23%, but it scores badly in all four categories. The report observes that the code to the latest “Honeycomb” version of Android has not been published. It also has this to say about the Open Handset Alliance:

When launched, the Open Handset Alliance served the purpose of a public industry endorsement for
Android. Today, however, the OHA serves little purpose besides a stamp of approval for OHA
members; there is no formal legal entity, no communication processes for members nor frequent
member meetings.

By contrast, Eclipse and Linux are shining lights. MeeGo and Mozilla are also praised, thought the report does mention Mozilla’s “Benevolent dictators”:

In the case of conflicts and disputes, these are judged by one of two Mozilla “benevolent dictators” – Brendan Eich for technical disputes and Mitchell Baker for non-technical disputes.

Qt comes out OK but has a lower score because of Nokia’s control over decision making, though it sounds like this was written before Nokia’s Windows Mobile revolution.

WebKit scores well though the report notes that most developers work for Apple or Google and that there is:

Little transparency regarding how decisions are made, and no public information provided on this

Bearing that in mind, it seems odd to me that WebKit comes above Mozilla, but I doubt the percentages should be taken too seriously.

It is good to see a report that looks carefully at what it really means to be open, and the focus on governance makes sense.

adobe, bbc, flash, google, linux, open source, professional, software development, Web

Cross-platform concerns as Adobe abandons AIR for Linux

Adobe is giving up on AIR for Linux – at least, in a fully supported manner:

To support the variety of Linux-based platforms across PCs and devices, we are prioritizing a Linux porting kit for AIR (including source code), which Open Screen Project (OSP) partners can use to complete implementations of AIR for Linux-based platforms on PCs, mobile devices, TVs and TV-connected devices. We will no longer be releasing our own versions of Adobe AIR and the AIR SDK for desktop Linux, but expect that one or more of our partners will do so. The last Adobe release of AIR for desktop Linux is AIR 2.6.

This is a curious message. OSP partners include ARM, Intel, the BBC, Google, Toshiba and other big names; but which of these might build an AIR SDK and on what sort of terms might it be supplied? Or it is more likely that, say, the BBC will deliver BBC iPlayer for LInux in a bundle that includes the AIR runtime? Or is it just wishful thinking?

Adobe’s open source evangelist Dave McAllister has a go at defending the decision, pointing out that the growing client operating systems are Android and iOS, not desktop Linux, and that AIR for Linux accounts for only a 0.5% download share. However, Linux developers observe that Adobe’s AIR for Linux effort has always been half-hearted and tricky to install, especially on 64-bit installations. AIR itself is still 32-bit, as is the Flash Player on all systems, though there is 64-bit version in preview codenamed “Square”.

Most people run Windows or Mac desktops, and will not miss AIR for Linux. That said, decisions like this do undermine confidence in the Flash platform as a cross-platform proposition. The problem is, Flash technology is not open source and ultimately whether a particular platform is supported is a matter for Adobe, with all the commercial and political factors that implies.

The risk for Adobe is that when it abandons smaller platforms, it make open standard alternatives and in particular the collection of web technologies we call HTML5 more attractive.

.net, cloud computing, microsoft, open source, php, professional, software development, web authoring, windows

Would you consider running PHP on Azure? Microsoft faces uphill battle to convince customers.

1 Comment

Yesterday Microsoft announced Windows Azure SDK for PHP version 3.0, an update to its open source SDK for PHP on Windows Azure. The SDK wraps Azure storage, diagnostics and management services with a PHP API.

Microsoft has been working for years on making IIS a good platform for PHP. FastCGI for IIS was introduced partly, I guess, with PHP in mind; and Microsoft runs a dedicated site for PHP on IIS. The Web Platform Installer installs a number of PHP applications including WordPress, Joomla and Drupal.

It is good to see Microsoft making an effort to support this important open source platform, and I am sure it has been welcomed by Microsoft-platform organisations who want to run WordPress, say, on their existing infrastructure.

Attracting PHP developers to Azure may be harder though. I asked Nick Hines, CTO for Innovation at Thoughtworks, a global IT consultancy and developer, what he thought of the idea.

I’d struggle to see any reason. Even if you had it in your datacentre, I certainly wouldn’t advise a client, unless there was some corporate mandate to the contrary, and especially if they wanted scale, to be running a Java or a PHP application on Windows.

Microsoft’s scaling and availability story around windows hasn’t had the penetration of the datacentre that Java and Linux has. If you look at some of the heavy users of all kinds of technology that we come across , such as some of the investment banks, what they’re tending to do is to build front and middle tier applications using C# and taking advantage of things like Silverlight to get the fancy front ends that they want, but the back end services and heavy lifting and number crunching predominantly is Java or some sort of Java variant running on Linux.

Hine also said that he had not realised running PHP on Azure was something Microsoft was promoting, and voiced his suspicion that PHP would be at a disadvantage to C# and .NET when it came to calling Azure APIs.

His remarks do not surprise me, and Microsoft will have to work hard to persuade a broad range of customers that Azure is as good a platform for PHP as Linux and Apache – even leaving aside the question of whether that is the case.

The new PHP SDK is on Codeplex and developed partly by a third-party, ReadDolmen, sponsored by Microsoft. While I understand why Microsoft is using a third-party, this kind of approach troubles me in that you have to ask, what will happen to the project if Microsoft stops sponsoring it? It is not an organic open source project driven by its users, and there are examples of similar exercises that have turned out to be more to do with PR than with real commitment.

I was trying to think of important open source projects from Microsoft and the best I could come up with is ASP.NET MVC. This is also made available on CodePlex, and is clearly a critical and popular project.

However the two are not really comparable. The SDK for PHP is licensed under the New BSD License; whereas ASP.NET MVC has the restrictive Microsoft Source License for ASP.NET Pre-Release Components (even though it is now RTM – Released to manufacturing). ASP.NET MVC 1.0 was licensed under the Microsoft Public License, but I do not know if this will eventually also be the case for ASP.NET MVC 3.0.

Further, ASP.NET MVC is developed by Microsoft itself, and has its own web site as part of the official ASP.NET site. Many users may not realise that the source is published.

My reasoning, then, is that if Microsoft really want to make PHP a first-class citizen on Azure, it should hire a crack PHP team and develop its own supporting libraries; as well as coming up with some solid evidence for its merits versus, say, Linux on Amazon EC2, that might persuade someone like Nick Hine that it is worth a look.

agile, open source, software development

QCon London kicks off with call to rediscover Agile, use open source

I’m at the QCon developer conference in London – one of my favourite developer conferences of the year because of its breadth and energy.

The opening keynote was from Craig Larman who spoke on doing lean and agile development – in particular, the Scrum methodology – with large multi-site teams. He means sizeable product groups of 500-1500 persons, though he also remarked that development on this scale is really a bad idea and that a team of 10 smart folk is much better.

Still, I guess large teams are an inevitability, and Larman has written books on the subject. I am not going to summarise the talk exactly, interesting though it was, but I am going to pick out a couple of asides which interested me.

Agile methodology is really about promoting communication; and one of Larman’s themes is that if you do what seems obvious, that is to break down a project into components and give one to each small team, then you end up with numerous teams that do not communicate well with each other. Agile becomes something you do in name only.

Larman spent a bit of time on which collaboration tools to use. One of his points was not to use any commercial tool that describes itself as being for agile project management or similar. I can think of several. He says these tools are just the commercial tool vendors repackaging their old non-agile tools. Whiteboards, spreadsheets on Google docs, wikis and other simple tools are his recommendation. For source code management he suggests Subversion, Git or other open source solutions. Never use Rational Clearcase, he added, it always causes problems.

In fact, he went on to say that any commercial tools cause problems when mutli-site development extends beyond to teams in developing countries. They cannot afford the licences, he says, so avoid them.

It seems to me that the common theme here is how easily agile development intentions become non-agile in practice, especially in these large project groups.

linux, open source, php, software development, web authoring

Accelerating PHP with the Alternative PHP Cache

I decided to install the open source Alternative PHP Cache on this server in order to improve performance. Interesting exercise. This server runs Debian Linux, and there are several ways to install APC:

1. Install the official package with apt-get install php-apc or similar

2. Install with the PHP Extension Community Library which goes something like:

apt-get install apache2
apt-get install libapache2-mod-php5
apt-get install php-pear
apt-get install php5-dev
apt-get install make
apt-get install apache2-prefork-dev
pecl install apc

The advantage over (1) is that you get the latest stable build, version 3.1.6, instead of the Debian package which is 3.0.19

3. Download the source and do something like this to install.

I started with option (2) though I came to regret it. The first problem is that the pecl installer will build with your currently-installed Apache, and if you later upgrade Apache it might break. Sticking with the official package is safer, even though it is very out of date.

I could live with the idea of re-installing APC every time Apache was updated if necessary, but I had another problem. I was up and running with APC 3.1.6 and pleased with the results, until after a while everything stopped working and my blog became a screen full of messages saying “Unable to allocate memory for pool”.

It looks like this bug, which was said to be fixed in version 3.1.5, but if you look to the end of the comments there is one from today with the same issue, and no suggestions about how to fix it.

The ancient version, on the other hand, has performed perfectly so far.

Another point of interest: I found it challenging to discover the best settings for APC. By default the install does no more than to enable the extension; but the default setting is unlikely to be the best one. The documentation tells you what each setting does, but not how to choose the best values for those settings. Should the cache be the default 32MB, or something much greater? Another thing to note: if you compile with MMAP support, which is the default, the value of apc.shm_segments is ignored, and the value in apc.shm_size will solely determine the size of the cache.

I found this Moodle article on installing APC in Windows helpful. What you do is first to find the file apc.php which the install put somewhere like /usr/share/doc/php-apc – in my case it was also compressed -  and put this on your website, preferably in a password-protected folder. This tells you the status of the cache. The aim is to have the cache just big enough that it does not become full and highly fragmented. Here is what I get after a short run with 128MB, which may be a little too much:

image

Another tip is to set apc.stat to 0. This means APC will not check for changes in PHP files since they were last compiled and cached. The downside is that every time you change a file you have to restart the web server; but the benefit is better performance, which is the goal after all.