.net, cloud computing, Notes from the field, tech

Odd behaviour from Azure App Service: production version sometimes serves app from staging slot

I am developing an application which is deployed to Azure App Service. It runs on .NET 5.0 on Linux. I have set up a simple DevOps process so that committing changes to GitHub runs an Azure DevOps pipeline that deploys the application to a staging slot on Azure App Service for Linux. Then I can use Swap in the Azure portal to update the production slot. Swap simply exchanges the content of the staging slot with that in the production slot, so there is a route back in the event of disaster. Swap also restarts the application and forces users to log back in.

Yesterday I fixed a bug, deployed the change to the staging slot, and performed a swap. Logged back into the application, but the bug was still there, though intermittent. That was the bit I could not figure out: what was causing the code to behave differently on different requests? I became suspicious that it was sometimes serving the old version. I proved this by refreshing a page that demonstrated the bug. My page has an application version in the footer, and I could see that when the bug appeared, the version was older.

image

image

Well this is odd. In the App Service Deployment slot settings I have traffic set to 100% for the production slot:

image

In general I tend to assume a bug in my code or an error in my configuration settings is more likely than an issue with the Azure App Service. This does look odd though: why, if traffic is going 100% to the production slot, does the application sometimes serve the old version?

The pragmatic fix was easy. A second deployment to the staging slot means both now have code that works. The bug no longer appears; but I have kept the version number different and can see that the issue is actually still occurring.

I will update this post when I have more information, just in case anyone else hits this issue.

audio, gadgets, hifi, tech

Some of my favourite earbuds and headphones of 2020: part 1

I love technology that endures and one of my most treasured gadgets is a pair of Sennheiser HD600 headphones. Mine are not quite that old, but this is a model that was first released in 1997 and remains on sale; they sound fantastic especially with a high quality headphone amplifier, and I use them as a kind of reference for other headphones. The ear cushions and headband perished on mine and I bought the official spares at an unreasonable price because I so much wanted to keep them going.

image

Still on sale today: Sennheiser HD600

They are as good as ever; but it happens that this year I have reviewed a large number of headsets and earbuds some of which I also like very much. First though, a few observations.

Passive headphones like my HD600s are in decline. I call them passive because they have no built-in electronics other than the speaker drivers. They can only work when wired to the output from an amplifier. Wires can be a nuisance; and as wireless options become cheaper, better and more abundant, wired is in decline (though it will never go away completely).

Once you go wireless, something else happens. The signal the headset receives over Bluetooth or wi-fi is a digital one. Packed into the receiving electronics is not just a pair of speaker drivers,  but also a DAC (Digital to Audio Converter) and an amplifier. Further, the amplifier can be optimized for the drivers. The output can be equalized to compensate for any deficiencies in the drivers or the acoustics formed by their case and fit. This is an active configuration and has obvious advantages, getting the best possible performance from the driver and also enabling smart features like active noise cancellation (if you add microphones into the mix). It should not be surprising that the better wireless devices sound very good. I have also noticed that headsets which offer both wired and wireless modes often sound better wireless, despite the theoretical advantages of a wired connection.

Another significant development is that off-the-shelf chipsets for wireless audio have got better. The leader in this is Qualcomm whose Bluetooth audio chipsets are packed with advanced features. SoCs like the QCC 3506 include adaptive active noise cancellation, 24-bit 96Khz audio, low power consumption, voice assistant support, fast pairing, built in DSP (Digital Signal Processing), and of course programmability for custom features.

It is because of this that there are now numerous budget true wireless earbuds and headsets from brands you might not have heard of, with these exact features.

The importance of the fit

A few years ago I attended a press event at CES in Las Vegas. Shure was exhibiting there and I got to try a pair of its premium IEMs (In-ear monitors). Until then I had been convinced that earbuds could never be as good as headphones; but what I heard that day changed my mind. The audio amazed me, sounding full, rich, spacious, detailed and realistic. I thought for a moment about it and realised I should not be surprised. IEMs are designed to be fitted directly to your ears; why should they not be of the highest quality? The best ones, like those at CES, have multiple drivers.

There is something else though. Getting the best sound from IEMs requires getting an excellent fit since they are generally designed to sealed into your ears; the once with expandable foam ear seals are perhaps the best for achieving this. If they are badly fitted then you will hear sound that is tinny or bass-light.

This means that it is worth taking extreme care with the selection of the right ear seals or ear sleeves, as they are sometimes called. Most earbuds come with a few sizes to accommodate different ear sizes. If the sound changes dramatically when you press the earbuds in slightly, they are not fitted right.

Something else regarding Shure, that I did not realise until recently, is that all its earbuds are designed to have a cable or (in the case of the wireless range) clip that fits behind your ears. The reason for this is that it makes it easier to get a good fit when the cable is not constantly pulling the IEMs out of position. Check out this video for details.

Earbuds that do not fall out

It is a common problem. You fit your earbuds and then go for a run or to the gym. With all that movement, one or both of the earbuds falls out. This can be a serious problem with wireless models. If you lose your earbud in the long grass or on a busy street; you might never find it again, or someone might trample on it.

There are a few solutions. The Shure approach makes it unlikely that the device will fall out. Another idea is to have a neckband that connects the two earbuds and hangs at the back of your head. I quite like this approach, since manufacturers can sneak some batteries into the neckband that give a longer play time than is typical with true wireless. It does not stop an earbud falling out, but it does mean you are less likely to lose it. Some people I’ve chatted to though feel it is the worst of both worlds, the battery and pairing issues of Bluetooth with the inconvenience or ugliness of wires. Up to you.

image
Airloop Snap 3-in-one

One brand, Airloop, claims to solve this by offering 3-in-one earbuds that can be used true wireless, or with a neckband, or with a lightweight “sports band” that has no batteries but does give a bit of security. The idea was good enough to raise hundreds of thousands in crowdfunding on Indiegogo and Kickstarter; they are nice devices but did not quite make my favourites list as I found the devices a little bulky for comfort, the firmware a little buggy, and the sound not quite what it should be at the price.

Don’t fret about the codec

One last remark before we get to my list of favourites. You will find a few variations when it comes to wireless standards and codecs. Headsets used for gaming generally use USB dongles for low latency. Other headsets generally use Bluetooth and support various codecs. The minimum today is SBC (low-complexity subband codec) which is part of A2DP (Advanced Audio Distribution Profile). Apple devices support AAC. Qualcomm’s aptX has advantages over SBC in compression, higher bitrate and lower latency. Sony has LDAC which supports higher resolutions.

In my experience though, the codec support is relatively unimportant to the sound quality. Yes, the better codecs like aptX or LDAC are superior to SBC; but compared to other factors like the number and quality of the drivers, the ease of getting a good fit (see above), and the quality of the electronics in other respects, the codec is less important. “As you may have noticed, it’s difficult to tell the difference between SBC and aptX by ear”, observes this article.

You can bypass these concerns by going wired; but when you consider the benefits of active amplification as well as the convenience of wireless, it is not a simple decision.

See part two (coming soon) for some of the headsets I have enjoyed in 2020.

.net, software development, tech, visual studio

A UI lesson: do not ask users to choose between Register and Login

1 Comment

I am developing a web site for playing bridge, a project which kicked off in March when lockdown caused bridge clubs everywhere to close. There are lots of sites where you can play bridge online, but not many options (particularly back in March) for clubs that wanted to run their own online sessions.

It’s going OK with a number of clubs now using it every week, though it is still in development. I have learned a painful lesson though. In order to proceed as quickly as possible, I started my project with the Visual Studio template for an ASP.Net Core application with ASP.NET Core Identity – the latter an easy decision since it handles all the complications of registration, password reset and so on. (I did end up having to re-plumb it to use int rather than GUID for the primary key but that is another story).

The default home page the template generates looks like this, with options in the menu to Register or Login.

image

Registration and login are fundamental concepts that have been part of the web forever. It’s simple for a developer to understand: you register to create an account, you login if you already have an account.

The painful discovery is that this is not obvious to everyone, particularly to an older demographic that did not grow up with computers. Another factor is that cookies, browser-saved passwords and single sign-on with Google/Facebook etc means that this whole area is a bit of a mess and there are people who just kinda expect web sites to know who they are (which in one way horrifies me but I do see the massive convenience).

The consequence is that a surprising (to me) number of people had difficulty knowing whether Registration or Login was what they needed. They would Register, then return to the site and hit Register again. Why is this site asking for my details again? Maybe a security thing? Oh no, why does it now say username not available?

This is because asking the user to make this choice is not good design. Registration is rare, login is common. Further, Register is a confusing word. We sometimes use the word register when accounts already exist. Create Account is better. And a better UI is just Login. I need to access this website. Then, underneath the username/password request, an option that says “I need to create an account”. The two options should not be equally prominent; and if you look at how many prominent sites design this, that is what they do:

image image

Lesson learned; but I wish this had occurred to me sooner!

book reviews, books, privacy, tech

Cyber Privacy by April Falcon Doss

1 Comment

This is a book about pervasive data collection and its implications. The author, April Falcon Doss, is a lawyer who spent 13 years at the US National Security Agency (NSA), itself an organization controversial for phone-tapping and other covert surveillance practices. Disturbing though that is, one of Doss’s observations is that “in democratic countries … the government doesn’t have nearly as much data as private companies do.” She argues that government-held data is less troubling since its usage is well regulated, unlike privately held data – though these safeguards do not apply in authoritarian regimes.

Government use then is just one piece of something much bigger, the colossal amount of personal data gathered on so much of what we do, our buying habits, what we search for on the internet, our health, our location, our contacts, tastes and preferences, all tracked, stored, and used in ways that we might not expect. Most of the book simply describes what is happening, and this will be eye-opening to anyone who has not followed the growth of data collection and its use in marketing and advertising over the last twenty years or so. Doss describes how a researcher analyzed his iPhone activity and found that “within seven days, the phone had exported data via 5,400 hidden app trackers.” – and Google’s Android is even worse.

How much do we care and how much should we care? Doss looks at this question which to me is of particular interest. We like getting stuff for free, like social media, search, maps and directions; but how aware are we of hidden costs like compromised privacy and would we be willing to pay in other ways? Studies on the subject are contradictory; humans are not very logical on the matter, and it depends exactly how the trade-off between privacy and cost is presented. The tech giants know this and in general we easily succumb to the temptation to hand over personal information when signing up for free services.

Doss makes some excellent and succinct points, as when she writes that “privacy policies offer little more than a fig leaf of user notice and consent since they are cumbersome to read, difficult to understand, and individuals have few alternatives when it comes to using the major digital platforms.” She also takes aim at well-intended but ineffective cookie legislation – which have given rise to the banners you see, especially in the EU, inviting you to accept all manner of cookies when you visit a web site for the first time. “A great deal of energy and attention has gone into drafting and implementing cookie notice laws,” she says. “But it is an open question whether anyone’s privacy has actually increased.”

She also observes that we are in uncharted territory. “It turns out that all of us have been unwitting participants in a multifaceted, loosely designed program of unregulated research,” she writes.

Personally I agree that the issue is super-important and deserves more attention than it gets, so I am grateful for the book. There are a couple of issues though. One is that the reason personal data gathering has escalated so fast is that we’ve seen benefits – like free services and personalisation of advertising which reduces the amount of irrelevant material we see – but the harms are more hidden. What are the harms? Doss does identify some harms, such as reduced freedom in authoritarian regimes, or higher prices for things like Uber transport when algorithms decide what offers to show based on our willingness to pay. I would like to have seen more attention paid though to the most obvious harm of the moment, the fact that abuse of personal data and social media may have resulted in political upheavals like the election of Donald Trump as US president, or the result of the Brexit referendum in the UK. Whatever your political views, those who value democracy should be concerned; Doss gives this matter some attention but not as much as it merits, in my opinion.

Second, the big question is what can be done; and here the book is short of answers. Doss ends up arguing that we have passed the point of no return in terms of data collection. “The real challenge lies in creating sufficient restrictions to rein in the human tendency to misuse information for purposes that we’ve collectively decided are unacceptable in society,” she writes, acknowledging that how we do so remains an open question.

She says that her ambitions for the book become more modest as the research continued, ending with the hope that she has provided “a catalogue of risks and relevant questions, along with a useful framework for thinking about the future” which “may spark further, future discussions.”

Fair enough, but I would like to have seen more practical suggestions. Should we regulate more? Should Google or Facebook be broken up? As individuals, does it help if we close social media accounts and become more wary about the data that we give away?

Nevertheless I welcome this thought-provoking book and hope that it does help to stimulate the future debate for which the author hopes.

BenBella Books (3 Nov. 2020)

development, Notes from the field, tech, web authoring

Debugging Safari on an old iPad

Someone was trying to use the bridge application I have in progress, using an iPad 2.0. There were a couple of interesting things about this. One was that I had to rethink the warning thrown up, base on Modernizr, which detects incompatible web browsers. The problem (obvious when you think about it) is that if you use some potentially incompatible features in the same page where you are testing for them, then with an old web browser the JavaScript fails with a syntax error and the warning does not appear. The fix: I now show the warning by default, and the compatibility check hides it.

Still, I was interested in the Safari error and wanted to debug it, in case it was something I could fix. How do you debug Safari on an iPad?  The way it is meant to work is this:

– On a Mac, enable the Safari Develop menu (in Safari preferences, Advanced, Show Develop menu).

– On iOS, enable Safari Web Inspector (Settings – Safari – Advanced – Web Inspector).

– Connect the iPad to the Mac via USB. You can now use Web Inspector on the Mac to debug the Safari iOS pages and scripts.

This did not work for me on my Catalina Mac. The iOS Safari did not show up in the Web Inspector on Safari Mac. I could get it to show briefly, by switching Web Inspector on the iPad off and on again, but after than, no go. I tried a few things, but none of the proposed solutions I could find for this issue fixed it for me.

I have an older 2011 Mac Mini in a drawer, so I thought that might work, being a similar age to the iPad. I fired it up, marvelled at how old-fashioned the UI looked (I had reset it to OS X Lion), and connected the iPad. No go. Same problem as with Catalina.

Surprisingly, what did work were the instructions here (more or less) for debugging Safari iOS on Windows. This is based on the RemoteDebug iOS WebKit Adapter described here, a project which originated as an internal Microsoft experiment.

image

I did find it amusing that I could do this on Windows, having failed with the Mac.

The next generation of this is Inspect. This is in private beta, though the GitHub page for RemoteDebug says it has been superseded and to use Inspect instead.

It worked for me though.

cloud computing, software development, tech

Point-in-time restore: a handy built-in feature in Azure SQL

I am working on a project that is hosted in Azure and I made a mistake, running a SQL script that was dependent on another SQL script that I had forgotten to run. It messed up the foreign keys and I would have to restore a backup … but my most recent backup was from the day before. Annoying.

But wait. Looking the Azure portal I saw this:

image

This is a plain Azure SQL instance with no extras, but look, you can restore the database from 6 minutes ago.

I did it; it restored to a second database. I deleted the bad one, renamed the restored one, ran my scripts in the right order, and all was well.

I recommend you do not run scripts in the wrong order … but if you do, or make some other error, this is a handy feature of Azure SQL which I was not aware of before.

development, tech

Wrestling with Azure DevOps Pipelines

Pipelines is an Azure service that enables a powerful feature: the ability to set up continuous integration. I have tangled with it before, in the context of trying Azure Kubernetes Service, but managed to avoid getting deep into the YAML which is the language of Pipelines. I am working on a web application and  trying to get it up to scratch as quickly as possible, especially as there are now a bunch of users who are being patient over glitches during development but whose patience may run out.

The application uses .NET Core which for the most part is working well for me. I am using Visual Studio 2019, with occasional forays into Visual Studio Code (VS Code), and deploying to a Linux Azure App Service. Everything was fine until one day when the Web Deploy feature in Visual Studio stopped working with “could not complete the request to remote agent … the operation has timed out.” I appealed for help but with no result yet.

All was not lost as I found that the VS Code Deploy to Azure extension worked pretty well. All I needed to do was to open the solution folder in VS Code, run:

dotnet publish -c Release -o ./publish

in the terminal, then right-click the publish folder and  then right-click the publish folder and choose Deploy to web app. There are a few annoyances but it solved the immediate problem.

One can do better though. Rather than manually deploying, you can create a pipeline using Azure DevOps (the thing that was once called Visual Studio Online and is the cloud version of Team Foundation Services). An attraction of using Azure Devops is that you get “1 Microsoft-hosted job with 1,800 minutes per month for CI/CD” free which seems decent.

I got started, creating an Azure DevOps project and adding a pipeline. You authorize it to access your GitHub repository (if that is what you have, as I do) and then end up in an editor that looks like this:

image

I soon got frustrated. The Pipelines service seems fundamentally excellent but spoilt by poor documentation and some odd behaviour – at least for .NET Core. It took me hours to achieve a basic setup that would upload, test and deploy my simple web application. Most of the time was spent observing pipelines fail to run and trying to figure out why.

When you run a pipeline you may notice that it uses .NET Core 2.1 and warns that 2.2 and 3.0 are end of life:

How do you get it to use .NET Core 3.1? You can add a snippet called UseDotNet@2 and specify the version. I put 3.1 and it was rejected as it likes a full version. I put 3.1.301 and it worked .

The most time-consuming thing for me was running tests. The application uses an Azure SQL database. It is unwise to put the database password in appsettings.json in the GitHub respository. How then do you connect to the database? The docs anticipate this and you can use a feature called variable substitution. In the Pipeline editor, you can add variables and mark them secret, so they are not included in logs. You can also use variables from Azure Key Vault. Then you can use the FileTransform@2 task to replace the connection string in appsettings.json with the one you need including the password. I do not think this is ideal from a security perspective – you are still putting the password in plain text in a configuration file – but it beats having it in the GitHub repository.

I had many issues. The main documentation on variable substitution is here. This is terrible. Note that if you look at the YAML example for JSON file substitution (which is what we need) it does not even use FileTransform@2. It uses AzureRmWebAppDeployment@4 which does a whole lot of other stuff as described here. Maybe I should have tried that. But FileTransform@2 looked like the right thing. Unfortunately it generally gives the error “Cannot perform XML transformations on a non-Windows platform.” No, I am not trying to do an XML transformation. Even if you specify the fileType as json and set enableXmlTransform to false, you still get the error. Later research suggests you can beat this error by setting xmlTransformationRules to an empty string. I gave up though and used FileTransform@1 (an older version of the task) which works as expected.

I still did not get the result I wanted though. All the tests using the database failed. Eventually I figured out that I had to set the folderPath to $(Build.SourcesDirectory). Then it works.

This was good. Now my tests run in Linux rather than on Windows, matching the deployed environment. In a full production environment I would use a second Azure SQL database for the tests, but for development this will do.

I then created a staging slot in the App Service and added a deployment step to deploy the application to that slot. Again, this is good. The application will not deploy unless it passes all the tests (this is a built-in feature of Pipelines, as each step does not run unless the previous steps succeeded). It deploys to staging which has a separate URL so you can try it out and not swap it to production until you are ready.

Overall, it is a better solution than the Visual Studio web deploy which it replaces, so perhaps the error did me a favour. It will work with Visual Studio as well as with VS Code, since it triggers automatically on every code commit. The Publish option in Visual Studio becomes redundant.

Note that Visual Studio also has an option to set this up automatically.

image

I tried it, letting the wizard do what it wanted including creating a new Azure DevOps project and a new App Service plan. Notable things:

– It created a pipeline using the Classic UI rather than the YAML based editor

– It uses an agent (the VM where the pipeline runs) called vs2017-win2016

– the pipeline did not get very far, failing on NuGet restore

No, I am not going to bother troubleshooting this.

This time yesterday I hated Azure DevOps pipelines. Nothing worked first time, YAML is a hostile editing environment (whitespace matters), and the documentation frustrated me. Now I feel pleased and I have this nice badge in my repository.

image

I am left with a nagging feeling though that all this is more difficult than it should be. It seems to me that what I wanted to do was commonplace: use .NET Core, use Azure App Service, have my pipeline build the project, run tests and deploy to staging. You could add, apply entity framework migrations in many cases. I did not find this documented in any one place and the result was that it took more time to figure out.

tech

Using Windows 10 on a 4K display: issues in multi-monitor setups

1 Comment

I made the mistake of reading this post where programmer Nikita Prokopov explains why it is time to upgrade your monitor, particularly if you are a software developer. “I optimize my setup to showing really, really good letters. A good monitor is essential for that. Not nice to have,” he says, going to to explain why standard 1080p (1920 x 1080 pixel) displays have insufficient resolution to display text nicely (unless the display is also small, such as on a 13” laptop). You can use the tool here to calculate the PPI (pixels per inch). You should aim for 150 or more PPI; a 27″ 1080p display will get you 81.59 PPI.

Prokopov’s point is that if you spend all day looking at text (and I do), then you should make the effort (and expense) of getting to display properly; your eyes will thank you and you can work with less strain.

One of my displays is dying (needs new capacitors I suspect) so I took the bait and stumped up for a 4K screen. I did not do what Prokopov also suggested which is to get a display with 120Hz refresh rate. I looked into it; but you need to get a TN (twisted nematic) display which involves some compromises in viewing angles and colours. I also did not want to spend £1700 or more. So I went for an 4K IPS display.

It has been educational. Prokopov is right; text looks much better. Note though that you cannot run at full resolution unless your display is huge; mine is 27” because it has to fit on the desk. It is quite fun at full resolution but the text is too small to read.

image

What you should do, says Prokopov, is to scale the display by an integer value. Therefore I scale 200% (Windows display settings). The display is now back to 1080p in terms of the size of most text but at higher resolution and text looks great.

There is a snag though, actually a couple of snags. One is that occasionally you hit an application that does not understand the scaling – like Open Live Writer – and the text is tiny. More significant for me though is what happens if you have multiple displays. Windows is smart enough to let you have different display settings for each screen. The problems come in two cases though:

– if you move the mouse from the 4K screen to the 1080p screen, it jumps vertically. Essentially, it retains the pixel coordinates from the 4K display and applies them to the 1080p display. So if the mouse is halfway down the 4K display, and you move it right onto a 1080p display, it jumps to the bottom of the screen.

– if you drag an application so it straddles the two displays, it all goes wrong. I cannot use a screen grab for this.

Exhibit one: what happens if the 4K display is set to 100% scaling and you drag an application to straddle across to a 1080p display (ignore the mottling effect, that’s just an artefact from snapping the screen):

image

Exhibit two: what happens if you have the 4K display set to 200% scaling and perform the same straddling act:

image

I appreciate the difficulties here, but possibly Windows could do better? Incidentally the weirdness fixes itself when you drag the window fully across to the 1080p display, it snaps back to normal.

The solution of course would be to get two or three 4K displays. An expensive solution though.

.net, azure, software development, tech

Developing software for playing bridge

3 Comments

I am a duplicate bridge player in my spare time and enjoyed playing in my local club once or twice a week. That was before COVID-19 and then, in March this year, lockdown. Bridge clubs were no longer able to meet. There are more important things in the world; but bridge is both a lot of fun and a welcome distraction from weightier matters, and my thoughts soon turned to what we could do to continue playing in these new circumstances.

The answer was to play online; but while there are plenty of ways to play bridge online, the existing systems were not designed with the idea of being a way for bridge clubs to meet in a new context. If anything, the reverse is true: online bridge site were designed for people who could not easily get to a club or wanted to play at any time with whoever else happened to be available. Clubs like my own, by contrast, wanted to replicate their face-to-face meetings with an online equivalent. A further complication back in March was that the biggest online bridge site, called Bridgebase, was immediately overloaded and declared that it was unwilling to allow new people to qualify as directors, people allowed to run online bridge sessions.

My immediate instinct was to build a new site for playing bridge. I was not quite starting from scratch. Back in the early days of Windows 8, I started work on a bridge game for Microsoft’s new and as it turned out ill-fated platform. I had got some way with it; I had created a bridge engine that understood about cards and hands and tricks and shuffling and scoring and all the various elements that go into playing bridge. It was written in C# and what is now UWP XAML. It is designed of course for a solo player. Here is the bidding screen:

image

and the play screen:

image

This is how it looks on Windows 10; it looked a bit better on Windows 8 though it would not win any prizes for design. My software could play bridge though; the reason I never finished it was that I never cracked getting the AI working. But for human to human play that did not matter. A weekend or two coding, I thought, and I could have a website up and running so our club could play bridge online. I made an immediate start, registering the domain name YourBridgeClubOnline.co.uk.

Well, three months later and here we are.

image

image

It is, I have to say, still under development. But it works and we have been able to play bridge again, as a club.

What took you so long? Ha! Much of my old bridge engine code remains untouched and has proved useful; it all runs fine on .NET Core. Even the (useless) AI has been handy, as I can test the mechanics of play without involving others. But I had, of course, wildly underestimated the problem of converting a game for solo play on Windows, to a multi-player web application. There is much to think about:

The UI. I am not a designer (I am sure you can tell) but spent ages puzzling over how to get a workable user interface in the browser for everything from tablets to desktops. Not smartphones yet but it is coming. I decided early on to take a view on compatibility. No Internet Explorer. JavaScript fetch API is required. When time is against you, it is easier to say, just use another browser, than to waste too much time supporting old browsers.

Messaging – both the API kind, and the chat kind. I am using C#, ASP.NET Core and SignalR. In general it works well. SignalR uses WebSockets as first preference, but falls back to Server Sent Events or long polling where necessary. In my first experiments I did my own polling and switching to SignalR was a great relief.

Registration and login. I am using the stuff that comes in the box, ASP.NET Core Identity. It has saved me a ton of work. It’s a bit annoying and not too well documented. I don’t really like using GUIDs for the primary key, for example, and I believe there is way to avoid it, but it isn’t top priority when you are going for Minimum Viable Product.

JavaScript. I’ve written tons of it and I don’t even like the language. I have a new respect for it though. The thing is, it is very fast and there is nothing you cannot do. The worst thing is the friction of doing some debugging in the browser, and some in Visual Studio. I am thinking of switching to VS Code for development since it works nicely with ASP.NET Core and is better for JavaScript than Visual Studio.

Scoring. My Windows software could score a hand of bridge. But duplicate is different; you have to compare the scores with others who played the same hands and work out the percentages, then export the results to standard formats for display on club websites and submission to the English Bridge Union. It was more work than I had expected and I am not done yet; the system only understands Pairs at the moment, not Teams (a different way of scoring).

Directing. Someone has to manage an online bridge session, settle any arguments, and fix errors like cards played by accident. It all needs coding and there was nothing like it in the Windows version.

Movements. Imagine you have 28 people playing bridge (or 14 pairs). They need to all play the same hands, but never play the same hand twice, and it has to be so arranged that each pair plays against other pairs in a defined sequence so it is balanced and fair. We call this the movement. Online, you have a bit more flexibility because you don’t need to share physical cards: everyone can play the same hand at the same time if you like. It is still quite fiddly though, and I did not do any of this in the old Windows version. I saved some time by writing an import function to enable re-use of movements made for EBUScore, a widely used scoring and bridge session management application. There is more to do though.

Claims. This is where, half way through the hand, a player says, “There’s no point in playing on, I’m obviously going to win all the remaining tricks.” A trick is a sequence of four cards played one from each hand, which is won by one of the pairs. This statement is called a claim, and has to be agreed by the other players. Getting this working was more difficult than I had expected – because built into my bridge engine was the idea that you could score by counting the tricks each side had won. But claimed tricks are never played. With hindsight, I should have allowed for this from the beginning.

Database. Every detail of play has to be stored on the server. I am using Dapper and SQL Server currently, though it is possible that PostgreSQL would work just as well. I started with Entity Framework Core, still there as it is used by ASP.NET Core Identity, but I am happier with Dapper.

Things that worked well

Three months is longer than I had thought it would take to get to a playable system, but I suppose as a spare time project it is not too bad. It would not be possible without the likes of ASP.NET Core and Dapper and SignalR doing so much for you. C# is a delight for coding. I am also using an Azure App Service for all this test and development and that has worked well. I am deploying to a Linux container of course; but the nice thing about App Service is that it will scale to a considerable extent without the hassle of Kubernetes. If the project succeeds and needs to scale up, there is an Azure SignalR service ready and waiting. I was nevertheless interested to see that AWS now offers .NET Core on Elastic Beanstalk, complete with some nice Visual Studio integration. Trying it there would be an interesting experiment, though I’m not sure AWS is so savvy about SignalR.

Open Source?

Could this have been done quicker by making it open source and seeking collaborators early on? Will it become open source? I need help for sure, though I also feel the code needs some cleaning up before it is fit to share more widely. You will recall though that I had started out thinking that it would be a small matter to convert my solo bridge game to an online multiplayer web application. I figured it would be better to get something working and then ask for help. But I am open to offers! Note: this is not a commercial project.

Rewarding

Most of the software projects I have been involved in have been business applications. Bridge is a lot more fun. I do see software development as a creative act. I recall starting work on the bridge game back in 2011 (I think); starting a new blank project in Visual Studio and thinking, hmm, I had better write a class to represent a pack of cards. From that beginning I ended up with an application that could play bridge, after a fashion, and now one that multiple people can play concurrently. It is rewarding and I will not regret the time spent on it, irrespective of how much actual use it gets.

software development, tech

StackOverflow survey 2020

The StackOverflow developer survey 2020 is out – surveys come out constantly but this one is worth more than most because of the huge reach of StackOverflow among developers. This one has 65,000 responses.  Every survey also has reasons why it is unrepresentative; there is no such thing as a definitive survey because you can never precisely compare like with like, humans are unreliable, and every sample has its biases. Every wondered what a survey of “developers who never respond to surveys” would look like, if you could cajole them into answering questions? I have.

image

Imperfect then, but still interesting. What is notable in this one? The questions that interest me are those in the technology section. I am also more interested in trends than in absolute rankings, because it is the trends that might tell us something about the future. So let’s have a look.

In “Most popular programming languages” the interesting stuff is well down the rankings, since the top places are little changed. Dart has gone from 1.9% last year to 4.0% this year; still small, but that is more than 100% growth, thanks no doubt to Flutter. Rust has also grown, from 3.2% to 5.1%. Swift has fallen slightly, from 6.6% to 5.9%. This no longer seems likely to become a top programming language, important though it is for macOS and iOS. Objective-C is down a bit too (4.8% to 4.1%) and I wonder if this suggests greater interest in cross-platform toolkits and/or web technologies for Apple platforms. Such as Flutter, of course.

What about web frameworks? React.js is up a bit, from 31.3% to 35.9%, and to my mind it does look all-conquering at the moment. jQuery is above it, but that is nonsense really, as jQuery is not an alternative to React.js, being more a low-level plumbing thing. The figures for both Angular and ASP.NET are confusing as hell, since last year there was no separate entry for ASP.NET Core, but this year there is; and this year Angular is split between Angular and Angular.js. So we cannot conclude anything about these technologies. There are signs of growth in JAMstack frameworks. Vue.js is up from 15.2% to 17.3% and Gatsby makes an appearance this year at 4.0%

In “Other frameworks” we see .NET down a little but .NET Core up a little, so probably no change there. Flutter up from 3.4% to 7.2% as noted above. Xamarin down a bit, from 6.5% to 5.8%. Xamarin has not been as popular as I had expected it to become when Microsoft acquired it; the reason seems obvious, which is that Microsoft has given developers mixed messages about whether to use it, with the Windows and Office teams seemingly preferring React Native.

In databases, PostreSQL had another good year, rising from 34.3% to 38.5%. More than MySQL, it seems a natural destination for those migrating from Microsoft SQL Server or Oracle. Though note too that SQL Server is up, from 32.8% to 34.8%. Microsoft pushes developers strongly towards SQL Server in its developer tools and frameworks, and the strategy seems to work (plus it is a pretty good database manager).

In platforms, both Linux and Windows have increased use among developers. Not so surprising when you consider that Microsoft now ships the Linux kernel with Windows 10. MacOS is going in the right direction too, from 22.2% to 24.0%.

StackOverflow stuffs cloud platforms into this part of the survey too, as well as things like WordPress (talk about not comparing like with like!). Still, note that AWS is up from 26.6% to 26.7% (well, hardly moved); Azure is up from 11.9% to 14.5%; and Google Cloud (GCP) from 12.4% to 14.1%. Oh yes, and Kubernetes is here too, up from 8.5% to 11.5%. All of this chimes with my perception that GCP is doing pretty well from a developer perspective, now only just behind Azure in this particular community.

Then there is the entertaining most loved, dreaded and wanted. Rust is still move loved by miles (86.1%, up from 83.5% last year). Not much else I want to say about this section, other than to note that Python tops the “most wanted” list by a bigger margin (30.0% up from 25.7%); and poor old VBA continues to be “most dreaded”, again by a bigger margin (80.4% up from 75.2%).

In web frameworks, the good news for .NET developers is that ASP.NET Core now tops the list of “Most loved”, whereas ASP.NET is well down the list (36.9%). Again you cannot really compare with last year. Angular.js (the old version) tops the list of “Most dreaded”.

Similarly, .NET Core tops the list of most loved “other frameworks, libraries and tools” at 71.5%, though it only manages  8.3% in “Most wanted.” Translation: .NET Core developers love it, but it is still not doing that well in terms of appeal to those not currently using it. Chef should be asking itself why it is top of “Most dreaded” for two years running, in fact more dreaded than last year (72.4% up from 66.7%). Puppet is not far behind. Ansible seems both better liked and less dreaded by developers.

The “Most dreaded platforms” list is also notable, with WordPress at the top (PHP spaghetti anyone?) and IBM “Cloud or Watson” second, both positions unchanged from last year.  Android as overtaken Windows as the most dreaded operating system. All the top cloud platforms are more dreaded than last year, and so is Kubernetes.

Make of all that what you will. The survey seems to me valuable as evidence of things we already know, but there are no huge surprises – and why should there be?

Tech Writing