Category Archives: cloud computing

AWS embraces hybrid cloud? Meet Snowball Edge

July 18, 2018 Tim Anderson

Amazon has announced Snowball Edge, an on-premises appliance that supports Amazon EC2 (Elastic Compute Cloud), AWS Lambda (“serverless” computing) and S3 (Simple Storage Service), all running locally.

Sounds like Microsoft’s Azure Stack? A bit, but the AWS appliance is tiny by comparison and therefore more limited in scope. Nevertheless, it is a big turnaround for the company, which has previously insisted that everything belongs in the cloud. One of the Snowball Edge case studies is the same general area as one used by Microsoft for Azure Stack: ships.

The specifications are shy about revealing what is inside, but there is 100TB storage (82TB usable), 10GB, 20GB and 40GB network connections (GBase-T, SFP+ and QSFP+), size is 259x671x386mm (pretty small), and power consumption 400 watts.

Jeff Barr’s official blog post adds that there is an “Intel Xeon D processor running at 1.8 GHz, and supports any combination of instances that consume up to 24 vCPUs and 32 GiB of memory.”

You can cluster Snowball Edge appliances though so substantial systems are possible.

Operating systems currently supported are Ubuntu Server and CentOS7.

Amazon’s approach is to extend its cloud to the edge rather than vice versa. You prepare your AMIs (Amazon Machine Instances) in the cloud before the appliance is shipped. The very fast networking support shows that the intent is to maintain the best possible connectivity, even though the nature of the requirement is that internet connectivity in some scenarios will be poor.

A point to note is that whereas the documentation emphasises use cases where there are technical advantages to on-premises (or edge) computing, Barr quotes instead a customer who wanted easier management. A side effect of the cloud computing revolution is that provisioning and managing cloud infrastructure is easier than with systems (like Microsoft’s System Center) designed for on-premises infrastructure. Otherwise they would not be viable. Having tasted what is possible in the cloud, customers want the same for on-premises.

cloud computing, microsoft, office 365, tech

Microsoft announces free version of Teams, ahead of Inspire partner conference

July 12, 2018 Tim Anderson

Microsoft’s partner conference, Inspire, kicks off in Las Vegas next week; and as part of the event the company has announced big news concerning Teams: a free version.

What is Teams? It is a collaboration tool for Office 365, or at least it was, since the new free version can be used with any email address and without Office 365. Here is what you get:

Chat
Audio and video calling
10GB online storage, plus 2GB for each additional team member (SharePoint/OneDrive)
Word, Excel and PowerPoint online
Ability to install unlimited additional applications

Teams is a strategic product for Microsoft – see here for the reason. A free version is way for the company to promote Office 365, and you will see an upgrade link in the user interface.

There are also new features coming to Teams. One seems minor, but will be popular. It deals with the problem of video conferencing from home, and not being sure what may happen behind you. You may remember this:

So now Teams video conferencing will let you blur the background. Here is Raanah Amjadi, Marketing Manager, Microsoft Teams, demonstrating the feature:

In addition, Teams is getting a new Live Events feature. This is where you broadcast a presentation or meeting to others in your company. Automatic speech-to-text will do close captions (so you can watch with the sound done, if you trust it enough), and this then enables text search of the event with index points into the video. Bing Translate is also included in Teams so you can have multi-lingual conversations.

Microsoft Workplace Analytics is getting enhancements including “My Analytics” which will give you AI-powered “nudges” in Outlook online. I am not sure I trust this to be much real-world use; but the example shown was intriguing: alert you if you try to schedule a meeting with someone out of their working hours.

Whiteboard, a collaboration canvas, is now generally available for Windows 10 and mobile.

Free Teams is available immediately here.

amazon, cloud computing, linux, tech

Ubuntu goes minimal (but still much bigger than Alpine Linux), cosies up to Google Cloud Platform

July 9, 2018 Tim Anderson

Ubuntu has announced “Minimal Ubuntu”, a cut-down server image designed for containerised deployments. The Docker image for Minimal Ubuntu 18.04 is 29MB:

Editors, documentation, locales and other user-oriented features of Ubuntu Server have been removed. What remains are only the vital components of the boot sequence. Images still contain ssh, apt and snapd so you can connect and install any package you’re missing. The unminimize tool lets you ‘rehydrate’ your image into a familiar Ubuntu server package set, suitable for command line interaction.

says Canonical.

29MB is pretty small; but not as small as Alpine Linux images, commonly used by Docker, which are nearer 5MB. Of course these image sizes soon increase when you add the applications you need.

I pulled Ubuntu 18.04 from Docker Hub and the image size is 31.26MB so this hardly seems a breakthrough.

Canonical quotes Paul Nash, Group Product Manager for Google Cloud Platform, in its press release. The image is being made available initially for Amazon EC2, Google Compute Engine, LXD, and KVM/OpenStack. The kernel has been optimized for each deployment, so the downloadable image is optimized for KVM and slightly different than the AWS or GCP versions.

android, azure, cloud computing, development, tech, Web

Pusher: a nice solution for sending messages and notifications to web and mobile apps

July 6, 2018 Tim Anderson

Pusher is a London company which runs cloud services for publish/subscribe in web and mobile applications. The idea is to deliver real-time updates, a concept that has many use cases. Examples include price updates in finance apps, status updates to track a delivery, news updates, or anything where users want to monitor progress or keep in touch with fast-moving developments.

The service passed my “get up and running quickly” test. I created a free account (limited to 100 connections and 200k messages per day) and a new channel:

I’m guessing it runs on AWS, looking at the datacentre locations:

I chose a JavaScript client and ASP.NET MVC for the back end. On my PC I pasted the JavaScript into a web page running locally on Apache (in Windows Subsystem for Linux). I also created a new ASP.NET MVC project and added the sample code with some trivial modifications. I was able to send a message to the web page; it triggers an annoying alert but of course you could easily amend this to update the UI in more user-friendly ways.

Of course you could roll your own solution for this but what you get with Pusher is all the plumbing pre-done for many different clients and automatic scalability.

Pusher also has a service called Beams (formerly Push Notifications) which lets you send notifications to Android and IOS apps.

Pusher or roll your own? As with many cloud services, you are putting a high level of trust in Pusher (security and reliability) if you use the service, and you will need a paid subscription:

You are saving considerable development time though, and as Google and Apple update their SDKs or change the rules, Pusher will presumably adapt accordingly.

Can Azure easily do this, I wondered? I headed over to Azure Notification Hubs. I noticed that the amount of admin you have to do to support each device is greater. Second, Microsoft promised to support “push to web” in March 2016:

… but has not done so nor even bothered to update those asking:

It is odd that Microsoft, with all its drive behind Azure, is still in the habit of leaving customers in the dark in certain areas.

cloud computing, database, microsoft, software development, tech

Inside Azure Cosmos DB: Microsoft’s preferred database manager for its own high-scale applications

June 27, 2018 Tim Anderson 1 Comment

At Microsoft’s Build event in May this year I interviewed Dharma Shukla, Technical Fellow for the Azure Data group, about Cosmos DB. I enjoyed the interview but have not made use of the material until now, so even though Build was some time back I wanted to share some of his remarks.

Cosmos DB is Microsoft’s cloud-hosted NoSQL database. It began life as DocumentDB, and was re-launched as Cosmos DB at Build 2017. There are several things I did not appreciate at the time. One was how much use Microsoft itself makes of Cosmos DB, including for Azure Active Directory, the identity provider behind Office 365. Another was how low Cosmos DB sits in the overall Azure cloud system. It is a foundational piece, as Shukla explains below.

There were several Cosmos DB announcements at Build. What’s new?

“Multi-master is one of the capabilities that we announced yesterday. It allows developers to scale writes all around the world. Until yesterday Cosmos DB allowed you to scale writes in a single region but reads all around the world. Now we allow developers to scale reads and writes homogeneously all round the world. This is a huge deal for apps like IoT, connected cars, sensors, wearables. The amount of writes are far more than the amount of reads.

“The second thing is that now you get single-digit millisecond write latencies at the 99 percentile not just in one region.

“And the third piece is that what falls out of this high availability. The window of failover, the time it takes to failover from one region when a disaster happens, to the other, has shrunk significantly.

“It’s the only system I know of that has married the high consistency models that we have exposed with multi-master capability as well. It had to reach a certain level of maturity, testing it with first-party Microsoft applications at scale and then with a select set of external customers. That’s why it took us a long time.

“We also announced the ability to have your Cosmos Db database in your own VNet (virtual network). It’s a huge deal for enterprises where they want to make sure that no data leaks out of that VNet. To do it for a global distributed database is specially hard because you have to close all the transitive networking dependencies.”

Technical Fellow Dharma Shukla

Does Cosmos DB work on Azure Stack?

“We are in the process of going to Azure Stack. Azure Stack is one of the top customer asks. A lot of customers want a hybrid Cosmos DB on Azure Stack as well as in Azure and then have Active – Active. One of the design considerations for multi master is for edge devices. Right now Azure has about 50 regions. Azure’s going to expand to let’s say 200 regions. So a customer’s single Cosmos DB table spanning all these regions is one level of scalability. But the architecture is such that if you directly attach lots of Azure Stack devices, or you have sensors and edge devices, they can also pretend to be replicas. They can also pretend to be an Azure region. So you can attach billions of endpoints to your table. Some of those endpoints could be Azure regions, some of them could be instances of Azure Stack, or IoT hub, or edge devices. This kind of scalability is core to the system.”

Have customers asked for any additional APIs into Cosmos DB?

“There is a list of APIs, HBase, richer SQL, there are a number of such API requests. The good news is that the system has been built in a way that adding new APIs is relatively easy addition. So depending on the demand we continue to add APIs.”

Can you tell me anything about how you’ve implemented Cosmos DB? I know you use Service Fabric. Do you use other Azure services?

“We have dedicated clusters of compute machines. Cosmos DB is a Ring 0 service. So it’s there any time Azure opens a new region, Cosmos DB clusters have provision by default. Just like compute, storage, Cosmos DB is also one of the Ring 0 services which is the bottommost. Azure Active Directory for example depends on Cosmos DB. So Cosmos DB cannot take a dependency on Active Directory.

“The dependency that we have is our own clusters and machines, on which we put Service Fabric. For deployment of Cosmos DB code itself, we use Service Fabric. For some of the load balancing aspects we use Service Fabric. The partition management, global distribution, replication, is our own. So Cosmos DB is layered on top of Service Fabric, it is a Service Fabric application. But then it takes over. Once the Cosmos DB bits are laid out on the machine then its replication and partition management and distribution pieces take over. So that is the layering.

“Other than that there is no dependency on Azure. And that is why one of the salient aspects of this is that you can take the system and host it easily in places like Azure Stack. The dependencies are very small.

“We don’t use Azure Storage because of that dependency. So we store the data locally and then replicate it. And all of that data is also encrypted at rest.”

So when you say it is not currently in Azure Stack, it’s there underneath, but you haven’t surfaced it?

“It is in a defunct mode. We have to do a lot of work to light it up. When we light up it on such on-prem or private cloud devices, we want to enable this active to active pathway. So you are replicating your data and that is getting synchronized with the cloud and Azure Stack is one of the sockets.”

Microsoft itself is using Cosmos DB. How far back does this go? Azure AD is quite old now. Was it always on Cosmos DB / DocumentDB?

“Over the years Office 365, Xbox, Skype, Bing, and more and more of Azure services, have started moving. Now it has almost become ubiquitous. Because it’s at the bottom of the stack, taking a dependency on it is very easy.

“Azure Active Directory consists of a set of microservices. So they progressively have moved to Cosmos DB. Same situation with Dynamics, and our slew of such applications. Skype is by and large on Cosmos DB now. There are still some fragments of the past. Xbox and the Microsoft Store and others are running on it.”

Do you think your customers are good at making the right choices over which database technology to use? I do pick up some uncertainty about this.

“We are working on making sure that we provide that clarity. Postgres and MySQL and MariaDB and SQL Server, Azure SQL and elastic pools, managed instances, there is a whole slew of relational offerings. Then we have Cosmos DB and then lots of analytical offerings as well.

“If you are a relational app, and if you are using a relational database, and you are migrating from on-prem to Azure, then we recommend the relational family. It comes with this fundamental scale caveat which is that up to 4TB. Most of those customers are settled because they have designed the app around those sorts of scalability limitations.

“A subset of those customers, and a whole bunch of brand new customers, are willing to re-write the app. They know that that they want to come to cloud for scale. So then we pitch Cosmos DB.

“Then there are customers who want to do massive scale offline analytical processing. So there is, Databricks, Spark, HD Insight, and that set of services.

“We realise there are grey lines between these offerings. We’re tightening up the guidance, it’s valid feedback.”

Any numbers to flesh out the idea that this is a fast-growing service for Microsoft?

“I can tell you that the number of new clusters we provision every week is far more than the total number of clusters we had in the first month. The growth is staggering.”

cloud computing, development, github, microsoft, tech

Microsoft and GitHub, and will GitHub get worse?

June 4, 2018 Tim Anderson 1 Comment

Microsoft has announced an agreement to acquire GitHub for $7.5 billion (in Microsoft stock). Nat Friedman, formerly CEO of Xamarin, will become GitHub’s CEO, and GitHub will continue to run somewhat independently. A few comments.

Background: GitHub is a cloud-based source code repository based on Git, a distributed version control system created by Linus Torvalds. It is free to use for public, open source projects but charges a fee (from 7$ to $21 per user per month) for private repositories.

First, why? This one is easy. Microsoft is a big customer of GitHub. Microsoft used to have its own hosting service for open source software called CodePlex but abandoned it in favour of GitHub, formally closing CodePlex in March 2017:

Over the years, we’ve seen a lot of amazing options come and go but at this point, GitHub is the de facto place for open source sharing and most open source projects have migrated there. We migrated too.

said Brian Harry.

Microsoft also uses GitHub for its documentation, and this has turned out to be a big improvement on its old documentation sites.

Note also that Microsoft has many important open source projects of its own, including much of its developer platform (.NET Core, ASP.NET Core and Entity Framework Core). Many of its projects are overseen by the .NET Foundation. Other notable open source, Github-hosted projects include Visual Studio Code, a programmer’s editor that has won many friends, and TypeScript, a typed superset of JavaScript that compiles to standard JavaScript code.

When big companies become highly dependent on the services of another company they may become anxious about it. What if the other company were taken over by a competitor? What if it were to run into trouble, or to change in ways that cause problems? Acquisition is an easy solution.

In the case of GitHub, there was reason to be anxious since it appears not to be profitable – unsurprising given the large number of free accounts.

Second, Microsoft is always pitching to developers, trying to attract them to its platform and especially Azure services. It has a difficult task because it is the Windows company and the Windows platform overall is in decline, versus Linux on servers and Android/iOS on mobile. Therefore it is striving to become a cross-platform company, and with considerable success. I discuss this at some length in this piece. Note that there is a huge amount of Linux on Azure, including “more than 40%” of the virtual machines. More than 50%? Maybe.

If Microsoft can keep GitHub working as well as before, or even improve it, it will do a lot to win the confidence of developers who are currently outside the Microsoft platform ecosystem.

Will GitHub get worse?

The tricky question: under Microsoft, will GitHub get worse? The company’s track record with acquisitions is spotty, ranging from utter disasters (Nokia, Danger) to doubtful (Skype), to moderately successful so far (LinkedIn, Xamarin).

Under the current leadership, I doubt anything bad will happen to GitHub. I’d guess it will migrate some infrastructure to Azure (GitHub runs mainly from its own datacentres as I understand it) but there is no need to re-engineer the platform to run on Windows.

Some businesses will be uncomfortable hosting their valuable source code with Microsoft. That is understandable, in the same way that I hear of retailers reluctant to use Amazon Web Services (since it is a platform owned by a competitor), but it is a low risk. Others have long-standing mistrust of Microsoft and will want to migrate away from GitHub because of this.

Personally I think it is right to be wary of any giant global corporation, and dislike the huge and weakly regulated influence they have on our lives. I doubt that Microsoft is any worse than its peers in terms of trustworthiness but of course this is open to debate.

Another point: with this acquisition, free GitHub hosting for open source projects will be likely to continue. The press release says:

GitHub will retain its developer-first ethos and will operate independently to provide an open platform for all developers in all industries. Developers will continue to be able to use the programming languages, tools and operating systems of their choice for their projects — and will still be able to deploy their code to any operating system, any cloud and any device.

It is of course in Microsoft’s interests to make this work and the success of Visual Studio Code and TypeScript (which also come from the developer side of the company) shows that it can make cross-platform projects work. So I am optimistic that GitHub will be OK.

Update: I’ve noticed Sam Newman and Martin Fowler taking this view, a good sign from a people I respect and who are by no means from the usual Microsoft crowd.

Official announcements

Press release: https://news.microsoft.com/?p=406917

Chris Wanstrath’s Blog Post: https://blog.github.com/2018-06-04-github-microsoft/

Satya Nadella’s Blog Post: https://blogs.microsoft.com/?p=52553832

cloud computing, microsoft, mobile, tech

Strong financial results from Microsoft as it aims for breadth of services

February 1, 2018 Tim Anderson

Microsoft reported a big quarter (in terms of revenue) for the three months ending December 31st, with revenue of $28,918 million.

What’s notable? Mainly the big jump in Microsoft’s recent success stories: year on year Office 365 up by 41%, Azure up by 98%, Dynamics 365 up by 67%.

Windows is flat/weak as you would expect, and Surface hardware is standing still. Xbox grew a bit following the launch of Xbox One X.

LinkedIn is growing: revenue of $1.3 billion and “sessions growth of over 20%” in the quarter. In the earnings webcast, Microsoft’s Amy Hood said that the LinkedIn acquisition has both performed better, and seems more strategic, now than it did at the time.

Hood also made reference to the company’s ability to up-sell cloud users to higher-margin services. “Office 365 commercial revenue increased 41 percent from installed base growth across all customer segments, and ARPU [Average Revenue per User] expansion from continued customer migration to higher value offers in the E3 and E5 workloads.”

This point is key and is the answer (from the provider’s point of view) to the lower margins implicit in moving from software to services. When Microsoft sells a licence for you to use Windows or Office, the margin is huge because reproducing the software, or providing it for download, costs almost nothing; whereas with a subscription there is significant cost to providing the service. However the subscription has advantages which offset this, in particular the continuing interaction with the customer that both provides data, which the customer as well as the provider can mine (subject to appropriate privacy controls), and gives opportunity for the provider to extend the relationship into new or upgraded services.

CEO Satya Nadella fielded a good question about Microsoft losing out to Sony in gaming and to Alexa and Google Home in voice devices. On gaming, Nadella referred to the PC alongside Xbox as a strategic asset. “PC gaming is a growth market,” he said, as well as software such as Minecraft now on mobile devices, giving the company a broad reach. He also remarked on Azure as a gaming back end.

As for Cortana in the home (or absence from), Nadella said that the focus is on the server-side cognitive services. He also talked about voice input and control of Office 365. The key point though was that Microsoft wants to work both with its own and other voice assistant devices so it can win on services even when competitor devices are in use. “One-turn dialogs on one speaker in one home, that’s just not our vision,” he said.

Nadella made another key point in the webcast, in answer to a question about how Azure Stack (a packaged version of Azure for installation on-premises) will impact Azure. “Computing is becoming more distributed, not less distributed,” he said. IoT and sensors play a large part in this. Everything goes to the cloud but computing on the edge (the new buzzword for local processing) is important for efficiency.

It is easy to see ways in which Microsoft could stumble. The PC will decline as the number of users who need a desktop or laptop computer diminishes. Microsoft’s failure in mobile could prove costly as competitors use synergy with their own applications and cloud services to steer customers away. There are opportunities such as home automation and payments which seem closed to the company now.

Then again, strong results such as these show how the company can succeed by continuing to migrate its business users to cloud services. It remains deeply embedded in business computing.

Here is my chart summarising Microsoft’s performance:

Quarter ending December 31st 2017 vs quarter ending December 31st 2016, $millions

Segment	Revenue	Change	Operating income	Change
Productivity and Business Processes	8953	+1774	3337	+284
Intelligent Cloud	7795	+1037	2832	+541
More Personal Computing	12170	+281	2510	-51

The segments break down as:

Productivity and Business Processes: Office, Office 365, Dynamics 365 and on-premises Dynamics, LinkedIn

Intelligent Cloud: Server products, Azure cloud services

More Personal Computing: Consumer including Windows, Xbox; Bing search; Surface hardware

cloud computing, google, salesforce.com, tech

Quick thoughts on Salesforce and Google Cloud Platform alliance

November 7, 2017 Tim Anderson

Yesterday Salesforce and Google announced a strategic partnership:

1. Salesforce named Google Cloud as “a preferred public cloud provider”. Salesforce says it “continues to invest in its own data centers”. However it will use public cloud infrastructure “for its core services” as well, especially in “select international markets.” Why is Google Cloud Platform (GCP) just a preferred partner and not the? Well, “AWS is a great partner”, as the release also notes.

2. New integrations will be introduced between Salesforce and G Suite (Gmail, Docs, Google Drive and Calendar for business), and there is a promotional offer of one year’s free G Suite for Salesforce customers. Note that the release also says “restrictions apply, see here”, with the see here link currently inactive.

3. Salesforce will integrate with Google Analytics.

Google has also posted about the partnership but adds little of substance to the above.

Why this alliance? On Google’s side, it is keen to build momentum for its cloud platform and to catch up a little with AWS and Microsoft Azure. Getting public support from a major cloud player like Salesforce is helpful. On the Salesforce side, it is an obvious alliance following the public love-in between Adobe and Microsoft Azure. Adobe competes with Salesforce in marketing tools, and Microsoft competes with Salesforce in CRM.

Google will also hope to win customers from Microsoft Exchange, Office and Office 365. However Salesforce knows it has to integrate nicely with Microsoft’s email and productivity tools as well as with G Suite. The analytics integration is a bigger deal here, thanks to the huge reach of Google’s cloud data and tools.

cloud computing, microsoft, tech

Microsoft Ignite: where next for Microsoft’s cloud? The Facebook of business?

September 25, 2017 Tim Anderson

Microsoft has futuristic domes as part of its Envision event, running alongside Ignite here in Orlando. Ignite is the company’s main technical event of the year, focusing mainly on IT Pros but embracing pretty much the whole spectrum of Microsoft’s products and services (maybe not much Xbox!). With the decline of the PC and retreat from mobile, and a server guy at the helm, the company’s focus has shifted towards cloud and enterprise, making Ignite all the more important.

This year sees around 25-30,000 attendees according to a quick estimate from one of the PRs here; a little bigger than last year’s event in Atlanta.

Microsoft will present itself as an innovative company doing great things in the cloud but the truth is more complex, much though I respect the extent to which the business has been transformed. This is a company with a huge amount of legacy technology, designed for a previous era, and its challenge has been, and still is, how to make that a springboard for moving to a new way of working as opposed to a selling opportunity for cloud-born competitors, primarily Amazon Web Services (AWS) and Google, but also the likes of Salesforce and Dropbox.

If there is one product that has saved Microsoft, it is probably Exchange, always a solid email server and basic collaboration tool. Hosted Exchange is the heart of Office 365 (and BPOS before it), making it an easy sell to numerous businesses already equipped with Office and Outlook. Email servers are horrible things to manage, so hosted has great appeal, and it has driven huge uptake. A side-effect is that it has kept customers using Office and to some extent Windows. A further side-effect is that it has migrated businesses onto Azure Active Directory, the directory behind Exchange Online.

Alongside Office 365, the Azure cloud has matured into a credible competitor to AWS. There are still shortcomings (a few of which you can expect to be addressed by announcements here at Ignite), but it works, providing the company with the opportunity to upsell customers from users of cloud infrastructure to consumers of cloud services, such as Azure IoT, a suite of tools for gathering and analysing data.

The weakness of Microsoft’s cloud efforts has been the moving parts between hosted services and Windows PCs, and legacy pieces that do not work as you would expect. OneDrive has been a persistent annoyance, with issues over reliable document sync and limitations over things like the number of documents in a folder and the total length of a path. And where are my Exchange Public Folders, or any shared folders, in Outlook for IoS and Android? And why does a PC installation of Office now and again collapse with activation or other issues, so that the only solution is removal and reinstall?

At Ignite we will not hear of such things. Instead, Microsoft will be presenting its vision of AI-informed business collaboration. Think “Facebook of business”, powered by the “Microsoft graph”, the sum of data held on each user and their files and activity, now combined with LinkedIn. The possibilities for better-informed business activity, and systems that know what you need before you ask, are enticing. Open questions are how well it will work, and old issues of privacy and surveillance.

Such things also can only work if businesses do in fact commit more of their data to Microsoft’s cloud. The business case for this is by no means as simple as the company would have us think.

Tim Anderson's IT Writing