I remember being impressed with the Azure docs... until I spend a week implementing something, only to have it completely fail when deployed to the test environment because GraphAPI did not work as documented. The beautiful docs were a complete lie.

These days I don't even bother looking at the docs when doing stuff with Azure.

I’ve worked with their consultants and they were lovely. They hate Azure too.

We migrated some services to AKS because the upper management thought it was a good deal to get so many credits, and now pods are randomly crashing and database nodes have random spikes in disk latency. What ran reliably on GCP became quite unpredictable.

>Is Azure really this unreliable? There are concrete numbers in this blog. For those who use Azure, does it match your external experience?

IME, yes.

I'm currently working as an SRE supporting a large environment across AWS, Azure, and GCP. In terms of issues or incidents we deal with that are directly caused by cloud provider problems, I'd estimate that 80-90% come from Azure. And we're _really_ not doing anything that complicated in terms of cloud infrastructure; just VMs, load balancers, some blob storage, some k8s clusters.

Stuff on Azure just breaks constantly, and when it does break it's very obvious that Azure:

1. Does not know when they're having problems (it can take weeks/months for Azure to admit they had an outage that impacted us)

2. Does not know why they had problems (RCAs we're given are basically just "something broke")

3. Does not care that they had problems

Everyone I work with who interacts with Azure at all absolutely loathes it.

In my experience Azure is full of consistency issues and race conditions. It's enough of an issue that I was talking about new OpenAI models becoming available via Bedrock on AWS and how convenient that was since I wouldn't have to deal with Azure and my colleague in enterprise architecture went on an unprompted rant about these exact issues. It's not the first time something like this has happened and I've experienced these issues first hand, so yes. I'd say reliability is a critical issue for Azure and it hasn't gotten better each time I've gone back to check.

Large orgs make decisions that prioritize short-term metrics over long-term quality all the time and nobody tracks whether those tradeoffs actually paid off. The decision to ship fast and fix later sounds reasonable in a meeting setting until articles like this surface and the reality comes through clearly.

I recall seeing some pretty damning reports from a security pentester that was able to escape from a container on Azure and found the management controller for the service was years old with known critical unpatched vulnerabilities. Always been a bit sceptical of them since then

Yes it is that unreliable. Even when given free credits, I would rather pay for the offerings from Amazon/Google.

The CEO is accountable to the board. If they are derelict in their obligations to the company, that's where you need to raise a stink so they can fix it.

Yeah I thought that was extreme. An engineer going to the board of any corporation let alone Microsoft is not normal or customary IME. That could explain why they got no response.

He is, I think, Swiss, perhaps a cultural difference?

Azure is when you have a different version of the same product/api in each region.

AWS and Google Cloud are both huge and are significantly better in UX/DX. My only experience with Azure was that it barely worked, provided very little in the way of information about why it didn't. I only have negative impressions of Azure whereas at least GC and AWS I can say my experiences are mixed.

> risks to national security

Microsoft is the go to solution for every government agency, FEDRAMP / CMMC environments, etc.

> People come in all the time crying that everything is broken and needs to be scrapped and rewritten but it's hardly ever true.

This I'm more sympathetic to. I really don't think his approach of "here's what a rewrite would look like" was ever going to work and it makes me think that there's another side to this story. Thinking that the solution is a full reset is not necessarily wrong but it's a bit of a red flag.

> Last thing, the final part 6 comes off as really childish, risks to national security and sending letters to the board, really?

That struck me too. Maybe i've never worked high enough in an org (im unclear how highly ranked the author of the piece is) but i've never been in an org where going over your boss's boss's boss's boss's head and writing a letter to the board was likely to go well.

That said, i could easily believe that both Azure is an absolute mess and that the author of the piece was fired because of how he went about things.

I think he did kind of point at the lack of seniority in the org, so I'm not sure he was trying to exaggerate with the titles.

I'm really struck that they have such Jr people in charge of key systems like that.

> People come in all the time crying that everything is broken and needs to be scrapped and rewritten but it's hardly ever true.

Or… you’ve just normalised the deviation.

One of the few reliable barometers of an organisation (or their products) is the wtf/day exclaimed by new hires.

After about three or four weeks everyone adapts, learns what they can and can’t criticise without fallout, and settles into the mud to wallow with everyone else that has become accustomed to the filth.

As an Azure user I can tell you that it’s blindingly obvious even from the outside that the engineering quality is rock bottom. Throwing features over the fence as fast as possible to catch up to AWS was clearly the only priority for over a decade and has resulted in a giant ball of mud that now they can’t change because published APIs and offered products must continue to have support for years. Those rushed decisions have painted Azure into a corner.

You may puff your chest out, and even take legitimate pride in building the second largest public cloud in the world, but please don’t fool yourself that the quality of this edifice is anything other than rickety and falling apart at the seams.

Remind me: can I use IPv6 safely yet? Does it still break Postgres in other networks? Can azcopy actually move files yet, like every other bulk copy tool ever made by man? Can I upgrade a VM in-place to a new SKU without deleting and recreating it to work around your internal Hyper-V cluster API limitations? Premium SSDv2 disks for boot disks… when? Etc…

You may list excuses for these quality gaps, but these kinds of things just weren’t an issue anywhere else I’ve worked as far back as twenty years ago! Heck, I built a natively “all IPv6” VMware ESXi cluster over a decade ago!

> risks to national security …really?

Really. Apparently the Secretary of War agrees with him.

> The post is so dramatized and clearly written by someone with a grudge such that it really detracts from any point that is trying to be made, if there is any

I guessed that from the title on the main hn page. Glad to see it confirmed.

He might sound like he has a grudge but you sound like you’re personally invested. Shill?

Scary is the understatement of the day. I can't imagine the environment where someone think that architecture is a good idea.

Like, what did the OP expect?

> 2 instances of Outlook

That's 2 too many.

GCP's support is abysmal. Our assigned customer support agent has changed 3 times in as many months. it's really a dice roll if our quota increase requests are even acknowledged or we can get clarification on undocumented system limitations.

In my experience Azure endpoint versus openAI endpoint was way faster and significantly cheaper.

> Even if you are an excellent engineer, making people do things, accept ideas, and in general hear you requires a completely different skill altogether - basically being a good communicator.

I was thinking like this for a while but, now, I think this expectation is majorly false for a senior individual contributor. Especially when someone who can push out a detailed series of blogposts and has tried step-wise escalation.

Communication is a two-way street. Unlike the individual contributors, the management is responsible for listening and responding to risk assesments by the senior members and also ensuring that the technical competence and experienced people are retained in a tech company. If a leader doesn't want to keep an open ear, they do not belong there. If there is a huge attrition of highly senior people from non-finalized projects, you do not belong leadership either. Both cases are mentioned in the article.

Unfortunately our socioeconomic and political culture in the West has increasingly removed responsibilities and liabilities from the leadership of the companies. This causes people with lackluster technical, communication and risk assesment mentality being promoted into leadership positions.

So outside of a couple completely privately owned companies or exceptionally well organized NGOs, it will be increasingly difficult to find good leaders.

Rust is nowhere close to Node in terms of package number bloat. Most Rust libraries are actually useful and nontrivial and the supply chain risk is not necessarily as high for the simple reason that many crates are split up into sub-crates.

For example, instead of having one library like "hashlib" that handles all different kinds of hashing algorithms, the most "official" Rust libraries are broken up into one for sha1, one for sha2, one for sha3, one for md5, one for the generic interfaces shared by all of them, etc... but all maintained by the same organization: https://github.com/rustcrypto/

Most crypto libraries are the same.

Maybe that bumps the numbers up if you need more than one algorithm, but predominantly it is still anti-bloat.

It really is about time that somebody do something about it.

Start with tokio. Please vend one dependency battery included, and vendor in/internalize everything, thanks.

From my old experience in IT - people just default to Microsoft for everything. They don't want to hassle with learning anything else and assume better the devil you know. Glad I'm out of that world but its wild what people will put up with.

Microsoft shops. Lots of C# devs gravitate to it naturally. I’m glad I abandoned the MS stack over a decade ago.

The VPs who think that they got a good deal by combining with o365

Environment is part of the package management. As it stand, it's better than npm only because it is in rust.

His writing style is fairly over the top (he is Swiss, and I have seen this before, but not most of the time), but most of the technical content seems true to me.

pardon?

It’s not inevitable. Absolutely this is true without significant effort, but if you’ve been around the traps for long enough (in enough organisations), you get to see that the level of quality can vary widely. Avoiding the mud-pit does require a whole org commitment, starting from senior leadership.

This story is more interesting, in my opinion, in how quickly things devolved and also how unwilling the more senior layers of the org were to address it. At a whole company level, the rot really sets in when you start to lose the key people that built and know the system. That seems to be what’s happening here, and it does not bode well for MS in the medium term.

Why do you speak about yourself in the third person?

Also, after this:

https://news.ycombinator.com/item?id=20341022

You continued to work at Microsoft and now there is this takedown?

I'm no friend of MS (to put it very mildly) but it seems to me your story is a bit inconsistent as well as the 7 year break between postings.

I downvoted this comment for sounding like a summarizing LLM, not adding anything substantial beyond the title of the post, before realizing you were the poster and author.

What's your assessment of AWS and GCP? Do you think it's likely they suffer from some of the same issues (eg the manual access of what should be highly secure, private systems, the instability, the lack of security)?