A fateful conversation
Back in the 90s, when cloud was bureau computing, and Microsoft was Word, Excel, and a cruddy windowing operating system, I was a cocky young product marketing manager at Oracle Australia. Imagine my surprise when Microsoft dropped the bombshell: they were diving into relational databases with SQL Server and taking Oracle head-on. The cheek!
A few weeks later, I was at Sydney airport in the Ansett Airlines lounge (that dates me!), and I happened to see my counterpart from Microsoft. Being overconfident and arrogant, I marched over and questioned him on how Microsoft had the audacity to take on the mighty Oracle! His response was enlightening and has lodged in my brain for 30 years. He said, “Well, Tim, we’re going to knock on someone’s door and ask them, ‘Do you want to buy a good cheap database?’ If they say anything other than ‘yes’, we’re going to politely close the door and move onto the next one. Trust me, we’ll find enough people to build a market.”
Sure enough, he was right. Microsoft did exactly that and has grown to become one of the most significant RDBMS vendors globally, currently holding the #3 spot – although at $7,000 a core for the Enterprise edition license, Microsoft’s pricing has moved with the times! Cheap it ‘ain’t.
Today, I feel like I’ve come full circle. While Ansett Airlines has long gone, SQL Server shows signs of age like me. If only I could return to that fateful meeting with Microsoft, I would acknowledge that SQL Server may be suitable for OLTP, but it’s limited for OLAP. This experience is underscored at Yellowbrick from numerous conversations with SQL Server customers. Only this week, we announced another SQL Server churn (United Energy Workers Healthcare Modernizes Analytics with Yellowbrick Cloud Data Warehouse). As Jacob Case, VP IT Systems & Applications, United Energy Workers Healthcare says:
“Analytics plays a pivotal role in our business, empowering us to tackle the unique challenges encountered within the home healthcare sector. With Yellowbrick, our data stays in-house in our cloud tenant. Yellowbrick’s unique cost model helps prevent runaway data warehousing costs. And Yellowbrick’s MPP architecture alleviates our concerns about scalability, which is critical with our ever-changing analytics demands and expanding data volumes.”
So, what’s the deal with Microsoft SQL Server when it comes to OLAP?
Microsoft SQL Server faces challenges with large (3TB+) datasets because it lacks MPP. Achieving optimal performance demands intricate query optimization, indexes, and similar techniques—all the hallmarks of ye olde databases. Whether working on-premises or in the cloud, you need a high level of expertise to ensure peak performance. In the cloud, configuring high-performance, high-scale, reliable SQL Server on Infrastructure as a Service (IaaS) is complex, requiring a brain the size of a planet. Additionally, when running SQL Server in the cloud, you may utilize more computing cores than on-premises. Regardless of the deployment method, a traditional SQL Server still requires active management of upgrades and infrastructure, and a typical configuration has numerous variants to lose sleep over.
SQL Server, as a general-purpose RDBMS, is not optimized for analytics workloads. Its reliance on caching data in limited memory for large datasets leads to performance degradation. To address these limitations, many users turn to SQL Server Analysis Services—a cubing and caching technology that requires ongoing maintenance and scheduling, ultimately introducing delays and leading to data staleness. Furthermore, these solutions predominantly rely on in-memory technologies, incurring substantial operational costs and necessitating a high level of expertise for development and operation.
Microsoft’s past endeavours, such as SQL Server Big Data Clusters and SQL Server Parallel Data Warehouse, faced challenges and setbacks, leaving a void in on-premises big data solutions. The latest offering, Microsoft Fabric, is exclusively available on Azure, potentially tying customers into a cloud-only ecosystem. This approach involves a consolidation and rebranding of various products and services. The eventual retirement of Synapse Analytics could compel customers to transition to Fabric, a move that may not align with the agility and cost-effectiveness demanded by businesses.
Even for those investing heavily in Synapse Analytics or Fabric, challenges persist. These platforms’ cost and scalability limitations hinder their ability to accommodate multiple teams, resulting in restricted access and creating expensive SQL database marts to cache data and prevent performance bottlenecks. This additional complexity introduces further deployment and maintenance expenses.
More details
For more details written by real experts, I refer you to Still Doing Battle With Your Monolithic Multi-terabyte Data Warehouse? And Brent Ozar, independent SQL Server Guru, The Real Problem with SQL Server’s Licensing Costs, and It’s Been 6 Months. SQL Server 2022 Still Isn’t Ready Yet.
But if you want to talk to me, I can generally be found in the lounge at SFO, LHR, or the Water Rat in Ripon this Friday night!
Conclusion: young and stupid
It often amazes me how overconfident I was as a young person. I don’t know whether one becomes more conservative with age, but I cannot believe some stunts I pulled in my youth, like dancing on stage to divert attention from Oracle7’s two-phase commit failure in front of 5,000 attendees at the product launch. I guess it’s a reasonable “nothing to see here” move. Right? I still cringe with embarrassment when I recall those moments but will draw comfort from this quote from renowned psychiatrist Thomas Szazs:
“The stupid neither forgive nor forget; the naïve forgive and forget; the wise forgive but do not forget.”
And that’s all I have to say about that.
