As the economic slowdown bites, our conversations with customers are shifting from talk of ambitious enterprise-wide distributed data clouds to something more basic: slashing cloud costs. Rather than having big plans for a single enterprise-wide data warehouse spanning multiple clouds (and integrating on-prem), companies are interested in Yellowbrick because our highly efficient architecture (something we call the Direct Data Accelerator) requires less consumption which equates to lower cost. Typically, these conversations occur when FinOps initiatives cap out, and organizations must consider architecture changes to achieve the required savings.
Reducing Cloud Data Warehousing Cost
Data warehousing gets more scrutiny because this technology is a high consumer of storage, network, and compute resources. According to Flexera’s eleventh annual State of the Cloud report for 2022, data warehouses top consumption, say 55% of the 753 respondents.
We’ve been exploring this topic and are investing in research and technologies to help rearchitect systems quickly and easily to offer solutions that complement existing cloud data warehouses, such as Snowflake, by offloading expensive high resource-consuming jobs.
We also explored the topic of cloud cost reduction at a recent LinkedIn Live with Kevin Petrie, VP of Research, Eckerson Group, Mark Cusack, CTO, Yellowbrick, and Heather Brodbeck, VP RevOps, Yellowbrick. Here are some of the talking points:
- Advanced analytics “has a lot of moving parts” and is variable, so an application prone to cost overruns.
- Advanced analytics is a “high cost” consumer of cloud services. An informal poll by Eckerson concluded that 48% of surveyed users thought cloud costs constrain analytics opportunities.
- More attention must be given to understanding the business value of advanced analytics, including quantifying exploratory and discovery use cases that yield new business opportunities. The business value of increased consumption needs to be modeled and understood. The argument that more consumption equates to greater business opportunities is weak.
- The IT organization’s traditional design and management processes that squeeze value from capped on-prem deployments are not being deployed with cloud technology. This is a missed opportunity.
- According to a recent FinOps Foundation survey, a third of respondents cited their biggest challenge as forecasting cloud costs. A third said their next biggest challenge was getting engineers to act on this!
- Application design and cost forecasting need to consider hidden costs such as cloud egress charges and API spending (a point also made in this SiliconANGLE article).
- Organizations need to conduct “analytics on analytics” utilizing observability and forecasting tools available to design for cost savings, e.g., executing low-priority jobs during off-peak hours.
- Tie workload to specific pricing models, e.g., predictable, persistent, budgeted workloads are better suited to subscription pricing, whereas shorter projects, and occasional spikes in workload are better suited to on-demand pricing.
- Workload management, monitoring, and forecasting tools are critical capabilities of any deployed cloud data warehouse.
- Kevin introduced the Eckerson “Cost Governance For Cloud Data Warehousing” framework, which combines FinOps service, observability tools, and best practice system design.
Managing Cloud Cost Control
- VentureBeat Report: 81% of IT teams directed to reduce or halt cloud spending by C-suite
- TechTarget: Cost savings among the biggest cloud fallacies
- Information Week: Cloud Spending: How to Get a Grip on Cost Overruns
- WSJ: Pricey AI Apps Drive Up Cloud-Computing Spending
- ComputerWorld: This would be a good time to test your cloud ROI
The recent Yellowbrick LinkedIn Live discussed FinOps’ best practices and cloud data warehouse capabilities that assist in cost management. In conjunction with CloudOps, FinOps is tasked with operational cost management initiatives, but where these initiatives fail to deliver the required cost savings, it’s time to re-evaluate the underlying architecture – something that Yellowbrick would be only too happy to discuss.