How TfNSW Lowered Their Data Lake Costs by 22% with CloudMonitor

Transport for NSW - Government Enterprise Customer Success Story

Transport for NSW (TfNSW) - Australia Government Agency

Transport for NSW (TfNSW) manages one of the largest fleets of vehicles in Australia, including Buses, Ferries, Light Rail, Trains and Metro. The real-time data generated by these vehicles provides valuable analytical opportunities to improve transport services by measuring service performance and optimising routes, however, the sheer volume of data presents data management challenges.

Data-Driven introduced CloudMonitor to TfNSW in an effort to improve cost visibility and enable chargeback to the various project teams within the program.

“TfNSW needed a solution to capture real-time data for every vehicle in motion across the state. This solution just gives us that so that we mine nuggets from this data. As you can imagine, this has resulted in a lot of data, which is costly to store!”
Sandeep - TfNSW FinOps Government
Sandeep Mathur
Program Manager, Transport for NSW

Problem Statement

Historical operational data too large and costly to store efficiently and analyze

Historical GTFS data has always been too large and costly to store efficiently and analyse. Each TfNSW vehicle sends its location every 10 seconds which results in a huge stockpile of data containing valuable insights (over 5TB per week and growing).

In the past, this data was not stored which made insights from past vehicle trips almost impossible to obtain and ruled out the ability to predict trip delays or optimise trip routes. It also made it difficult to report on and analyse service performance and reduced the ability to improve customer services.

This scenario underscores the essential role of FinOps in balancing the operational need for continuous data with the financial imperatives of cost and efficiency, framing it not merely as a data storage issue but as an integrated challenge that intersects with several key domains within the FinOps practice.

Key FinOps Challenges

56 Bus at Martime Museum 2 scaled 1 • Cost Optimization

The FinOps Solution

CloudMonitor focused on "low-hanging fruit" and identified zombie resources and orphaned services.

The consultant conducted audits using CloudMonitor to find and remove orphaned resources — those resources that were no longer in use but still incurring costs. By systematically identifying and deleting these resources, they prevented unnecessary expenses.

It recommended right-sizing virtual machines and services based on actual demand.  Additionally, it advised on the purchase of reserved instances, which offered notable discounts compared to pay-as-you-go options. This was done for Storage and Databricks compute.

The Program Manager was given the CloudMonitor Reporting suite to be able to subscribe to monthly Budget versus Forecasting reports with real-time cost anomaly alerting via the CloudMonitor Teams bot.

Results for the Business

A 22% Reduction in the Data Lake Azure Spend

TfNSW achieved a significant transformation by integrating FinOps principles into their data management practice. The impact of this alignment was profound, resulting in a 22% reduction in Data Lake costs, enabling more efficient data storage and analysis while improving transport customer services.

Right-sized Resources​

Recommendations for over-provisioned resources ensure the system cost is minimized.

Cost Anomaly Detection

Real-Time cost anomaly detection catches cost spikes quicker, which means less bill shock.

Improved Cost Governance

The CloudMonitor reports allow the whole project team to subscribe to cost reports.

Related FinOps Domains

Arraya found that Customers first and foremost wanted to Understanding Cloud Usage and Cost.

As Azure expenses started to rise, the need for efficient cloud resource management became crucial. The goal was to match the real workload requirements with the cloud services used, ensuring that only appropriate resources were accessed when needed, to best serve the business. This highlighted the significance of Cloud Usage Optimisation.

Reservation/RI Recommendations are part of the Cloud Rate Optimisation domain.

More Success Stories

How Clinic to Cloud lowered Azure costs by 46% with CloudMonitor | Customer Success Story

About Transport for NSW

Transport for NSW is a government-run enterprise responsible for the delivery and development of safe, integrated and efficient transport systems for the people of NSW; including transport planning, strategy, policy, procurement and other non-service delivery functions across all modes of transport: Buses, Ferries, Light Rail, Trains and Metro.

NSW Transport works hand-in-hand with operating agencies, private operators and industry partners to deliver customer-focused services and projects in order to make NSW a better place to live, work and visit.

Learn more about Transport for NSW

Download CloudMonitor and find out how much you can save on your cloud costs

CloudMonitor logo - Azure Cost Optimization and Cost Management
Connect Power BI Reporting adn Analytics Engine

Subscribed! We'll let you know when we have new blogs and events...