Case Study · Public sector · Government

How Transport for NSW lowered data-lake OpEx by 22%.

The TfNSW data platform serves real-time transport data to millions of journeys per day. CloudMonitor cut its operational cost while protecting performance.

Demo

Talk to Sales →

About the customer

Transport for NSW

The state government agency that runs public transport across New South Wales — buses, trains, ferries, light rail and metro — serving millions of journeys every day.

Industry: Government
Headquarters: Sydney, NSW, Australia
Company size: 10,000+
Website: transport.nsw.gov.au

Results at a glance

Transport for NSW · by the numbers.

22%

Data Lake Azure spend reduction

5 TB+

Vehicle telemetry per week

10 sec

Vehicle location ping interval

The Problem Statement

Historical operational data too large and costly to store efficiently and analyse.

Historical GTFS data has always been too large and costly to store efficiently and analyse. Each TfNSW vehicle sends its location every 10 seconds which results in a huge stockpile of data containing valuable insights (over 5 TB per week and growing).

In the past, this data was not stored which made insights from past vehicle trips almost impossible to obtain and ruled out the ability to predict trip delays or optimise trip routes. It also made it difficult to report on and analyse service performance and reduced the ability to improve customer services.

This scenario underscores the essential role of FinOps in balancing the operational need for continuous data with the financial imperatives of cost and efficiency, framing it not merely as a data storage issue but as an integrated challenge that intersects with several key domains within the FinOps practice.

Before CloudMonitor

The Key FinOps Challenges

Four practical blockers were making cost decisions slow and painful on the data platform.

Immature tagging policy which meant it was hard to allocate costs.
The business had no tooling with which to view platform spend.
IT would need to be involved to calculate and distribute cost reports.
Many zombied and orphaned resources in DEV and UAT with no known owners.

The FinOps Solution

CloudMonitor focused on "low-hanging fruit" and identified zombie resources and orphaned services.

The consultant conducted audits using CloudMonitor to find and remove orphaned resources — those resources that were no longer in use but still incurring costs. By systematically identifying and deleting these resources, they prevented unnecessary expenses.

It recommended right-sizing virtual machines and services based on actual demand. Additionally, it advised on the purchase of reserved instances, which offered notable discounts compared to pay-as-you-go options. This was done for Storage and Databricks compute.

The Program Manager was given the CloudMonitor Reporting suite to be able to subscribe to monthly Budget versus Forecasting reports with real-time cost anomaly alerting via the CloudMonitor Teams bot.

The Results in the Business

A 22% reduction in the Data Lake Azure spend.

TfNSW achieved a significant transformation by integrating FinOps principles into their data management practice. The impact of this alignment was profound, resulting in a 22% reduction in Data Lake costs, enabling more efficient data storage and analysis while improving transport customer services.

Key results

Right-sized resources

Recommendations for over-provisioned resources ensure the system cost is minimised.

Cost anomaly detection

Real-time cost anomaly detection catches cost spikes quicker, which means less bill shock.

Improved cost governance

The CloudMonitor reports allow the whole project team to subscribe to cost reports.

Related FinOps Domains

Transport for NSW found that customers first and foremost wanted to Understand Cloud Usage and Cost.

As Azure expenses started to rise, the need for efficient cloud resource management became crucial. The goal was to match the real workload requirements with the cloud services used, ensuring that only appropriate resources were accessed when needed, to best serve the business. This highlighted the significance of Cloud Usage Optimisation.

Reservation/RI Recommendations are part of the Cloud Rate Optimisation domain.

TfNSW needed a solution to capture real-time data for every vehicle in motion across the state. This solution just gives us that so that we mine nuggets from this data. As you can imagine, this has resulted in a lot of data, which is costly to store!