By Insight Editor / 28 Feb 2022 / Topics: Microsoft Azure Analytics Cloud Migration
Facts at a glance
Client industry
Public sector
Challenge
Migrate from the existing Oracle database into a modern cloud platform to better support real-time and long-term reporting needs.
Solution
A modern data architecture and real-time reporting dashboard for up-to-date insights on emergency events, as well as status and location of equipment and response personnel
Insight provided:
- Automated resource deployment of Modern Data Warehouse (MDW) components
- Full implementation of Insight LENS — a metadata-driven data ingestion and processing framework
- Power BI reports based on MDW data
- Training of fire department staff on LENS and creation of Power BI reports
- Real-time ingestion framework based on Azure Event Hubs and Spark Structured Streaming
- Azure cost models for all deployed resources
Insight services:
- Consulting services
Outcomes:
Download the client story
- 2X faster emergency response times
- Improved safety and service for 4 million+ major metropolitan population
- Real-time data supporting 3,000+ uniformed personnel
- Optimized staging of vehicles and assets 15,000
- Simplified IT management across data systems
- Strategic foundation for future predictive analytics capabilities
Fire departments across the U.S. depend on data to protect the public. The ability to quickly identify the status of personnel and equipment ensures the effective direction of resources — and rapid response times in the event of an emergency.
Our client, one of the largest emergency services providers in the country, had been operating with a legacy data system and decided it was time to pivot. By modernizing its data environment and shifting to a cloud-based infrastructure, leaders hoped to better position the 135-year-old organization to address current and future challenges.
For many years, the department had relied on an Oracle® database to manage all incoming data and outbound reporting. This included information collected from emergency calls, first responders and GPS-tracked equipment. But a high volume of records, numbering in the hundreds of millions, created bandwidth and performance challenges.
And while they had the ability to generate basic reports, IT leaders realized that leveraging their system for broader analytics didn’t align to best practice. The organization decided it needed a modern, cloud-based data platform — one that would enable a unified view of all assets and events while supporting both real-time and long-term reporting needs.
With this partnership in place, an Insight architect and senior systems engineer began implementing a Proof of Concept (PoC) to establish critical features and feasibility for the project.
They worked closely with the client’s IT team to pull data from the legacy Oracle database and other peripheral systems into a centralized data lake. Azure® Data Factory was leveraged for ingestion, with Databricks used to support processing before landing the data in Delta Lake. The data was then written to Azure SQL, and a semantic model was developed through SQL Server® Analytics Services, allowing sample reports to be served up through Power BI®.
The PoC was successfully completed within four weeks. IT leaders immediately recognized the value of the unified reporting platform and the ability to bring in real-time data to support data science projects.
With the PoC approved, Insight’s team began implementing the full data modernization project at scale. They leveraged Insight’s LENS framework to automate significant portions of the infrastructure deployment, data cleansing and configuration processes. This dramatically fast-tracked project timelines, and the new data architecture was up and running in a matter of weeks — rather than months.
The modern data architecture greatly improved the way the fire department managed its data. Now leaders had additional goals to leverage their data in support of daily operations. They wanted to add a real-time channel to the reporting infrastructure to serve up critical information through a centralized dashboard, including:
The primary data sources for the dashboard included the city’s 911 system, as well as GPS trackers from fire department equipment and personnel. This data is reported at regular intervals to track activity and movement over time. The goal was to display this information on a map that would automatically update with locations and incidents. This view would need to be as accurate and up to date as possible to facilitate rapid decision-making on resource and equipment deployments.
Insight’s team leveraged Azure Event Hubs to ingest event data at rates of up to 100+/second. Databricks® analytics platform with Spark Structured Streaming was then used to stream events from the ingestion platform into Delta Lake. This served as the high-performant, big data ACID store to capture and store processed data, allowing it to be merged with previous records. Finally, Power BI was used for real-time visualization with automatic updates triggered directly by data changes in Delta Lake.
In just a few weeks, the PoC was complete and approved for full production. The full-scale solution was quickly delivered, providing a single-pane-of-glass view of citywide emergency response data.
With the modern data architecture and real-time reporting dashboard in place, the fire department now has greater visibility into current events and long-term trends.
Today, fire chiefs regularly rely on real-time information to optimize “move up” operations. As events appear on the map, nearby resources are quickly rerouted to respond to incidents in progress or prepare for potential future incidents. This has enabled improved staging of 15,000 vehicles and assets — and a dramatic 2X improvement in emergency response times across the city.
Furthermore, this system lays the groundwork for future predictive capabilities. With the incorporation of machine learning, the organization may soon be able to analyze historical trends and automatically identify preemptive placement of resources.
By building on its work with Insight, the fire department is not only equipped to better serve its community today, but also to transform the way it responds to challenges in the future.
Innovating is the only way to stay relevant in today’s uber-competitive market. Our unique approach and deep knowledge put you on the path to true innovation.
Discover reports, stories and industry trends to help you innovate for the future.
