. Data Analytics and Big Data Management - EWSOLUTIONS

Teaching Data Management Since 1998

Data Management University

Request a free consultation with a DMU Expert

Search DMU Library


2 - 3 Minute Data Management Videos

Data Analytics and Big Data Management

10 October, 2018 | Martin Frappolli, CPCU, FIDM, AIC | Data Management

Data and business analytics, blockchain, data science, big data management, and other trends in data management and technology will enable efficiencies in risk management and insurance for many years.


In Part One of this series, we spoke of how the great volume of new data types, such as data generated by the Internet of Things, will be a catalyst for improved risk management, loss prevention, and insurance. Part Two of this series focused on new technology for storing the waves of external data, such as the cloud and Blockchain.

So, we know that there is a virtually unlimited volume of new data types generated by the Internet of Things, and we’ve taken a look at both traditional and newer, evolving technologies that can store, process, and share that data. One thing has not changed, though, from the very first days of data processing – the need to convert data into information.

The data alone is just a huge pile of ones and zeroes. Data managers, data scientists, and other talented analysts need efficient and effective ways to store and process the data on its journey to becoming useful information.

We’re now at the early stages of a shift that will expand the traditional methods of data generation, data storage, and data analysis. Those three factors are the drivers of the digitization of risk management and insurance in a way that will reduce friction, reduce costs, improve safety, reduce accidents, and streamline insurance contracts and claim settlements.

To know where we might be going with modern advanced data analytics, it is useful to look back and see how risk managers and insurance organizations have managed data historically. Actuaries have existed long before computers. These skilled professionals examined the costs of losses incurred on covered exposures. Those loss costs were projected into the future through the application of factors that attempted to account for future inflation, updated policy language, and evolving court decisions. Actuarial science has been defined as a “discipline that uses mathematics, statistics, and financial theory to measure, manage, and mitigate financial risk and uncertainty.”

There is a tremendous volume of new data, much of it coming from external sources like the Internet of Things. We have new ways to store and process that data, like the cloud and blockchain. Can we still rely on actuarial analysis to lead us to mitigate financial risk and uncertainty? Following are three broad areas of skill that will be essential to manage all this data.

Figure 1: Data Science Venn diagram

Here we see a simplified version of Drew Conway’s Venn diagram on data science. The traditional analysis handled by actuaries required risk management subject matter expertise (domain knowledge) and a mastery of mathematical and statistical skills. As we noted, actuaries practiced their craft long before computers made the job faster and easier.

Just a few decades ago, most of the number crunching was performed on centralized mainframe computers, with the ability to store much structured data in specific files or databases. The era of the desktop computer and decentralized processing brought with it applications and programming languages for more expeditious and powerful data analysis. Many actuaries began to acquire some of that IT skill. At the same time, risk management organizations began waking up to the value in their data, and to the need for connections among those data sets. The Data Manager / Data Analyst, once considered the keeper of a lowly backroom operation, was elevated (and rightly so) to a profession of profound importance.

While the data manager needed to possess the same domain knowledge as the actuary, the other essential skillset was in the IT/data arena. The data manager needed to know, at a minimum, how to translate data needs into technology applications. As much as any person in the insurance and risk management business, the data manager needed to speak both IT and the language of the business.

So the actuary has two overlapping areas of expertise: domain knowledge and math skill. The data manager has two overlapping areas of expertise: domain knowledge and IT skill. Working together, they have all three spheres covered. However, the relentless wave of new data, and the recognition of its value, has given rise to a new professional charged with making sense of it all: the data scientist.

Some have argued that data scientists sometimes possess only two of these skill areas: IT and math. They can acquire a lot of education in those two areas, but they rarely arrive on the job with the same kind of domain knowledge that the actuary or data manager / data analyst has.

Actually, the real sweet spot is the intersection of all three spheres, and the most valuable data scientists possess domain knowledge, IT skills, and math skills. Right now, insurers that are fully leveraging the waves of data with teams of data scientists are finding pockets of profitability that the traditional data analysis would never discover. Michael W. Elliott, CPCU, author of Big Data Analytics for Risk and Insurance says that data science is “an interdisciplinary field involving the design and use of techniques to process very large amounts of data from a variety of sources and to provide knowledge based on the data.”

Will the data scientist replace the actuary? That may be an academic argument. If a newly schooled data scientist makes the effort to acquire domain knowledge in his or her field, or the traditional actuary acquires the IT skills to perform modern data analysis, aren’t they doing the same job regardless of the hat they are wearing? The CAS Institute is fully aware of the expanding skills that actuaries will need in the future, and has introduced a specialty credential in Certified Specialist in Predictive Analytics (CSPA).

The digitization of risk management and insurance can be compared to “Moneyball” in sports. As was described in the book and the movie, a low-budget baseball team was able to beat the better-financed teams because the successful team management analyzed all available data about players and their relevant statistics over their careers. The management team did not rely on hunches and intuition to develop winning lineups and design a plethora of scoring scenarios. The digitization of insurance means that more decisions can be data-driven, minimizing the hazard of bad guesses.


Big data is here and always expanding. Smart insurers embrace this information to mitigate risks, control costs, and more accurately project future loss costs and loss reserves. More directly, though, we must remember that insurance is simply one tool in the risk manager’s toolbox. Even better than efficiently paying for losses is avoiding the loss altogether, and data is key for that. The fully realized data scientist is going to be a highly-sought resource, because the combination of skills is both rare and essential to compete in the era of big data, cloud storage, and blockchain.



View Comments

Request a free consultation with a DMU Expert


View all podcasts

View Our Podcasts

DMU provides regular podcasts of our best webinars, expert speaking events and our 3 minute Data Management Moment teaching videos.

View Our Upcoming Webinars

Click here to signup for our upcoming expert webinars and to view our previous webinars

WordPress Image Lightbox