Key Takeaway
We’re only accessing about 20% of our available data on a regular basis. There is an unprecedented amount of information hiding in our dark, or currently unleveraged, external data. Today we finally have the tools to help us uncover those insights and shine a light on dark analytics to help us form a better, more predictive strategy.
According to research from Deloitte, the world’s data creation today is doubling in size every 12 months, and is expected to reach 44 zettabytes (44 trillion gigabytes) by 2020. At this point it will contain nearly as many digital bits as there are stars in the universe.
This unstructured data requires the help of advanced machine learning technology to extract anything meaningful. IBM indicates 80% of data is currently dark, and is expected to rise to 93% by 2020.
Today the vast majority of our data is what many consider “dark”. According to the Wall Street Journal, dark analytics includes exploration of “unstructured and hidden or undigested data” with a goal of unearthing “highly nuanced business, customer, and operational insights that structured data assets currently in their possession may not reveal.” But what is dark data and why should enterprise leaders be paying attention?
Going dark
Business intelligence tools have been in place for a number of years, helping decision makers sift through their own sales and financial data, supply chain numbers and internal data. As well, the importance of insights from social media and news data is increasingly taking center stage. But there are a number of other data points filled with forward-looking insights that are not yet being utilized.
In simple terms, dark data is the data out there you’re not yet analyzing, and the insights your business is missing out on.
Dark data comprises data you already have but you’re not tapping into, unstructured data that requires advanced technology to interpret, and data found within the deep web.
According to Deloitte, dark data extends into information that can be found “in the deep web and the dark web – which comprises everything online that is not indexed by search engines, including a small subset of anonymous, inaccessible sites known as the ‘dark web.’” In capturing insights from the deep web, you can cast a wider net to discover signals which may currently go untapped. But its sheer size and lack of structure can make it difficult to search, without the help of machine learning and AI tools that can help executives extract the real value.
How can an enterprise better leverage dark data?
According to Abhishek Budholiya, in a recent report from Future Market Insights, “Valuable insights from this data are gained by solving statistical inference problems at massive scale…Dark analytics assist in recognizing better unused opportunities mainly in sales and marketing processes by analyzing customer behavior insights.”
The key to utilizing this data effectively is having a clear intention and problem to be solved. Knowing what the question is will help you to better anticipate the sources from which you may find the answer.
One of the largest use cases for adoption of dark data is in the healthcare industry – from better leveraging historical public databases, to offering healthcare companies a better understanding of their consumers by evaluating external data, enabling them to predict their needs and behaviors.
Case study: dark data offers predictive indicators for the healthcare industry
According to EMC2 and IDC, healthcare data is growing at an annual rate of 48%, and is one of the fastest growing data segments in the world. Scientists are working to tap into this data, while healthtech companies are looking for new ways of implementing data insights to improve care, insurance and proper diagnoses.
The Deloitte team cited a case from Indiana University Health, who is looking to use nontraditional and unstructured data to personalize health care and improve overall health outcomes. “IU Health needs a 360-degree understanding of the patients it serves in order to create the kind of care and services that will keep them in the system,” says Richard Chadderton, senior vice president, engagement and strategy, IU Health.
Taken in aggregate, customer data – from within the healthcare system as well as habits and indicators taken from sources like social media, regional buying habits and income levels, news and political events, and other socioeconomic factors – can offer healthcare providers an unprecedented opportunity to predict needs and better manage healthcare both from a population perspective and at an individual level.
In this case, IU Health is using this information to examine how “cognitive computing, external data, and patient data could help identify patterns of illness, health care access, and historical outcomes in local populations.”
This sort of external information can be used to help leaders in industries across the board better understand their customers and their own efficiencies or shortcomings, better predict their competitors’ next moves and plan more effectively for the future.
Outside Insight was built to help business leaders better leverage external insights from their currently untapped data sources. Download a sample report for your industry here.