Desilo, detangle, deduplicate: A better way to solve a complex data problem.
A comprehensive guide to overcoming your company’s trickiest data record challenges—through contextual Master Data Management.
The Challenge You Face
After decades of data collection, your company’s volume of data will have grown exponentially. But most of this data is collected by multiple applications, which often results in duplicate entries in different (siloed) repositories.
That makes it difficult to get an accurate, timely view of the data your organization relies on to operate—whether for customer service, information management or decision analytics.
And the result?
Internal problems
- Your records are incomplete and may include outdated, inaccurate information
- Your data teams are overloaded with mounting manual remediation tasks
- Revenue opportunities are missed
- Potential risks are overlooked
Customer problems
- Services and products are offered to customers who already have them
- Customers are frustrated at repeating details and information to different customer service agents operating in different data silos
- Customer experience takes a nosedive due to inaccurate, missing or misrepresented information
Regulatory problems
- An increasing number of regulations require reliable, consistent and well-managed data
- Regulators receive information submissions that are inconsistent and do not tie together
- Inspections demand ever-higher standards for data quality and lineage
- Enforcement actions involve fines and restrictions on business activities, affecting P&L and reputation
The fact is this: most organizations have the data, but not the technology needed to manage master data and harness its value.
Without a way to deduplicate records and accurately chart data, individual entities—like customers, vendors and related parties—can’t be properly understood and analyzed.
To truly make the most of your data, you need to be able to:
Consolidate and deduplicate it
Resolve discrepancies
Have full visibility —in context of related data
Master Data Management allows companies to tackle this issue. But poorly implemented or limited solutions are often ineffective at best and harmful at worst.
Read on to learn more about Master Data Management, its potential shortfalls, how you can overcome them with best practice – and the pay-off when you get it right.
Contents
Introduction: The Challenge You Face
Companies everywhere are facing a huge data challenge—and it’s only getting bigger.
01. What Is Master Data—And Why Does It Need Managing?
What is Master Data Management, exactly? And why is it so important?
02. The Inherent Problems of MDM
A closer look at the shortcomings of traditional Master Data Management and what it could mean for your organization.
03. Building a Single, Golden Point of Truth
How entity resolution helps to add context back to your master data, so you can build an accurate single entity view populated by rich, insightful golden records.
04. The Benefits of Contextual Master Data Management
A deep dive into what you can achieve when you implement best practices and get Master Data Management right.
05. Get Started
A few next steps, for companies looking to get their Master Data Management journey started on the right foot.
CHAPTER 01
What Is Master Data—And Why Does It Need Managing?
Master data is a record of entities in the real world that are important to your business. It’s stable, enduring and heavily-used.
Master data is a record of information about:
Parties, such as: customers, suppliers, associated parties, citizens
Places, such as: addresses, locations
Things, such as: products, assets, vehicles
Master data underpins and enables a whole enterprise, and is referenced by every other piece of data—such as transactions, trades, contracts, orders, deliveries and repairs.
Which is why it’s a big problem when master data is incorrect or ambiguous.
The master data vs real world entities gap
In the real world, each thing is a single ‘entity’. It could be a specific person, organization, address, phone number, bank account or device. But, online, each entity can—and often does—exist in multiple formats and records. Different sources may have different pictures of the same entity.
For an organization that wants to be able to have a “single version of the truth” to refer to these entities, this can be problematic. To be able to trust your master data, you’d ideally have just one record for each real world entity.
That’s what Master Data Management (MDM) aims to solve.
However, even the aim of “single version of the truth” can be challenging – since there are often different ways of looking at real world entities. For instance, an organization can simultaneously be seen as a brand, a collection of legal structures and a group of people. There are multiple views on real world entities that can seem conflicting but are equally valid based on your perspective.
Master Data Management: The process of managing critical data
There are many definitions for MDM; but, put simply, MDM refers to how an organization ensures its shared master data is consistent and accurate. MDM covers how companies govern, manage, and use their data to create true-to-life master data records.
When people talk about MDM, they could be referring to the discipline or the technology solutions enabling it.
As a discipline, the principles of MDM center on good data governance, with the goal of creating a trusted and authoritative view of a company’s data.
As a technology, MDM solutions integrate, reconcile and resolve data, bringing together information from multiple sources to create single-view master data records that can be referred to whenever needed.
Master data records are an organization’s go-to reference for the truth about entities. As a result, they need to be trustworthy and as accurate as possible. That’s why strict rules around data governance need to be enforced in regards to it. Master data should be governed, quality-managed and made available for operational and analytical use.
The difference between master records and golden records
People often refer to the ‘golden record’ when they talk about Master Data Management. But there can be confusion about where that fits into the picture when there is just as much reference to master records. So, what’s the difference?
In short, there is none. Both refer to the same thing: a trusted view of an entity, created by linking multiple data sources. A master—or golden—record gives you access to the most pure, validated and complete picture of single entities.
The fields included on master records differ by organization and need. Companies may choose to have as many fields as they like; but the fewer the fields, the easier the records will be to maintain.
Why is Master Data Management so important?
Information discrepancy is a big problem, especially in large organizations. There’s now more data—collected in more places—than ever before. But because data changes over time and is collected multiple times in different places, it’s easy for it to get out of sync and become fragmented, inaccurate and inconsistent.
The result is that companies are having to find ways to keep up with an evolving stream of data sources; but the solutions they’re using just aren’t up to the task. And that’s a big problem for larger companies with millions of records, who have had to retroactively adapt their systems to support digital formats.
When master data records are wrong:
- Your view of customers is corrupted
- You cannot trust your management reports
- Business decisions are made based on partial information
- Things get sent to the wrong places
- Different customers get mixed up in billing
- Customer experience takes a nosedive
- Big opportunities are missed
How do Master Data Management solutions work?
MDM solutions ingest data from multiple sources and use it to update and improve existing master data records.
The Simple Matching Method
In traditional MDM solutions, this is done through record-to-record matching, where the attributes of two records are compared and contrasted to find whether they match.
If it’s determined that two (or more) records match, then values are picked based on survivorship rules—which discards all values apart from one to create a golden record.
We’ve pioneered a slightly different way of doing it. You can skip below to find out more about Quantexa’s powerful, more accurate method of using entity resolution to deliver contextual MDM. Or, read on to find out why traditional MDM falls short.
CHAPTER 02
The Inherent Problems of MDM
The aim to have one master record for each real world entity has always been hard to achieve—and it’s becoming even harder.
Companies are contending with:
- Multiple internal applications—many of which will contain different versions of the same master data record
- Numerous external data sources that provide additional—sometimes contradicting—information about companies and individuals
Bringing together all these views of master data is incredibly difficult, because of:
The data quality problem
Traditional MDM has an inherent data quality problem. And it affects your decision making, regulatory compliance, business effectiveness and efficiency.
Traditional MDM solutions don’t focus on solving data quality issues. In fact, they tend to fail at the first hurdle of matching data—because they struggle to join data from across disparate data sources such as multiple internal applications.
When you add the volume and variety of data from external sources it’s even further beyond their capability. Instead, they use roughly the same matching algorithms they’ve used for the past 20+ years, which relies on record-to-record comparison that is very fragile when key attributes are missing or different.
And that’s a problem. Because crucial information goes unreported when your MDM solution can’t catch essential links between data, and obscure relationships and connections are often overlooked.
It also makes your MDM implementation very high risk.
Bad data quality means:
- Data remains trapped in silos and is duplicated across channels
- Master records aren’t accurate and true-to-life
- Bad decisions are made, due to the fact they’re based off incorrect or delayed data
- Key information and critical links go unnoticed
- Opportunities are missed
The transformation challenge
MDM is both a business and a technology transformation challenge.
Here’s how it usually goes.
You start out with multiple users updating the master records in separate applications, all with their own ways of working. Inevitably, it’s messy, it’s haphazard and it results in duplicate records, inconsistencies and confusion.
So, to combat this, you decide to standardize things. Everyone is to use just one application, with a standardized set of rules on how to input data, how data should be formatted, what records look like and more.
It’s a good idea—in theory. But the problem is that the real world rarely plays out quite so neatly.
So what you end up with is this:
- A logistical nightmare, as you try to migrate decades of data and make it conform to your ideal record format.
- Confused and frustrated users who need to be transitioned to the new service—with all the training and business change support that entails.
- Backwards-compatibility issues, as information moves to new places and takes on different—unrecognizable—formats, meaning users and business applications can no longer find information.
- Challenges updating future records. If your records only track A, B and C, what happens when users later need to add D and E? Is it updated across the entire data store? And what happens to data that is initially discarded for non-conformity, but is later needed?
It’s a big job. If you don’t have a plan for data migration, then your initiative is doomed. And even the best, most precisely executed migration plan can’t save you if the tools you rely on aren’t up for the task.
The varying needs of different data consumers
For an MDM initiative to be considered successful it needs to be able to serve data to consumers across the organization. For instance:
-
Analytics teams who need to link data sources for decision intelligence
-
Fraud monitoring applications
-
Customer services or relationship managers who need rich views of their customers
-
Finance teams who need to aggregate risk reporting
Different business units often have different views on the data they require. So, when an MDM initiative attempts to standardize master data attributes across the organization, it may mean dropping attributes that these data consumers rely on—which, naturally, can result in tension and impact business performance.
MDM needs to be able to present a rich and deep view of data to areas of the organization that consume it, while on the journey to standardizing key attributes. But that often runs counter to the way a lot of existing MDM software works, which relies on a fixed view of data that needs to be adhered to from day one.
The need for governance and control
In large organizations there are often many applications that hold customer data—each controlled by different business units. And that means a wide range of stakeholders with different priorities.
Implementing traditional MDM often requires each business unit to give up control of their applications and data to a central initiative—which can result in a great deal of pushback and agitation.
To successfully implement an MDM initiative, you’ll need to be ready to address the political challenges that come with it. A lot of that relies on bringing people together around a vision of a service that will benefit them, within a transformation program that can actually deliver.
CHAPTER 03
Building a Single, Golden Point of Truth
The key to resolving the traditional MDM data quality issue lies in powerful entity resolution that retains context and doesn’t force data into a standardized format.
We call this contextual MDM.
How contextual MDM stands apart
Originally built to tackle financial crime, contextual MDM (cMDM) ingests data from both internal and external sources to build an accurate, connected and enriched single-entity view using entity resolution and network generation technology.
This is different from traditional MDM solutions, which rely on record-to-record matching—a method that does not work well on disparate records, as it relies on many attributes matching.
At Quantexa, we use an expanded range of data—including address, phone, email, country, and third party data—to make further connections and enrich your data. Which is why our solution can make connections between records even when data quality is poor.
Traditional MDM
Traditional matching does not work well on sparsely populated records—because it relies on many attributes matching.
Records can only be accurately linked if a number of fields match (for example, if two records have the same name, date of birth, and address on file).
The lack of additional contextual data makes deduplication difficult and leaves questions unanswered.
Traditional MDM also relies on you to set rules.
If the rules are too rigid, records will be under-linked (meaning duplication is more likely to go uncaught).
If the rules are too loose, records will be over-linked (meaning different records are more likely to be mistakenly deduplicated, even when the entities in question are different).
Quantexa cMDM
With our entity resolution software, connections can be made intelligently across records.
Using additional fields and an expanded range of records from any number of internal and/or external sources, our software makes it possible to accurately determine when multiple records exist of a single entity—and to turn duplicated records into a single, enriched entity view.
Resolved entities can also be seen in context with their networks—so you can see how different entities relate to each other.
With cMDM, data across different records is iteratively updated to enrich all sources, leading to better match rates and higher quality records data.
CHAPTER 04
The Benefits of Contextual Master Data Management
With contextual MDM, you gain:
A single, complete view of connected data
One trusted source of truth across all systems and resources
A foundation for trusted data
Enrich and improve quality of your master data records
Flexible and open architecture
With open APIs and original-format records, joined-up records can be easily consumed and integrated
Low risk implementation
Without the need to transform data, it’s quick and easy to deploy
The power to make better decisions
Information is accurate and updated in real-time, offering more visibility and greater understanding
Consumer oriented views of master data
Different applications consuming data are able to connect data according to their needs
So you can
Create and update accurate master records, in real time
Spot hidden risks and identify high-value growth opportunities
Share essential data among your teams
Offer frictionless digital-first experiences for your customers
Develop good data practices and upkeep organizational data hygiene
Scale your business easily
CHAPTER 05
Get started
Find out more about contextual MDM in our ebook
Or, if you’re ready to take a closer look, why not book a demo