Quantexa

What is Entity Resolution and How Does It Transform Data Into Value?

Your essential guide to Entity Resolution: what it is, how it works, and why your organization needs it.

Quantexa
Quantexa
Jul 19th, 2023
15 min read

What is Entity Resolution?

Entity Resolution is the process of working out whether multiple records are referencing the same real-world thing, such as a person, organization, address, phone number, bank account, or device. Entity Resolution takes multiple disparate data points — from external and internal sources — and resolves them into one distinct, unique entity.

Every decision your organization makes relies on accurate and complete data.

And while we have access to more data than ever before, connecting today’s infinite data points and turning them into actionable, valuable insights presents a considerable challenge.

This essential guide will tell you how to overcome some common data challenges, why many current approaches to data matching no longer work, and which capabilities are the most important to look for when investing in data software.

You’ve got all the data. You just need the right technology to harness its value. Read on to learn how to effectively use Entity Resolution to transform your organization.

Imagine you have a customer named John Citizen, and a corporate data record for ABC Inc. that lists a director named Jonathan Citizen. Are they the same person?

If you can’t answer that basic question, how can you make an accurate decision about the risks or opportunities associated with John Citizen?

Enter Entity Resolution

Entity Resolution is the best way to connect billions of data points spread across multiple systems into a single, accurate view.

It creates a complete, meaningful view of data across an enterprise that reflects the real-world connections between people, places, and organizations.

It also builds a contextual data foundation that enables you to enhance decision-making across the customer lifestyle, uncover hidden risks, and discover unexpected opportunities.

CHAPTER 1

Solve your biggest data challenges

You have billions of data points spread across multiple systems — but without the right technology to harness them, you can’t create value.

Data should yield opportunities — but for many organizations, more data means more problems. From duplicate customer records to never-ending data science projects, there are numerous challenges. Trying to overcome these challenges is difficult: Connecting disparate datasets into a comprehensive single view of data is time-sensitive and labor-intensive, and brings a maze of security issues.

Without a reliable data foundation, it’s impossible to build a 360-degree view of customers, prospects, partners, and organizations. And without this complete view, there’s no way to turn this data into insights.

Fewer than a quarter (22%) of decision makers believe their entire organization trusts the accuracy of available data.

Why is connecting data so difficult?

Block img

Siloed data
Data is trapped in silos across internal and external systems, resulting in teams repeating work and customers receiving a disjointed experience. Siloed data makes it impossible to see a full picture — which leads to inaccurate decision-making.

Block img

Varying requirements
Each unique user or use case requires specific data sources and has different requirements about how fuzzy or strict the matching may be. Sensitivity of sources (in financial crime use cases, for instance) complicates these requirements further. Multiple views are necessary for different requirements.

Block img

Data quality challenges
Poor data quality results from duplicate entities, missing information, and intentional manipulation by criminals. Traditional record-to-record matching struggles to deal with sparse, inaccurate data and is ill-suited to natural variations in data sources.

Turning volumes of data into commercially-useful, business-oriented, valuable insights is a significant challenge.

But it’s one that Entity Resolution can help resolve. Here’s how.

CHAPTER 2

How does Entity Resolution work?

Turning records into reality

This animation shows three unique customer records in an organization’s database: John Smith, John Citizen, and J. Citizen. Using Entity Resolution, the organization can connect these records to gain a single view of the data, showing that John Citizen and J. Citizen are actually the same person.

Entity Resolution overcomes variations in addresses, phone numbers, and company name records to match them — and then draws on these attributes to merge John Citizen’s and J. Citizen’s customer records into a single entity. The process also reveals a connection between John Citizen and John Smith that wasn’t evident before.

Internal versus external data

Entity Resolution brings together data from a number of different sources — both internal and external.

Importing data on a project-by-project basis isn’t efficient. Instead, use a central integration point to make integrated internal and external data available to all decision-makers.

Internal data
Internal data is any information that your organization creates internally and manages, 
such as customers, transactions, products, 
or communications.
External data
External data is any data amassed outside of your organization, such as corporate registry sources or external watchlists.
CHAPTER 3

Why use an Entity Resolution tool?

Current approaches to data matching

With ever-expanding volumes of data, a lack of consistent data quality, and a wide range of use cases, traditional approaches just aren’t good enough.

Traditional data matching

  • Compares data records directly using a paired matching approach

  • Relies on many attributes matching to produce a match score that’s high enough

  • Struggles with sparsely populated records or those where there is variance in the information

Entity Resolution

  • Uses an iterative matching approach that continually enriches records with additional data to provide the most accurate view possible

  • Makes connections between data even when quality is low or where there has been manipulation

How Entity Resolution differs from traditional matching

Traditional matching

  • Compares data records directly using a paired matching approach

  • Relies on many attributes matching to produce a match score that’s high enough

  • Struggles with sparsely populated records or those where there is variance in the information

Entity Resolution

  • Uses an iterative matching approach that continually enriches records with additional data to provide the most accurate view possible

  • Makes connections between data even when quality is low 
or where there has been manipulation


CHAPTER 4

The Entity Resolution imperatives

There are a few principles to follow when you look for an Entity Resolution tool. Some may be more important for your organization’s needs than others — but having all seven will give you a solid foundation.

The best Entity Resolution tools resolve entities in the same way a human would but automatically, at massive scale, and with a full understanding of the data. They don't need to be programmed to know how common a name is or how large a business is. Instead, they mine this information from the data itself.


Accuracy

Above all, your Entity Resolution tool must be accurate. Look for independent validation, client testimonials and proven metrics to ensure it meets the highest accuracy standards. Look for a solution that’s been proven in the fraud and financial crime space, as these are built to overcome challenges like intentionally manipulated or poor-quality data


Transparency

With a trusted foundation of data, you can do everything from improving operational agility to automating decision-making. But you need to understand and trust how your system works. Choose an Entity Resolution tool that’s white-box by design, ensuring that the underpinning logic is accessible, transparent, and explainable.


Real-time and batch ingestion

Batch ingestion enables large scale resolution for data science use cases, while real-time ensures you’re always getting the most up-to-date and accurate view possible. To ensure you get the best of both worlds when it comes to data processing, choose a tool that offers both real-time and batch.


Granular security

Security is one of the main reasons why Entity Resolution tools are deployed within specific areas of the business rather than across the entire enterprise. Look for software that supports dynamic processing, which resolves entities based on the data each use case requires and ensures the user has the right to access.


Scalability

Entity Resolution is designed to bring all your data together into a single view, so it's crucial that your software doesn’t hit limits as you bring in more data. Look for a solution that's proven and in production at large Tier 1 organizations. Also, ensure it can scale linearly with hardware via a distributed architecture — otherwise, you'll end up with long batch times, delayed insights, and ugly interim processing workarounds.


Time to value

One of the main challenges with Entity Resolution is the time required to onboard new data. Some solutions require all data to be normalized into a standard schema, which wastes time. An Entity Resolution tool that is able to accept data in almost any format and contains out-of-the-box integrations to a wide range of trusted external data sources will give you a quick, accurate, complete view of every entity.


Use case flexibility

While many organizations use Entity Resolution for one initial use case, it's actually a foundational layer suited to many parts of the business — from faster customer onboarding, to sharper financial crime detection, to single view augmentation for MDM. As each use case has different requirements, it's vital to implement Entity Resolution software that can support different matching and data source requirements.

CHAPTER 5

Dynamic Entity Resolution: the next evolution

Real-time Entity Resolution may keep your single entities up to date, but it doesn’t have the flexibility to provide value across multiple use cases. Dynamic Entity Resolution is the next evolution of real-time Entity Resolution.

Rather than keeping an existing single view of an entity up to date, Dynamic Entity Resolution re­-generates the entity in real-time from the underlying raw data.

This lets the software dynamically include or exclude particular data points and allows users to specify the match confidence they require for their specific use case.

This unique capability allows you to deploy Entity Resolution across your entire organization for any use case.

The evolution of data matching

  1. Traditional matching
    Focuses on direct matching, which results in poor match quality

  2. Batch Entity Resolution
    Offers improved matching, producing a single-entity view across all data

  3. Real-time Entity Resolution
    Keeps the single view up to date as new data comes in

  4. Dynamic Entity Resolution
    Adds the ability to re-generate entities at the time of request from the underlying data

What makes Dynamic Entity Resolution different?

“One size fits all” is rarely a good thing — especially when it comes to Entity Resolution. Other systems assume that a single view of “John Citizen” can be utilized for every use case within your enterprise.

However, it can't. Here are two reasons why relying on that single view isn't sufficient:

  1. Different use cases require different levels of matching and have varying restrictions around different data sources. For example, if you use a highly sensitive source in the Entity Resolution process, the resulting entities will also be classified as highly sensitive-and therefore can't be used elsewhere. The capability to distinguish source sensitivity is critical if you have external data that's only been licensed for use within one part of the business.

  2. Multiple use cases lead to multiple instances. Other systems can support different use cases — but only by replicating data. This might seem like a reasonable approach, but as you start using Entity Resolution more broadly across your enterprise, replication quickly becomes unmanageable. Plus, when you add the matrix of data source permissions to the mix, a “multiple instances” approach quickly becomes unwieldy.

A Dynamic Entity Resolution tool builds on demand, at the time of request. 
This means you can:

  1. Deploy one instance of the platform to serve all the needs in your organization

  2. Specify the level of fuzziness required per use case, at the time of request

  3. Control access to data sources depending on what the user or use case can see

CHAPTER 6

Dynamic Entity Resolution: Benefits and business value

Dynamic Entity Resolution is the only way to create an enterprise-wide, trustworthy, resolved data foundation that can support multiple use cases. It helps you resolve a growing number of use cases in a rapid and secure way, harnessing and connecting all the data to which you have access.

The results speak for themselves. This isn't just about creating a single view across your enterprise — it's about improving efficiency, laying the foundations for accurate analytics, and scaling to your needs.

60x
Faster data resolution
99%
Entity Resolution accuracy
20%
Record deduplication

Dynamic Entity Resolution lets you:

  • Create a single, complete view of customers, prospects, and organizations across your business.

  • De-duplicate records across your business, giving you a single source of truth.

  • Improve the quality of your data and automatically fill in missing information.

  • Centralize access to both internal and external data throughout your organization.

  • Drive up productivity by providing access to accurate, consolidated information.

  • Create the foundation for data-driven decisions and automated/augmented decision-making.

Gain a single customer view across your enterprise to drive growth

Here’s a snapshot of what’s possible with a complete view of your customers and prospects:

CHAPTER 7

Empower AI-driven decision-making with Entity Resolution

Entity Resolution lets you connect and resolve tens of billions of internal and external data points in one place. This means your organization gets an enterprise-wide view of people, places, organizations, and events across supply chains, empowering you to make faster, more accurate decisions throughout the customer lifecycle.

With an accurate, trusted single view in place, you can:

  • Understand the connections and relationships between entities, and form networks of connected data.

  • Use both the entities and networks within analytical models to enable better decision-making across your enterprise.

With Decision Intelligence, your organization can:

  • Drive automation and deliver greater business value from enterprise data.

  • Make faster and more accurate decisions.

  • Spot hidden risks and identify high-value growth opportunities.

80%

of organizations believe data intelligence is key to decision-making

75%

of organizations believe data intelligence is tied to to financial performance