Overcoming data challenges and harnessing the value of big data starts with building a reliable and accurate data foundation. Entity resolution is the process of connecting data, wherever it is, regardless of quality, into a single complete view – providing the foundation for better decision-making.
But this single view is only complete if it can scale across your entire enterprise, encompassing an ever-growing amount of data. A successful deployment of entity resolution must encompass as much data as possible – meaning it needs to scale indefinitely.
Existing Technologies Can’t Achieve Enterprise Scalability
Traditionally, there were two competing approaches to entity resolution:
- Batch-based systems: all entities are resolved across all data sources in parallel. This is useful for analytical model building (where a data scientist wants to build a model for all customers). However, the biggest pitfall with batch processing is when operationalizing these models, decisions are being made on outdated data.
- Real-time systems: These technologies allowed the entities to be kept up to date as new data arrived. This makes them suitable for streaming use cases and ensures they are always up to date. However, these architectures are optimized to process one record after another, making them extremely slow for full-scale batch runs.
So how can organizations benefit from the speed of batch entity resolution and the timeliness of real-time?
The Answer is Dual Architecture
To overcome this challenge, Quantexa built a dual architecture platform that allows for both batch and real-time processing. As a result, Quantexa is able to do the initial priming of a system faster than any competing technology, while its real-time capability means data is continuously up to date.
Quantexa is also able to scale linearly with data and infrastructure. This means if you use twice the amount of hardware, it’ll run for the half the time; if you double the amount of data and double the amount of hardware, it’ll run for the same time.
Many existing entity resolution technologies can not offer linear scalability with data and hardware – and those that do, operate in either batch or real-time.
The Best of Both Worlds: Quantexa’s Performance
To put the benefit of the Quantexa dual architecture in perspective, it is worth looking at a real-world comparison. Using an alternative real-time only engine, 1 billion records can be processed in roughly two weeks, with a cost of $30,000 on hardware. By comparison, Quantexa’s platform can process 1.5 billion records (50% more) in under 3.5 hours (100x faster), costing only $183 (160x cheaper). Fundamentally, this is because a real-time engine is being asked to do something it is not suited to: batch-based processing.
When comparing market leading batch-based entity resolution engines, Quantexa is able to process records 4.5x faster which generally means an equivalent reduction in cost of infrastructure.
Dynamic: The Next Evolution of Entity Resolution
One of the biggest challenges with traditional technology is there is of an assumption that it is possible to create a single view for all use cases across the enterprise. Unfortunately, this assumption is invalid for two reasons:
- Different requirements on match confidence
The single view that is required for a Master Data Management (MDM) use case is stricter than one required for financial crime. This is because in MDM, there is zero tolerance for over-matching, whereas in financial crime the focus is on ensuring you do not miss risk. - Data security/compliance
Each use case will have different requirements around what data can be used. For example, a fraud watchlist cannot be used in marketing, and an individual who has enacted their GDPR “right to be forgotten” must not be used in marketing but can be used in financial crime detection.
The result of this means traditional technology ends up being deployed multiple times for each use case, resulting in multiplication of hardware, run time and cost.
Quantexa’s Dynamic Entity Resolution technology solves these problems. Rather than keeping an existing single view of an entity up-to-date, Dynamic Entity Resolution regenerates the entity in real-time from the underlying raw data. This unique capability is fundamental to scaling entity resolution across your enterprise and allows a single instance of the platform to serve all of the use cases across your organization.
You may be interested in…

Why You Need A Holistic View of Integrity Risks Within The Supply Chain
Businesses that support organizations through their supply chain face a growing number of risks. Learn how Decision Intelligence can help.

How to Achieve Growth & Manage Risk with Real-Time Customer Insights
Discover how forward-thinking organizations are deploying new technologies to create a dynamic, contextual understanding of their customers.

The Quantexa Community: Where Customers and Partners Can Learn, Share and Collaborate
Find out more about The Quantexa Community, a global professional network built to bring Quantexa users together.

Quantexa Opens State-of-the-Art Technology & Analytics Hub in Malaga
Quantexa has opened the doors to its Technology & Analytics Hub, set in the very heart of Malaga’s thriving Tech Park. Learn more today.

It’s Time To Upgrade Your Early Warning Signal Systems – Here’s How
Find out why risk managers are turning to CDI technology to provide more accurate early warning signals that anticipate risk changes earlier.

How to Build Additional Context into Your Machine Learning Algorithm
Learn how building additional context into your machine learning algorithm can help your organization detect risks at speed.
Related Solutions

Tax Authorities
Reduce the tax gap, identify fraud and non-compliance, and operate as efficiently as possible with limited resources.

Anti-money laundering
Reveal hidden risks and detect criminal activity faster. Reduce false positives to manage the cost of compliance. And improve investigations to make faster and more consistent decisions at scale.

Customs Agencies & Border Control
Contextual Decision Intelligence enables faster decisions, increased revenue collection and enhanced compliance. The Quantexa platform enables Customs and Border agency teams to analyze data successfully, automate and accelerate decision-making, and achieve improved results.

Fraud
Identify potentially fraudulent activity by looking at people or transactions in isolation. Understand the context surrounding the organizations you do business with to make fast, accurate decisions.

Fraud, Waste & Abuse
Empower your team with the best tools available for today’s challenges to identify and prevent fraud, waste and abuse with contextual decision intelligence software.

Credit Risk
Understand your customers, their business structures and supply chains. Make better lending decisions, faster. And support digital risk transformation.

Customer Intelligence
Generate a complete view of the context around your customers and prospects to build better relationships, reduce attrition and find hidden opportunities.

Revolutionize Your Financial Crime and Fraud Detection

Investigations
Enhance the efficiency, effectiveness and consistency of your operational and complex investigations to empower your teams to expose and understand risk faster.

Master Data Management
Connect all data—internal and third party—to create a joined-up, contextual view of all the relationships between your customers and every other domain.

Compliance
See how we help to reduce costs and improve coverage for financial crime compliance.

CDO
See how our platform uses contextual analysis to turn data into a high value asset.

CIO
See how our platform uses financial crime technology to enhance your existing IT ecosystem.

Healthcare
Reduce the tax gap, identify fraud and non-compliance, and operate as efficiently as possible with limited resources.

Contextual Monitoring
Reveal hidden risks and detect criminal activity faster. Reduce false positives to manage the cost of compliance. And improve investigations to make faster and more consistent decisions at scale.

Unified CRM Solution

Know Your Customer
Reduce significant manual effort across onboarding, refreshes and remediation. Automate checks, implement continuous monitoring, and focus on contextual decision making.

Growth and Retention

Contextual Engagement
Generate a complete view of the context around your customers and prospects to build better relationships, reduce attrition and find hidden opportunities.

Data Management
Connect all data—internal and third party—to create a joined-up, contextual view of all the relationships between your customers and every other domain.

Connected Customer View
Generate a complete view of the context around your customers and prospects to build better relationships, reduce attrition and find hidden opportunities.