Microsoft Dynamics 365 Customer Experience Analyst : Configure duplicate detection
Duplicate detection in Dynamics 365 is a feature that helps maintain clean and accurate data by identifying and preventing duplicate records from being created. It works by using duplicate detection rules, where administrators define which fields should be compared (like email, phone number, or account name) to check for possible duplicates. When a user tries to create or update a record, Dynamics 365 evaluates it against existing records and displays a warning if a potential duplicate is found. This ensures data quality, avoids confusion in customer interactions, and supports better reporting and decision-making by keeping the Dataverse environment consistent and reliable.
Technically, duplicate detection in Dynamics 365 is a rules-based validation system built into Dataverse that enforces data quality, prevents redundant records, and supports reliable integrations.
Duplicate Detection Rules
- Configured at the entity (table) level in Dataverse.
- Rules define which fields to compare (e.g., Email, First Name + Last Name, Account Name).
- Support exact match, case sensitivity, and partial matching (depending on field type).
Entities Supported
- Works with most standard and custom entities.
- Commonly used for Leads, Contacts, Accounts, and Opportunities.
Execution Triggers
- Create/Update: Checks for duplicates when saving a new record or updating existing data.
- Import: Validates incoming data files during data import.
- Bulk Data Merge: Useful when merging or consolidating records.
System Jobs
- Duplicate detection runs as an asynchronous system job (can be monitored in System Jobs under Advanced Settings).
- Jobs can be scheduled for bulk detection across selected entities.
Security Model
- Only users with appropriate privileges (e.g., Manage Duplicate Detection Rules) can create or publish rules.
- Users without these privileges can still see duplicate detection alerts when interacting with records.
Limitations
- Performance considerations: Complex rules on large datasets may slow down record creation or imports.
- No fuzzy logic: Out of the box, it relies on exact matches; advanced matching requires custom plugins or third-party solutions.
Integration with Data Quality
- Ensures data integrity in Dataverse, reducing risks in workflows, Power Automate processes, and reporting.
- Helps prevent issues in integrations with other systems (e.g., marketing automation, ERP).
Steps to Configure Duplicate Detection
1. Enable Duplicate Detection (System Setting)
Before creating rules, duplicate detection must be enabled globally.
- Navigate to Settings → Data Management → Duplicate Detection Settings.
- Select the following options based on requirements:
- Enable duplicate detection.
- Check when a record is created or updated.
- Check during import.
- Check when Microsoft Dynamics 365 for Outlook goes from offline to online.
- Save settings.
This ensures the system will run duplicate checks wherever applicable.
2. Create a Duplicate Detection Rule
- Go to Advanced Settings → Data Management → Duplicate Detection Rules.
- Click New.
- Fill in the following fields:
- Name → e.g., "Duplicate Contact by Email".
- Entity Type → Choose the table (e.g., Contact, Account, Lead, or a custom table).
- Base Record Type and Matching Record Type → Usually the same (e.g., Contact vs Contact). You can also compare across entities (e.g., Lead vs Contact).
3. Define Rule Conditions (Fields to Compare)
- In the Duplicate Detection Rule Conditions section:
- Select Field → (e.g., Email, First Name, Last Name).
- Select Criteria → Exact Match, Same First Characters, etc.
- Add multiple conditions with AND/OR logic.
Example:
If Email (exact match) OR (First Name + Last Name exact match), then it’s considered a duplicate.
4. Publish the Rule
- After creating the rule, click Save and Close.
- Then click Publish to make it active.
- Rules must be published before they can be applied.
5. Run Duplicate Detection
You can run detection in several scenarios:
- On Record Create/Update → If user tries to save a duplicate, a warning is displayed.
- During Import → The system validates incoming data and flags duplicates.
- Manual Run (Bulk Check):
- Go to Data Management → Duplicate Detection Jobs.
- Create a job to scan for duplicates across entities.
- The results will list duplicate records, and you can take action (merge, delete, update).
Example: Duplicate Detection for Contacts
- Entity: Contact
- Rule Name: "Duplicate Contact by Email"
- Conditions:
- Contact Email = Exact Match
- OR (First Name = Exact Match AND Last Name = Exact Match)
- Result: If a user tries to create a new contact with an existing email or same name, the system prompts:
“A duplicate record has been detected. Do you want to continue?”
Best Practices
- Keep rules simple → Too many fields can slow performance.
- Use unique identifiers → Like email, phone number, account number.
- Cross-entity rules → Example: Prevent Lead with same email as existing Contact.
- Schedule jobs → Run bulk detection jobs regularly for data cleanup.
- Train users → Ensure sales teams know how to handle duplicate prompts.
In summary:
Configuring duplicate detection in Dynamics 365 involves enabling it at the system level, creating detection rules with field-based conditions, publishing them, and running detection jobs to clean and maintain reliable data in Dataverse.
Published on:
Learn more