Manage Duplicates: Merge Duplicates Part I - Merge System Identified Duplicates

Manage Duplicates: Merge Duplicates Part I - Merge System Identified Duplicates

1. This tutorial will cover how to merge potential duplicate contacts that the system has identified.
2. In addition to preventing duplicates from being created (see our Manage Duplicates: Prevent Duplicates Part I - Duplicate Prevention Settings tutorial for information on preventing duplicate records), your Fundly CRM system will run an automated quality assurance check once per week or month to look for records it thinks are potential duplicates using the automated search's Search Preferences (and NOT your duplicate prevention settings).
3. These records will be flagged and presented to your on the Review tab of the Duplicate Prevention screen. Please Note: this function is DIFFERENT, separate, and independent of the real-time duplicate prevention happening based on your duplicate prevention settings as detailed in the tutorial linked above and your Hygiene rules as details in the Prevent Duplicates Part II tutorial. Fundly CRM provides two levels of duplicate prevention check: real time prevention and a secondary, offline, quality assurance check of your data. In addition, users can manually identify and merge duplicate records (see our Merge Duplicates Part III tutorial for more information on manually merging duplicate records).
4. To use the automated quality assurance check, start by clicking Contacts on the Main Menu and select Manage Duplicates.

   

5. Before using the automated quality assurance check for the first time, you will need to configure it.
   • On the Review tab, indicate if you want the system to run the automated quality assurance check every week or every month. You will only need to do this once (though you may change this preference as often as you like. After setting up the automated check, you can skip to step #x below to manage the records that have been flagged as potential duplicates).

     

   • Next, click the white Search Preferences button to set up the criteria the automated search will use to identify records as potential duplicates.

     

   • In the resulting pop up window, indicate what fields the system should use to flag records as potential duplicates.

     

   • This criteria can be different than your regular duplicate prevention settings, and, in fact, you may want to make it less strict than you real-time duplicate prevention settings. Unlike the real-time duplicate prevention which will live update records that the system thinks are the same as an incoming record, the quality assurance check will only flag records as potential duplicates for your review (that is, the quality assurance check doesn't make any changes to records). So, with the quality assurance check, there is no danger of a record accidentally being updated with the wrong data if you use less strict criteria (such as only 3 or 4 characters of last name instead of the entire last name).
   • Of course, if you make the criteria too broad, then too many records will be flagged as potential duplicates. Too strict, and the system will overlook obvious duplicates.
   • Review our Prevent Duplicates Part I tutorial to see best practices and suggestions for criteria settings.
   • In the Email Notification box, indicate the email address(es) of the staff member(s) who should be notified every time the search runs/the search results are ready for review. You can enter multiple emails separated by commas. If you leave this field blank, an email will be sent to your system admin.

     

   • When you finish setting up your criteria, click the black Save and Close button to save changes, close the pop up, and be returned to the main screen.

     

   • Your duplicate prevention quality assurance search is now set up. You do not have to repeat any of the above steps in future. The system will now run an automated quality assurance check every week or month based on your settings. However, feel free to change the above settings as often as you like/need to.
6. In addition to the automated quality assurance check that will run based on the above settings, you can also manually initiate a search by clicking Search Now as often as you like between the background/automatic searches the system will conduct.

   

7. Any potential duplicates the system finds (based on your Search Preferences set in step #5 above and not your duplicate prevention settings set on the Prevent tab of this screen) either through the automated search or a manual search you trigger by clicking Search Now will be flagged and shown here, on the Review tab by the type of fields that matched.

   

8. In the left hand panel, you will see the total number of flagged records as well as the breakdown by match fields. Keep in mind that the flagged records are potential duplicates (based on your Search Preferences settings); not all of the records may be duplicates.
9. To review each list of potential duplicates, click on that category in the left hand panel. A list of the contacts in that category that have potential duplicates is displayed in the right hand panel.

   

10. To start going through all potential duplicates at the same time click the blue Merge Now button in the bottom right corner to go through all the records in the list, one set/pair at a time OR to go just one set of records, click the View Details button next to any duplicates in the list.

    

    OR

    

11. You will be presented with a side-by-comparison of the (potential) duplicate records. Keep in mind there may be more than 2 records in this pairing and you may have to scroll to the right to see all of the records. In our second example below, there are 4 records that are all potential duplicates of each other. You can view the list of records being presented in this merge on the left to verify how many records you're merging together.
    

    

    

    

12. Keep in mind, also, that individual contacts and organizational contacts cannot be merged together. Individuals can only be merged to individuals and organizations to organizations.
13. Compare the records that are presented. They will be presented, from left to right in order of Contact ID (generally, this will mean the oldest record/first created will be on the left, but not always). The name that shows at the top of Merge screen, however, will be the one that is alphabetically first in the pairing/set.
14. If they are not duplicates/should not be merged together, then click the Not Duplicates button. The system will mark internally that these records should be ignored as potential duplicates in future.

    

15. If you have more than 2 records in one set and some of the records are not duplicates but others are, uncheck the box next to the names on the left that you want to take out of this merge and click the Merge Separately button. This will uncouple the set and allow you to handle each pair of records in the set separately. The ones that are checked will remain in this set/merge and then the remaining items that you unchecked will be presented together in the next set. You must uncheck at least two items to proceed.

    

    

    

16. To merge records, starting by indicating the name to be kept by clicking the radio button next to the name (by default the name and all associated data of the record on the left will be selected). This will also be the overall record that is kept - all other records in the set will be merged INTO this record.

    

17. This means that, if, for instance, you do not select a mailing address to be kept, the mailing address on the kept record will be kept by default. The active/deceased (active/inactive for organizations) status of the kept record will be kept/applied to the merged record as well.
18. Go down the list of available fields and indicate the data you wish to keep in the resulting merged contact. You can keep only one mailing address, one billing address, one other address, but any/all emails and phone numbers.

    

19. Regarding the merging of addresses, the system uses the following logic:
    • For Mailing Address, the selected to keep mailing address is kept as a mailing address even if it is uncategorized. If the mailing address of the duplicate/not selected to keep record is kept, it will overwrite the mailing address (if any) on the active/selected to keep record. The mailing address of the duplicate/not selected to keep record will be discarded UNLESS it is a home or work address. If it's a home or work address, see below.
    • For Home Address(regardless of whether the home address is in Mailing or Other), the system checks if there is home address on the active/selected to keep record (the record belonging to the name you selected to keep).
      • If there is a home address on the active/selected to keep record, then the system discards the home addresses of duplicate/non-selected record.
      • If there is no home address on the active/selected to keep record, then system picks the home address from the duplicate/non-selected records. If there is more than one home address among the duplicate contacts, then the system picks the first one it comes to. If the active/selected to keep record has a mailing address, then the home address taken from the duplicate/not selected to keep record will be placed in the Other Address field.
    • For Work addresses the system moves all the work addresses of the duplicate/non-selected records into Other Address block of the active/selected to keep record (that is, ALL work addresses are kept, but moved to the Other Address block).
20. Some fields, indicated with a radio button can only have results taken from a single contact.
21. Other fiends, indicated with a check-box can have results taken from more than one contact.
22. Data not indicated on the merge duplicates screen is saved from both/all records in the merge by default (all donations, all membership information, all event information, all tasks/interactions, etc.).
23. Once all desired fields have been selected click on Next to see a preview of the contact information selected.

    

24. If you wish to make changes to the fields selected click on Review Contact and you will be returned to the previous step.

    

25. If the contact details are correct click on Save & Merge Next to set the contact to be merged and move onto the next record.

    

26. A pop up window will appear asking you to confirm that you want to proceed with merging the records.

    

27. If you want to receive an email when the merge of these records finishes, click the check box. Leave the box unchecked if you don't want an email notification. You will know when the merge completes by searching for the contact and only finding one record or by checking the Contacts: Manage Duplicates: Merge History tab and seeing the contact listed there.

    

28. Click Merge Now to proceed with the merge.

    

29. The pop up box will close. All records set for merge will be merged via an offline process that will take anywhere from 10 minutes to a couple of hours depending on system load and complexity of the merge (records with many transactions and engagement history entries will take longer than records with very little data). We recommend NOT entering any data into the records in the process of being merged as data could be lost if the merge has already started behind the scenes and/or it will slow down the merging process (and make the system start over merging the records). You can continue working in the system in other records while the merge is in process.
30. The contact will be removed from the list of flagged records. If you used the Merge Now button to enter into merge duplicates, you'll be taken to the next record in the list to be merged. If you clicked View Details to enter into the merge flow, you'll be taken back to the Review tab.
31. Continue reviewing and merging records until you have gone through all of the flagged records. It is important to stay on top of potential duplicates to keep your data clean and error-free. Duplicate records can cause problems for users logging into to public profiles, renewing memberships, and receiving member discounts on event ticket purchases.
32. Sometimes when you bring up a pair of records for merging, you will receive one of two error messages:
    • An error indicating that a contact you are attempting to merge is in a partially processed communication. This means that there is a draft communication batch (usually a donation acknowledgement) on your Communications: Pending tab that has this contact in it OR in a failed communication batch in History tab. The contact must be removed from the batch (or the batch processed) before you can proceed with the merge. The batch status will be generated, but won't be sent. Find the batch by name (as indicated in the error message) and remove the contact from the batch (delete), delete the entire batch, or process the batch to proceed with the merge.
      

      

    • An error indicating that a contact is already in the process of being merged with another record. You can not proceed with the new merge until the previous merge completes. Click the Remove button to cancel the merge request/remove the contact from the list of potential duplicates. Exit out using the Back button to leave without taking action on this merge request (so you can return later and complete it, once the previous merge has finished processing).

      

33. If you wish to see a history of all contact records that have been or will be merged, click on the Merge History tab in the upper right of the Manage Duplicates page. There is no search field on this page, but you can search by name (or part of name) by using your browser's search function (usually called "Find on this Page" - on Firefox for Windows/PC, it's on the Edit tab). Merged (Duplicate Prevention Key) means that the duplicate was merged during creation/update. Merged (Name/Email/Address) is when the system search on the basis of your duplicate search preference.
    

    

34. To view the details of what records where merged for a given contact click on View Details for that record.

    

    

35. If any merges happen that day, your system admin will get a notification overnight of a list of all merges that have happened that day. This is so you can verify that the records merged together by staff were correctly merged/will be an additional layer of quality assurance checks for you (and so that you can contact Fundly Support immediately if anyone on staff improperly merges any contacts together).

    

36. In addition, you will see on the change log and the "last updated field" the entry "merge" when updates to contact records happen due to a merge.