Business Rules for Data Cleansing

Data cleansing can be done according to the business rules provided separately in subsequent section. Various parameters defined by administrator are as below.

• Should not contain any alpha character
• Should not be less & more than 'N' digits
• Should start with/without specific numeric or characters
• Defining minimum, maximum, and exact length
• Should be mandatory or specific credentials
• Remove or include spaces
• Should not contain punctuation mark, any special characters, and specific special characters
• Should not have numeric between statements
• Should not have special character between statements and/or number
• Should not have character between numbers
• Should not contain any space, punctuation mark or any special character (except given special characters).
• Roman numeral allowed
• Case sensitive conversion rule
• Conditionally appending or replacing repeat characters

Lexicon Rules for Data Enrichment

• City validation and enrichment
• State validation and enrichment
• Pin-code validation and enrichment
• Defining default value based on other dependent value.
• Conditionally appending or replacing repeat characters
• Spelling should be in sync with the reference data
• Mapping with Master & Slave data
• Blank data will be populated based on other dependant data

Key Benefits

• User defined cleansing parameters and its data rejection categories
• Maintaining transaction logs
• Works very well on even bad data
• Works with any database on any PC platform
• Helps to standardize customer contact data like name, address and other data
• Helps to identify duplicates

Process

• Increasing the value of information by appending and replacing the exact information
• Verify and maintain data consistency using our inbuilt lexicons (ex: if city is given, pin code can be added) or enriching components in address field (ex: bld to building, rd to road).
• Data validation is process to identify quality of data, and is been purified by verifying Customer Name, Address, Pin-code, Mobile / Alternate contact nos., Date of Birth, ID Proofs, email, etc.

Our StarDQ-Pro extracts and analyzes identities, represented by names and/or addresses. Incorporating cutting edge search technology and applying innovative and powerful algorithms to find inexact matches in names and/or addresses along with ID Proof, Address Proof, Date of Birth, and Email ID which are optional parameters.

De-duplication Rules

• Inherits the same cleansing rules
• Flat file / Database connectivity
• Identify the specific data columns and grouping the columns with respective components
• Duplicate identification rules like In-String / Phonetic Algorithm
• Defining Threshold percentage along with conditional threshold adjustment

Data De-duping Engine – Architecture Diagram

Batch De-duping: Full historical data of customer profiles can be processed and results shared. This process can be done on entire data from scratch to create Master unique and Master dupe tables. As this is a major activity, it may require a minimum of three iterations of DEDUPE process with different threshold parameters. The results of the each iteration exercise is helpful in finalizing the standard threshold percentage to be set for the MASTER Database.

Thereon, StarDQ-Pro requires only the periodical incremental data in order to compare the same with the Master Database. This expedites the process and deliver immediately instead of redoing the Full Historical Data, a fresh every time.

Incremental Batch Process: The Incremental Data can be extracted either from Flat File which is placed in FTP location or directly from source database. The extracted incremental data can be compared with MASTER Data to find any duplicated records already existing with the same customer profile.
Incremental batch process can be on a Daily / Bi-weekly / Weekly / Fortnightly / Monthly basis

(or)

Incremental Online Instant Process: End-user has the option to provide Incremental Data on-line by either data-entry for each customer detail or uploading excel file. Soon after submitting the data, our StarDQ-Pro Invoker will initiate the de-duping and display the matching customer record with the count of cluster members details on a real time basis.

Benefits

• Improve brand penetration by enabling stronger identification of cross-sell opportunities.
• Support effective tactical and strategic decision making through more accurate analytics.
• Increase revenues through knowledge that could not otherwise be realized.
• Single view of the customer: Better call centre throughput for outbound lead solicitation calls. More strike rate and reduced call times.
• Less bad debts and improved customer receivables recovery with the support of call center who are empowered with better customer data.
• Field sales force are better equipped to handle potential customers because of better selling skills, arising from back up data on past customer behavior, knowledge of customer groups and improved customer visibility