GoDBUniquer - High-Speed Email List Cleaner & Duplicate Remover

GoDBUniquer is a high-speed Windows command-line tool designed to clean email lists, remove duplicate records, validate email formats, and maintain a persistent local database of previously processed addresses.

Unlike basic duplicate removers that only compare records inside a single file, GoDBUniquer stores processed email addresses in a local database. This allows the software to detect duplicates across multiple files and future processing sessions.

Key Features

• Persistent duplicate database: processed email addresses remain registered between program runs.
• Cross-file deduplication: detects duplicates across multiple input files, imported databases, and previous sessions.
• Email format validation: filters malformed addresses and checks whether the domain uses a recognized ICANN public suffix.
• High-speed multithreaded processing: file-processing and record-processing thread counts can be configured separately.
• Preload database support: existing email lists can be imported into the local database without adding them to the output file.
• Original record preservation: when a unique email is found, the complete associated input record is written to the result file.
• Automatic progress statistics: displays processed, invalid, unique, duplicate, and imported record counts.
• Crash recovery: pending unique records can be recovered after an unexpected shutdown, helping prevent lost output.
• File status markers: processed and currently locked input files are tracked to prevent accidental repeated processing.
• Custom character cleanup: unwanted characters can be removed from input records using the configuration file.
• Optional generic string mode: the tool can be configured to deduplicate arbitrary strings instead of validating email addresses.
• Local processing: input lists are processed on your computer and are not uploaded for deduplication.

Supported Input Formats

Input data must be stored in plain-text .txt files, with one record per line. The software can extract email addresses from several commonly used formats:

• user@domain.com
• user@domain.com:password
• user@domain.com;password
• user@domain.com=name
• sender@domain.com|sender_name|user@domain.com|name
• Four-column tab-separated records
• Arbitrary strings when accept_any_string mode is enabled

How It Works

1. Place existing reference lists in the preload database directory if they should be registered as known records.
2. Place the lists that need to be checked in the configured input directory.
3. GoDBUniquer reads each record, extracts the email address, and validates its format.
4. The extracted address is checked against the persistent local database.
5. New addresses are reserved in the database and written to the result file together with their original record data.
6. Previously registered addresses are counted as duplicates and excluded from the result.
7. Successfully processed files receive status markers to prevent accidental repeated processing.

Preload Database

The preload feature allows you to import existing master lists before processing new data. Imported addresses are added directly to the local database and will be treated as duplicates if they appear in future input files.

Successfully imported preload files are automatically archived with a timestamp, making it easier to identify completed imports.

Configurable Options

• Number of record-processing threads
• Number of simultaneously processed files
• Number of database-import threads
• Result dumping interval
• Import statistics interval
• Input, preload, result, and database directories
• Custom database and output prefix
• Optional character replacement rules
• Email-validation or arbitrary-string mode

Ideal For

• Cleaning large email datasets
• Merging multiple customer or subscriber exports
• Removing records already present in a master database
• Preparing unique records for authorized marketing systems
• Maintaining a reusable suppression or processed-address database
• Deduplicating account lists and structured text records

System Requirements

• Microsoft Windows
• Permission to read and write files in the configured directories
• Sufficient storage space for the local database and result files
• An internet connection for license verification
• A valid GoDBUniquer license key

Important Information

• GoDBUniquer is a command-line utility and does not include a graphical user interface.
• Email validation checks address structure and public domain suffixes. It does not connect to mail servers or guarantee that a mailbox exists, is active, or can receive messages.
• Processing speed depends on CPU performance, storage speed, database size, thread settings, input structure, and the number of unique records.
• For best performance, keep the database and result directories on a local SSD or NVMe drive.
• Always keep backup copies of important source files and databases.
• Use the software only with data you own or are legally authorized to process. Purchasing this tool does not grant permission to contact email address owners.

License Notice

This is a paid digital software product. A valid license key is required to start the application. The software performs online license verification and generates a hardware identifier for license validation.

GoDBUniquer provides a practical, persistent, and configurable solution for removing duplicate email records across large collections of text files.