Dupemerge 0.999

Most harddisks contain quite a lot of completely identical files, which consume a lot of disk space. This waste of space can be drastically reduced by using the NTFS filesystem hardlinks functionality, if identical files aka dupes are hardlinked together.
Dupemerge searches for identical files on a logical drive and creates hardlink among those file thus saving lots of harddisk space.

Dupemerge creates a cryptological hashsum for each file found below the given pathes and compares those hashes to each other find the dupes. There is no file date comparison involved in detecting dupes, which might cause troubles.

To speed up comparison only files with same size get compared to each other. Furthermore the hashsums for equal sized files get calculated incrementally, which means, that during the first pass only the first 4 kilobyte are hashed and compared and during the next rounds more and more data are hashed and compared.

Due to long runtime on large disks some files, which have already been hashsumed, might change before all dupes to that file are found. To prevent false hardlink creation due to intermediate changes, dupemerge saves the file write time of a file when it hashsums the file and checks back if this time changed when it tries to hardlink dupes.

If dupemerge is run once, hardlinks among indentical files are created. To save time during a second run on the same locations, dupemerge checks if a file is already a hardlink, and tries to find the other hardlinks by comparing the unique NTFS file-id. This saves a lot of time, because especially checksums for large files need not to to be created twice.

Dupemerge has a dupe find algorithm, which is extremly tuned to especially perform well on large server disks, where it has been tested in depth to guarantee data integrity.