PC Applications forum

General discussion

Find duplicate files

Hello all,

I make regular backups of my data and now my external hard drive is almost getting full. I would like to do some cleanup and retain only one copy of all the files. Keep one backup of these unique files on HDD and also take a backup on DVDs. Once I am done, the next cycle of back-up begins but I am stuck with the duplicate files.

I was wondering if anyone can suggest me any utility where it makes one copy of multiple files at destined location on HDD and deletes all rest copies. This double file checking should NOT only be by the name as I have updated few files and backed them up as regular backups so the name remains same but not the data.

I know there are many such utilities out there but looking for suggestion from someone who uses them or have used them without any hassle. (This data is my living..)

Discussion is locked
You are posting a reply to: Find duplicate files
The posting of advertisements, profanity, or personal attacks is prohibited. Please refer to our CNET Forums policies for details. All submitted content is subject to our Terms of Use.
Track this discussion and email me when there are updates

If you're asking for technical help, please be sure to include all your system info, including operating system, model number, and any other specifics related to the problem. Also please exercise your best judgment when posting in the forums--revealing personal information such as your e-mail address, telephone number, and address is not recommended.

You are reporting the following post: Find duplicate files
This post has been flagged and will be reviewed by our staff. Thank you for helping us maintain CNET's great community.
Sorry, there was a problem flagging this post. Please try again now or at a later time.
If you believe this post is offensive or violates the CNET Forums' Usage policies, you can report it below (this will not automatically remove the post). Once reported, our moderators will be notified and the post will be reviewed.
Collapse -
One I use.

In reply to: Find duplicate files

Collapse -
Have you tried Google?

In reply to: Find duplicate files

Collapse -
Did you read my post

In reply to: Have you tried Google?

properly? I have googled and also on download.com but many a times the explanation doesn't say if it meets my requirement. I don't want to use some utility because author says so and then repent...Looking for sort of tested and approved kind of utility

Collapse -
Tested and approved.

In reply to: Did you read my post

For use by me. I however will not test it to see if it meets your spec.


Collapse -
It wasn't you

In reply to: Tested and approved.

@ Bob,
I DON'T expect anybody to test for me. If anybody is using such application and happens to have same requirement as mine then reply was expected. As a matter of fact, reply wasn't posted to your post..

Bury it now..

Collapse -
Let me be blunt.

In reply to: It wasn't you

The app I noted seems fine to me. I leave it to you to see if you'll try it.


Collapse -
Did I read your post? ... YES

In reply to: Did you read my post

I have read all your post. In your first post, this thread, you never mentioned you Googled at all. In fact, you never mentioned you never said you even searched at all. All you said was that you wanted a suggestion, and I suggested a Google search which uncovers a lot of similar tools. There is no reason to be rude.

Since you say that "this data is my living" I would imagine that you might be more interested in getting information than getting 'back at someone.' I would imaging that no matter what tool gets suggested you would still read and test as to what it does, since, after all, this data is your living.

In a later post you mention "I DON'T expect anybody to test for me."
Fine, then just look over the specs and see what each of the uncovered tools say they'll do. It is not that hard to do.

I have two free dup finding tools that I use a fair bit, that not just do a duplicate name find, but also do a date and size, and even byte-for-byte compares to see if the located files are truly exact duplicates. Of which I can even filter down to include or exclude those (name, date, size) properties.

They even allow me to view the discovered files, with the viewer of my choice. I would have been glad to have shared the names of those tools, or even have sent you a copy ... notice that is past tense, though.

Here is what I will not share; How did I find my free tools?
I did a web search (Google wasn't around then - which would have made it much simpler) and read through the features of the ones listed and tried them out. It doesn't take all that long. Actually I think I got one from Download.com, long ago, but while WinXP did not exist at the time, it still works fine on WinXP.

Some people do simple one word searches in Google, not knowing how to add phrases and exclusion words in the search box. I did not bother you with the details, or quiz you about your knowledge of doing some of those types of searches, I just provided you with a link that, when clicked on, would get you started. Some people also only look at the first page of Google hits, where the 2nd or 3rd page might be more to their liking. Google is good, but not always so good that it will find exactly what you want on the first page (the first 10 hits).

Good luck on your endeavors. I am sure that someone else will be more than happy to help you, just don't spit on them first.

Collapse -

In reply to: Did I read your post? ... YES

Apologies posters if I sounded rude to anyone.

Before I post any question on these forums, I do multiple google searches, this forum searches, try different websites like download.com. Sometimes lifehacker.com has very good tips. Once I end up getting nothing at these places, I land in these forums.

I am not a geek so I am always worried about 'stable' windows machine so I am pretty afraid to 'give a try' just by looking at specs. This is the whole reason why I turn to 'used and trusted' advice. I think, I just didn't articulate the words properly to get my message across and that led to confusion.

Thanks for your posts anyway, I will certainly give a try with this robocop.

Collapse -
Robocopy is a good utility from Microsoft

In reply to: Find duplicate files

I have used Robocopy to "synchronize" files between two or more hard drives. It is a manual process, but you can create a small text-based batch file that you can schedule with the Microsoft Scheduler utility so that it runs at night. Here is a sample batch file:

c:\robocopy\robocopy "d:\data\is projects" "h:\Home\a_fairw\odrive" /e /r:3 /w:1 /log:f:\robocopy.log /np

As you can see, it is a command with several parameters defining the specific behavior I want; I picked the parameters based on the end result I wished to achieve.

Robocopy is a free utility included in the Windows Resource Kit packages available from microsoft.com just search for robocopy.

If you are not comfortable with editing and using batch files, I recommend using the utilities that come with many external hard drives, such as "Autoback".

I have used Robocopy for several years now, and I am confident in its use.

I might also recommend a different approach: create your copies in dated folders, and delete the folders that are more than a month old. After all, if you are copying the entire contents of "My Documents" on a weekly basis, then a copy that is a month old is simply a waste of disk space.

Also, I recommend multiple copies of the valuable files. It sounds like you perform regular DVD copies, which is good but certain DVD brands do not work well for archival purposes. I own four external hard drives, and keep four different copies of my valuable data (music, photos, and other documents I do not wish to lose or cannot recreate). Redundancy, redundancy, redundancy is my motto - helps me sleep better at night.


PS - Bob has offered good advice, for you and for others on this web site, please do not be rude to him. He is my friend, and he has a good heart. He meant no offense.

Popular Forums

Computer Newbies 10,686 discussions
Computer Help 54,365 discussions
Laptops 21,181 discussions
Networking & Wireless 16,313 discussions
Phones 17,137 discussions
Security 31,287 discussions
TVs & Home Theaters 22,101 discussions
Windows 7 8,164 discussions
Windows 10 2,657 discussions


This one tip will help you sleep better tonight

A few seconds are all you need to get a better night's rest.