All Collections
Uploading, Importing and Exporting
Uploading
How can I detect and remove duplicate assets in Dash?
How can I detect and remove duplicate assets in Dash?

Learn about duplicate detection and removal

Charlie Pinker avatar
Written by Charlie Pinker
Updated over a week ago

How does Dash define a duplicate asset?

We only consider an asset to be a duplicate if it is an identical file (based on an MD5 hash). This means that the entire contents of the file must be identical. The filename is not considered part of the check. Similar assets are also not considered, e.g. a cropped copy of an asset or a copy of an image that has been edited.

There are two methods of removing duplicates in Dash:

1. Identifying duplicates on upload

Whenever you upload assets into Dash, Dash will compare the assets to other assets in your upload and assets that already exist in your Dash account.

Once the duplicate detection is complete, you will have the opportunity to remove the duplicates from the upload before you submit the rest of your assets into Dash.

2. Identifying duplicates of already-uploaded assets

If you already have assets in Dash that you want to deduplicate (for example, if you did a large import), we can run a deduplication on your account for you.

What happens when you run the deduplication on our account?

We will look for all duplicate assets as defined above. For each that we find, we will merge these assets into a single asset. If both assets contain different information in the same field, we apply the following logic to decide what information to retain:

  • Single value text field - the longest length text string is kept.

  • Single value Date or DateTime field - the oldest Date is kept.

  • For multi-value fields like folders and tags, values from duplicates are added to the original’s set.

Here's an example of this logic in action:

⚠️ Be aware that, when multiple duplicate assets are merged into a single asset, only one of the assets is retained and it is is arbitrary as to which one. As the metadata is merged, the retained asset will appear in all the right folders. However, information about collections, embed and share links is not merged. This means an asset may disappear from a collection, embed or share link after the deduplication has completed.

I'm ready to run the deduplication!

Deduplication is irreversible. Any deleted assets will not be recoverable once complete, so please make sure you understand the implications listed in this article above.

Once you are ready, just get in touch using the in-app chat, confirm you've understood the above, and we'll run the job on your account.

Did this answer your question?