How to filter out Duplicate search results?

Ashish Narnoli asked on October 14, 2014 11:40

Hi all,

I have setup Lucene search for one of our website and it is working fine. There are some products which are duplicates. When i search for these products on our search page, the search result displays duplicate results (products).

My question is what approach should i follow to remove out Duplicate results?

As a thought, i tried to clone the search results Webpart and removed duplicate records from the Dataset which is returned by SearchHelper.Search(...) method. But, in this approach returned result is perfect but the Total number of results is Incorrect.

I am working on v7.0

Suggestions are invited!!

Thanks in advance. Ashish Narnoli

Recent Answers

Dawid Jachnik answered on October 14, 2014 13:08

Hello,

These products are copied or you just link to existing product ?

if you want to remove the linked document you can simply add another condition to searchresult web part:

-NodeLinkedNodeID:[0 TO 999999999]

and this one should remove linked documents from you results.

2 votesVote for this answer Mark as a Correct answer

Ashish Narnoli answered on October 14, 2014 13:12

Hi Dawid,

Thanks for your quick response :)

I already using -NodeLinkedNodeID:[0 TO 999999999] for the Linked documents.

Here, duplicate products are not Linked Documents, they are simple documents. There are some special scenario in which we need to create that Document twice in the application.

Thanks! Ashish Narnoli

0 votesVote for this answer Mark as a Correct answer

Yehuda Lando answered on October 14, 2014 15:12

You'll need to handle the global event DocumentEvents.GetContent.Execute (more info here).

And then you can add a custom field to the search index using code like this:

if (e.Node.IsLink)
{
    e.SearchDocument.Add(
        "linked",
        "true");
}

I use this to mark linked docs (the -NodeLinkedNodeID:[0 TO 999999999] doesn't work for me correctly).

You can also test different queries using Luke

0 votesVote for this answer Mark as a Correct answer

Dawid Jachnik answered on October 15, 2014 12:57

Don't you have separate NodeAliasPath for two duplicated documents ?

2 votesVote for this answer Mark as a Correct answer