Smart Search - Can you index categories?

Deb Apprille asked on March 20, 2017 18:36

I have implemented Smart Search on my site using various indexes that I built. All of the indexes work as expected and I am happy with the functionality. My issue is that I would like to create a "category" index to include in the search, but I can't figure out how to create an index for data that is not present in the CMS Tree Menu.

View screenshot with search term 'Amplifiers'

Our tree is set up with parent Manufacturers and child Products such as:

Home > Manufacturers > Ophir > 4061 Amplifier

We also reference the same products through the category application (which is not part of the tree, but is native to Kentico):

Home > Categories > Amplifiers > 4061 Amplifier

Our Smart Search returns both manufacturers and products but not categories. What I need is a third segment called Categories under which "Amplifiers" would appear, if someone were to search for amplifiers. All attempts at creating an index for category have failed. When I set the Indexed Content to path: /% it only takes results that are present in the tree and nothing outside of it.

Can anyone help me make a custom index that will include the category application?

Thanks,

Deb A.

Correct Answer

Deb Apprille answered on May 18, 2017 19:44

Update on this. I fooled around with the smart search tool and noticed I could make a custom index off of Object Name "Content Category" which includes all of my categories. So simple! I can't believe I couldn't find documentation on this. My only problem now is getting the correct URL (rather than the root) when each result is clicked.

0 votesVote for this answer Unmark Correct answer

Recent Answers


Zachary Perry answered on March 20, 2017 18:51

When you go to indexed content on the search index, do you have categories checked?

You can read the documentation on creating a custom index.

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on March 20, 2017 19:29

Hi Zachary, I did check the "Include categories" checkbox inside the products index I created. It doesn't seem to change anything, though. That's why I began trying to make a category index on its own.

0 votesVote for this answer Mark as a Correct answer

Rui Wang answered on March 20, 2017 21:41 (last edited on March 20, 2017 21:41)

Hi Deb, The categories are typically used as filters for search results, not as search keyword. If that's the confusion point.

0 votesVote for this answer Mark as a Correct answer

Zachary Perry answered on March 20, 2017 22:33

You would have to create a custom search index like in the documentation I linked, and instead of getting files you would have to get the products you want joined with their categories.

You would end up with something like this:

  public void Rebuild(SearchIndexInfo srchInfo)
{
    // Checks whether the index info object is defined
    if (srchInfo != null)
    {
        // Gets an index writer object for the current index
        IIndexWriter iw = srchInfo.Provider.GetWriter(true);

        // Checks the whether writer is defined
        if (iw != null)
        {
            try
            {
                // Gets an info object of the index settings
                SearchIndexSettingsInfo sisi = srchInfo.IndexSettings.Items[SearchHelper.CUSTOM_INDEX_DATA];

                // Gets the search path from the Index data field
                string path = Convert.ToString(sisi.GetValue("CustomData"));

                // Checks whether the path is defined
                if (!String.IsNullOrEmpty(path))
                {
                    //Get Products Here
                    var products = new DataQuery().From(new QuerySource("").LeftJoin("", "", ""));

                    foreach (DataRow product in products.Tables[0].Rows)
                    {

                        // Creates a new Lucene.Net search document for the current text file
                        SearchDocumentParameters documentParameters = new SearchDocumentParameters()
                        {
                            Index = srchInfo,
                            Type = SearchHelper.CUSTOM_SEARCH_INDEX,
                            Id = Guid.NewGuid().ToString(),
                            Created = product["CreatedOn"]
                        };
                        ISearchDocument doc = SearchHelper.CreateDocument(documentParameters);


                        //Add the search fields

                        // Adds a content field. This field is processed when the search looks for matching results.
                        doc.AddGeneralField(SearchFieldsConstants.CONTENT, product[""], SearchHelper.StoreContentField,
                            true);

                        // Adds a title field. The value of this field is used for the search result title.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_TITLE, product[""], true, false);

                        // Adds a content field. The value of this field is used for the search result excerpt.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_CONTENT,
                            TextHelper.LimitLength(product[""], 200), true, false);

                        // Adds a date field. The value of this field is used for the date in the search results.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_DATE, product[""], true, false);

                        // Adds a url field. The value of this field is used for link urls in the search results.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_URL, product[""], true, false);

                        // Adds an image field. The value of this field is used for the images in the search results.
                        // Commented out, since the image file does not exist by default
                        // doc.AddGeneralField(SearchFieldsConstants.CUSTOM_IMAGEURL, "textfile.jpg", true, false);

                        // Adds the document to the index
                        iw.AddDocument(doc);
                    }

                    // Flushes the index buffer
                    iw.Flush();

                    // Optimizes the index
                    iw.Optimize();
                }
            }

                // Logs any potential exceptions
            catch (Exception ex)
            {
                EventLogProvider.LogException("CustomTextFileIndex", "Rebuild", ex);
            }

                // Always close the index writer
            finally
            {
                iw.Close();
            }
        }
    }
}
1 votesVote for this answer Mark as a Correct answer

Trevor Fayas answered on March 20, 2017 22:45 (last edited on March 20, 2017 22:46)

Categories (as long as set in the Smart Search -> [Your Index] -> Indexed Content) are included in Smart Search indexes. Using Luke 4.0.0 (a lucene index analyzer) you'll find the categories are in the "documentcategories" field and can be referenced by using filters such as

+documentcategories:development

Additionally there is a documentcategoryids that lists the numerical values.

Last note is by default it also includes any PARENT categories to the ones selected, so if you select "Additional Categories > My Category" it will add both "Additional Categories" and "My Category" to the lucene index. You can use a custom search indexer to create your own field and adjust how you wish!

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on March 20, 2017 22:55

Thanks guys! I will read all of this over tomorrow at work. :) I truly appreciate the guidance!

0 votesVote for this answer Mark as a Correct answer

Peter M answered on March 21, 2017 02:57 (last edited on March 21, 2017 02:59)

Your smart search autocomplete shows 2 indexes on your screenshot: Products and Manufacturers. Assuming you don't want change anything in your current setup (category menu web part and these 2 indexes), you just need to have the 3rd index there.

Here is my 2 cents: The problem is that you have a categories but you don't have a pages for them. I would create bunch menu items (not visible in navigation) to represent the categories you want to index (i.e. Amplifiers), connect them to these categories. In the navigation settings: Menu Action -> Redirect to URL put your category url (i.e. /category/amplifiers). Put all those menu items for categories in some folder in the tree and create your smart search index on them.

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on March 21, 2017 15:07

Peter, thanks for the input. I would ideally like that third index, yes. But it doesn't appear to be very easy! I thought about re-creating the categories as hidden pages (as you suggested), but we have hundreds of categories that are constantly being renamed and moved around so that option is out. I wonder if there is some way I could implement a single page that would work as a dynamic version of the category page, that would somehow iterate through the categories and be usable in the Smart Search index. I'm totally reaching now, but I am a dreamer. :)

0 votesVote for this answer Mark as a Correct answer

Trevor Fayas answered on March 21, 2017 15:28 (last edited on March 21, 2017 15:30)

Deb, i would just look at making a Custom Search Index, there you can use Kentico's API to pull in all the categories and add them to an index.

https://docs.kentico.com/k10/custom-development/miscellaneous-custom-development-tasks/smart-search-api-and-customization/creating-custom-smart-search-indexes

Here is the code that would index all hte categories, you can modify and adjust as you wish!

using System;

using CMS.Search;
using CMS.DataEngine;
using CMS.IO;
using CMS.Helpers;
using CMS.EventLog;
using CMS.Base;
using CMS;
using CMS.Taxonomy;

[assembly: RegisterCustomClass("CategoryIndex", typeof(CategoryIndex))]
public class CategoryIndex : ICustomSearchIndex
{

    /// <summary>
    /// Fills the index with content.
    /// </summary>
    /// <param name="srchInfo">Info object representing the search index</param>
    public void Rebuild(SearchIndexInfo srchInfo)
    {
        // Checks whether the index info object is defined
        if (srchInfo != null)
        {
            // Gets an index writer object for the current index
            IIndexWriter iw = srchInfo.Provider.GetWriter(true);

            // Checks the whether writer is defined
            if (iw != null)
            {
                try
                {
                    // Gets an info object of the index settings
                    SearchIndexSettingsInfo sisi = srchInfo.IndexSettings.Items[SearchHelper.CUSTOM_INDEX_DATA];

                    foreach (CategoryInfo catInfo in CategoryInfoProvider.GetCategories().Where("1=1"))
                    {
                        // Creates a new Lucene.Net search document for the current text file
                        SearchDocumentParameters documentParameters = new SearchDocumentParameters()
                        {
                            Index = srchInfo,
                            Type = SearchHelper.CUSTOM_SEARCH_INDEX,
                            Id = catInfo.CategoryGUID.ToString(),
                            Created = catInfo.CategoryLastModified
                        };
                        ISearchDocument doc = SearchHelper.CreateDocument(documentParameters);

                        // Adds a content field. This field is processed when the search looks for matching results.
                        doc.AddGeneralField(SearchFieldsConstants.CONTENT, catInfo.CategoryDescription, SearchHelper.StoreContentField, true);

                        // Adds a title field. The value of this field is used for the search result title.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_TITLE, catInfo.CategoryDisplayName, true, false);

                        // Adds a content field. The value of this field is used for the search result excerpt.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_CONTENT, TextHelper.LimitLength(catInfo.CategoryDescription, 200), true, false);

                        // Adds a date field. The value of this field is used for the date in the search results.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_DATE, catInfo.CategoryLastModified, true, false);

                        // Adds a url field. The value of this field is used for link urls in the search results.
                        doc.AddGeneralField(SearchFieldsConstants.CUSTOM_URL, "TheLink", true, false);

                        // Adds an image field. The value of this field is used for the images in the search results.
                        // Commented out, since the image file does not exist by default
                        // doc.AddGeneralField(SearchFieldsConstants.CUSTOM_IMAGEURL, "textfile.jpg", true, false);

                        // Adds the document to the index
                        iw.AddDocument(doc);
                    }

                    // Flushes the index buffer
                    iw.Flush();

                    // Optimizes the index
                    iw.Optimize();

                }

                // Logs any potential exceptions
                catch (Exception ex)
                {
                    EventLogProvider.LogException("CustomCategoryIndex", "Rebuild", ex);
                }

                // Always close the index writer
                finally
                {
                    iw.Close();
                }
            }
        }
    }
}

Just tested this out myself and works, i was able to search for my categories and pull them up.

1 votesVote for this answer Mark as a Correct answer

Peter M answered on March 21, 2017 15:49 (last edited on March 21, 2017 16:05)

if you have "hundreds of categories that are constantly being renamed and moved around", then it is a bit unusual. Categories are more like a taxonomy. I have thousands but they say intact for long time. Pages are being changed but the taxonomy stays. If your changes to the categories are frequent, you might try to go with a custom table (and smart search index on it). I don't know your process and how categories management is done. You can do it either with a sql trigger on cms_category (if change occurs - reflect this chahge in my custom table) or with a system event on "category change" (smth like CategoryInfo.TYPEINFO.Events). The system event is a better option here. Whenever category changes - you update your custom table (auxiliary table).

What you can do as well is go to modules->categories->fields. Create some sort of a bit field (IsInCategoryIndex) to mark the ones you need in your index. So all category like Amplifiers will have it set to true by editors. You can add some extra fields there as well like Path etc i.e. the info you want in your index. And then you can go with custom search index (Zachary and Trevor gave a couple hints above).

When you build your custom index you just do
foreach (CategoryInfo catInfo in CategoryInfoProvider.GetCategories().Where("IsInCategoryIndex=1"))

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on March 21, 2017 16:50

Yes, the changing of categories is unusual. My coworkers like to tweak the wording and organization of our site ... often. Kind of frustrating for me since I am in charge of SEO, but it is what it is.

Peter, the bit field is an interesting idea. But, looking at scalability... they want to index ALL categories (and as you know, those change) so perhaps I could reference a pre-existing field (which all categories would have) instead.

I'm gonna try the custom search index... that sounds like it could be the solution.

Thanks again, everyone, and thanks so much for the code snippets!

0 votesVote for this answer Mark as a Correct answer

Trevor Fayas answered on March 21, 2017 18:10

Deb, keep in mind that with Custom Search Indexes, you can (through Kentico) when you configure it you can pass it a textarea "Custom data" that you can then leverage the same indexing logic on multiple indexes, modifying the behavior via the custom data.

An example is you can pass it the actual Category "WHERE" Condition so you can control which categories are indexed by that particular instance of your custom index. Just a thought!

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on May 30, 2017 22:23

Update:

I made a custom index with these settings:

General

  • Index type: General
  • Analyzer type: Subset

Indexed Content

  • Object name: Content category (native choice in Kentico's select menu)
  • Where condition: CategoryEnabled = 1

The categories are showing up great. The problem is that the destination URL is missing.

For example: http://74.62.189.21/ -- in the search bar, search for preamplifiers

You will see that the manufacturers and products will have valid URLs but the category just points to the root. It seems as though only "Document" URLs are being returned.

Sidenote: The following is not currently on the dev server, but I did try messing around with the search entities (title, date, content, image) in the Search fields section. I assigned CategoryNamePath to "Content" in hopes that I would be able to use it to build a URL. I then went into Predictive results > Search result transformation and added <%# Eval("Content")%> but what got returned was the CategoryDescription and not the CategoryNamePath. So... I feel like the search field mapping is not actually being used.

Thanks for any further help you guys can give. I post here a lot and I really appreciate your answers!

-Deb

0 votesVote for this answer Mark as a Correct answer

Trevor Fayas answered on May 30, 2017 22:32

Looking at the documentation, what you're doing SHOULD work.

Did you go to Modules > Categories > Classes > Category > Search and configure the fields there? That is where you would customize this General Index for Categories.

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on May 30, 2017 22:48 (last edited on May 30, 2017 22:50)

Trevor ~

I had not gone there, and you're right, it responds when I change the 'Content' value!

Now I just have to figure out if there is a field that will get me the NodeAlias. I am afraid it's in another table... :( I guess I can try using a function to pull the value from the CatId. I'll post updates.

0 votesVote for this answer Mark as a Correct answer

Trevor Fayas answered on May 30, 2017 23:08

Categories don't have a "Node Alias" i'm afraid, what i would recommend though is in the Smart Search Results Transformation for this, use Text/XML and you can use macros to try to do a look up, or a custom transformation to find that.

Don't forget to look at the Cache system to make sure your results get cached, so you don't have to do a look up on each category each time it shows.

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on May 30, 2017 23:15

Thanks, I suppose there is no NodeAlias since it is not in the document tree. Foiled again!

The good news is that I have noticed (that for my data at least) the CategoryName matches the URL so I can build it as /category/CategoryName once I figure out how to do that programmatically!

0 votesVote for this answer Mark as a Correct answer

Deb Apprille answered on May 31, 2017 16:55

Ok, I got the thing working. Thanks for the input! :)

0 votesVote for this answer Mark as a Correct answer

   Please, sign in to be able to submit a new answer.