Pause Smart Search Index during nightly data sync

Jonathan Bowen asked on September 1, 2017 13:50

I am using Kentico 9. I have a nightly process that syncs data (using a Kentico scheduled task) over to Kentico to make use of the smart search feature. The smart search index looks like it is trying to update while the sync process is updating pages it has indexed, slowing down the process. Is there anyway to programmatically pause the smart search index build and then rebuild/optimize the index once the sync is done?

Recent Answers


Brenden Kehren answered on September 1, 2017 14:16

Check to see if your scheduled taks is already processing, if it is, wait for that to succeed, run your process, and rebuild the index.

SearchIndexInfo searchIndex = SearchIndexInfoProvider.GetSearchIndexInfo("SearchIndexCodeName");
if (searchIndex != null)
{
    if (searchIndex.IndexStatus == IndexStatusEnum.OPTIMIZING || searchIndex.IndexStatus == IndexStatusEnum.REBUILDING)
    {
        // index is busy
    }
    else
    {
        // perform your work
        // once it succeeds rebuild the index
        SearchTaskInfoProvider.CreateTask(SearchTaskTypeEnum.Rebuild, null, null, searchIndex.IndexName, searchIndex.IndexID);
    }
}
0 votesVote for this answer Mark as a Correct answer

Jonathan Bowen answered on September 1, 2017 15:30

My data sync task can take a very long time to run. I've had the Smart search index page open in the cmsdesk while the data sync has been running, and I can see a spinner indicating the index is updating itself. Is there anyway to pause all updates to the index?

0 votesVote for this answer Mark as a Correct answer

Chetan Sharma answered on September 1, 2017 15:53

How long does your Scheduled task for content update takes on an average? Can you not keep the average time + 1 hour between them? If you don't want to do it programmaticaly?

Let's say its starts at 1 am and takes 2 hours to finish (This is a too big number) then add another 1 hour and start the index update and optimization process to run at 4 am every day!

I am sure this is not the best way but trying to get away from writing a code.

0 votesVote for this answer Mark as a Correct answer

Brenden Kehren answered on September 1, 2017 16:12

There isn't an enabled or disabled glad on a search index, you simply check the status. The downside of this is if a page gets published dring that time the index will kick off again.

0 votesVote for this answer Mark as a Correct answer

Jonathan Bowen answered on September 1, 2017 17:12

Is there anything I can do? In my use case, Kentico isn't the source of truth for data. Another SQL Server database is. That's why I'm syncing data over, which can take several hours. The index updates are slowing things down more.

0 votesVote for this answer Mark as a Correct answer

Brenden Kehren answered on September 1, 2017 17:33

You may have to be more specific on what data you're syncing over? Any page data inserts or updates will trigger the index. Only way I can think of right now is to remove the index and recreate it each time you run your scheduled task.

0 votesVote for this answer Mark as a Correct answer

Jonathan Bowen answered on September 1, 2017 18:01

Thank you for all of your help today. I really appreciate the quick responses!

The sync is adding/updating/removing pages in the content tree. Is there anyway to disable automatic index updates in general/site-wide? That way indexes can only be updated manually (either scheduled task or programmatically at the end of my sync process). Right now, this is the only smart search index.

0 votesVote for this answer Mark as a Correct answer

Brenden Kehren answered on September 1, 2017 18:10

Your question "Is there anyway to disable automatic index updates in general/site-wide?". Yes, delete the index each time you run your scheduled task.

0 votesVote for this answer Mark as a Correct answer

Jonathan Bowen answered on September 1, 2017 18:50

The only downside is the search won't work at all while the sync is running.

I found a disabled scheduled task using CMS.Scheduler.SearchIndexOptimizer. Does that help in any way?

0 votesVote for this answer Mark as a Correct answer

Brenden Kehren answered on September 1, 2017 19:35 (last edited on September 1, 2017 19:36)

IF the scheduled task is already disabled, it will not help at all. As I mentioned previously, anytime a page is updated it triggers the index to update with that new content. So only way I see it working is to delete your index each time your task runs. If you task is running for a few hours you might look at some other options to optimize that and maybe only bring over only items which have been updated vs all of them.

Another option to help with processing is in your scheduled task to stop logging on a few things using the code below.

using (new CMSActionContext { LogEvents = false, LogSynchronization = false })
{
    // do some work in which you don't log event logs for or log sync tasks for.
}
0 votesVote for this answer Mark as a Correct answer

Peter Mogilnitski answered on September 1, 2017 19:37

My 2 cents:

Assuming you are updating pages of a certain types (Articles in my example). For the sake of argument for all your types where you do your sync update try uncheck this: Image Text

In theory it should prevent creation of update tasks for search indexes when you are running your document updates. See if that works then we can find a way to do it pro grammatically :).

0 votesVote for this answer Mark as a Correct answer

Brenden Kehren answered on September 1, 2017 19:40

Still don't believe that solves the problem Peter only because unchecking it will only tell the index to not index it next time it is built. BUT since those page types are already included in the index, it will update when the page is updated.

1 votesVote for this answer Mark as a Correct answer

Jonathan Bowen answered on September 1, 2017 19:49

Thank you for all of your help today. This information will definitely help me come up with a long term solution.

0 votesVote for this answer Mark as a Correct answer

Peter Mogilnitski answered on September 1, 2017 19:55 (last edited on September 1, 2017 19:55)

Brenden, I know, but he has to try it first:)

The other way I see is to delete index update task: .

SearchTaskInfo.TYPEINFO.Events.Insert.Before += SearchExclude;
private static void SearchExclude(object sender, ObjectEventArgs e)
{
    If (AmIRunningMyUpdatesCondition) {
        SearchTaskInfo sti = (SearchTaskInfo)e.Object;
        if (Regex.IsMatch(sti.SearchTaskValue, @"\d+;CMS.Article", RegexOptions.IgnoreCase))
        {
            if (sti.SearchTaskStatus == SearchTaskStatusEnum.Ready)
            {
                // remove the task that update searchindex when you update a document
                sti.Delete();
            }
        }
    }
}

I know this is kind of crazy because for each document update a search task will be created by the system and deleted by your code.

0 votesVote for this answer Mark as a Correct answer

   Please, sign in to be able to submit a new answer.