How to import data via custom scheduled task async?

Anna Pengal asked on December 8, 2022 16:24

I am using Kentico 13 Core.

I have a custom task that creates or updates Kentico pages based on json data that I want to improve performance on. There are several thousand records in the json object that need to be either created or updated.

My initial thought was to make the scheduled task async to improve performance, but the below code throws an error because ITask does not seem to support async methods.

public async Task<string> Execute(TaskInfo ti)
    {
        var ItemsToUpsert = fileName.LoadJson<List<CustomItem>>();

        List<Task> listOfTasks = new List<Task>();

        foreach (var item in ItemsToUpsert)
        {
            listOfTasks.Add(UpsertItemAsync(item));
        }
        await Task.WhenAll(listOfTasks);

        return returnMessage;
    }

I then tried versions of this code that look like the two below, but both throw errors about having multiple datareaders open and then connectionstring changes respectively.

public string Execute(TaskInfo ti)
    {
        var ItemsToUpsert = fileName.LoadJson<List<CustomItem>>();

        Task.Run(Parallel.ForEach(ItemsToUpsert, item =>
        {
            UpsertColorwayAsync(item);
        }));

        return returnMessage;
    }

Second version

    public string Execute(TaskInfo ti)
    {
        var ItemsToUpsert = fileName.LoadJson<List<CustomItem>>();

        Task.Factory.StartNew(CMSThread.Wrap(() => (Parallel.ForEach(ItemsToUpsert, items=>
        {
            UpsertColorwayAsync(items);
        }));

        return returnMessage;
    }

I have tried to look up the CMS.DataEngine.AsyncWorker class to see if I could use that, but there is limited documentation about it.

Recent Answers

Brenden Kehren answered on December 12, 2022 19:02

I would suggest simply checking the box to run your task in a separate thread. This will execute the task in a different thread outside of the main thread. As long as you don't need any Context objects you should be good to do this. Is there a particular reason you need to run a task async?

1 votesVote for this answer Mark as a Correct answer

Jeroen Fürst answered on December 13, 2022 08:16

Hi Anna,

Please bare in mind that updating pages via the API can be slow, especially when Workflow is enabled. Is there a specific reason that you are storing the data as pages?

Cheers, Jeroen

1 votesVote for this answer Mark as a Correct answer

Sean Wright answered on December 13, 2022 16:24

A few things to note:

Async in ASP.NET apps doesn't improve speed of single requests (in fact, it slows them down my a very small amount). It does, however, improve concurrency by allowing your application to handle more requests at once.

Since a Scheduled Task is processed by a single request/thread, making it async won't speed it up.
SQL Server traditionally only supports 1 active query/result per connection at a time. You can enable MARS in certain scenarios, but it comes with lots of caveats that you need to understand and Xperience doesn't support it for most use cases.

So, Parallel.ForEach is not really speeding up your task processing since your single db connection can't be used from multiple threads.

If you want to improve the performance of database operations through Xperience's API, you might want to look at CMSActionContext which can let you turn off features that you aren't using for the operation:

using (var ctx = new CMSActionContext())
{
    ctx.LogIntegration = false;
    ctx.LogSynchronization = false;
    ctx.LogEvents = false;
    ctx.LogWebFarmTasks = false;
    ctx.TouchCacheDependencies = false;
    ctx.CreateSearchTask = false;

    // or ctx.DisableAll();

    var ItemsToUpsert = fileName.LoadJson<List<CustomItem>>();

    foreach (var item in ItemsToUpsert)
    {
        UpsertItem(item); // not async / Task-based
    }
}

2 votesVote for this answer Mark as a Correct answer