How to load a large amount of page data efficiently

Summit Pump asked on October 9, 2023 17:56

I am currently creating an ecommerce portion for our website and we currently have over 9000 products that I have made pages for using a page type I created that represents a product. I have a main store page that lists all of the products in a table and was wondering if there was a more efficient way of loading the page data as currently it takes a bit for the table to load since it has to get over 9000 products' page data. I am using a jquery plugin called DataTables for the table.

In my main store viewmodel I am creating a list of the page type Parts which is the product's page type. In the main store controller I call a method from a repository I created that uses IPageRetriever to get the pages, limiting to the four fields that I need using Columns(). I have this method cached which helps a bit with performance but I was wondering if there was a way to attack the problem at the source.

Recent Answers

Jono Rickard answered on October 10, 2023 22:44 (last edited on October 10, 2023 22:45)

Hey there,

Yeah what you'll want to do is update the IPageRetriever query to page the results.

Usually on the frontend you'd also only display 10-20 at a time and include a list of pages or a load more button.

To do that it would look something like this:

int currentPageIndex = 0; //Youd pass this in as an argument - perhaps via a ?currentPage=X query string
const int pageSize = 20; //This would be set once and is up to you
int totalRecords = 0; //This is set to the actual number of records down below.
var results = pageRetriever.Retrieve<Part>(query => {
    query.Columns(...); //Your existing Columns line

    query.Page(currentPageIndex, pageSize); //This is the key line 

    totalRecords = query.TotalRecords; //This field will give you the actual count in the database without retrieving them all (So in your case this would be set to 9000. 
});

Then all you need to do as you want records, you change the currentPageIndex. In this example itd get the first 20 of 9000 results, but if it was 1, it would return the 21st to 40th items, and so on

This will be substantially faster than getting all 9000 at once.

0 votesVote for this answer Mark as a Correct answer