Performance tips – ObjectTransformation

Hi there,

It happens to everybody. You need to display some content, let’s say pages, along with related information referenced from the original data just by a foreign key. Now how would you do that?

The first thing that probably comes to everyone’s mind is to just put code into the transformation that gets the additional data for each item, and renders it. Believe it or not, doing it this way is one of the most common mistakes when it comes to performance, even if you set up caching for the loaded data.

When dealing with performance (or any other mission-critical parameter), it’s important to think about the worst case scenarios, not the most common ones. For performance, the worst case scenario is always the first load when no cache is available at all, which pretty much takes your caching efforts away for that single moment. Caching is great for improving performance, but its primary purpose is to help make the website more scalable, not to give super-fast performance on all possible occasions. So always think about both caching and the basic efficiency of the process used to get the data.

For the purpose of this article, let’s just narrow down the example to the displaying of products.

If you’re displaying a detail of a product together with some related information (e.g. a status, or the user that created it), the system only queries the related information once. So if that call is efficient enough, you are not in that much trouble when the data is not cached.

A very common mistake in performance optimization is that people tend to focus on optimizing individual tasks, and often forget about what we can call the “multiplication factor”.

Yes, you not only display details of single products, but most importantly lists of products. That means that if you want to display the same related information in a list, the same operation multiplies by the number of items in the list, introducing many round trip times and parsing on both the database and application level. Again, you need to think about the worst case, as if each of the products has different related information and the cache can’t be shared between them. Even if the related data is loaded by a provider that caches individual items internally, the cache isn’t available for all items on the first load. As you call the API, the provider queries the items one-by-one, since it doesn’t know the future API calls that will be made for the same request.

We actually have two best practices for making this process efficient. One is public and you may have already learned about it from our support and consulting. The other we use mostly in the Kentico admin UI, but I thought it would be useful to share it with you. You can use the second approach for developing both custom admin UI and the live site.

Practice 1 - Use custom queries and JOINs

A typical solution used in most of the products out there is to get more low-level and make sure that you already load the additional data in the initial database query.

A typical process that our clients use is the following:

• First build the listing with regular components, without displaying the related information, and capture the SQL call that gets the data from the database (through the Kentico Debug, SQL Server Profiler or other tools)
• Then, either use a Custom query repeater, or a combination of a Basic repeater and Custom query data source to achieve the same while using the previously captured query
• Modify the query with JOINs (preferably), or nested queries to add the required additional data to the result on the SQL server side
• Sometimes it may be a good idea to build a View using the query, and then load data from that view
• Modify the transformation to display the related data using the columns in the data source

This kind of low-level solution is great for performance, but comes with some drawbacks:

• It is strongly tied to the Kentico database structure, therefore it may not be completely upgrade-proof in future major updates
• Some features, such as permission checks for pages, are done on the application layer, not the database layer, so you are not able to effectively use them in a custom query
• If you already have an existing integration with an external system, or plan to have one in the future, this approach introduces additional maintenance that you will need to handle over time; the more views or possible ways how data gets from the database to the application you have, the more things you need to figure out and test

I am not going to describe this particular approach in any more detail, as solving things this way is quite common.
Instead, I want to show you a treat in Kentico that solves this problem more efficiently with all the above problems in mind.

Solution 2 – Object transformation

If you think about my introduction a little more, technically the only problem in the listing scenario is that the system cannot predict future calls and therefore cannot bulk-query the related data. If we solve this single problem, then there are pretty much no other issues left.

We thought about this in Kentico and introduced the Object transformation concept. In addition to better performance of the UI, it helped us eliminate unnecessary Views and JOINs, which opens easier integration options on the data level.

Before I get into details about how to use object transformation, I want to give you some high-level background information about the feature.

What is it?

Simply put, ObjectTransformation is a web forms control capable of displaying related data and swarming. If you have more of them on a single page, they group together in their effort and get the data together in a single joint query.

How do you set up one particular instance?

It has three main properties:

• ObjectType – The object type of the target object, typically just the lowercased class name for most objects
• ObjectID – Foreign key (ID) pointing to the target object
• Transformation – Either a column name of the target object or text containing K# macros that define how the data will be rendered

Here is an example of how you can use an ObjectTransformation control in the product list transformation on the sample Corporate website:



In this case, I am using just the "FullName" column name for the output. If I wanted something more complex, it could look like this using K# macros:  "{% FullName %} ({% UserName %})"

What about the life-cycle?

As I said, these controls swarm to share a single round-trip to the database when querying data. To be able to do that, the process uses the Web forms life-cycle.

The life-cycle utilization is the following:

• You place the control onto a page and set it up, typically in the Init / Load / DataBind phase
• During the OnPreRender phase, the controls ensure a shared data source for each object type that you use, and register their related object IDs within it
• During the Render phase, the data source gets the data for all registered items at the same time using an optimized method BaseAbstractInfoProvider.GetInfosByIds(objectType, ids), providing each transformation with an instance of the object
• In the same phase, the transformation text is resolved using the standard macro resolver and the provided data

And that is pretty much all the magic behind the scenes. It’s not rocket science, right?

Results

Just to show you the outcome, I used the previous example on the sample Corporate website to efficiently display the owner of pages. The code inside the transformation including the object transformation is the following:


 
And the presentation is the following (note that I changed the page owners from the default ones to display more variable information):

What about caching?

I have mentioned the method BaseAbstractInfoProvider.GetInfosByIds. This is the most efficient way to get a strongly typed list of objects while leveraging as much provider caching as possible, and keeping the need to instantiate new info objects to a minimum.

What this method does is that when the provider has “Use ID hashtable” enabled (if you go generate code for a class within the new Modules interface, there is a checkbox for that), it first attempts to get all objects already cached by the provider. Then it takes all the remaining IDs and gets the data for all the remaining objects from the database using a single query. It also caches all new data and returns all that data joined together. This means that each subsequent call is efficient, even if it includes previously used IDs.

For a non-cached provider, it just gets data from the database and instantiates objects on-the-fly. No caching is involved (as of today).

So technically, if you use an object type that has this type of caching enabled, everything is set for you with the maximum level of caching. Object transformation itself does not currently provide any caching capabilities. If you use a provider that does not cache objects by IDs, the most appropriate type of caching would be the closest level above the ObjectTransformation control, which is partial caching of the web part output as a whole. There are also some alternatives, such as leveraging a custom data source for object transformation, or enabling caching on the provider level, but I will get back to that in other articles.

How does the performance compare to the low-level approach?

Doing things low-level will always be the best performing solution, so for really critical operations, it may still be the best way to go. You always need to find a balance between performance and ease of use.

As the Kentico API on its own does not have as much overhead as regular LINQ (all forms of it) thanks to how DataQuery works, instantiation only adds a very small amount of overhead when compared with the low-level access. Note that for larger target objects, object transformation loads all object data, and you can’t select just specific columns to be retrieved, so that may be another aspect to consider.

The decision is up to you, try both approaches and decide for yourselves.

How do I use this thing in the UI as you do?

I mentioned that we use object transformation in the admin UI and you can as well, either by placing the controls onto pages, or using the built-in support in listings (UniGrid). You can use two different approaches for listings:

• Return the control within OnExternalDataBound handlers of UniGrid – just create a control that you set up as above and return it
• Use the built-in support in the UniGrid configuration

I believe the first way is clear (if not, see this article), so let me comment on the built-in support. In general, if you search the whole web project for the text “#transform” in .aspx, .ascx and .xml files, you can find several typical examples.
The basic formats that you put into the externalsourcename property of the column XML definition are:

#transform: <object type>.<column name>

#transform: <object type> : <transformation text with macros>

In both cases, the object ID is taken from the value of that column (which must be a foreign key to that related object).
Here is an example from the Contact management interface:


 
As I said, probably the easiest way to learn is by example. Take a look at how we use the feature in the web project.  If you need more information about built-in transformations, see the documentation for the <column> element in UniGrid definition reference, and Adding custom UniGrid transformations.

Object transformation form control

In addition to the previously mentioned usage in listings, you can also use this concept in your editing forms to display related information. We provide the Object transformation form control to allow this. When you assign the form control to a foreign key column in a form, and provide the object type and transformation, you can easily display related information in that form.

You can see an example in Modules > E-Commerce > Classes > Order > Alternative forms > Update general > Fields > OrderCustomerID.

Can I use object transformation in MVC?

Not really, that is not how MVC works. Unless you use the WebForms view engine with late binding, and that would make it more of a hybrid solution rather than pure MVC. But if you want to use something similar in MVC, you can collect IDs from the input data while you are preparing your model, query the data using the GetInfosByIds method, and add that related data to the model.

Wrap up

Today, you learned about the powerful concept of Object transformation, which you can leverage to develop performance-efficient listings with ease. Think carefully in each situation, and decide whether you want to get things done using this approach or custom queries.

Once I get into further details about module development in upcoming articles (as a follow up to the Kentico Connection sessions), I will also show you some more representative examples of using object transformation while developing the admin UI, including a couple of advanced scenarios.

We plan to further improve this model, so any feedback is appreciated.

Enjoy!