Forums (Obsolete)

Member

matt.barry-stockhouse - 6/19/2012 5:28:58 PM

Continuous integration?

Hi there:

So I've gained some experience with Kentico, building page layouts, web parts, etc. Now I would like to set up automated builds, unit tests and deployments using a CI server such as Team City or CruiseControl.NET. I want to do the following:

1) Ensure that my website builds so that user's don't encounter errors on a dynamic page compilation.
2) Run automated tests using tools such as nUnit, FxCop and Sellenium.
3) Add web.config transformations so that I can maintain different settings for debug, test, release, etc.

Obviously these things are going to require a web application project instead of a website project. So I re-ran the installer and created another instance of Kentico using the Web application project. No problems there. The issue I have is that each time I build the solution, the first load of each page is horribly slow. What is Kentico doing? Aren't the code-behinds compiled into an assembly in the bin folder when I build the solution? My development machine isn't a rocket, but it's got 8 cores and 16GB of memory. No wonder Kentico recommends a website project!

So here are my questions:
1) Is anybody successfully doing continuous integration using Kentico?
2) Is anybody developing using the web application project type?

Thanks in advance!

Kentico Consulting

kentico_borisp - 6/22/2012 2:54:47 AM

RE:Continuous integration?

Hello,

We recommend the website project because the development time is much shorter and you don't need to compile the site after each small change. The first load is caused by saving the content in the .NET cache. This shouldn't be an issue for your visitors, since the first load is usually performed by you as a developer after testing the changes on the page. Afterwards the loads should be significantly shorter.

Best regards,
Boris Pocatko


Member	nico.leroux-bmf.com - 8/16/2012 12:29:21 AM RE:Continuous integration? I'm currently trying to implement CI into my web application using teamcity. The problem I'm facing is the sheer size of a Kentico build coupled with pushing something to a staging/production environment where I need to make sure files are removed and new files updated. HAve you had any luck in doing this? Thanks!

Member

matt.barry-stockhouse - 8/16/2012 12:09:56 PM

RE:Continuous integration?

Not much luck. CI with Kentico is like pounding a square peg into a round hole.

Here's what I've learned:

1) Most of the Microsoft infrastructure to perform builds and deployments is set up for Web Application Projects (WAPs).
2) Although you are given the option of installing as a WAP, Kentico recommends installing as a WebSite Project (WSP). This is because development is much quicker using a website. You do not need to re-compile the entire project every time you make a change.
3) If you intend to do any unit testing (e.g. nUnit) you need an assembly to run your tests against. Obviously this won't work with a Website project. You would need to use a WAP.
4) You don't really "build" a WSP the same way you build a WAP. You can run the aspnet_compiler process on the website in order to verify or pre-compile it. However the compiler is slow and you should expect it to take 20+ minutes. Not ideal if you're looking to set up a continuous build.
5) If you do intend to pre-compile your website, you'll have to manually go into the CMS and export all of the portal engine objects to disk. Otherwise you'll get errors when you visit the pre-compiled site.

Here's where I'm at:
1) I went with a website project. I found developing a WAP to be painfully slow.
2) I'm trying to keep my logic in separate class library projects. That way I can run unit tests against the class library assemblies. Better than nothing...
3) I haven't solved the pre-compilation problem. I might try taking the code from the CMS that exports all of the virtual path objects and put it in a custom MS build task. I don't know if it will work without an HTTP context.
4) I'm using the RoboCopy msbuild task to sync files between my build server and my development server.
5) I intend to look at Web Deployment Projects. I think they can be used to package up a WebSite Project for deployment.
6) I still have no idea how I'm going to version and deploy changes to CMS objects (CSS, layouts, page templates, etc.). The portal engine makes this very difficult.

Hope that helps,

-Matt

Member

nico.leroux-bmf.com - 8/16/2012 6:45:13 PM

RE:Continuous integration?

Thanks for that Matt.

I got my WAP running OK and after uninstalling some modules and languages got it to a smaller size (still not exactly reasonable at around 150MB) Pushed my references into a solution items folder and having a go at using a web deployment project now. I will do the same using a seperate class library with my code.

Remote servers still running a little slow when reloading the assemblies for the first time, but is important that we have multi-developer capabilities on our projects, especially managing the gap between front-end jquery/images and back-end customization.

Member

matt.barry-stockhouse - 8/16/2012 7:11:01 PM

RE:Continuous integration?

I did the same thing with my WSP, removing unused modules and languages. It knocked a couple minutes off of my compile time. I'd be curious to try it again with a WAP.

Multi-developer is important to me as well. I noticed the following new feature in Kentico 7:
Check-out/check-in support for page templates, transformations, CSS stylesheets and other development objects which will help avoid modifications being made by multiple developers at the same time.

And you might be able to use the IIS auto-start feature to load those assemblies before a user hits your site.

Anyways, let's keep this thread going...

Member

nico.leroux-bmf.com - 8/20/2012 2:34:32 AM

RE:Continuous integration?

Thought I'd share a big (well for me) breakthrough today.

Thought all hope was lost as I ran into Web application project (or more correctly pre-compiled websites) are unable to use the VirtualPathProvider. This was quite a setback as it removes the ability of tranformation and layout management on the CMS side instead of the files.Would just be too taxing to have the front end developers checking code in and out and using CI for every change.

Did find a "work around however" at link below. Had to put my thinking cap on and replaced the first line with this:

VirtualPathProvider providerInstance = new CMS.VirtualPathProvider.DbPathProvider();

http://sunali.com/2008/01/09/virtualpathprovider-in-precompiled-web-sites/

Seems to be running ok again, but will need to keep an eye on this.

Currently have only necessary directories synchronising with build server and FTP'ing this with changed bin folder up to working environment. Pretty quick (normal 3 minutes CI process)

Not too concerned about IIS restart as developers would normally double check their work by loading the site.

Anyway, thought I'd document as someone else might be running into this issue at some point.

Member

matt.barry-stockhouse - 8/21/2012 1:08:41 AM

RE:Continuous integration?

That's interesting. Where did you put the code to create the providerInstance?

The developer's guide says something like this:

Since the virtual provider cannot run in a pre-compiled web site, you need to store the physical files to the system before compilation. You can save all the virtual objects to the file system in CMS Site Manager -> Administration -> System -> Deployment interface by clicking the button “Save all virtual objects to disk”.

I found the code that does this in KenticoCMS\CMSModules\System\System_Deployment.aspx.cs. I had planned to copy it into a custom MSBuild task and see if I could run it as part of my build.

Member

nico.leroux-bmf.com - 8/21/2012 3:12:16 AM

RE:Continuous integration?

I register it at Application_Start in CMSHttpApplication in Old_App_Code. I remember there might be a better place to plug your custom registration in somewhere else, but this is where it lives now.

Needing to have physical files on the site severely cuts down abaility to develop through the CMS (portal mode). This would have meant developing EVERYTHING from transormations to layouts and I think CSS too from development box and push using continous integration to servers.

Now the database handles everything other than physical .Net file changes. In my mind the content of the site sits in the database, whilst the filesystem itself runs through continous integration.


Member	matt.barry-stockhouse - 8/21/2012 12:39:18 PM RE:Continuous integration? I'm thinking about moving to physical files for CSS (and maybe more): 1) The portal engine doesn't support Sass or LESS 2) The portal engine doesn't allow me to commit my changes to version control 3) The portal engine doesn't prevent me from overwriting another developer's changes (atleast until Kentico 7 is released)

Member

nico.leroux-bmf.com - 8/21/2012 8:05:41 PM

RE:Continuous integration?

Sounds like a good way to do it. Main reason I'm keeping the database managing content is we have a hectic timeline on the project, and templates/content/styling/functionality all need to kinda happen at the same time by different people (Some non-technical).

Think your structure would work nice if you're able to run the Administration -> System -> -> Deployment -> Save all virtual objects before pushing the files up to your relevant system

Member

matt.barry-stockhouse - 9/10/2012 7:09:08 PM

RE:Continuous integration?

I managed to get this working, but instead of registering in Application_Start, I did the following:

1) Added a new class library project to my solution.
2) Added a new class called DbPathProvider that derives from Kentico's CMS.VirtualPathProvider.DbPathProvider.
3) Overrode the Register() method to call the RegisterVirtualPathProviderInternal (as per the article).
4) Added the CMSVirtualPathProviderAssembly appSetting to the web.config and pointed it to my custom virtual path provider assembly.

Everything seems to work as long as I pre-compile with the "updatable" option. Did you get it to work without "updatable"? I suspect this would be faster.


Member	george-partial - 9/8/2012 9:58:32 AM RE:Continuous integration? Kentico CMS is not ready for CI :( I've tried to setup up CI on Kentico CMS several times with different versions and haven't yet got how to do it :( Try to ask somebody from Kentico (maybe CTO), but my uderstanding is, they don't know about CI, TeamCity etc. well ....

Member

martinh - 9/10/2012 4:08:17 AM

RE:Continuous integration?

Hi,

You are right, we are not using any of these things yet, in fact we are currently at the very start of that, and these are quite unexplored waters for us.

Also, except for few of you (out of several hundreds of others active clients), there was no real traction to get this done before, so we put priority on more obviously wanted things but we would like to allow this as well. I agree that the current support for it is lousy.

Not knowing this much, it doesn't help up very well to hear what is wrong, because we can't really get the whole picture out of that. We need to get an idea what the ideal scenario would be, which parts we currently provide and which we are missing, and where it doesn't fit your needs and how to change that.

.. or, alternatively we can adress your issues and provide a best practice on how to do it with current version.

If you could provide that, we can start moving things towards that point.

Thank you

Member

@davey_lad - 11/7/2012 9:04:53 AM

RE:Continuous integration?

Finally... I've found a thread that confirms that i'm not going mad!

A good Source code control strategy and CI have really given me headaches since I started using Kentico to the point where I've pretty much given up (on CI at least). I've been spending more time on this than actual "real work" lately. I'm glad (sort of) I'm not the only one.

In my opinion it all boils down to the dreadful compilation times for the Web Application project type. If Kentico optimised/refactored their platform so that this was the default (and recommended) project type it would be a start. Individual developers would then figure out their own development and deployment strategies themselves.

As an earlier comment mentions, everything now (plus a few years) is geared towards WAP. The website project type is something that should be assigned to history and labelled with "lesson learned". As it stands, it the only option in Kentico that is actually usable.

Member

matt.barry-stockhouse - 11/7/2012 11:54:59 AM

RE:Continuous integration?

The first thing you want to do in CI is to get your code under revision control. The portal engine makes this difficult because it encourages you to keep your development assets inside the CMS. Think about page layouts, page templates, CSS, web part containers, transformations and settings. These things exist in the database where you can't really version them.

I think there needs to be an option to either create these items as files or export them (maybe as XML). Then, from MSBuild, you need to be able to deploy the changes. Say, for example, you could serialize your page templates to XML. There would be a command-line API to import them into the target environment. That way you could deploy your changes without overwriting the whole database.

To be honest, I'm not sure if there are any CMS solutions that really get this stuff right.

Member

martinh - 11/9/2012 10:48:52 AM

RE:Continuous integration?

Hi,

Regarding using source control, this new feature in v7 should help you:

http://devnet.kentico.com/Blogs/Stepan-Kozak/September-2012/Develop-Team-Built-Websites-More-Efficiently!.aspx

I know it is not perfect, but it may be a good thing to start with for now.

Just so you know, we are spending significant time on v8 revising the architecture, and redefining the core of the platform, including implementing Unit tests and basically get rid of the main constraints from the history of our code, so it should get much better with the next version. If you have any particular suggestions what we should not forget to include, please let us know.

Member

Accepted solution

ehuggins-pinnacleofindiana - 11/16/2012 1:24:02 PM

RE:Continuous integration?

I am working with developers on our first Kentico project and am implementing Continuous Integration as follows (for now):

1. Web Application Project
2. Visual Studio for application development
2. Microsoft Team Foundation Server (TFS2010) for code and database source control
3. Red Gate SQL Tools for database versioning
4. Microsoft Web Deploy (MSDEPLOY) for deploying WebApp
5. TeamCity 7.1.1 for Continuous Integration management
6. Three Team City Build Agents for builds of MVC3, MVC4, Sharepoint, and other .Net and C# projects.

I currently have over 125 active builds in our TeamCity Continuous Integration environment, most of which use MSDeploy for Web Application and Web Service deployments. We just started using RedGate for SQL management, with the Kentico project our first.

I setup the development Kentico database for Source control yesterday and configured the new RedGate TeamCity plugin for Schema Synchronization last night - deploying WebApp and DB to UAT via TeamCity. All worked pretty much the same as our other environments, with the addition of the DB component.

Deployment to UAT from Development is now a chained build which first generates the MSDeploy Package and Synchronizes the DB schema from Dev to UAT. The deployment then picks up the database documentation for the updated UAT environment and the MSDeploy Package and updates UAT. Total deploy time to UAT about 10 minutes.

The missing piece was data synchronization, which I worked through, at least partially this afternoon using the Red Gate Data Compare tool. I already have an automated build component I can migrate over to the Kentico project to automatically synchronize databases as well.

While still a work in progress - I believe we will be able to address most of the issues I can see ahead of us with this set of tools in order to maintain our focus on Continuous Integration.

I have been very happy with TeamCity and MSDeploy. The RedGate tools look like another big piece of the CI puzzle.

Next up - automating Functional testing. Currently evaluating Selenium, Telerik Test Suite, and Microsoft Test Manager.

Happy to share ideas, successes, and frustrations with any and all.

Eric Huggins
Continuous Integration Manager
Pinnacle of Indiana


Member	matt.barry-stockhouse - 11/19/2012 10:56:03 AM RE:Continuous integration? Thanks for sharing! I'm definitely curious about which Red Gate tools you're using and how you have them configured. The database piece is the missing link for me. I need to figure out how to get dev changes to production without blowing away content. And I'd like to be able to roll back if anything goes sideways.