Amazon S3 provider for the CMS.IO namespace

   —   
I’m really happy to announce the brand new provider for the CMS.IO namespace. This feature enables you to store all files in your Kentico CMS web application to Amazon S3 (Simple Storage Service). This service is part of the Amazon web services cloud service. This blog post guides you through the installation and configuration of this provider.
We have included this provider into the latest hotfix package 6.0.18. After applying this hotfix, you still have to go through several steps to make it work. Installation of the provider depends on the type of project you are using.

Web site project

No special steps needs to be done.

Web application/Windows Azure project

  1. Add reference to CMSApp project to CMS.AmazonStorage.dll.
  2. Include ~/CMSPages/GetAmazonFile.aspx into CMSApp project.

Source code

  1. Include the AmazonStorage project into your Kentico CMS solution.
    1. Go through Visual studio migration wizard if you are using Visual studio 2010.
    2. Set .NET version to 4 if you are using this version of .NET in the rest of the solution.
  2. Add reference to CMSApp project to AmazonStorage project.
  3. Include ~/CMSPages/GetAmazonFile.aspx into CMSApp project.
Configuration of the storage is the same for all types of projects and it’s all done via application settings keys in the web.config file:

<add key="CMSStorageProviderAssembly" value="CMS.AmazonStorage" /> - Specifies that the Amazon S3 storage assembly is used.

<add key=”CMSExternalStorageName” value=”Amazon” /> - Specifies a provider name (this value is used in URL generation logic).

<add key=”CMSAmazonBucketName”  value=”YourBucketName” /> - You need to specify the name of the bucket where you want to save the files. This bucket must already exist. A bucket name must be unique in the whole Amazon namespace so the provider doesn’t create it by itself.

<add key=” CMSAmazonAccessKeyID” value=”YourKey” /> - ID of an Amazon access key which is used for the authentication to the S3 storage.

<add key=”CMSAmazonAccessKey” value=”Your secret” /> - The Amazon access key which is used for the authentication to the S3 storage.

All the keys above are mandatory. There is also a set of optional keys:

<add key=”CMSAmazonTempPath” value=”PathToTemp” /> - Path to the directory where temporary files are stored. If you don’t specify this key, the temporary directory will be created in the OS temporary directory.

<add key=”CMSAmazonCachePath” value=”PathToCache” /> - Path to directory where cached files are stored. If you don’t specify this key temporary directory will be created in the OS temporary directory.

<add key=“CMSAmazonEndPoint“ value=“endpoint“ /> - Endpoint to the Amazon S3 storage. Here you can also specify the endpoint of the CloudFront content delivery network. If you don’t specify this key, the endpoint will be in format <yourbucketname>.s3.amazonaws.com.

<add key=“CMSAmazonPublicAccess“ value=“true/false“ /> - Specifies whether objects (files) in Amazon S3 uploaded through Kentico CMS are accessible for public users. The default value is true when an end point is specified. Otherwise, the default value is false.

I hope you will like this new provider. If you have any questions or remarks, let me know through the comments.
Share this article on   LinkedIn

Dominik Pinter

I'm a fan of cloud computing (primarily Windows Azure) and I really like to dig into web application security. My blog is focused on everything related to Kentico, .NET Framework, Cloud platforms and web application security.

Comments

kentico_dominikp commented on

Hi Adrian,

1) Is the website hosted in standard server or on EC2? Can you check if folders D:\ImageCache and D:\ImageTemp exists and contains some data? Is the project hosted in standard server or on EC2? Do you have JS minification enabled? If so can you turn it off and try to upload a picture again?

2) Do you see proper link in CMSDesk? If you insert an image through "Insert image or media" the link for the image is generated in that moment and saved into the database with other content (image is not generated everytime when someone visits a page). You need to regenerate your link the image (delete the image and insert it again) in order to create proper link.

3) What exactly does not work in Singapore region? Files are not accessible by generated URL? We did not test all regions but we did tests mainly on EU region (Dublin) and everything seems to work correctly there.

You can answer my questions here or write me an e-mail to dominikp (at) kentico (dot) com.

Thanks.

Dominik

Adrian Keevill commented on

Hey Dominik. Great news on the plugin - however, not having much success so far.

Web.Config keys (obfuscated some portions):
<add key="CMSStorageProviderAssembly" value="CMS.AmazonStorage" />
<add key="CMSExternalStorageName" value="Amazon" />
<add key="CMSAmazonBucketName" value="thenameofmybucket" />
<add key="CMSAmazonAccessKeyID" value="AAAAAAAAAAAAAAA" />
<add key="CMSAmazonAccessKey" value="BBBBBBBBBBBBB" />
<add key="CMSAmazonTempPath" value="D:\ImageTemp" />
<add key="CMSAmazonCachePath" value="D:\ImageCache" />
<add key="CMSAmazonEndPoint" value="http://cname.to.my.cloudfront.url.com" />
<add key="CMSAmazonPublicAccess" value="true" />


1) The issue I'm facing is that no Images are getting created in the Bucket after upload (via the Editor) - only a folder structure like:

app_data-->cmstemp-->multifileuploader-->f8-->f826d7e3-3716-4218-a600-224a1cdb8e58 <-- Empty Folder

and

__metadata-->app_data-->cmstemp-->multifileuploader-->f8-->f826d7e3-3716-4218-a600-224a1cdb8e58.META

The .META file contains: "LastWriteTime;7/20/2012 4:27:48 AM#"

2) When browsing to the site, the image is still displayed. When I look in Firebug, the image URL is still http://siteurl.com/folder/image.aspx. Shouldn't this now be the URL of CMSAmazonEndPoint?


3) This plugin only seems to work with the US Standard S3 store - we use the Singapore Region, but I couldn't find anything in the solution to easily change it. Perhaps a config key "CMSAmazonRegionURL" (if not specified, default to s3.amazonaws.com) would be a nice addition to the next release :-)

Any help you can provide would be much appreciated :-)

Cheers,

Adrian

kentico_dominikp commented on

Hi thanga,
yes you are right, I apologize for typos in my blog post. I have updated the post. I hope that it's all correct now.

thanga commented on

I figured out the issue. The correct config key are: Note, there NO ".dll" in CMSStorageProviderAssembly

<add key="CMSExternalStorageName" value="Amazon" />
<add key="CMSStorageProviderAssembly" value="CMS.AmazonStorage" />

thanga commented on

Is there any update on this issue? I am also getting the error that the assembly cannot be found

kentico_dominikp commented on

Hi Rick,
currently, you can use only 1 bucket for all your files. In future (probably next release of Kentico CMS), you will be able to map buckets to certain paths in file system.

You can inherit from classes in AmazonStorage and override its behavior, this is the only way how can you customize provider. Do you want to change some specific thing or you were asking in general?

Rick Seinfeld commented on

Perfect! What if I would like to utilize more then one bucket (one for video streaming, one for public data storage and one for private data storage), how am I supposed to do that? Is there any way I can customize the provider now?

kentico_dominikp commented on

Hi David,
are you using WebProject or WebApplication? For WebApplication, you must add reference to CMSApp project (WebApp project) on CMS.AmazonStorage.dll.

Setting key <add key="CMSStorageProviderAssembly" value="CMS.AmazonStorage.dll" /> should set up IsExternalStorage to true and also set up ExternalStorageName to "amazon".

At first point, it seems that you configured settings correctly. If you don't success please write me an e-mail (dominikp (at) kentico (dot) com) and we will figure out what's wrong.

David Conder commented on

Hi Dominik,

This is great to see! However I'm running into an issue. If I used the web.config settings you specify, whenever I try to upload a file, it doesn't work. When I click the "Save" button, the upload screen just refreshes and nothing happens.

I checked the value of: CMS.IO.StorageHelper.IsAmazonStorage and it's set to False, but CMS.IO.StorageHelper.IsExternalStorage is set to true.

I then changed the app setting CMSExternalStorageName to Amazon, and now CMS.IO.StorageHelper.IsAmazonStorage is true. However, I still can't upload any files, it just refreshes the page when I hit Save.

I finally tried adding the app key <add key="CMSStorageProviderAssembly" value="CMS.AmazonStorage.dll" /> but I'm getting an error that the assembly cannot be found (even though it's in the /bin folder).

Here are my actual web.config settings...can you take a look at what is causing the issue?

<add key="CMSExternalStorageName" value="Amazon" />
<!--<add key="CMSStorageProviderAssembly" value="CMS.AmazonStorage.dll" />-->
<add key="CMSAmazonBucketName" value="<bucketname>" />
<add key="CMSAmazonAccessKeyID" value="<access key>" />
<add key="CMSAmazonAccessKey" value="<secret>" />
<add key="CMSAmazonTempPath" value="c:\dev\kentico6_2\temp\" />
<add key="CMSAmazonCachePath" value="c:\dev\kentico6_2\cache\" />
<add key="CMSAmazonEndPoint" value="<cloudfront url>" />
<add key="CMSAmazonPublicAccess" value="true" />