Configuring Azure storage - how do you migrate existing media?

Mark McDonald asked on July 30, 2021 16:43

So we've set-up Azure storage for some existing sites, and mirrored the settings to work with the same folder structure. We have tested this and new uploads, once connected, go into Azure in the expected location and show in the media libraries.

We now have to figure out how to migrate existing media to Azure, the documentation seems to miss this scenario completely. The biggest part of the problem looks to be that the way Kentico uses Azure it sets everything to lowercase - so if you just copy mixed case file and folder names into Azure storage they are not seen by the CMS.

As such I'm guessing we need something to go through the folders and files and covert to lowercase, and are there any other gotchas like special characters that could cause problems when migrating media?

Correct Answer

Kadir Ergün answered on August 2, 2021 20:26

Hi Mark,

We have been there ,Kentico documentation explain how to integrate Azure storage from beginning. Windows is not case sensitive but blobstorage is. Situation here, Kentico treats all media library files like lowercase when blobstorage activated.

Help of Microsoft Turkey team in multiple Azure migration projects we run small pice of code that convert all media files to lowercase in your blobstorage folder recursively.

I'am sharing the code blow, than you can easly change some properties and run the Console app. Runtime depends on your media library size.

class Program
{
    //change YOUR_ACC_NAME and YOUR_ACCOUNT_KEY
    const string AzureStorageConnectionString = "DefaultEndpointsProtocol=https;AccountName=YOUR_ACC_NAME;AccountKey=YOUR_ACCOUNT_KEY;EndpointSuffix=core.windows.net";

    static CloudStorageAccount storageAccount = CloudStorageAccount.Parse(AzureStorageConnectionString);

    //Your container name where cms puts media files
    const string containerName = "";  //ex : cmscontainer

    //Your website name lowercase
    const string websiteName = "";  // ex: medioclinic

    static void Main(string[] args)
    {
        Console.WriteLine("Started!");
        CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

        CloudBlobContainer blobContainer = blobClient.GetContainerReference(containerName);


        ListBlobsHierarchicalListingAsync(blobContainer, websiteName).Wait();
        Console.WriteLine("Ended!");
        Console.ReadKey();
    }

    private static async Task ListBlobsHierarchicalListingAsync(CloudBlobContainer container, string prefix)
    {
        CloudBlobDirectory dir;
        CloudBlob blob;
        BlobContinuationToken continuationToken = null;

        try
        {
            // Call the listing operation and enumerate the result segment.
            // When the continuation token is null, the last segment has been returned and
            // execution can exit the loop.
            do
            {
                BlobResultSegment resultSegment = await container.ListBlobsSegmentedAsync(prefix,
                    false, BlobListingDetails.Metadata, null, continuationToken, null, null);
                foreach (var blobItem in resultSegment.Results)
                {
                    // A hierarchical listing may return both virtual directories and blobs.
                    if (blobItem is CloudBlobDirectory)
                    {
                        dir = (CloudBlobDirectory)blobItem;

                        // Write out the prefix of the virtual directory.
                        Console.WriteLine("Virtual directory prefix: {0}", dir.Prefix);

                        // Call recursively with the prefix to traverse the virtual directory.
                        await ListBlobsHierarchicalListingAsync(container, dir.Prefix);
                    }
                    else
                    {
                        // Write out the name of the blob.
                        blob = (CloudBlob)blobItem;

                        await RenameAsync(container, blob.Name, blob.Name.ToLower());
                    }
                }

                // Get the continuation token and loop until it is null.
                continuationToken = resultSegment.ContinuationToken;

            } while (continuationToken != null);


        }
        catch (StorageException e)
        {
            Console.WriteLine(e.Message);
            Console.ReadLine();
            throw;
        }
    }

    private static async Task RenameAsync(CloudBlobContainer container, string oldName, string newName)
    {
        CloudBlockBlob source = (CloudBlockBlob)await container.GetBlobReferenceFromServerAsync(oldName);
        CloudBlockBlob target = container.GetBlockBlobReference(newName);


        await target.StartCopyAsync(source);

        while (target.CopyState.Status == CopyStatus.Pending)
            await Task.Delay(100);

        if (target.CopyState.Status != CopyStatus.Success)
            throw new Exception("Rename failed: " + target.CopyState.Status);

        await source.DeleteAsync();
    }

}

I hope it works for you. Please test it somewhere else than production, than use it.

0 votesVote for this answer Unmark Correct answer

   Please, sign in to be able to submit a new answer.