What are the best practices for sending a high volume of email as quickly as possible?

Ed O'Brien asked on September 11, 2019 21:12

For reference, we've got Kentico running on two load-balanced servers in Azure - each has 840 ACUs (4 cores) and 14GB RAM. We're currently using the Default SMTP Server to relay the emails to a single SendGrid instance for delivery. Our Email Processing Batch Size (in Settings > System > Emails) is 50.

Our first send to 170,000 contacts took about 7 1/2 hours from start to finish. It took about 75 minutes for Kentico to generate the emails and send them to the Email Queue. The only macros in the email were for the unsubscribe and opt-out links. Looking through the Sent Email log, the emails were sent continuously (4-9 emails/second, averaging about 6) throughout the 7 1/2 period without any breaks.

We can probably live with the email-generation times (for now), but not the delivery time - we will frequently be sending time-sensitive and/or limited availability promotions, and would like all recipients to get their email at close to the same time. Compared to the ExactTarget platform I've had previous experience with, email generation and delivery to the same size recipient list completed in minutes, not hours.

Based on my research, it looks like the fastest way to send emails using Kentico/SendGrid would be to create a custom Email Provider. Does anyone know what throughput/volume of email is realistic using a custom provider vs. the default SMTP server? Or, is there a better/faster way to deliver email from Kentico?

Recent Answers


Michal Samuhel answered on September 12, 2019 09:53

Hi Eddie, I will put in excerpts of my two emails, so apologies about formatting or grammar.

Per my recent tests, you should be able to get out about 50-70 emails in a second. Which version are you using? There was slight problem with SQL query and paging on email generation part which was adjusted in one of the hotfixes in v12.

If contact data is collected emails are inserted into internal in memory newsletter email queue. This queue is processed by ThreadEmailSender. This thread is collecting data from queue and only during send an email template is retrieved. Afterwards our internal class calls EmailQueueManager.SendEmailsViaQueueManager. This method and subsequent calls and applies few sets of filters and macro resolvers over email content. Only after this, all of data is send into database email queue. DB email queue is than handled by ThreadSedner which picks up emails based on their importance level in batches and sends them to available SMTP server. In case of multiple SMTP server, round robin is applied to distribute between SMTP servers which are available.

Most drawbacks, I had seen were during macro resolving (our email widgets are built on top of macro engine as well) and this causes longer email generation. As most of these can be personalized or use contact context we can not cache any of these. Can you enable macro debug and see whether there are any delays, or make sure, there are no computing heavy operations, or for example no extra DB calls inside of email content? Contact context should for example be present or loaded by ID from contactInfo hashtables. This is another step, which should be checked.

Can you make sure, that ThreadSender is always running within the application? Can you check how often is procedure cms.email.fetchemailstosend called to SQL server? This should fetch new emails from email queue into data set, which should be processed by threadSender. Each email from retrieved set should be set to 'sending', objects should be enqueued into Queue and processed with callbacks and rather heavy logic.

Another one, would be email queue itself and communication across the board with email servers. Can you check both DB and application servers resource consumption? I want to make sure, there is are no spikes in either DB nor application server.

Macros do slow down the process, but that does not interfere with email queue. Email queue as it is does hold already generated emails. If email generating is not a problem at this point, I think we can focus on queue itself. Just few points to add to CPU consumption, macros have to be resolved and that can be slightly more consuming on resources. For sake of user experience we created email builder and widgets are using macro engine to be created and to enter and save data into emails and to populate dynamic values. We try to resolve all global values just once, however, contact data and personalization is unique and has to be resolved per email.

EmailQueue should alone have a better throughput, than email generation. It is serviced by ThreadSender thread which will try to take emails out of it and send them in batches of 10 emails. This is a singleton thread, which should process queue as long as there are any items in it. It fetches batches per size set in settings. It queues them and processes this in memory queue by calling SMTP servers and sending emails. I see a few things in this process and internal mechanism which could cause problems. One is that if number of items in above mentioned memory queue is over treshold ManualResetEvent is triggered to stop fetching more emails from DB. Threshold is hardcoded to 100 items, so larger batches could trigger extra resets. Another one would be static or restarted application. If no traffic is coming to the site, environment can shut down or restart the application and thus killing all running threads including mail sender. So keep application alive as well.

0 votesVote for this answer Mark as a Correct answer

Ed O'Brien answered on September 12, 2019 22:01

Thanks for your very detailed response, Michal - it's encouraging to learn that we should be able to send 50-70 emails/second. That throughput should be sufficient to keep the marketing department happy. We are current on our K12 hotfixes, so we'll start monitoring to see where our bottle-neck is - I'll update this question with my findings once we've resolved the issue.

0 votesVote for this answer Mark as a Correct answer

Ed O'Brien answered on October 16, 2019 19:18

We were able to increase our throughput significantly (50,000 emails in under 30 minutes) by cloning our SendGrid SMTP server in the SMTP Servers app, increasing the frequency of the Send queued emails scheduled task, and allocating more CPU in Azure. Per the SendGrid documentation, "you may open up to 10 concurrent connections from a single server" and "send up to 100 messages per SMTP connection". We created 50 clones, set the scheduled task to run every 5 seconds (our batch size in Settings > System > Emails is 1,000), and increased our CPU by 33%. Since we made several changes, it's not clear how much each change contributed to the increased throughput - we may dial some of these changes back to find the sweet spot, but hopefully this helps someone experiencing similar issues.

0 votesVote for this answer Mark as a Correct answer

   Please, sign in to be able to submit a new answer.