Handling of custom tasks

I believe only one of the servers should run the task (it may not always be the same one).

From my experience running locally connected to the DEV database (similar scenario), tasks will only run on the web server, not my local. If you want to run a task on local, the only way to do it is to disable the task, then manually execute it. Then it will run from whichever web server made the request. I am not 100% sure, but for some reason I am thinking it runs on the server with the domain that is set up in the site.

You can also specify server name in task settings, that should execute a task.

see TaskInfo properties http://devnet.kentico.com/docs/8_1/api/html/AllMembers_T_CMS_Scheduler_TaskInfo.htm I guess you can use either TaskServerName or TaskSiteID

Technically speaking Stefan, unless you have a multiple server license or a web farm license, your setup is not in compliance with Kentico licensing (unless you're doing an active/passive setup). If you're using a web farm setup, you can specify which server (by name) it runs on in the actual task configuration.

Since each server has their own running IIS instance, they may execute at any time. For instance server 1 runs a task now and in 5 minutes the other server is scheduled to look and see if it's time to run that task. If the caching is different because of restart times in IIS, it could cause one task to run more or less frequently than desired.