Small Tip - Adding one more point to Dawid's answer as I personally faced this problem, unless you will not delete default favicon from the Kentico's root installation it will keep appearing on your wesbite even if you add a favicon through a HEAD HTMl webpart.
In that case your webiste will have HTML for two favicons like
<link rel="icon" type="image/png" href="/lolinc/media/Images/yourwebsite.ico" />
<link rel="icon" type="image/png" href="/favico.ico" />
The default icon will always render later than your website's icon and will appear no matter what.