My answer would be combination of Jan's and Milan's approach. Both are good in their own way.
- I would not default not delete favicon file in case my server has multiple websites.
I would upload my favico.ico(making sur that it is "16 by 16", 32 by 32, 48*48 file and optimized) to site specific media library and use it either through HEad HTMl webpart or write code directly in the Master Page's head section
<link rel="icon" type="image/png" href="/website/media/Images/website.ico" />
<link rel="shortcut icon" type="image/png" href="/website/media/Images/website.ico" />
In your question, you do not mention the mobile browsers. Most of them will ignore the favicon.ico file. Although your site may be dedicated to desktop browsers, chances are that you don't want to ignore mobile browsers altogether??
I hope this helps.