Limit file types to media library per directory

I would use global event handler and catch the media file object type - insert before event. Check the folder, check the uplaoded file and then either continue and let the file to be saved and uploaded or cancel the operation.
Or, another option would be creating a fully custom module with custom logic, where you will do the check before you call the media file API. So the users will have access to this custom module only.

Well, not sure how to even begin doing that since, like I said, I can only edit the existing Kentico CMS files manually because I don't have a project/solution file for the CMS code itself.

Is there a specific file (e.g. MediaFileUpload.ascx.cs) that I can modify to achieve this?

Or failing that, even easier would be to add text to the file upload frame at the top near the [Upload] button to remind users not to upload doc files into the image directories and vice versa.

Well, for global event handler you need to at least build the solution, e.g. using MS Build tools.
You can see what page is displaying given part of UI e.g. like described here. But it is possible that the UI page is driven by UI element and DB content. And if you are using web appliction project type, you need to build the change too.
But since oyu said you do not have all the solution files, it looks like the project is precompiled so the changes made to the code files will most likely not apply. You need to get the original, full solution with all the files to do the changes.