Ah, then this would be my fault, i didn't put the ~ at the beginning of the URLs, i never personally use sub sites so i guess it's a bad practice of mine.
Quick fix is to edit the \CMSFormControls\Custom\AdvancedCategorySelector\AdvacnedCategorySelector.ascx and replace lines 5 6 and 7 with the below:
<script type="text/javascript" src="~/CMSGlobalFiles/CMSFormControls/Custom/AdvancedCategorySelector/js/jquery.js"></script>
<script type="text/javascript" src="~/CMSGlobalFiles/CMSFormControls/Custom/AdvancedCategorySelector/js/AdvancedCategorySelector.js"></script>
<link rel="stylesheet" type="text/css" href="~/CMSGlobalFiles/CMSFormControls/Custom/AdvancedCategorySelector/css/styles.css" />