Sounds like you're missing an attribute on the class.
In the instructions, when you define your API class, you are suppose to put this attribute:
[assembly: RegisterCustomClass("MyClassName", typeof(CustomClass))]
Did you perform this step? You may need to call a different "Register" method other than "RegisterCustomClass" as well, but i believe that should be the issue.
https://docs.kentico.com/display/K8/Loading+custom+classes+from+App_Code