You should add the gif file you like to use as an icon of custom document into the directory below:
Then rename the file according the code name of the document type. Follow the example below how to rename the file:
So the name consists of namespace and document type of that document separated by underscore.
UPDATE for v5.5:
the location of the icon files was changed to \App_Themes\Default\Images\DocumentTypeIcons</span>