Hi,
Thank you for your message.
That is correct, according to Media_Library table our libraries are bounded to a particular SiteID so inside CMS desk you would need customization in order to show all media libraries. However that doesnt mean that you could not get the media libraries files in API by selecting the them manually from Media_File table.
Let us know if you have further questions.
Kind regards,
Richard Sustek