In Azure you should have 3 separate web apps. Production is most likely yourdomain.com while the others are named with an alias based on the web app's name. Something like dev-webapp.azurewebapps.net.
You should be able to access the site at that Azure alias URL and receive a Kentico related message stating you don't have a key for this domain. You can then log in and enter a new license key and update the sites domain.
If you're getting a 404 or Azure blue page, they you don't have your web app configured properly. If you FTP'd your website contents to the web app, then you need to go into the configuration of the web app and set the path mapping to the site\wwwroot\CMS
directory. If you're using a repository to deploy your project, then it should do this for you automatically.
I'd suggest FTPing into your instance and looking at the file structure. Then going to the web app and