I upgraded the site version 10 to 11 completed.
I am unable to login the site in existing user and getting popup window.
I clear the cache and clear the browser history.
Try clearing your browser cache. If that doesn't work, resign your macros in System > Macros. Most likely your hash-string changed in your web.config
If you are getting a popup, this sounds like Windows AD authentication is configured and you should provide your AD credentials to get in.
Please, sign in to be able to submit a new answer.