Well, this information is maybe bit hidden in the e-commerce customers related setup and registration. It can be found in this documentation page, all the way at the bottom:
"If you want to use a customer's email address as their user name for signing-in, you need to solve different sets of allowed characters. For example, email addresses are by default valid with plus symbols (+), but user names cannot contain this character. You can adjust the behavior with the CMSUserValidationRegEx web.config key."
And there is also an old KB article on this topic too.
Ohh and of course, you need to add that settings key into the appsettings.json in the front end app too. I forgot doing it and scratched my head for 10 minutes! :-)