We added the fields to the User Class (Modules -> Membership -> Classes -> User). We definitely aren't adding them to an alternative form.
I've also noticed that when selecting these custom fields the Field name, Field type and Field Size properties are disabled. It seems to me these custom fields are now recognized as system fields.
One thing that may be related - I clicked the "Generate default form definition" button (to attempt solve a different issue). Could this have caused it?
Is there a way to set these fields as "custom" instead of "system"?