Although nowadays quite accurate, I wouldn't 100% rely on the geolocation by IP. If your ISP is located in a different location it might show up not what you expected.
Also not sure how you test it, but as you can read in the following documentation it's not updating your contact details on a 2nd visit:
What you could do for debugging is output the current location (to screen or event log) to see if it has the correct one.
Secondly depending on the scenario you might want to introduce other ways to determine a user is from a specific location. A form you bind to the contact location, a CTA button on your homepage...