I think security settings are based on node, which is shared across all culture versions of the particular document. I'd recommend to utilize custom event handler for document events and implement AuthorizeDocument
event, where perform appropriate check of document culture and user culture. More details here.