What are you using to display the navigation on the page? If you're not using one of the default "menu" webparts, I don't believe this works by default. You'd need to add in your where clause:
DocumentMenuItemHideInNavigation = 1
to exclude those items that are not checked.