Looks like the issue is with my templates calling individual css and js files. If I put some cache busting on the links, the css and javascript seem to load fine. Appending say "?v=1" to the link loads my cache/javascript fine with debug mode set to false.