Turns out, the issue was that Cloudflare was setting the x-frame-options: SAMEORIGIN header.
This is under Rules > Transform Rules > Managed Transforms > Add Security Headers. Turning on that off fixed it for me. I added the other headers that were missing manually and added a CSP. They're listed in the Cloudflare Docs about this if you're curious.