I would prefer them to be either supernatural beings or space aliens but not both.
I liked the idea that the Norse gods were space aliens and that the Bifrost was some kind of wormhole connecting nine locations in real space (as opposed to other dimensions or planes of existence like in mythology). I feel like the MCU has muddied the waters a lot lately. It would seem that normal Asgardians are just humans from another world but their leaders are actual gods, not just space aliens with advanced magitech.
Moon Knight showed us that the Egyptian gods at least dwell in a different dimension ... but all these other Earth and alien gods just hang out on what is essentially a space station in outer space somewhere?
OK, there can be different types of gods, but I just find it all a bit difficult to get my head around.
EDIT: This article sums it up pretty well:
The MCU’s gods mythology is a complete mess