The blank page. That's the part that's the biggest 'struggle' for me. For whatever reason, when I need to come up with a scenario/module carte blanche, the ideas don't flow or I get concerned "will that be good/original/interesting/complex/etc enough?" And I can spin my wheels for quite some time.
But once over that hurdle, even with only a small seed or suggestion, I can go from there and elaborate it out. Ditto if cribbing from a prewritten adventure and re-working over half of it. No problems. It's just that initial bit that vexes me.
On the premise of the thread's inquiry, while I do enjoy GMing, on the whole I much prefer to play than GM. Yet I very much agree with bsss above, it's a weird premise to me that if there's even one aspect of GMing that you don't find enjoyable or find hard or a chore then you shouldn't GM at all. If you're suffering over it, then yes, stop until you can clear the suffering (note, that doesn't mean clearing the chore or hard work or etc, it's clearing the suffering about/around it). If you dislike the majority of GMing, then sure, switch to being a player only or do it in smaller doses until you gain enjoyment. But a binary "you must like everything or else you are doomed to failure and suffering" is unnecessarily reductive (and not a good recommendation to approach our lives).