Honestly, as someone who has read a lot of cozy fantasy... I think it is easy to misunderstand it.
"Can't Spell Treason without Tea" is a book in the genre, and it has plenty of intrigue and fighting, the one main character fights off a flight of dragons, the other is dodging spies and investigating an ancient mystery. But the emotional core of the story, the meat of it, is two women building their lives together with a simple tea shop/library away from all the struggles and hardships they have lived through. It isn't that there is no threat, no conflict, it is that the goal is a peaceful life.
In a way, I feel like the genre is "what happens during downtime", to put it in DnD terms. It isn't "build my empire to conquer all and carve my name in history!" but more "Hey, there was that old warehouse... maybe I can fix it up, get some students, teach what I've learned and find a girlfriend.
This isn't to say it is where I feel the game is going, I don't think DnD is going to be a Cozy Fantasy game. But... well, haven't players since the dawn of the game found baby animals to raise? Dragon eggs, young griffins, ect. Haven't we often had squires or apprentices that we have taken under our wings and helped guide the growth of? I wouldn't want to make a campaign out of it. But I do like the acknowledgement that the downtime, the space between adventures... matters. That not everything needs to be a metal album cover. That the wizard can sit in the market and use minor illusions to play with the kids while the fighter tries to teach their baby phoenix that they can't eat all the fruit being sold by the vendor.