I speculate that this is the new 5.5 "you need a magic weapon or a spell to hurt this powerful monster" rule.
It's similar, but not the same in a very important (and positive, for once!) way. More below.
This damage threshold of 10 (for an example) can easily be defeated by tier 2 heroes! Now, weak attacks like cantrips may not be reliable,
Cantrips are unlikely to be affected unless the characters themselves are weak. A 10 damage threshold would be pretty vicious for my current party...
except me and our Barbarian, because the two of us dish out pretty hefty hits. As a Warlock, I'm doing 3d6+5 on a melee hit (greatsword pact weapon + 1d6 invocation bonus + 20 Cha) and 1d10+5 via EB+AB. More than half my EB hits will get through, but it'd still be
rough to lose 4/10 of successful non-crit hits, since it's pretty unlikely to roll only 2, 3, or 4 on 2d10 if it crits. Essentially all (anything but the two lowest possible results) of my melee attacks will go through.
But our Monk? Yeah this would kinda suck for them. They'd probably need a +1 weapon just to break even on dealing damage. Low-damage-dice characters like Monks, dual-wielders, and Rogues are likely to be unhappy about facing this sort of thing.
Thing is...other cantrips aren't gonna be as negatively affected, because they're all-or-nothing stuff.
Fire bolt is 2d10
or nothing (and can crit!), and most other strong offense cantrips are like that too. Certainly by tier 3, a damage threshold of 10 becomes pretty plausible for most cantrip hits, and essentially all spells of 3rd level or higher.
Am I correct in predicting this?
I think you are correct, but only technically, not productively. See, there's a key difference: you aren't guaranteed to get a magic weapon. You are guaranteed to get more feats/stats and/or class features that increase your damage. Hence, a damage threshold is less inherently punishing than requiring magic weapons (which, itself, is less onerous than requiring
specific weapons e.g. only silver, only cold iron, etc.)
BUT--and this is the big but!--the damage threshold can scale while those other things mostly don't. (IIRC some editions may have required specific +N numbers or else do not pass go, do not collect $200? But I don't remember.) A threshold of, say, 8 would still be pretty rough for even 5th level characters (expected +4 modifier means you must roll at least 4 on your damage dice, so d6 weapons only have a 50% chance of actually dealing damage
even when you hit!) A threshold of 15 would be crippling for nearly everyone even at very high levels because most weapons don't deal d10/d12/2d6 damage, and even for those that do that's an incredibly high bar for anyone to pass, even magic-users.
If I'm wrong, would it be a nice house rule? It would favor "a few big hits" vs the "many small hits" type of martials...
I think it has good chance to be helpful, but one must be
very very careful not to over-use it. 5e is kind of replete with over-using rules (see: advantage) to its detriment, so this sort of thing should only be used where it's really warranted, and should be kept to pretty conservative numbers. As noted, 15 is crippling to nearly anyone (e.g. that makes EB+AB essentially useless, as I could only deal damage if dealing
maximum damage, so only 1/10th of my non-crit hits actually land.) 8 is good for, as you say, making a "heroic"-scale monster, where it shrugs off the blows of most mortals. That is, assuming a shortbow and a +0 damage modifier, that means zero non-crits can make it through, and less than half of crits will make it through too. Even a longbow only does damage if you roll maximum, so only 1/8th of non-crit hits do anything, and only 2/3 (67%) of crits will do any damage either, again assuming a +0 damage modifier.
Another option could be a defeasible damage threshold, like how zombies don't get their "I survived a lethal blow" thing if they take radiant damage (or a crit). That way, ordinary arrows are useless, but specific damage types, magic weapons, or specific trigger conditions can make it much more vulnerable. This would likewise emphasize a heroic tone, but rather more a trickster or guileful hero rather than a mighty-thews hero.