Yes, it's the general sections on Magic Armor and Magic Weapons that say a weapon or suit of armor cannot have more than a +10 total effective enhancement. So the rules are unclear about whether or not Greater Magic Weapon spells would allow overcoming that limit, however temporarily; by the rules as written, however, it seems that you cannot.
Greater Magic Weapon does supercede the weapon's normal enhancement bonus if GMW's bonus is higher, but it appears that you still cannot raise the weapon's total effective enhancement total beyond +10. So GMW cast at 20th-level could bump up a +1 longsword to a +5 longsword temporarily, but it would only boost a Holy +2 Vorpal Longsword up to +3, since such a weapon already has a +9 virtual total beforehand.
Anyway, there's no rule saying that special abilities are capped at +5 total.
There just aren't any NPCs with magic weapons/armor loaded with special abilities because NPCs normally have much lower wealth by level, beyond the first few levels where they're strangely rather wealthy. A +7-equivalent weapon (such as a Holy +1 Keen Longsword of Flaming Burst and Ghost Touch) costs 98,000 GP and change, which only an NPC of 17th-level or higher is even capable of affording at all, per the DMG guidelines on NPC wealth/gear.
Though a 20th-level NPC has considerably more wealth, even they would have to dump almost half of it on such an item. Since only 19th- or 20th-level NPCs can afford such an item AT ALL without giving up almost everything else (a 17th-level NPC would have almost nothing left after "buying" such a weapon), you're not likely to see them lying around.
Heck, it's tough to even find anything more than a simple +4 or +5 magic weapon on any NPC, even the high-level ones, and only a few NPCs in the EPIC LEVEL HANDBOOK even have any weapons with more than a +5 total enhancement equivalent; those that do, well, the jerk DM would surely just dismiss as simply possessing epic magic items made with epic item creation feats, since they're similar in power to some artifacts (and, of course, some of those epic NPCs do carry artifacts).
Vlaakith the Lich-Queen has [sblock]a +5 Dancing Vorpal Silver Sword (special githyanki magic greatsword) in Dungeon #100 (one of the Dungeon/Polyhedron combination issues) [/sblock] as noted in her statblock for the Githyanki Incursion campaign. But she's also an epic-level NPC (with an epic item creation feat, but not Craft Epic Magic Arms & Armor, curiously enough; one must wonder where she got that epic magic weapon! Perhaps from an earlier Vlaakith?).
Although magic armor is cheaper, I still can't find anything so far with more than simple +4 or +5 armor right now.
