The other player is wrong by core rules, although if you've houseruled and forgotten it, maybe he's right...?
But aside from preposterous possibilities, no, masterwork absolutely doesn't qualify. The DMG makes it clear that the cost is a "+1 bonus" as in, you now have a base of +1 magical item and you build from there. A +1 to hit is not the same as a magical +1 bonus. The DMG is deliberately trying to make sure that these awesome weapons are not bought cheap.
A 5th level rogue with a flaming shocking rapier is wrong, too. At 5th level, the DMG says you get 9,000 gp. A flaming shocking rapier costs 18320 gp. It's simply beyond a 5th level character to have such an item unless you or the modules you're playing offered it. I mean, if you have a house rule of allowing such items, then the player isn't breaking any rule at all. Of course, the fact that you're here posting that you don't think it's right clearly indicates that you've not condoned what the player is doing. So, stop that player.
If it helps, note that PCGen will figure the math out for you. Go to the inventory section, find the rapier, right-click on it, select custom item, and you'll note that "flaming" isn't even offered until after you give the item a real +1 bonus. Then flaming will appear, along with shock. And the cost really is over 18,000.