Yes, there do need to be limits and as DM and referee sometimes we have to say no. It depends on campaign style and assumptions. In FR for example, there's apparently a dozen or so organizations (I can't keep track of them all) that seemingly span the known world(s). With a network like that...