There isn't a single answer to this, IMO. For instance, if the party were in a rural area, a large gem wouldn't be worth squat--what would the local peasants use it for? In a trade town 50 miles from the mine, it would be worth a certain amount. However, in a wealthy and decadent metropolis 4000 miles away, it might go for 10 times that much. And if the gem has uses as a spell component, or even in alchemical formulae, then it might be worth a great deal more to practitioners than to a run of the mill jeweler.