I have messed around with the idea of using 2d10 but never a single d10 due to the fact that things are scaled off of the numbers 1-20... 2d10 gets rid of the one and skews the rolls closer to a 10. Using 3d6 is one of my favorite variant rules simply because it averages a 10.5 and features the simplicity of using d6s, of course you have to change around crits and stuff. I made some charts about this , i'll see if I can find them.
I found my picture that shows you all the distributions. It should be uploaded. Anyway back to your d10 idea. I have always been a fan of remodeling distributions and using different dice to achieve a different flavor (for lack of a better word) to your game. Now for OGL stuff , since everything is based on a 1-20 roll so just off the top of my head if you wanted to use a d10 there are a few things you could possibly do to make the game work for you.
(Again this is without any real analysis , im just talking about what pops up in my head at this point) .
Since the average roll of a d20 is 10.5 and the average roll of a d10 is 5.5 I figure you can basically lower all the things based on d20 rolls by 5 or 6 depending on which way you want to round. So in essence instead of AC being 10 + X you would have to calculate 5+ X. All DC's should be reduced by 5 as well. There is something that you might want to look out for though when dealing with skills. Since the skill systems are based on a d20 that means one "skill point" raises your chance of succeeding by +1 , this changes though if you are using a d10 because each skill point will be roughly equivalent to a +2 bonus assuming you lower all DCs by 5 (If I'm thinking about this correctly, which again I may not be).
It seems like all of this may be more trouble than it's worth since you are trying to use a single d10 to reduce the math in your game. Of course you could make a handy table to keep track of what average DCs are for each level of play.
You MIGHT be able to use a single d10 if you dont monsters at your party that are much higher level than yours. In fact you may only be able to use monsters lower than your party so the die range of hitting an AC or something is within reach. You would still be faced with the problem of things like spells where you have to roll some sort of save ,because magic saves are usually based on caster level checks which may outpace your saves since the game is assuming you will roll a 10 (ish) instead of a 5(ish).
So yeah... it seems to be more of a headache than it's worth. But who knows, there's a good chance I may be completely wrong about all of that and it might work fine /shrug.
You'll probably want to get rid of critical hits and auto-fail on 1 (since they happen twice as often on 1d10). You'll probably want to keep the auto-hit on 10, though, because it'll help compensate for the big problem (see below).
The advantage of this is that you'll have created more consistent results from a given skill. Instead of characters seeing a 20 point variance in their results, they'll only see a 10 point variance.
The problem, however, is that narrowing the range on key figures like attack bonuses and save bonuses is going to seriously start mucking with balance. First, you'll have a lot less flexibility in the CR of monsters that you can throw against the PCs (because monsters that could be hit 50% of the time on a d20 roll suddenly can't be hit at all on a d10 roll).
Second, you're going to find yourself running into one of the mechanical breaking points of the D20 system a lot sooner: Once the guy with the lowest attack bonus falls 10 points behind the guy with the highest attack bonus, you've created a situation where any given opponent must be either an auto-hit or an auto-miss for one of those characters. At that point it becomes very, very difficult to balance challenges.
If you're going to use your 1d10 variant, you'll probably want to also adopt something like E6.
Your main problem here is going to be that D&D is designed under the assumption that every +1 is a 5% chunk. You'll run into some problems turning those into 10% chunks, as large differences in bonuses will rapidly lead to auto-successes and auto-failures.
I'll demonstrate with a low level example, a first level fighter in fullplate fighting his twin. Let's give him +1 BAB, +3 Str bonus, and weapon focus. That is a grand total of a +5 attack bonus.
Defensively let's give him fullplate (+8), a shield (+2), and dex (+1) for an AC of 16 (decreasing all ACs and target numbers by 5, as [MENTION=63245]Evenglare[/MENTION] suggested, since you are rolling d10's).
You'll notice that the fighter can't hit the opponent, unless a natural 10 is considered an auto-hit, and even then a 10% chance to hit someone of the same level is pretty sad.
Now we outfitted that fighter pretty well defensively, but giving him banded mail instead of fullplate still gives him an AC of 14 (5+6+1+2), which still only gets hit 20% of the time.
That same match-up using d20s, as it was designed, has 25% and 35% success rates, respectively. Substantially higher, though still something of a grind.
It only gets worse as levels increase and BAB and save bonuses diverge, unfortunately. If you thought you had it bad as a monk with only a +15 BAB at level 20 versus the fighter's +20, imagine how much worse that is now that it represents a 50% to-hit differential instead of 25%!