How do I go about convincing my players to learn the rules for their characters in such a way that it'll feel less like I'm giving them homework and more like something they actually want to do?
First off, let me say that you're in a prime position as DM if your players don't know the rules. Let your players focus on the story and look to you for help on rules.
You ask them what they want to do, while you're playing, and they tell you. Then, you simply tell them what to roll.
For many DM's, this is a dream situation.
But, that dream situation is not alwasy possible. Most players like to know the rules. They like to know what's possible. And if you're playing a d20 game (or one of the other D&D versions) chances are your players will need to know some of the game rules in order to make good decisions when building their characters (and guiding their characters for prerequisites for future builds).
I was in a similar situation about a year and some months ago. My players were AD&D 2E gamers. Nobody had ever played a 3/3.5 E D&D or d20 game (except one of them played Neverwinter Nights on the PC).
These players were not new to gaming. They had 20+ years of gaming each. But, they were new to the d20 system, which is a big step, in a lot of ways, away from 2E AD&D.
Here's what I did...
First, I picked a game that I knew we'd all like. As GM, I had to be passionate about the universe, and I was ready for something "new". One of my players really doesn't get into anything other than fantasy, so I knew I had to pick a game in that genre. I couldn't go with science fiction or espionage, two of my top picks.
I settled upon the Conan RPG. I'm a huge fan of Conan. The game is fantasy, but it is much different from Tolkien-esque D&D. The Conan RPG is swords and sorcery fantasy--darker, gritier, and just different enough to attract everybody.
The next thing I did was spend a long time with the rules myself. I read the rules closely, and tried to understand as much as I could. These are big, complicated rule sets, these d20 games, so I'm still looking at some concepts in the game. But I didn't involve the players until I felt I had a pretty good grasp on the rules myself.
Next, I decided that every Player Character would be of the same class. In the Conan that that's easy to do. It's much easier to believe a clan of Cimmerian Barbarians than it is an exclectic, D&D-ish group composed of a lot of different classes, with races from all over the world.
Picking the one race and the one character class narrowed down the scope of our learning (besides making a lot sense story-wise).
If you're playing a D&D game, try limiting the classes. You could set your game centered around a Thieves Guild and just allow Fighter and Thief characters. Or, you could set it around a church, and just allow Fighters and Clerics. Or, maybe there's a mage (NPC) who hires some help.
Limiting the classes really take a load off your plate when you're learning a new game. As long as it makes sense story-wise, you'll probably be glad you did it.
The next thing I did was go about creating a home for the PCs, the root of my sandbox. I created a village, some major NPCs, the PC's family, and some story threads that I could devlop later.
If you want to check out some of the stuff I'm talking about here, click on this link:
CIMMERIAN BLOOD
Now, this part is the major answer to your question: Once I was ready to play our first session, I didn't start with the players having full characters. If you look at my outline for our first game sessions, you'll see that we didn't cover all the aspects of character generation.
FIRST GAME SESSION OUTLINE
I had the characters roll stats and decide on a SOME skills. That's it. I didn't go further with character generation.
If you look at one of those links above, you'll find mention of the Ras Croi. The inspiration for this race was really just to give the players something to do in order to learn how to use the skill system.
I designed a race, with skill checks for running, jumping, climbing, throwing, and swimming--all the physical skills.
Here's the post that describes the race:
RAS CROI
And, as you will see, I worked it into the story.
The characters learned the game by playing through the story.
During game sesson two, we completed character creation.
GAME SESSION II OUTLINE
So, over two game sessions, the characters were created.
This is how I did it. Different folks have different tastes. But, the two game sessions when a long way towards the players getting the players to learn what the needed in order to play the game.
Hope that helps.
The next game sesssion