It all comes down to the needs/desires of everyone involved. Are you the odd guy out, as everyone else around you has a blast and you don't? Or is the DM the only one having fun?
We can talk about how your brother is a nitwit all we want, and it could very well be true, but if he's playing...