I haven't played it yet but you only count the 3s and something special can happen if you roll 3 threes. I have the special dice where the number 3 is in red, so it is very easy to see how many you have.
Most of the time, you only need one 3 to succeed, the rest are mostly bonuses.