It's never bad to have character restrictions. In the campaign world I'm in dwarves were all but killed off, so they're not common. (We've had exactly one dwarf in the party, and he died...) It can make things more interesting.