Using pastels is a bit different here to paints; although you can mix them, you need more than 3 primary colours. A mixed set of 30 half sticks is probably enough to get started with. You do not need the whole range, although that would be lovely!
Don’t be fooled into buying a set of cheap, ‘student quality’ pastels to get started, as they will not be made using pure pigments. They will not be light fast, and you won’t have a good range of colours and tones. You get what you pay for with art materials.