It's going to depend on the powder you choose, but most of them are polyester and pigment. Since kids aren't dying from chewing on polyester fleece onesies, the bulk of the resin isn't a significant health hazard. However, pigments used in industrial powder coat generally don't have to pass CPSIA, so you'd need to know 1) what specific pigments are in the specific coating you're dealing with, and 2) whether the pigments can leach out of the ground powder coat.
I suggest reading the MSDS for the specific powder coat products (and colors) you intend to work with.
Having said all that, lots of people grind powder coat off as welding prep, and it's not known as a health threat. (NOTE: "not known as a health threat" IS NOT THE SAME AS "known to not be a health threat".) Also, lots of hobbyists do small-lot powder coatings.
If you are spraying anything, you should keep it out of your lungs.