It could be that the knurling wheels are not tracking properly and therefore shred the peaks already formed.
Tip: start the knurl off the end of the part so perhaps only 1/4 of the width of the wheels are engaged. Start the spindle at very low rpm because you don't want the part to make a full revolution before you've set the tool to depth. Crank that mother in hard and fast, and it should track. Now, it will likely be excessively deep at this point and you will have to back off the pressure a little to proceed. The paramount feat to achieve is to get a good first revolution to teach the wheels the proper path to follow.
I most often set my knurling wheels at a bit of an angle, with the leading edge farthest into the part (so you can knurl nicely up to a stop). IMO, a full width broad faced setting of the tool doesn't produce enough focused pressure to get the job done.
BTW, you can dial the spindle speed back up to whatever you want as you proceed with the operation.