The purpose of the wedge has been well explained above.
But you say the taper will not stay in the spindle. This is probably due to damage to either the taper adapter on the chuck or to the female taper in the spindle. This is probably a Morse taper which is designed to be self holding. When these mating tapers are new, they will bind together with a moderate amount of force as when the chuck with it's adapter are thrust by hand into the spindle with moderate force/speed. Often that will be followed by pressing the spindle down with the chuck resting on a block of wood to increase the hold. Some people will use a hammer with a block of wood against the chuck, but I do not recommend that. Hammer blows can damage ball and roller bearings in the spindle.
If the tapers do not hold together with moderate force, there are likely some dings on one or the other surface or both. If it is not hardened, a small ding on the male taper can be removed with a small file. Try to file just the ding, and not very much of the original surface around it. I like a round, chain saw style file for this as it can remove just the ding. If the taper is hardened, do not use sand paper but a fine stone, USED CAREFULLY and GENTLY can work. A small diameter, round stone would be preferred. If you have a precision, flat ground stone, that can be ideal for this kind of repair but not many shops have these.
If the female taper is damaged, you can purchase Morse taper reamers of the appropriate size and again, CAREFULLY use it to restore the internal taper in the spindle. But the damage is more often on the male taper.