What I did a few decades ago when using a DP to mill, is take a correct arbor (in my case, MT2 solid socket milling holder) and heat the tang to soften it. Then drill and tap it on an angle, so that a socket head capscrew with a taper turned on the head and end, would thread into it.
The arbor tang was also ground (freehand) so it would fit in the spindle socket in any orientation, to rotate it so the screw threads lined up with the knock-out slot. The solid collet was put in the spindle, threaded hole aligned with the slot, and then given a good "thump" to seat it. Then the taper head socket screw was inserted and tightened to continue pulling up on the shank. Never had a problem.
Obviously, given the small amount of play to work with, the slightly angled threaded hole has to be carefully placed, and the screw chosen and tapered appropriately. Note also, that for the taper to work, the threaded hole in the tang points "downwards" from the end the screw is introduced to. You kind of have to remember this when setting up; that the threads point "up" from one side and "down" from the other. But it is very slight. (I guess it also depends on checking that the bottom of the slot in the spindle was square across, as mine was)
For a machine like that, I'd probably install an ER collet chuck and just leave it in. Can mill or drill with it. Then maybe keep a couple sizes Jacobs drill chucks with straight shanks to put in for quick, less precise drilling jobs.
smt