I have a couple Slater rotary broaches, and make the punches myself.
I have thought about the shape you illustrate, and am pretty sure it can be done. What may cause some thinking/set up considerations is that the center hole for the punch needs to be centered on the form. IOW, the hole you need to drill, needs to be offset from the centerline of what would be a complete circle instead of a D shape.
So making the parts might be more conducive to working on a mill, say, rather than a lathe. Or use an offset collet or 4 jaw in the chuck.
If you've never done it before, pay attention to what Doc notes about centering the "focus" (my term) of the wobble at the end of the punch or it will give you no end of consternation as to why it won't cut easily and smoothly. Though both terms are relative.
You can also "wobble broach" with a ball bearing center in a lathe, center stuck in the center of the punch, and the TS off set to whatever gives ~1° over the working length of the punch (for a 1-1/2° taper each side on the punch itself). I wouldn't try it the first time with my best/high dollar center, though.
Point being, whether you use a factory tool or make the tools yourself, it takes a _lot_ of pressure to cut shapes with a lot of material removal (as that D shape would be) once they start getting over maybe 1/4" - 5/16" dia. Slater and Somma both publish charts of pressure required for some common shapes. The process is also a little like tapping : Use a large chamfer to center and start the punch. Then, if you can use a clearance hole that is even a couple/few thousandths larger than right on size, the pressure will go way down and speed of production up.
smt