If I understand your question correctly using the term "lathe tool holder" than that suggests carbide inserts and a single cutting edge. You can use carbide but I've never been a fan of this due to the shock of an interrupted cut which chips the brittle cutting edge. While spindle speeds are much slower I prefer using HSS for use in a fly cutter as it resists shock much better and touching up the cutting edge is much less work. Fly cutting demands you tram the mill head fairly close or you'll be cutting an angled and/or concave surface. Hot chips are thrown everywhere so guards are important or you'll catch a hot chip down your shirt or have a large area around the machine to sweep up. The other type some call a fly cutter is a multi insert tool that does use carbide inserts. This is useful for harder materials, stainless steel, or have a large plate to deck down to thickness. The multi insert type isn't as susceptible to shock like the single bit are but still have comparatively short life for the inserts, particularly in a mill like a Bridgeport. If you're new at this I suggest a "lathe tool" type of single bit using a HSS bit to get familiar for the lowest cost. Do make sure the set screws are locking the bit in for safety. Use cutting oil, set your spindle speeds per diameter and material (use guards when cutting) and find what works for you. You can always spend more money later on more expensive tooling if you need/want to. Carbide isn't always the best choice for cutting metal but it is more expensive.