IMO a retrofit knee mill is a good middle ground if you're already familiar and happy with manual knee milling, but there are serious drawbacks. I think a more accurate question isn't what you'll miss from a knee mill, but what you'll miss not having a VMC. True you don't have the hands-on aspect of making simple parts, but that's only because it's a different way of making simple parts. Most holdouts against VMC's do so because they are not familiar with programming enough to be able to tap out simple commands quickly, or keep their programs and set-ups organized well enough that it's easy to be super dynamic with yours tasks. A lot of the speed of using a manual comes from having your vise or indexer right there trammed in and ready to go, or having your tools in a rack 2 feet away from the spindle. Apply that same logic to a VMC and become ultra-familiar with the control and you're set.
Now that said, I too am a proponent of keeping manual mills in the shop, but If I had to choose between them? The one thing we really must have our Bridgeport for is it's table/knee/turret configuration makes it fantastic at getting ahold of odd or large parts for small operations. Some of it you can do in the VMC with some creative fixturing, but ultimately faster and more ridged to do on the knee mill. At one time we had 6 vertical knee mills around the shop. After we got into VMC's that number went down to one knee will with a 2 axis Trak control because for the jobs they excel at, that's all we needed. We also have a jig bore and a horizontal with a vertical head that I use for certain jobs that the time isn't so much an issue as I would rather not stop the other machines from chugging out parts. I'm trying to get the horizontal tooled up to make gears because for one odd gear, I wouldn't want to set up the VMC or bother with the program, but that same time and resources applied to the manual mill cost less in their inception and age better, even though it takes more spindle and man hours. At home I have a little manual round ram Bridgeport. Any more than that would be a waste for how it's used, so there really is no right answer for everyone. Each machine has a place and you just have to consider your space jobs and skills, and if changing any of that space jobs and skills would help or hinder your quality and output at the end of the day.