The Victory line is significantly more modern regarding production standards and quality of engineering than the HD line, except the V-Rod, which is a pretty modern powertrain.
Victory has been producing MCs for enough years now that I don't think they are going to go boots-up in the foreseeable future, unlike the various attempts to resurrect Indian/Norton/etc.
Also, the bikes are built to be ridden, not just looked at. They're not high-tech show pieces, such as Confederate, which I love to look at (gasp, I just admitted publically that I like a cruiser!) but, aside from the cubic-dollar price tag, I just can't imagine riding for more than 30 minutes.
ScrapD, it's possible that you've overlooked one "axis" of interest in MC ownership: the culture of, and availability of customizing options. I like tinkering with bikes, and mostly build my own stuff (a hobby unto itself), but some folks prefer to bolt on stuff. That will be better supported for the HD than the Victory. This may or may not be a selling point for you, but bears examination.
Regarding "Smiles Per Hour", I have found that stepping downward in outright performance to a Dual Sport bike has been great fun. I notice that you're in Arizona: you certainly have the territory to use a DS bike! Maybe they're better as a second bike - you'll not want to ride it to LA & back. They can be had cheap.