ISSUE SOLVED! Hooray!
Ok, so this morning I moved the parts in CAD the tiny bit needed to make them perfectly symmetric around the Y axis origin:
Here is a sketch showing the models are symmetrical around the origin:
And here is a backplot of the gcode output with coordinates showing that the parts are mirror images of each other:
Unfortunately this did not solve the problem. In my mind this is actually a good thing... It would have been very unintuitive if this solved the issue
Ok so onto the next item: changing all the tools.
I did this one as kind of just a 'belt and braces' item because the tools were still producing good finishes and drilling to size and so on. I wasn't really expecting it to fix the issue, and yet IT DID!?
Looking at the part and program I think I know why. As G00 Proto suggested I think it was drill bit wander.
I think of G10 as a pretty soft material (which it is) but it also is pretty good at dulling tools. The drill bit I was using was a screw machine length solid carbide drill held in an ER collet so it should be pretty stiff, but it had been in there a long time because I'd never had any reason to change it...
I think the drill was pulling/walking off position just as it started the hole. I have never had this issue before so didn't really think to look for it. I have changed the program so the counter-bore happens first to ensure a perfectly flat starting spot for the drill, and I will make sure to change that drill more regularly!
In the end it was definitely a head-slapper. I discounted a potential problem because I 'ass-umed' it could not affect the part the way I was seeing... Lesson learnt!
Thanks to all for the feedback and ideas!