Well, I decided to try a 8.5mm drill bit rather than hint down a 8.4 as I guessed that if one would work, so would the other.
I have drilled out all the holes of the center assembly to 8.5mm and so far it seems to have worked. Still a little tight but the stainless will now at least work in the X & Y channels where as before it would not.
I tried with the pedestal drill but found the drill bit tended to grab the PLA when pulling the bit out and skewing, so I switched to my faithful cordless and on low speed, drilled each hole out, carefully reversing the drill to take the bit out. The PLA and the PLA+ both seemed to tolerate this and a small amount of tidying up with a needle file was required. (I printed the parts with 3 outer layers if that makes a difference to the success of the drilling.)
There were 4 holes in the assembly that were drilled, but when putting things back together I realized would not make any difference as the bracket coupling up to it would need drilling as well which I have not. I am hoping the other 2 bearing holes being drilled will be enough. Still some deflection but have not measured it yet.
Not Ideal I know, but hey, what have I got to lose.
I will try taking the assembly to the next stage now with regards to alignment squaring up etc. and report back.