The only time I think we’ll really need to surface the spoil board is when it gets really chewed up from cutting. We really shouldn’t be cutting that deep into it, unless something went wrong, so taking the top 2 or 3 mm should bring it back to newish. If you want to do inlay into ply, I’d just get the proper diameter downcut endmill. They’ll have shorter cutting areas so they are less likely to snap off on you. Rotozip like bits have a really long cutting area, so there’s not that much metal there and will flex under a load.