How tall is your Zaxis and leg length? That has the biggest effect on rigidity. If you are having issues I suspect your z axis is very tall. Most non comercial cnc machines have about 3″ of z travel. Before any upgrades were ever made I was still able to mill aluminum with a flex shaft dremel, shown in the first few vids I put up. You shouldn’t need to upgrade but it does help you cut faster.
The roller motor mounts have the biggest rigidity improvement, not the actual rollers.