OK, the problem pretty much has to be outside the Arduino/RAMPS since the issue stayed put when you swapped the wires. If the X and Y wiring were swapped and the problem was on the board the problem would have moved. Try swapping both ends of the wiring this time, you can’t do it directly with the stepper motor, since its wires are built in but you can do it from the connector. This will show if the problem is in the wiring. If there’s still no change then you likely have a bad stepper or a bad wire coming off the stepper.
I’d put my money on the wiring connecting the stepper wires to the board, since if it were a bad X stepper the other side would still try to work.