Pi’s take a bunch of current. I don’t know much bout the zero, but the full size ones are all 1A or so.
You can buy cheap DC to DC converters that are large input, like 5V-15V and output 5V, or variable voltage. You can find them on eBay, Amazon,z or alibaab type places. They are a buck or two each. Just make sure the amperage rating is 2x what you plan to use it for or so.