The early GPIO should only be needed in cases where some interrupts
are routed via FPGA. In most cases doing the init with arch_initcall()
should be fine.