[POWERPC] mpc52xx_psc_spi device driver must not touch port_config and cdm