omap: serial: Fix the boot-up crash/reboot without CONFIG_PM