atmel_serial: Fix build on avr32 with CONFIG_PM enabled