colors: Fix bug with emphasis turning off when moving around in buffer