Also dispose of the layer if we are replacing a previous back buffer