hp_accel: fix race in device removal