migration: Move return path cleanup to main migration thread