Lock src when modifying idle_source