Being distracted isn’t always a bad thing
If you let the distraction mean you don’t get anything done, then yes, it can be a problem. However, if you’re distracted, if you’re letting your mind wander, sometimes it takes you places you wouldn’t otherwise get to go. These places can be as much an instigator of inspiration as anything.
Let the distraction lead you. Just don’t go overboard.