Like spray them with water? Sure, if they’re in the sink or on the counter. If you mean just swat them away, I wouldn’t trust that to kill them. And as soon as it doesn’t kill them, and they find their way back to the nest, there’s a chance they’ll lead others back, if they found so much as a scrap of food.