Uber drivers accepting rides then delaying

I’ve noticed that some Uber drivers accept a ride then message me asking where I’m going. Then they go silent and eventually I have to cancel. I presume they do this when they think my ride is not worth their while. As a result I land up late for meetings. Any ideas on how to avoid this?