Maybe not as much as you think. Here is the abstract from a new paper (AEA gate) by Saurabh Bhargava and Vikram S. Pathania:
We investigate the causal link between driver cell phone use and crash rates by exploiting a natural experiment induced by the 9pm price discontinuity that characterizes a majority of recent cellular plans. We first document a 7.2 percent jump in driver call likelihood at the 9pm threshold. Using a prior period as a comparison, we next document no corresponding change in the relative crash rate. Our estimates imply an upper bound in the crash risk odds ratio of 3.0, which rejects the 4.3 asserted by Redelmeier and Tibshirani (1997). Additional panel analyses of cell phone ownership and cellular bans confirm our result.
Here is another way to put the puzzle:
Cell phone ownership (i.e., cellular subscribers/population) has grown sharply since 1988, average use per subscriber has risen from 140 to 740 minutes a month since 1993, and surveys indicate that as many as 81 percent of cellular owners use their phones while driving—yet aggregate crash rates have fallen substantially over this period.
I don’t want you to start or continue this practice, as numerous other studies do find significant risk. Still, maybe this matter is not quite as settled as many people think.