One in seven drivers (14%) have been forced by another driver's overtaking to swerve, pull over or brake to avoid a collision in the past year, the survey found.
Brake, the road safety charity, and Direct Line insurance are urging drivers to hang back, slow down and chill out on rural roads to avoid tragedies.
The survey found that four in five drivers (80%) have felt endangered by an overtaking manoeuvre, either of their own, their driver, or another vehicle.
Almost all drivers (94%) have witnessed a risky overtaking manoeuvre, and more than half (53%) see them monthly or more often.
One in five drivers (18%) admitted they had overtaken another vehicle when they were not certain if there were any hidden vehicles or hazards they could have hit during the manoeuvre.
Brake deputy chief executive Julie Townsend said: "Overtaking on single carriageway country roads is a huge risk, and one that ultimately just isn't worth it. Why risk it and rush?
"You could cause a devastating, high speed, head on crash that ends lives and ruins others."
In another recent Brake and Direct Line survey, risky overtaking came fourth on a list of UK drivers' road safety concerns, cited by two thirds (66%) of respondents.
Overtaking is a particular problem on single carriageway rural roads, where high speeds and blind bends often make it impossible to be sure the road ahead is clear.
This creates a risk of devastating head-on collisions.
It is one of the reasons rural roads are, per mile travelled, the most dangerous for all types of road user, accounting for more than half (52%) of fatal crashes on UK roads .
Brake is urging all drivers to think twice before overtaking on country roads, and only to do so if absolutely essential, for instance to overtake an extremely slow moving vehicle.
Otherwise, it says, drivers should hang back and relax.
Rob Miles, director of motor at Direct Line, said: "People die on rural roads in the UK every day and many of these fatal crashes could be prevented.
"Our own data suggests that young drivers and their passengers are even more likely to be killed on this type of road. Drivers should remember that patience is a virtue.