[to be on the safe side] {adv. phr.}
To take extra precautions; reduce or eliminate the possibility of a mistake, an error, or even danger.
* /Dad always keeps his valuables in a bank's safe deposit box, just to be on the safe side./
Compare: JUST IN CASE.