On the safe side is used to mean being careful to avoid risk. This option is correct because it matches the common idiomatic use of the phrase.

Examples: Take an extra pen to be on the safe side. We left early to stay on the safe side.