The European Union currently has strict pandemic safety rules barring entry to most countries outside the union with high Covid rates.
Dominic Raab acknowledged Brits will be beholden to what "the EU decides".
The Sun reports Raab told BBC Radio 4: "Covid restrictions will depend on the combination of what the EU decides, but also member states.
"We have already got challenges with that and we have put our own restrictions in place."
He added: "I’m afraid restriction on travel, inevitably, is going to be something that is kept under review."