The region ‘Rest of the world’ is defined as a default and is valid for all those countries which are not explicitly listed in their own region. This way you don’t have to record every single country if you ship worldwide.
Uncheck ‘Rest of the world’ if you only ship to certain countries. Then people who specify a different country in the delivery address will not be able to place an order.
You cannot delete the option, you can only deactivate it.