A referee must send off a player if the player commits an act of violence against any person. One of the ref’s responsibilities is also to write a report about it after the game, so that the relevant authorities can punish the acts of misconduct.
In the video below, you can see that a player kicks his shoe to a referee assistant. However he claims either he did not want to do that or he regrets, referee is doing his job properly by sending him off the field:
The video below also shows how the referee decides correctly. A goalkeeper beats up a fan running to the field of play and referee shows the keeper a red card because of his act of violence: