If it was an asphalt track you would essentially be off track. The games idea is prepared surface versus outside surface. The punishment is supposed to be the same as grass even though generally on a track flat inside would still be paved mostly.
Rather than put a wall there for you to hit you have a bit of an area to run off. For sand/dirt it might seem odd but its a video game
No clue on sand versus dirt racing type stuff. But you do use sand to slow down cars when they go off track in some cases. I am sure you could have a different sand mixture for a racing surface than a run off surface in reality.
I've had stuff like that happen in so many games, I just sigh in disbelief and try to carry on. So many times, in so many games... *screen slowly fades into a nostalgic glaze of rippling water and a slowly descending fog...*