From what I understand, this is because once the mortar begins its 2 second "wind-up" it has to go through the entire process of firing a shell. There's a 2 second "wind-up", the shell firing and then also a 2 second "wind-down". The entire process can easily take 5 full seconds or more to complete even after you press retreat. Easily enough time to get a grenade on the mortar team.
You could argue that heavily reducing or removing these times and substantially increasing "reload" could make mortars too difficult to wipe since they can retreat. I'm sure there are ways to combat that problem though.
![Honor icon](/images/replay/honor_icon.png)