I'm sure they were utilized, which means "make practical and effective use of". It only makes sense. What doesn't make sense is squads running around shooting rockets, which you can't call 'utilize'. I'm sure this wasn't the case.
Have you played lately? It's pretty reliable. You will do damage, because even if you miss, there is an explosion that damages every model, not just the one you missed. I lose models every game from anti-vehicle rockets, and every time it's costing me the engagement.
Still, that's not the problem. It subtracts from the immersion, the same way snipers in scout cars did.
If you're looking for immersion grounded in realism, COH2 is not the game for you or anyone who thinks likewise. It's a game where men with rifles will repeatedly miss firing at point blank range. It's a game where walking into a hail of machine gun fire will simply force your squad the hug the ground for 15 seconds before they suddenly can't move. It's a game where players can slap a few sandbags on the ground behind a wall and then abandon construction to deny the enemy cover; where often the most effective way to eliminate partially constructed razor wire is to have your men fire their rifles at it; where hand held AT grenades, panzerfausts and AT rifle grenades are all heat-seeking and will bend around trees and corners of buildings to hit a tank in just the right spot to disable it, as long as it's taken 10% damage.
I personally don't take issue with any of the above (except ghosting with sand bags) because it's a game that has to follow some rules that don't 100% click with reality. Given all of that though, if you're coming at this whole thing with "broken immersion" being the backbone of your argument, are bazookas really THE thing that demands your attention, that rattles you out of your "zone"?
Bazookas will occasionally kill models in the same way the panzershrecks from a Volks blob and the occasional AT gun round will. If you're consistently losing engagements and, should we infer(?), even games because of the Captain unit with his two bazookas, I really don't think the issue is that the bazooka is overperforming against infantry. No one sees the Captain hit the field and thinks, "Shit, he went Captain! Pull everything back, EVERYTHING BACK!"