I'd call not being able to move a pretty large drawback.
Depends on the time it takes to hull down, de-hull (or w/e the correct English word is).
If it takes a good chunk of time to hull down and can not instantly exit the hull down mode, then it has a drawback. If the de-hull time is instant, then there is no point in flanking as the tank can just exit hull mode, maneuver and go into hull again.
So the overall power of Hull-down depends on setup time.
What is the setup time for it?
How long does it take to exit Hull down?
These two questions should be asked. As you've said, not being able to move is a substantial drawback, but not if it takes a couple of seconds to setup or exit.
The tank bonuses are fine. Elefant is probably the only tank that should not be able to hull down because it's got a wide enough cone, to be able to put far enough to cover a whole lane with it's range. Because you know, the further, the more comes into your field of "view".
Other than that, a Panther is fine with it's bonuses. Tweaking the time needed is enough to balance hull down for team games.