Correct. Grabbing is a very mild and passive action in 4th Edition. It has only two effects:
1. The grabbed individual is "Immobilized", meaning that he cannot change his location on the grid until he escapes the grab. He can still, however, be pushed, pulled, or slid by another creature without interference. In this case, he leaves the grabber behind and escapes the grab automatically. The grabber can also be forced to move by another creature as normal, also ending the grab.
2. The grabber may move the victim up to half the grabber's speed as a standard action using Strength vs. Fortitude. Note that the victim does not provoke any opportunity attacks during such movement, but the grabber does.
Nothing else happens.
This means that the grabber can still do whatever else he wants on his turn (as long as he keeps spending a minor action to maintain the grab) and the victim can use all of his actions without restriction or penalty (so long as he doesn't try to leave his current location). The victim can, for instance, swing a sword, shoot a bow, cast a spell, avoid attacks, pick a lock, or even juggle - all while grabbed and all without any penalty of any kind.