You both grab him by the feet and hang on for dear life. This yanks him back down to the ground and you all go toppling down. He growls and gives you an evil look as he tries to struggle to his feet.
If I understand correctly, those last grapple rolls were just to get him out of mid-air rather than properly grappling him. So, doing that now, after raging.