The Dire template is just too good to not be worth some kind of level / CR adjustment.. sorry.
You could homebrew a Dire Man, - to Int, Wis and Chr though. Want this one?
[+1 Str, +1 Con, +1 initiative, Weapon Focus (Unarmed), Improved Natural Attack (1d4), Improved Trip, Improved Grab, Skill Focus: Survival (+3), Unarmed Rake for 2d4 + (1.5 x Str mod) after successful grapple (using an illegal wrestling hold that maims). -2 Int, -2 Wis, -2 Chr. No bonus feat for being human, no bonus skill points. Bad temper. Eats everything, loves children. Free to loving home.]
Otherwise we start looking at +1 level adjustments. (Apply a +1 level adjustment to the example, and the -2 to Int, Wis, Chr transforms into -2 Wis, Illiterate. Still no Human bonus feat or bonus skill points.)
If you wanted to take a primitive Goliath and file off the serial numbers (now renaming it a "Dire Man", fabricate a new story to explain) that would work - if you are ok with that +1 level.