I'm not super familiar with the lore surrounding Goliaths; it's been a long time since I looked at Races of Stone, let alone any of the 4e material. In general though, the most important question to ask is "What role do you want them to have in your game?" That's going to go a long way towards figuring out exactly where to find them on a map or describe their origins.
That said, here's some of the ideas I've been thinking about (several of them mirroring what's already been suggested here):
*Goliaths are descended in some way from the Giants (either directly or as experiments), and thus hail from the mountainous regions of Xen'drik. This certainly lends itself to the Incan civilization feel mentioned earlier.
*Goliaths are originally from the Frostfell, but stayed behind as the dwarves migrated south to Khorvaire. They're more likely to remember dwarves (living as they do among their old civilization) than dwarves are to remember them. There's also, if I recall correctly, an Overlord with more than a little bit of influence up in Frostfell, which creates some interesting possibilities)
*Goliaths start emerging from the depths of the ancient Mror Holds. Do they know what happened to ancient dwarves down there? Are they them, somehow changed? Who they truly are and how much they know will likely shape the future of the Mror Holds.
*Something something Sarlona. They have docile ogres over there; perhaps those are what have become Goliaths? Perhaps Goliaths are those ogres from Adar who accepted the rebel quori spirits, as the original Kalasthar did? Would be an entirely different kind of lore and backstory for them, but it could be work.
*The Demon Wastes are home to all sorts of creatures most in Khorvaire couldn't fathom. Perhaps Goliaths are natives of the Labyrinth, allies of the Ghassh'kala in an endless war against the Carrion Tribes.