I don't think it was meant to be a challenge at all; it (one CR 1/4 goblin vs 1st level PCs) was a social/roleplay/atmosphere encounter, not a real combat challenge.
Your personal recognition of what encounters should be has little bearing on what others want for their game. My players only recognize threats when those threats are credible. A lone goblin that poses little threat to anyone isn't some horrid monster to them, it's just more comic relief hinding in a closet scared of the family dog.
<snip> I'd like to be able to present a challenge that lies between "not a challenging encounter" and "challenging encounter" which 4E doesn't handle well.
Just a question and a couple of thoughts as I'm entirely unfamiliar with the source material. From what I'm reading it seems that this is a stand-alone encounter with no stakes or relevance related to the greater story arc, yes? If that is true then I would suggest distilling precisely what you're looking to get out of this scene and adjusting the premise and mechanics to meet that goal.
I could easily see this being a non-combat encounter whereby the goblin is a deserter from a raiding party who didn't have the stomach for butchering. Perhaps he's holed himself away in this closet and sneaks out at night, stealing the remnants of table scraps and squirreling himself away in the closet or perhaps the cellar. One day the family dog catches whiff of him and all hell breaks loose. The terrified family is now outside with the creature trapped in the closet or the cellar with the dog ceaselessly, vigilantly barking. Or maybe the goblin, being discovered by the child, has snatched him and is holding him hostage (without any exit strategy plan for any of it...he just doesn't want the door to open and the dog to chew him to bits). The stakes are different here as is the thematic relevance. You've got the potential for a non-combat social encounter and possibly the difficulty of disarming the dog's protective instincts. You have questions of mercy and property rights that would need to be addressed and would help shape the characters as they move forward into the campaign.
Or, I could easily see this turning into something more ominous. Perhaps the closet or cellar is a portal to the Far Realm or the Feywild. Maybe its not a goblin. Maybe its a Quickling or a diabolical abomination who has been performing reconnaissance on the town/city (in preparation for a raiding party or full scale invasion) via this portal and the dog has spotted the creature going back in and shutting the door. The dog is rabbidly barking at the door...the PCs carefully open it. The dog sniffs around wildly, uncovering every crevice...but there is nothing. Perhaps unlocking the mystery of the portal and the potential raid is its own mini-adventure.
Or it could be a singular goblin, afflicted by some form of madness (that makes it more powerful) holed up in the closet, eating shoes, coats, flesh, gnawing on floorboards...whatever it can get its mindless grubby hands on. A bloody encounter with it could set off a madness pestilence (Disease Track) that spreads to the PCs and the town proper. Then a proper Exploration Skill Challenge to find the means to cure it is set in motion by the town's healer/medicine man.
4e would do all of those quite well. Again, I don't know the stakes or relevance of this creature but if its detached from the greater story (and just color or a stand-alone adventure) then you should be able to employ any number of options.