Each phase has its own random tables of events/weather. They serve the same purpose—to affect the challenge that players/characters face (in either direction)—with different skins for tone. The camp and town event tables have their ample share of depredations and calamities (rockfalls, raids, plague, fires), but they are more prominently suited to the environment and the activities the PCs will be engaging in. For example, the weather table might have an entry that affects Dungeoneer or Pathfinder skill tests, because those are things you you do in Adventure phase, while the town events table might affect your Steward or Resources tests, because those are the things you do in town. Anything else in the text is slight window dressing, leaving deep exposition for purposes of tone or story to the GM (sometimes overtly so).To respond and consider your point, I think that the reason you don't roll for weather in those scenarios is because the weather isn't setting the tone or playing a role in constructing the outcome of those scenes-- town is safe from the depredations of nature, particularly in the milieu Torchbearer emulates, and so the tone of the story demands that weather be more impactful outside of town than within.
See the link in my previous post about wandering damage tables.One of my key ideas tends to be that "game elements" like mechanics, procedures, feats, whatever, have a feel to them, a texture that, through the player's utilization, experience, or use of those elements conveys theming-- a kind of playable literary device if you will, and that most conventionally simulative and gamist elements are actually textural (of or pertaining to texture, as opposed to textual, to be clear with the semantics.)

Torcherbearer is based on Burning Wheel, but the point stands. Anyhow, the thing is that, while Torchbearer makes a nod to the texture by having events or weather in the appropriate phases, and that does contribute to the tone, the players focus very quickly on the mechanical challenges presented—which also contribute to the tone of dreariness, exhaustion, or whatever.In other words "What feeling does this element create, and how does that inform the feel of the story?" In this instance, the gap that you've highlighted seems to be drawing the participants attention to the weather in some instances, but not others, probably to shift the tone of those instances when it does matter. Doubly so if it has mechanical impact, it is a means by which the environment-- Tolkien's "Wilderness" to borrow the concept, makes itself known to the participants. If say, things are harder to do in the rain, that drives home it's impact on the scene-- the dreariness and exhaustion, which sounds like the point of grind mechanic, from what you're saying (and also knowing Torchbearer is FITD, I'm imagining it as not being entirely dissimilar to BITD's Stress mechanics, conceptually.)
The section of the book containing towns is titled, "Safe Havens and Other Poor Assumptions". In Torchbearer, town is not a refuge, it's just a different pile of crap to deal with.Meanwhile in the tone of the milieu, town is a place of safety, where the players can find comfortable refuge away from the rain, they don't have to worry about it there, so its touch is gone, along with the cloud it puts over the narrative.

I'm not sure what you mean by "unnevenness of the mechanic", but if it's simply the fact that we have different wandering damage tables for camp, town, and journey/adventure, I think that was just done for variety. The tone remains one of desperate scrabbling amid adversity and danger, regardless of where you are.In other words the uneveness of the mechanic creates a kind of "Ludonarrative Harmony" between the narrative ideas of the wilderness as a place of stress /town as a place of safety, and the player's feelings of tension and frustration during each portion of the game. Similarly, from a simulation perspective, it doesn't have meaningful effects anymore because you're safe and can just go inside-- the game doesn't need the information for the simulation to impose logical effects on you, because there are none to impose, its giving the weather variable a null value while you're in town.