There's nothing wrong with an evolving internal mythology in a show, if you balance it will more accessible faire. The X-Files, which had an extremely elaborate mythology, balanced that by having about half the episodes almost completely unrelated to the continuing plot, but still using the show's overall themes (mystery, paranormal, etc).
BSG slides by, despite having no "unrelated" episodes, by repitition. The characters are relatively simple in concept, and their basic traits are repeatedly asserted in every episode. There's always a little bit of exposition reminding you what happened last time, and the main characters practically slap you in the face with the main plot points. BSG, though excellent, is not subtle.
On the other end of the spectrum, Star Trek: TNG was wildly popular in its day beyond the core fan group because it so rarely had a continuing arc. Although the characters grew and changed slightly throughout the show, you almost never had to know what had happened last time to make what happened this time make sense. Even if you did, there was a convenient montage that reminded you of all the key points.
Star Trek: DS9 took a step away from TNG's casual approach, and has thus won itself a number of diehard fans, because it rewarded the loyal more clearly. Thus, it's generally considered part of the Babylon 5 crowd. It's certainly frequently compared in places like this. Its internal mythology was hardly as elaborate, but it was a Trek that took a step in that direction.
Farscape is deeply intricate, character changed a lot, there were almost no "casual" elements of the show, which made it extraordinarily different to attract new viewers. In fact, the show's format encouraged viewer attrition, since you were punished for failing to follow along. Once you missed an episode or two, there was little point in coming back. To travel toward the main stream, Lost is the same way - very intricate, with little point in tuning in unless you've been there since the beginning.
Now, if this is all the case, why is a relatively intricate show like Battlestar Galactica considered so successful? I'd say that the rise of TiVO/DVR has made intricate shows more potentially succcessful. It's easier to stay current on a show, since it's easier to record.
Anyway - YMMV. That's my idea.
