Front page wonky

[DaveStebbins]

Is there a reason for the venom?

I have insulted no one, and only complained that the site appears to be less friendly at the base screen level?

Right, you made that complaint in post #2 of this thread. Also posts #5, #7, #12, #15, #18, #21, #23, #29, #32, #33, #38, #40, #43 and #46. A full 30% of the first 50 posts in this thread were yours.

Perhaps the 'venom' was a response to your... enthusiastic and persistent... method of complaining?

 

[BrooklynKnight]

You know, in further retrospect, my venom was due to a number of contributing factors.

A. My cranky mood.

B. Wystans annoying persitence

C. I think I was trying to say what Morrus couldnt, or wouldnt. He's got far too much tact to speak to members of the site that way. I'm not saying I was speaking for him, mearly that If i was him thats how id react.

In any case the moment is past.

 

[resistor]

I just wanted to take a moment to apologize to Morrus for being a pain earlier. I was mostly just in a really bad mood at the time.

In all honesty, though, this change does have a major impact on my browsing habits. I haven't been reading nearly the number of threads that I used to read, just because I don't see them while browsing the Recent Posts. I have also (prior to today's performance issues) not seen any improvements on my end, so, at the time, the change seemed unjustified to me.

So, while I would still like to see them come back, I would like to redress some of my fiercer words earlier. Instead, I'll add myself to those who calmly support its reinstatement.

Sorry again, Morrus.

 

[Pinotage]

On a tangent, however, I find that the recent reviews panel is something I never use. If I want to read a review of a particular product, I'll look for it, but don't see the relevance of the list of recent reviews. Maybe I'm in the minority on this point, but I'm thinking that either
a) replace the recent reviews with the recent forum posts

-- or --

b) if removing the recent forum posts increased speed, why not remove the recent reviews as well and increase speed even more?

Did the Recent Reviews take up as much CPU time as the Recent Posts? I can't imagine that it did. I certainly miss that one. Oh, well. Thanks, Morrus, in any event, for speeding things up.

Pinotage

 

[Twin Rose]

Here is what Recent Posts was doing:

Loading up however many million posts on EN World, and then organizing them from newest to oldest, showing the most recent few of those. This process was repeated for every person viewing the front page in any given few seconds, which is typically quite a few, 80% of which are "guests". (About 8% of the total number of people online were viewing the front page at the time of this post, aproximately 125 people).

The number of database connections this made was creating a situation where the system was unable to create any more connections, therefore timing people out. Something that normally takes 1-2 seconds to load was taking 10-20. In some areas, where multiple things are being done, that drags those on for 40, 50 or even more seconds.

Reducing the number of database connections at any given moment that aren't needed is a huge part of speeding EN World up right now. It's simply that all the connections made on any page, are multiplied by the number of people viewing it... and with news, reviews, recent posts, polls, and anything else that requires database queries on the same page multiplied by the number of people using it, it can become a huge drag.

Some things can be cached - the front page doesn't list the 'newest' stuff at EN World, rather specific things that are chosen, to reduce the number of database queries needed to display it. I'll be looking at more ways to streamline that, and create more caches, perhaps updated hourly that can bring some of these features back to the front page.

It's 3 am here as I post this, and 1500 people are on.. On a tuesday night! (or wednesday morning). That's a huge jump from where it was a few months ago. In the last 30 days, there's been 13 million page views, and over 1.3 million unique visitors. That's a lot. (Well, 31 days, it's after midnight now). And considerably higher than it was. We have to find ways to streamline, since there are other sites running on this server.

In a perfect world, I could find a way to make it run fast AND have every possible feature, but until the upgrade is complete, there isn't a way with the amount of people online. Many threads have cropped up of people saying "EN World is slow!"

 

[Twin Rose]

I do want to point out that there seem to be occasional, intermitant slow downs, and I'm working to find the situations that put the most drag on the system and number of database connections.

One of these I know is that at midnight the system sends out a LOT of email to people for subscribed forums and threads. The sad part is, the server also has to process a TON of returned mail about 5 minutes later, because MANY people have the wrong email address in their EN World account.

I've been working on it steadily since I got home, and Morrus (obviously) was working on it the whole time I was at Gen Con - removing the heavy drain of the recent posts box, for example. I'll work to streamline as much as I physically can, however. With enough work it's possible we can get those features back up before the upgrade, but I can't promise anything - it will vastly depend on performance vs. hassle. I don't think ANYONE likes it when EN World is having a bad day speed wise.

 

[francisca]

Hey Mo-

As far as I'm concerned, you can pare it down even more. I actually dislike all the extra "features" hanging around the periphery of the page. I'd really like a "lite" presentation of ENW. I don't want the latest poll stuck over there, the newest posts over here, etc.. The toolbar/links near the top of the page is way more than I need to find what I want.

Keep up the good work, guys.

 

[Dimwhit]

One of these I know is that at midnight the system sends out a LOT of email to people for subscribed forums and threads. The sad part is, the server also has to process a TON of returned mail about 5 minutes later, because MANY people have the wrong email address in their EN World account.

Personally, it wouldn't break my heart to see the email subscriptions go away. I know that's heresy, because a lot of people use that feature, but email notifications on a board like this are a serious drag on the server load.

I stopped using it when I realized that if I just enter the site through my user cp, all my subscribed threads with recent posts are right there for me.

Anyway, that's one insignificant man's opinion on that matter. And I'm kinda with francisca...the lighter the better. But I'm sure we're in the minority on that one.

Thanks for the work!

 

[Nellisir]

Just a thought, and an obvious one at that -- can the "new posts" feature be set to only feature posts from certain forums, say the top three? Or would those still be too much?

Just a thought,
Nell.

 

[resistor]

Loading up however many million posts on EN World, and then organizing them from newest to oldest, showing the most recent few of those. This process was repeated for every person viewing the front page in any given few seconds, which is typically quite a few, 80% of which are "guests". (About 8% of the total number of people online were viewing the front page at the time of this post, aproximately 125 people).

That just seems like an awfully inefficient way to implement the recent posts thing. I feel like there's got to be some better way to do it.

As an idea, I would suggest looking into using stored procedures (also called triggers) to fetch the most recent posts. Basically, these allow complicated queries (like finding the most recent posts across all forums) to be written in scripts within the database server. This would likely be a speed gain, since it removes the need to transmit large intermediate datasets (all the posts on the forums) from the server to the client. Rather, the server only transmits the final product of the procedure.

I'd be willing to help figure out how to do it, but I'm not familiar with the database model ENWorld uses.