Database lost?

[jcfiala]

I just wanted to throw in my own two cents - as someone who does both programming and dba work, I'm shocked at how old a backup this is. What am I donating money for if the admins aren't taking care of what I'm paying for?

 

[Twin Rose]

I just wanted to throw in my own two cents - as someone who does both programming and dba work, I'm shocked at how old a backup this is. What am I donating money for if the admins aren't taking care of what I'm paying for?

There have indeed been other backups, but those had some problems as well. Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed. Obviously, this is something that can't be done every week.

In the past, a backup every 3 months or so seems to have done the trick - but this time, there's been problems for the last month or so with the database. The most recent backup was, in fact, also a bit corrupt - and I suspect it could lead to another crash in a couple weeks or so.

Also remember that the previous admin left us recently, and I'm still rather new at the gig - if assiduous in it.

What donations go for is the hosting and bandwidth. Even still I'm planning to donate one of my other hard drives to the server so we have more free space for more regular backups. (If I can manage it, i'll be trying to keep weekly backups instead).

In the next few months, I'd prefer to buy a very large hard drive to put in down there. Hopefully, that means through money gained at the EN World GameStore.

 

[GlassJaw]

Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed.

2 gigs isn't that much space - why does it take so long? That doesn't seem right.

What donations go for is the hosting and bandwidth. Even still I'm planning to donate one of my other hard drives to the server so we have more free space for more regular backups. (If I can manage it, i'll be trying to keep weekly backups instead).

In the next few months, I'd prefer to buy a very large hard drive to put in down there. Hopefully, that means through money gained at the EN World GameStore.

Why not pay to have the site hosted rather than spending the money on hardware? That seems a lot more reliable to me. I was in the network industry a few years ago but haven't really kept up on fees but I would guess that hosting a site that is primarily a message board (albeit a somewhat active one) shouldn't be too server intensive for even an average webhost.

 

[James Heard]

I think ENWorld eclipses the "somewhat" active status by quite a lot. Crothian's posts alone would tally up the sum total of most of the most active of the lesser boards I take part in, and maybe beat them out. It might be worth it though, to try batting doe eyes at a corporation that really DOES have a lot more traffic. I'm picturing Google ads

 

[IronWolf]

I will say up front that its always easier to be afforded the luxury of an armchair quarterback than to be the one that actually is doing the work. I certainly don't have all the details of the situation, which can certainly change observations to one way or another.

There have indeed been other backups, but those had some problems as well. Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed. Obviously, this is something that can't be done every week.

As Glassjaw said, 2GB isn't that much space really. Is this a compressed or uncompressed backup?

20 hours to download 2GB? Someone in another thread I believe mentioned that seems like a long time to download. I don't know the bandwidth allocation on the hosted side for EN World though, so maybe it is the bottleneck. Most cable modem connections and even a fair number of DSL connections should be able to come in well under 20 hours.

Downloading aside, I would think you could keep at least a couple of backups on the server itself. Then those can be shuffled off to an offsite location automatically as needed for extra safety. Even if it took 6 hours in the middle of the night to shuffle them offsite, that shouldn't be too big of a deal. All of this would be automated, so once its setup it wouldn't even require that much maintenance save for a test restore here and there to be sure the backups were good (and even that can be automated to some degree).

In the past, a backup every 3 months or so seems to have done the trick - but this time, there's been problems for the last month or so with the database. The most recent backup was, in fact, also a bit corrupt - and I suspect it could lead to another crash in a couple weeks or so.

If decisions were made that 3 months loss of posts, new memberships, etc was reasonable then I can see why that might have done the trick. As with everything one must weigh the options that one has and determine what acceptable risk is. I would think though that with existing resources one could pull off a weekly backup. As mentioned before though, I am only operating on knowledge I have of this place picked up from various threads over the past 2 years.

 

[buzz]

Why not pay to have the site hosted rather than spending the money on hardware? That seems a lot more reliable to me.

The bandwidth, traffic and services offered on ENWorld easily merit a dedicated server. A hosted solution, even a leased server, with consulting, would probably cost way too much.

 

[Morrus]

The bandwidth, traffic and services offered on ENWorld easily merit a dedicated server. A hosted solution, even a leased server, with consulting, would probably cost way too much.

Yep. A co-loc dedicated server is waaaaay cheaper. EN World used to be hosted on a leased server, but the cost became excessive.

 

[BSF]

Downloading aside, I would think you could keep at least a couple of backups on the server itself. Then those can be shuffled off to an offsite location automatically as needed for extra safety. Even if it took 6 hours in the middle of the night to shuffle them offsite, that shouldn't be too big of a deal. All of this would be automated, so once its setup it wouldn't even require that much maintenance save for a test restore here and there to be sure the backups were good (and even that can be automated to some degree).

That might be a worthwhile approach. Bring the boards down long enough to backup the database to disk; bring the boards back up; then download the backup on disk. Minimum time for the boards to be down and perhaps some pain with the download competing with traffic. But that might be worth it.

Of course, I am not sure how admins access lower level functions. The options will be much different if you have full CLI access as opposed to access via a browser tool interface. But the idea is certainly a decent option that Morrus could consider.

As you said though, most of us don't really know what's going on with the backend. Some of our suggestions might not make sense, or might be a heavy burden for volunteers.

 

[Staffan]

There have indeed been other backups, but those had some problems as well. Backing up the entire database takes something like 2 gigabytes worth of space - approximately 20 hours to download even at high speed. Obviously, this is something that can't be done every week.

2 gig taking 20 hours? If I have calculated correctly, that's something like 30 kB/s, which is pretty much what I get with low-end DSL. A 10 Mbit connection (which is what I would consider "high speed") would get it done in half an hour.

 

[BSF]

2 gig taking 20 hours? If I have calculated correctly, that's something like 30 kB/s, which is pretty much what I get with low-end DSL. A 10 Mbit connection (which is what I would consider "high speed") would get it done in half an hour.

Geographic diversity and personal budgets stand in your way. My last house couldn't even get DSL and it was in a well-populated area of the city. I could get ISDN or I could get cable internet access. I don't watch much television and just to get hooked up for cable internet was $60+/month. In short, not an expense I would want to afford.

My new house can get DSL and we do have that. Even so, it is considerably more expensive than comparable DSL rates in other areas of the country and world.

I don't know what anybody's personal budget and geographic limitations are for greater than modem access is, but it is entirely possible that Twin Rose, MM and others would be administering from a DSL connection.