Large Aus Forum Loses 4 yrs of data

[glend]

Just a reminder that the "Cloud" is not perhaps as reliable as the marketing people would like us to believe. The Ausjeep forum (the big Jeep owners forum in Australia), and where I am a moderator, has today rolled back to May 2010, with complete data, and member loss for a four year period. How is this possible? The Adminstrator trusted the hosting supplier with the backup and restoration of the board and the supplier had a key server crash which held the backups, and I believe there was a cascade of corruption as a result. If the Administraor had not had an old hard copy on DVD (dated May 2010) they would be completely gone and had to rebuild from scratch.

Don't take your internet forum content for granted.

[ZeroID]

Adobe Cloud went AWOL today, no one is immune to a crash

[AG Hybrid]

Epic. Just epic! No backups of the backups.

[The_bluester]

Ouch...

The big time delay of NBN type services has put some focus on our own backup issues at home. I did have plans for buying a new NAS and rehoming my existing one to my sisters place where we could swap backup space. Our in home backup replicated over the net on a share on the old NAS and her in home backup on our old NAS replicated on a share on our new one.

Problem is it relies on decent upload speeds to be workable and while I have something passable (NBN Fixed wireless with around 4mb/sec real world speed tests) she does not. I suppose that we could do the initial backups in house than move the other one over.

[Kunama]

That would be a right pain Glen. I run twin external backups with TimeMachine doing its stuff at all times.

[killswitch]

Bummer...

Even if the host guarantees 99.9% redundant backup, you should still do a overnight or weekly external backups and store it on a fireproof safe.

[Steffen]

More importantly, you need to test your DR plan regularly, at the very least test your backups. Otherwise you may end up with years worth of corrupt data, all neatly tucked away on discs or tapes, and never know it until it's too late.

Cheers
Steffen.

[multiweb]

The cloud.... LOL. It does sound good, almost serene... until it all comes tumbling down. There's no two ways of thinking of data. Back it up, know what you're backing up and know where it is. Then figure how quick you can recover it. That will never change.

[marc4darkskies]

I'd just ask the NSA for their backup of that cloud to be restored!! :lol:

[tlgerdes]

Backup is never the problem, restore is the problem. When was the last time you tried to recover what is in your backup?

Most backup applications rely on a functioning system to restore too as well.

[FlashDrive]

Have never ' trusted ' the so called ' Cloud ' or Skydrive ' ... don't want my data ... ' out there somewhere ' .

Bought myself a Western Digital 1.5 TB ' My Cloud ' and connected it to my High Speed Modem Router N600 ..... can access it through my network on any Laptop or Tower ' wirelessly ' and it's fire-walled and password protected.

Transfer speed is not to bad either....67mb - 90mb p/sec on most file types.... ( akin to typically USB3 Speed )
I also have a 1TB Sata 6 Hard drive in my tower ( D- Drive ) as a backup also.

You can never be too careful :question:

Col.....:)

[glend]

Well they have apparently been able to restore it to last Saturday's state. Not sure how it was done, it was not the hosting company, but some of the forum members who are experts in internet technology. It had something to do with Google Caching.

Here is what the Administrator had to say:

"Many members came to the aid, some working to try and restore some of the 4 years of data by using Google Cache, and other, with expect knowledge, talking to our host to offer more knowledge and possible solutions."

I did not realise that Google is just caching things all the time.

[multiweb]

Bare metal restore, the only way to go.

[mithrandir]

99.9% means it could get lost once every three years.

When I was working the client wanted 99.999% uptime - quite reasonable in a 24x7 multinational business. That's an unscheduled outage of less than a second per day, or if it was backups, one single day failure in 300 years.

If you are going to back up to a NAS it has to be at least RAID-1 or RAID-5.

[el_draco]

You can never be to careful. I have triple redundancy in back-up including an off-site, one I carry with me and the system itself. TWICE I have got to the point of plugging in my last back-up, sweating and trembling all the way.... Twenty years worth of everything!! Makes you wonder why we ever left paper behind!

[rally]

Glen,

Who was the Cloud Service provider ?
They should be named so we can all avoid them.

I find it unfathomable that a commercial service provider who is being paid to perform but one single service - to provide safe storage and access to data files - finds itself incapable of doing so in the normal course of routine business.
How on earth would we expect them to look after our data in a real emergency.

To not have their own incremental backups and offsite storage simply reeks off colossal ineptitude and I would say borders on professional negligence.

We are now to well under $0.10c per gb of hard disk storage cost - down from nearly $1,000,000 per gb 30 years ago !
No doubt SAS and Tape options for hyper data storage is even less.
There simply cannot be any valid excuse for not having backups - no user could possibly expect any less.

Phew - glad I got that off my chest !

[issdaol]

The unfortunate fact is that many cloud or hosted service providers rely on one or more unverified steps in their backup & restore processes. Simply due to the extra time and cost to undertake those procedures.

Also the promise of xx.xxx % up-time does not mean too much unless you have a guarantee of where your last restore point was. Your systems may be down for the promised up-time period only to find that you lost a day or more of data.

The reality is for genuine guarantees of both up-time and minimal loss of data to work the cost are usually higher than most people want to pay.

[glend]

I will try to find out.

[Shano592]

Incredible, isn't it? I was reading on Whirlpool last week, that a similar thing happened with someone's business data. Hosting server had 'backups,' but when the hosting server crashed, all of the backups went with it. Good to see that the forumites were able to do some fancy footwork and recover so much, Glen.

Our web host lost 8 years of our emails, when their email server was crushed under a collapsing wall. Luckily we sent everything to Gmail as redundancy. We lost about 2 hours of emails that hadn't caught up.

@Rom - 20 years of paper backups is a lot to carry around! And hard to cross-reference! :lol:

@Col - We have the same modem, and do the exact same thing. I also backup from my system with a second hard drive, for redundancy. USB3 is a glorious thing!

Glen's example just shows that you can never be too careful with your data. More chance of restoration is always better.

[The_bluester]

If that was pointed my way, yes, I agree, I use a four drive array in RAID5 for my NAS and any other I build or buy will be the same. It has saved us once already when a drive died (Only my second in 25 years) and the NAS flagged itself as having an issue, a hot plug and overnight rebuild later and it was back to normal. At that time it mostly stored media so it was stuff that would be inconvenient to loose rather than a real problem.

The February bushfires gave us a really good scare and I want offsite backup as well for all our photos and so on before next years fire season just in case we are not at home if it all goes pear shaped again meaning we don't get to unplug the NAS and clear out with it under our arm. We were so frazzled by the time we evacuated this year that I did not take it as I left so all our important data was in the lap of the gods when the fire went through.

[glend]

The hosting services for the Ausjeep forum, which failed, were provided by Panthur Hosting Services, an Australian company. They are one of many budget hosting services.

This is them: www.panthur.com.au

It is pretty amazing how cheap hosting is these days.

[Lee]

Excellent point.... I often wonder how those with 500-1000 GB backups on cloud servers would ever restore their data after a catastrophic loss....