RAID array recovery

[pete376403]

I know this is a bike forum but there seems to be a lot of IT knowledge here.

Who in NZ can recover data from a failed RAID5 array? Got hit with a double whammy of a failed backup AND a lost array configuration. The disks are (afaik) ok, as everything was working fine right up to the point the config was lost in the controller. No recovery attempts have been made. The config was R5 + hot spare - 5 x 300G SAS disks in total. We have older backups but nothing for the week immediately prior.

And yes management agree that spending money on a better backup solution is likely.

[Akzle]

oh fucking chortle.

Throw reiners testdisk at it, for a start. That may allow you to rebuild your config and good-o.

If not, and i assume shit's striped, and pariity is everywhere, you're fuckeroo.

Just googluhd up reclaimefree raid recovery, too.

[Akzle]

oh fucking chortle.

Throw reiners testdisk at it, for a start. That may allow you to rebuild your config and good-o.

If not, and i assume shit's striped, and pariity is everywhere, you're fuckeroo.

Just googluhd up reclaimefree raid recovery, too.

[TheDemonLord]

Step 1: Spead Cheeks
Step 2: Bite Pillow

Really - there aren't much other options - RAID recovery from a reputable company is in the Thousands, plus if you try and do it youself, you run the the risk of further corrupting the data (if it is recoverable)

[wharekura]

I assume the failed backup is on a non-raided drive? If so and non encrypted, maybe easier to recover from than the Raid5'd disks?

You sure you're not hit by the crypto virus going around?

[Tazz]

Step 1: Spead Cheeks
Step 2: Bite Pillow

Really - there aren't much other options - RAID recovery from a reputable company is in the Thousands, plus if you try and do it youself, you run the the risk of further corrupting the data (if it is recoverable)

They are onto a good wicket with that. I got a quote a few years back, something like $3,000 for a 500gb drive :shit:

[aderino4]

Cameron from Datalabs http://www.datalab.co.nz/ is the man you want to see.

Be prepared to spend 1200+ for RAID recovery tho.

I've only ever had 1 drive he couldn't recover fully and even that he managed to get close to 45% of the content. The drive was completely stuffed. Scratched platter and fallen head.

[pete376403]

Thanks all, I'll contact Datalabs.
No not Crypto (had a brush with that before, and recovered from, back when the backups were working)

I'll let management decide of the cost of recovery is more or less than the cost of loosing a weeks work.

Stuff that I have got back so far shows me that, among other things, people keep a lot of unimportant, out of data shit in their user folders.

[TheDemonLord]

Stuff that I have got back so far shows me that, among other things, people keep a lot of unimportant, out of data shit in their user folders.

Welcome to the world of IT....

[oneofsix]

Thanks all, I'll contact Datalabs.
Stuff that I have got back so far shows me that, among other things, people keep a lot of unimportant, out of data shit in their user folders.

In the physical world it is called hording, but there is no cost to keeping stuff in IT, right? :bleh:

[p.dath]

I know this is a bike forum but there seems to be a lot of IT knowledge here.

Who in NZ can recover data from a failed RAID5 array? Got hit with a double whammy of a failed backup AND a lost array configuration. The disks are (afaik) ok, as everything was working fine right up to the point the config was lost in the controller. No recovery attempts have been made. The config was R5 + hot spare - 5 x 300G SAS disks in total. We have older backups but nothing for the week immediately prior.

And yes management agree that spending money on a better backup solution is likely.

If you have only lost the array configuration, and the drives themselves are fine then you may be able to tell the controller to re-read the configuration off the drives. Many controllers support this so that you can transport drives sets between servers with the same controller.

Does the controller still show that there is a RAID set, or does it show nothing at all?

[Gremlin]

Welcome to the world of IT....
I keep telling a client (on hour rate) to tidy up their data. I mention that it would be way too expensive to pay me to do it, but they should tidy up, they'd save big, retention of backups as well etc etc.

Otherwise... yeah, I can keep buying more hardware to store it but it's like building more highways for cars....

[TheDemonLord]

I keep telling a client (on hour rate) to tidy up their data. I mention that it would be way too expensive to pay me to do it, but they should tidy up, they'd save big, retention of backups as well etc etc.

Otherwise... yeah, I can keep buying more hardware to store it but it's like building more highways for cars....

I remember a NAS implementation and migration - over half of the Data that was stored on the old Server wasn't work related (used a tool to find all the MP3, Iphone Backups, Video files etc.) The Customer was understandably a little annoyed at their users when they realized what was going on.

[Stirts]

We are currently rolling out ScarePoint and the shit we are finding on clean-up prior to migration is un-fucking believable. Backing that up...what a fucking waste of money :crazy:

[TheDemonLord]

We are currently rolling out ScarePoint and the shit we are finding on clean-up prior to migration is un-fucking believable. Backing that up...what a fucking waste of money :crazy:

Aaaahhhh SharePoint migrations - hours upon hours of my life that I will never get back

[Big Dog]

Once upon a time at a company I shall not disclose...
I was asked to tidy up some files for prior to a hardware migration.
I created a script to remove all the files that failed the specified business rules.
Thinking I was smarter than the average bear but not smart enough to not make a mistake I thought I know, I will ship all the detritus to this storage over here that is scheduled for destruciton in 30 days instead of listening to the honchos and just shredding it. So I created a locked down folder that only I had access to and away I went...

75% of the data on the source was moved to my hidden location and deleted from the source.
All sorts... full cds, itunes collections, family albums etc.

So a a couple of days later I get a panicked call from one of the execs... omg all my files are gone... I didnt think our files counted in the clean up etc.
No worrries I will put them back in a folder labelled recovered files, you sort what you need onto a drive you can take home and I will delete it again in a week.


Sounds great right?

Well someone asked their wife to sort the files onto a drive and take them home. I guess he was too busy to do it himself...

To bad he forgot about the videos of him banging his secretary on his desk then eh?

[Akzle]

Does the controller still show that there is a RAID set, or does it show nothing at all?

a quick googluh shows raid five stripes parity. ie it's not all on one disk.

it also shows that fucking two disks fucks your array.

[Gremlin]

a quick googluh shows raid five stripes parity. ie it's not all on one disk.

it also shows that fucking two disks fucks your array.
p.dath is pointing more towards the controller. ie, we haven't actually been told that the disks are fucked. It's just the controller that is fucked, so the disks could actually be perfectly fine...

On a side note, I never really liked roaming profiles, it fucked up stuff like offline file sync blah blah and made login/logoff times long. We always prefer to make our clients aware that they should save on a network location, because that's the server and it's backed up etc etc rather than hiding stuff, re-directing folders and so on. It makes perfect sense now when you consider Apple loves saving everything into the profile...


There was the one time staff at a client wanted to share some movies and didn't have the space to put them somewhere for others, so thought the server share was the perfect place... until it ran out of space and the backup software scheduling fell off the earth because the hourly incrementals had grown by double digit GB, disk IO started complaining and generally their simple idea turned my life into a mess...

The files were immediately deleted, much to their disappointment :bash:

[pete376403]

If you have only lost the array configuration, and the drives themselves are fine then you may be able to tell the controller to re-read the configuration off the drives. Many controllers support this so that you can transport drives sets between servers with the same controller.

Does the controller still show that there is a RAID set, or does it show nothing at all?

That sounds a little bit modern. This is an eight year old Intel SSR212MC. The disks show as there, but unconfigured.
Also, I talked to Datalabs, they sound pretty sure they can get it all back at somewhere between $2k5 and $4K5. Also they will look at the disks and if they cant salvage them, there's no charge.
Over to management.

[wharekura]

Once upon a time at a company I shall not disclose...
...Well someone asked their wife to sort the files onto a drive and take them home. I guess he was too busy to do it himself...

To bad he forgot about the videos of him banging his secretary on his desk then eh?
Auckland City Council?

[wharekura]

...There was the one time staff at a client wanted to share some movies and didn't have the space to put them somewhere for others, so thought the server share was the perfect place...
In a prior company we had an admin that gave his server shares heaps of space and us plebs buga all - despite having sql server backups, log files etc. after asking for more space he refused. So being the labyrinth the network array of folders are, I played a joke. Told said admin in passing, that I implied that I no longer had to worry about shared space as there was heaps of space in a particular folder of his and walked off not mentioning which folder. I did not actually use any of his folders.

I am sure he would of spent time checking logs...

[Big Dog]

Auckland City Council?
I can neither confirm nor deny.
It wasn't but I could just be saying that.
It was but I could just be saying that.

2 of the above could be considered true.

Sent via tapatalk.

[Akzle]

The Tao of Backup (http://www.taobackup.com/)


((just found it. it's lulz, this seemed the most appropriate thread to hijack)

[Big Dog]

The Tao of Backup (http://www.taobackup.com/)


((just found it. it's lulz, this seemed the most appropriate thread to hijack)
That's pretty good.

Sent via tapatalk.

[ledaero]

As it's an Intel... I can't offer you any promises, but steps 1-3 won't hurt anything and may very well return to you a working array.

1) Go into the Intel Matrix Storage Manager in BIOS.

2) Mark all the drives in the RAID as "non-Raid." This warns you that all your data will be lost, but really is simply re-writing the partition table, which is ALREADY messed up anyhow...

3) Re-Create the RAID using the EXACT SAME settings as the original RAID that you are trying to recover. Disks must be in the same order as before, and the strip size must be the same as before. Note: Having the disks in the wrong order will not destroy your data, it will just mean that step (4) will not work... If (4) doesn't work, you will need to repeat steps 1-3 until you have the disks in the proper order...

4. Download and run TestDisk from http://www.cgsecurity.org/ according to steps on the web site. If your HDs are connected in correct order, TestDisk should find the partition(s) within a few seconds. If it doesn't - look again at the array configuration. Get it right and it will recover - there's nothing wrong with the disk.

5. (optionally - if you need to) have TestDisk write the fixed partition table to the array drive.


A Raid 5 Array

[pete376403]

That looks interesting, thanks. The disks are still installed in the original positions. No recovery attempts have been made, because I was able to get back almost everything that mattered from backups. Not actually sure what to do with the disks, as mentioned they hadn't failed, but they are near 8 years old so probably past their use-by date, and the whole volume was only 900gig (300G 16Krpm SAS drives).

Do organisations replace drives on a regular basis, as preventive maintenance, or just as they fail?

[jonbuoy]

That looks interesting, thanks. The disks are still installed in the original positions. No recovery attempts have been made, because I was able to get back almost everything that mattered from backups. Not actually sure what to do with the disks, as mentioned they hadn't failed, but they are near 8 years old so probably past their use-by date, and the whole volume was only 900gig (300G 16Krpm SAS drives).

Do organisations replace drives on a regular basis, as preventive maintenance, or just as they fail?

"If its not broken don´t fix it" or "Nothing prevents maintenance like preventive maintenance" Your call :crazy:

[Akzle]

That looks interesting, thanks. The disks are still installed in the original positions. No recovery attempts have been made, because I was able to get back almost everything that mattered from backups. Not actually sure what to do with the disks, as mentioned they hadn't failed, but they are near 8 years old so probably past their use-by date, and the whole volume was only 900gig (300G 16Krpm SAS drives).

Do organisations replace drives on a regular basis, as preventive maintenance, or just as they fail?

ill take those scuzzies off your hands...

Typically run to failure. With sufficient redundancy, it is no problem. Find the company that can afford to bin perfectly good disks and ill show you one that doesnt need my custom.

[Gremlin]

Do organisations replace drives on a regular basis, as preventive maintenance, or just as they fail?
Try to replace whole servers on a cycle to prevent outages, but companies do try to run out as long as possible. Virtualised environments make migration much easier. Disks holding backups are regularly tested and any issues and they're replaced. Too important to risk issues there, especially when requesting massive amounts of data from them can put them under the most strain...