Network Attached Storage recommendations [Archive]

MarkH

12th July 2015, 15:49

So what would people recommend? I have no real requirements but ability to add drives wouldn't be a bad thing

What would people recommend? The smart ones would make recommendations with considerations to the requirements.
If you have no real requirements then I'd recommend you save your money until you have a need to spend it.

For those with some requirements:
I bought a couple of Dlink 2-bay NAS boxes for $150 each. They aren't blazingly fast but give me much convenience and are fast enough. I've plugged my USB printer into one and can print to it easily, saving having to plug the printer into one of my laptop's USB ports.
I have 3 x 4TB and 1 x 2 TB hard drives in my NAS boxes - that is enough to hold my TV programs.
I do like the convenience of having all that data available without having to find the right drive and plug it into one of my USB3 enclosures.

I'd love to get 4 x 6TB HDDs and a 4-bay NAS box so I could set it up in a RAID 5 array, but that would be quite a lot of money to spend and I have 2 motorcycles to support.

bogan

12th July 2015, 15:58

Personally, I'd go with a single board compy as a lounge media server/player over a simple NAS. SATA ports plus a few mechanical drives, open frame if you've got a tv cabinet to put it in. Would take a bit more config, though there are probably prebuilts out there to be had, or cloned.

Scuba_Steve

12th July 2015, 22:34

What would people recommend? The smart ones would make recommendations with considerations to the requirements.
If you have no real requirements then I'd recommend you save your money until you have a need to spend it.

By "no real requirements" I meant special ones. Space is the main thing.
Currently I have a 4TB HDD connected to a wireless router via USB but this is filling up (think we're at 3.67TB of the 4TB limit) so looking at replacements.
The setup I have now requires the replacement of the drive in order to expand so thought I'd see what other people are running/recommend and as mentioned being able to add rather than replace would be an advantage as this replacement is going to take the NAS out of commission for awhile as I copy data across.

TheDemonLord

12th July 2015, 22:46

Synology NAS

Do it.

I run multiple ones for my Company - they are lovely to work on.

pete376403

12th July 2015, 22:50

RAID5 is no longer considered desirable, especially when you start using large (2tb or more drives). Its good while everything is working but turns bad when a failing drive is replaced and it's rebuilding - a single bit error is enough to make it fail. RAID6 is better but if the data really matters use RAID 10 (of course that means a whole lot more disks)

http://www.avsforum.com/forum/26-home-theater-computers/1078476-raid-5-raid-6-bit-rot.html

Gremlin

12th July 2015, 23:12

Horses for courses, is all your data as precious, do you need it redundant, how much do you need.

2011 I bought a 6TB (2x 3TB) WD NAS (My Book Live Duo) and configged a raid 1. Then I began running out of space in 2012 and bought a second (same config). Couple of weeks ago the first one came up with the red light of doom (well, that's what some refer to anyway) but essentially it looks like it's dead (suspect the drives might be fine tho). Haven't got around to seeing if the data can be salvaged (sure hope it can) but I need space to store the data on anyway.

Any of the consumer grade NAS are relatively expensive (easy to see the cost if you exclude the drive cost), and stepping up to anything more enterprise grade has laughable prices (as in, haha, they can't be serious).

So enter the original plan which I dismissed in 2011 as I wanted something easy to manage. FreeNAS. Current stable version is 9.3 and it's software you load onto a USB key, and all the drives are used as storage. Hardware is whatever you build, but they do have recommendations of what you should be running. Feel free to choose something more basic, but you may encounter random issues. However, this is not consumer type stuff (iXSystems does have TrueNAS appliances you can purchase) and you have to be technical... even then delving into Linux isn't something I do, so the learning curve has been... steep. :wacko:

Old desktop has been re-purposed, with a new one built... still trying to wrap my head around the configs before I load data on it, but it has 5x 4TB disks (Raid Z1 usable capacity is just over 14TB), read/write cache disks, and then probably another drive or two later for a bit of extra storage for FreeNAS itself.

Akzle

13th July 2015, 07:29

this:

So enter the original plan which I dismissed in 2011 as I wanted something easy to manage. FreeNAS. Current stable version is 9.3 and it's software you load onto a USB key, and all the drives are used as storage. Hardware is whatever you build, but they do have recommend...

Old desktop has been re-purposed, .

also, if you can get your head around winblows' horrendous workings, nix is a cakewalk.
Linux will get all your files back (photorec, testdisc, scalpel). Rewrite the fstab first and the drives may go bootable.

Mate runs a gigabit netgear hooderdacky. Missed some config and blanked 7T when installing the drives. Serves it vpn through win2k3 i think.

Best standalone hardware imo is lacie, but it comes with the pricetag.
Seagate drives are good. I've baked too many WDs to have any faith in them. hitachi rank best but come with the price.
Ymmv, because i hear people blag seagates heavily.

As for raid, very litlle advantage for end user, i have no idea about 5 or 6 or 10. You want striping with redundancy, which was about 3 iirc.

But yeah. Id bung a headless server under your staircase,(linux, obviously) lots of drivebays, lots of mobo ports. Could prob pick up a dual cpu on tm cheap. Not sure about running sata controller cards as bus is too slow unless theyve changed technology in the last 10 years :sweatdrop:

TheDemonLord

13th July 2015, 08:01

As for raid, very litlle advantage for end user, i have no idea about 5 or 6 or 10. You want striping with redundancy, which was about 3 iirc.

RAID 6 is your best option, gives the best option in terms of not having silly amounts of disks, very good redundancy and performance.

Whilst you can make a NAS from an old PC (which technically isn't a NAS, its just turning the PC into a glorified File Server) it does come with increased running costs (in terms of Power usage)

Akzle

13th July 2015, 10:08

RAID 6 is your best option, gives the best option in terms of not having silly amounts of disks, very good redundancy and performance.

manual/scheduled offline tape backups are your best option, actually.
IMO the only benefit of raid for eu is faster access (not really notable for most people/applications, especially if it's a music/movie dump...doesn't have high bandwidth requirements) and the OS "seeing" one partition rather than multiple. -:woohoo::rolleyes:

jbod. raid1 with backups. job done.

Whilst you can make a NAS from an old PC (which technically isn't a NAS, its just turning the PC into a glorified File Server) it does come with increased running costs (in terms of Power usage)
you realise NAS is an glorified file server, yeah? the overheads on a box pc are fucken negligible, while i cant point you to a watt-per-gig cost, i'm picking most NAS now-a-days run comparable cores and PSUs...

...and/or you could use it to heat your growroom :shifty:

Gremlin

13th July 2015, 14:09

Seagate drives are good. I've baked too many WDs to have any faith in them. hitachi rank best but come with the price.
Ymmv, because i hear people blag seagates heavily.
Hah, I wouldn't be making such blanket statements. One Seagate Enterprise range we used (the good old .11) ran at a 50% failure rate (inside warranty). Funnily enough their website still said it was 0.0something %. :scratch:

The majority of failed drives for us (tracking near on 700 drives now) have been Seagate, but then we used more Seagate in the past. Now we use more WD. WD does have a habit of running out of spare sectors ie, sectors failing too quickly.

you realise NAS is an glorified file server, yeah? the overheads on a box pc are fucken negligible, while i cant point you to a watt-per-gig cost, i'm picking most NAS now-a-days run comparable cores and PSUs...
Depending on NAS obviously, the main thing around TrueNAS for example is using very specific hardware the whole time. Gives a consistent response, stability etc. New desktop is using an GA-H97M chipset... Gigabyte lists Intel NIC, driver on website is Intel (desktop doesn't have an optical drive) but after some head scratching why I couldn't enable the NIC, the driver disk came with several NIC manufacturers on it and actual NIC was Realtek... Thanks Gigabyte :brick:

Then, with old desktop (also Gigabyte, EX58) it wouldn't boot from the USB stick. Whenever I inserted it (pre post, post post etc) it froze. After a lot of swearing and testing, it turned out that when FreeNAS set itself up on the stick, it was creating a flag the motherboard didn't accept. Changed the flag to BOOT (using Linux, you'd like that Akzle) and it was all happy... :blink:

TheDemonLord

13th July 2015, 14:16

Hah, I wouldn't be making such blanket statements. One Seagate Enterprise range we used (the good old .11) ran at a 50% failure rate (inside warranty). Funnily enough their website still said it was 0.0something %. :scratch:

The majority of failed drives for us (tracking near on 700 drives now) have been Seagate, but then we used more Seagate in the past. Now we use more WD. WD does have a habit of running out of spare sectors ie, sectors failing too quickly.

I'm the opposite - all my Seagates have been great, but have had WD drop its MBR for no reason and other WD do weird shit,

Gremlin

13th July 2015, 14:19

I'm the opposite - all my Seagates have been great, but have had WD drop its MBR for no reason and other WD do weird shit,
Brings us to the #1 rule... backup backup backup. :laugh:

Doesn't matter what your preferred flavour is, they all have equal chance of failure.

Except OCZ. Those fuckers should make coasters.

TheDemonLord

13th July 2015, 14:29

Brings us to the #1 rule... backup backup backup. :laugh:

Doesn't matter what your preferred flavour is, they all have equal chance of failure.

Except OCZ. Those fuckers should make coasters.

Lol - I know them feels - actually used getdataback for NTFS to 'fix' those WD drives, before they were sold and replaced with Seagates....

MarkH

13th July 2015, 21:34

RAID 6 is your best option, gives the best option in terms of not having silly amounts of disks, very good redundancy and performance.

I Can't see it to be honest.
4-Bay NAS box running RAID 6 would give the data storage of 2 drives - half the storage you paid for.
That's the same as RAID 10 while being slower in most cases both for read & write.
Maybe if you have RAID 6 over 8 drives it can make a lot of sense, but for what the OP is talking about it seems to me that RAID 6 would be pretty shit TBH.

With RAID 5 you could at least get 3 drives worth of storage out of 4 drives.
For better redundancy you might as well just bite the bullet and go RAID 1 for 2 drives or RAID 10 for 4 drives.

Another option is to forget RAID and just go with a scheduled backup of data, of course this also requires twice the data storage as the space you want to use. It is hard to get around this if you want to protect your data, you just can't have redundant data without purchasing extra drives to store it.

Gremlin

14th July 2015, 02:25

I Can't see it to be honest.
4-Bay NAS box running RAID 6 would give the data storage of 2 drives - half the storage you paid for.
That's the same as RAID 10 while being slower in most cases both for read & write.
Maybe if you have RAID 6 over 8 drives it can make a lot of sense, but for what the OP is talking about it seems to me that RAID 6 would be pretty shit TBH.
So summing it up, raid config depends on your config (number of disks, desired protection and speed). Certain numbers of disks do indicate raid configuration.

3 disks - Raid 5
4 disks - Raid 1 (x2) or Raid 10, Raid 5 for more capacity (trade off being speed)
5 disks - Raid 5 or 6
6 disks - Raid 1, 5, 6, 10 etc....

Once you have enough disks you can then make multiple groups instead of throwing them all into one group.

2, you have no choice, 3-5 some choice, and once you're going higher then yes, multiple parity options are available.

Gremlin

14th July 2015, 02:33

I have no real requirements but ability to add drives wouldn't be a bad thing
Mmm, saw this Steve. This is a tricky one and different NAS handle this differently. One thing to replace a drive, another entirely to dynamically add more disks. If you're using a consumer setup, it would have 2 or 4 bays for example, you can't put 3 into 2, or 5 into 4 bays.

If you have a raid 1 on 2 disks in a 4 bay, you can't add 1 disk because it's not the right number. Then you factor in whether you're striping or mirroring (or parity) and this affects the config very definitely. So you'd add 2 disks, but to get a raid 10, you have to stripe 1 and 3, 2 and 4, but the data isn't striped. You'd be better (if it lets you) to add another raid 1. Otherwise, you're removing all the data and re-configuring the raid.

If it can do it on the fly (lets say raid 1 to raid 5), then you're risking the data during the process. When messing with raid config I prefer never to have the only copy of data in it... it can easily go wrong.

If you're going to keep it longer, then you need to check if you can upgrade disks, ie, replace 1TB disks with 2TB disks. Can you increase the volume once disks have all been replaced... and so on :D

Akzle

14th July 2015, 06:10

hdd docks. Can get 3-4 slot bastards, hot swappable. Usb3. No raid faggotry.

eldog

14th July 2015, 07:26

NAS BACKUP SOFTWARE

Anyone found a decent automated backup software that can be configured and actually run at a reasonable pace without stopping 3/4 of the way through or taking forever?

Tried Norton 360 which I could configure to select my data files but would fail to completely copy some random time in the middle of the night for no apparent reason (I often wondered if this was Windows glitch or accessing 'updates')

Tried Western Digital Smartware - setup again good but took months with the computer continuously on and still didn't finish

I currently use File History but it seems a bit vague.

I have about 1 TB of files to back up, this is from a Windows 8 computer with a 2 TB RAID 10 drive setup (4 disks full disk space 2 TB)
It has run very reliably for the past 5+ years - Its a CAD number cruncher

I have multiple Western Digital My Cloud Mirror 6 TB drives which are nice and when used directly have quite a rapid file transfer (I like to think so), but its the backup software I am really concerned about. I would like to swop these on a weekly basis and use file history in between (or something)

Like my bike I just want it to be there for me and to start when I turn the key-off we go....

I expect the usual KB wealth of information and comments :cool:

Gremlin

14th July 2015, 13:40

Latte

14th July 2015, 13:54

Synology NAS

Do it.

I run multiple ones for my Company - they are lovely to work on.

Deployed these years ago back when I was in the SME space. Synology were always ahead of the curve with features and ease of use as you say. DS101's and DS107's from memory. I beleive the new ones have a *nix or bsd OS ? Able to run torrent clients / TV Managers etc.

Big Dog

14th July 2015, 15:19

Big Dog

14th July 2015, 15:21

But if your after an out of the box no mucking about solution buy exactly that.
Synology is a name I hear over and over from people with a better disposable income than me.

Sent via tapatalk.

TheDemonLord

14th July 2015, 15:30

Deployed these years ago back when I was in the SME space. Synology were always ahead of the curve with features and ease of use as you say. DS101's and DS107's from memory. I beleive the new ones have a *nix or bsd OS ? Able to run torrent clients / TV Managers etc.

Yep - all the old ones ran a customized version of *nix - I think its called Synology DSM now.

But yes - they are Ver nice

I prefer them over the Netgear ReadyNas and the Qnap QTS NAS(or whatever it is called)

Scuba_Steve

14th July 2015, 16:17

Gremlin

14th July 2015, 16:36

OK so further thought into this, I don't really need redundancy as 86.93% of what this is will be backups for the PC's (currently 2x looking to expand to 4x) so a backup of a backup in a home environment to me seems maybee overkill... JBOD could be a way to go as it'll allow me to keep the 4TB & add to it with whatever size is flavour of the day; Only question being really how flexible is expansion on it does anyone know??? I'm getting conflicting stories on Googley
Depends on the hardware. WD for example in the Live Duo range allows for Raid 1 or spanning (not Raid 0). However, this means that some data can still span disks, ie, lose a drive and that data is lost (not all in most cases).

For a cheap option, I'd consider adding a disk or two to an existing PC then backup to that disk with file sharing + permissions. Nothing flash, just extra manually controlled space (so you know which backup is stored where).

MarkH

14th July 2015, 16:54

OK so further thought into this, I don't really need redundancy as 86.93% of what this is will be backups for the PC's (currently 2x looking to expand to 4x) so a backup of a backup in a home environment to me seems maybee overkill... JBOD could be a way to go as it'll allow me to keep the 4TB & add to it with whatever size is flavour of the day; Only question being really how flexible is expansion on it does anyone know??? I'm getting conflicting stories on Googley

I've gone JBOD with a couple of DLink 2-bay NAS boxes.

DLink 2-Bay: $150 on special, nothing amazing in terms of performance but does the job OK. Uses EXT3 file system so can't just plug in an already formatted NTFS drive.
Advantage: cheap.
Disadvantages: lower performance than some more expensive NAS boxes, only 2 bays.
For expansion: Put in a drive and initialise it, copy data to it, easy. To expand storage you add a 2nd drive and initialise that, you can then copy data to either drive as desired.
Currently 4TB drives are cheap, in a few months 6TB drives (currently pricey) should be coming down in price.

If you want to spend more there are 4-Bay NAS boxes by QNAP or Synology you could look at.

So:
How much do you want (are willing) to spend?
How many drive bays do you want/need?
How fast a system are you willing to pay for?

Scuba_Steve

14th July 2015, 17:18

For a cheap option, I'd consider adding a disk or two to an existing PC then backup to that disk with file sharing + permissions. Nothing flash, just extra manually controlled space (so you know which backup is stored where).

Problem is my current PC's consist of a full server, a laptop & a All-in-one... not exactly expandable in themselves; The 4th PC to come will have space but also I'd rather have the backup external to a gaming PC (which the 4th will be)

So:
How much do you want (are willing) to spend?
How many drive bays do you want/need?
How fast a system are you willing to pay for?

As little as possible, they're not for media creation/mass file-sharing/Webserving or anything exciting
Would like 4x just for future expandability but 2x would probably suffice for awhile
Don't need speed, it's the space I'm after... Just as long as it can keep up with N wireless or faster I'm pretty happy (most stuff is done from laptop)

Latte

14th July 2015, 17:19

Of course you can go the other extreme.
Old box as a file server with no raid.
Ftp for browsing and uploading directly those files you don't want to keep locally.
Scheduled robocopy or other fancier backup from your hosts to the box and from the box to a larger backup drive and replicate to at least one other hdd.
Throw up a media streamer and stream your music, movies etc within your LAN.
Cost if you're patient? $0

Sent via tapatalk.

This is what I have now, an old E8500, onboard HD (intel 3500?) , 10TB storage. has Sonarr, Couchpotato, kodibuntu, and Deluge. Great for TV - add a show I want to watch via a web page. it downloads the entire series to date then grabs new eps as they arrive, updates xbmc library and notifies me. Missus and the kids can use it like a set top box with a standard remote. I can browse my tv and movies with my iphone. Fiddly to set up exactly how I wanted it , but pretty reliable now.

Gremlin

14th July 2015, 18:11

Wait, you're backing up over wireless? :eek:

Please don't.... use cables...

Scuba_Steve

14th July 2015, 19:00

Wait, you're backing up over wireless? :eek:

Please don't.... use cables...

yea, yea... It's working fine at the mo has been for the past couple years :sweatdrop
Wiring up the house with Giggity etherwebs now; Cat6 :wings: tho to be honest with the care (or lack there of) I'm giving it it'll prob only comply with Cat5e... but hey it's as good as 70% of "Cat6" installations out there according to 1 study & I didn't pay thousands of $$$ for it so I'm still 1 up :laugh:

Gremlin

14th July 2015, 19:08

Wiring up the house with Giggity etherwebs now; Cat6 :wings: tho to be honest with the care (or lack there of) I'm giving it it'll prob only comply with Cat5e... but hey it's as good as 70% of "Cat6" installations out there according to 1 study & I didn't pay thousands of $$$ for it so I'm still 1 up :laugh:
Cable cost difference between 5e and 6 is quite negligible now, but 5e won't automatically cap your transfer at 100mb. One thing you don't want is dodgy wiring where devices can't connect properly etc.

eldog

14th July 2015, 19:29

Do you want a fancy GUI type software, do you want specific backup software that will encapsulate all the data into one backup file (specific to that software) or do you actually want replication?

Specific backup software: ShadowProtect (paid), Macrium (think there is a free consumer version)
Replication: Rsync, cwRsync derivative, BeyondCompare etc etc.
Prefer replication over specific one backup file.
want to also sync with a vista based laptop
mostly cad.photos.video (someone elses who wants it edited and written to dvds-a number of projects which have me at a loss at the MO. Its holding up my riding)
dont mind paying as long as its reliable
Have several WD NAS drives for rotation

Gremlin i will look at your suggestions

Akzle

14th July 2015, 20:20

Akzle

14th July 2015, 20:23

Prefer replication over specific one backup file.
want to also sync with a vista based laptop
mostly cad.photos.video (someone elses who wants it edited and written to dvds-a number of projects which have me at a loss at the MO. Its holding up my riding)
dont mind paying as long as its reliable
Have several WD NAS drives for rotation

Gremlin i will look at your suggestions

rsync. Learn it.

Its free as shit and linux > winblows all day long.

Cygwin your shit, too, windows gui is almost as uteless as its (former) cli.

Scuba_Steve

14th July 2015, 20:32

Cable cost difference between 5e and 6 is quite negligible now, but 5e won't automatically cap your transfer at 100mb. One thing you don't want is dodgy wiring where devices can't connect properly etc.

Wiring is all good, just not expecting it to pass any Cat6 certification for termination; but then I wasn't planning 10Gig anytime soon, the 1Gb of Cat5e or whatever standard my wiring would meet is plenty for now & still faster/more resilient than the non-duplex N-AC wireless I have as alternative
As for cost... well only thing I've bought so far is a punchdown tool (tho might have to buy some RJ45 plugs too)

Gremlin

14th July 2015, 20:54

As for cost... well only thing I've bought so far is a punchdown tool (tho might have to buy some RJ45 plugs too)
Better you than me. I can't terminate for shit :laugh:

Scuba_Steve

14th July 2015, 21:12

Better you than me. I can't terminate for shit :laugh:

haha it's easy with practice.
I used to run a network lab, ALOT of etherwebs running/terminations got more advanced at that than anything else; fuck I even got pretty proficient at terminating solid RJ45 plugs onto stranded cable (or was it the other way round?) due to incorrect order & urgency on cables

eldog

15th July 2015, 07:19

rsync. Learn it.

Its free as shit and linux > winblows all day long.

Cygwin your shit, too, windows gui is almost as uteless as its (former) cli.

will look at rsync
I know Linux is supposed to be better than Windows (which seems to have endless amounts of crap protecting itself)
but
its another thing I have to learn, so much crap I am dealing with at the MO, I really don't need extra.

Cygwin?

cli = command line interface
windows gui is great occasionally but rarely delivers on promise

and I am over all the 'Security' updates - I try and keep the network off line as much as possible

Latte

15th July 2015, 08:33

If you want to stay on windows check out xcopy or robocopy. Not in the same league as rsync but close enough for your purposes.

Akzle

15th July 2015, 10:25

If you want to stay on windows check out xcopy or robocopy. Not in the same league as rsync but close enough for your purposes.
xcopy is a whore. Robocopy sounds third party.

Akzle

15th July 2015, 10:44

will look at rsync
I know Linux is supposed to be better than Windows (which seems to have endless amounts of crap protecting itself)
but
its another thing I have to learn, so much crap I am dealing with at the MO, I really don't need extra.

Cygwin?

cli = command line interface
windows gui is great occasionally but rarely delivers on promise

and I am over all the 'Security' updates - I try and keep the network off line as much as possible

this is all you need to learn:

rsync myshitiwanttobackup whereiwantitbackedup

most other shit happens automatically, replaces old versions of files with newer ones, doesnt waste bandwidth. Just fuken works.

Only thing it doesnt do it backup two remote locations, but theres grsync (?) for that.

Big Dog

15th July 2015, 16:54

xcopy is a whore. Robocopy sounds third party.
Robocopy is baked in for win 7 and server 08.
Can be added to any windows version later that xp or server 2003.

I use it a lot because it is recursive, can be throttled, auto scales and retries up to the level you specify.

I have used it and seen it cope with over 100,000 media files 6tb.

When my box is online (unplugged at the Mo for renovations) my ftp data files are mirrored via this as a backup which means I don't need to fuck about with matching spindles partitions etc.
Files that match my rules are backed up to a third drive daily. Weekly snapshots of that are backed up onto a fourth drive. Allowing for a lot of regression with only important files backed up.
The box is a 2000ish dell with xp pro and 2gb ram.
Provides backup and media backup for three domestic users and movie streaming.

Also operates as a print server and shared DVD burner.

Except when streaming my 15yo box trickles along at 15% CPU. Sure I could open my wallet and get something with better burst. But why?
Because I would be able to transfer a tb of data quicker? Not for domestic use.

As an interim measure I have similar scripted chips of my user data on my local box. Couple of externals and a couple of old internals with anything valuable going of site.

https://technet.microsoft.com/en-us/library/cc733145.aspx

Sent via tapatalk.

eldog

15th July 2015, 18:35

Big Dog

15th July 2015, 19:13

Ta Akzle/Big Dog - food for thought :clap::not:

Did try Robocopy - results seemed a big vague - I was trying to synch Laptop (Vista) and Windows 8 with approx. 1TB of data

I should have done a small section to make sure but I was under pressure. I was getting worried I would loose everything as previous backups had failed to finish for unknown reasons in the middle of the night.
Even worked along side it for 12 hours no problem, went home crashed approx. an hour after WTF!

I have a very small network here I want to automatically backup 5 computers and swopp a NAS / synch it with at least another while doing FileHistory at the same time.

If its raining on the weekend - I will give it a run again and look at rsync/grsync as well.

The WD NAS Mirror drive is much quicker when used as a straight drive via a network cable direct to computer.. Its a good system for novices like me.
home use would be a piece of cake.
I have all but given up on the Smartware (Too many files - 1.8M over 930 GB space, lots of file changes per day 500 per day typical, I think it struggles to keep track of it all.)

Big Dog, link is a great help - I didn't know about all the multiple commands - I remember some of the DOS ones but the CPM ones are a bit vague:rolleyes:

Save what you want to do to a text file.
Change the extension to CMD
Schedule it as a task, add any desired triggers.

E.g, When my box was on line I had a laptop (since scarificed for parts) that backed up the my documents every 15 minutes to the box. To make sure my weak ass laptop didnt bog when first connected the script ran with a bit of a delay in between writes and only 2 threads allowed that ran on home lan or home wifi connect.
I have read where others have set it to trigger on close of certain apps but I prefer to catch all at this point.

If you want you can then add a zip task to the job to compress your backup.

IF that works you add a date to the file name.

PM me your email if you want a sanitised version of a script to modify to your needs.

pete376403

15th July 2015, 19:30

xcopy is a whore. Robocopy sounds third party.
It is and I've not had much success with Robocopy even though its supposed to be good.
Teracopy works for me every time and is free. There is a paid, pro version but so far I haven't found the need (windows of course, not Linux)

Akzle

15th July 2015, 20:02

It is and I've not had much success with Robocopy even though its supposed to be good.
Teracopy works for me every time and is free. There is a paid, pro version but so far I haven't found the need (windows of course, not Linux)

linux is free all day long.

Gremlin

15th July 2015, 22:57

One thing to watch out for in replication or backups is silent corruption. You're diligently backing up everything... except some of it is useless... seen it before. It's a bitch :(

Big Dog

16th July 2015, 00:30

One thing to watch out for in replication or backups is silent corruption. You're diligently backing up everything... except some of it is useless... seen it before. It's a bitch :(
Nothing makes your anus quite as water tight as restoring an 80gb db from backup and the restore finishes with an error that the tail was not properly closed.

Depth of iteration on redundant but indépendant disks is your friend.

Raid might be faster but involves constantly manipulating the data.

If you really have to be sure you need to regularly sample your data.

Boring. But less so than trying to explain why the data is gone

Sent via tapatalk.

Skiwi

17th July 2015, 15:45

I'm very happy with my Synology DS114. I use it to serve media to various tablets, laptops, desktops and phones on the family network, back up our devices and store our photos, videos, music etc.

Downloading once and share to all has cut down our downloads to the point our data limits are no longer relevant.

Having the right software to meet your needs is key. Figure out what you want then it will become clearer.
Cheers
Skiwi

ledaero

11th September 2015, 12:54

For a home-cloud I have two - a Mac Mini with 4x 3TB WDD USB 3.0 drives in 2x mirrored pairs for 6GB of storage - it serves media.

I also have an Airport Time Capsule with 3TB that does nothing but back up our laptops, phones and tablets. But it's not NAS and you shouldn't try and use it like one.

For the more important stuff I have a wee PC running Linux and FreeNAS. The case is a 8-bay Coolermaster tower with 6x SATA interfaces and 4x IDE (one of which runs the OS). It's rock solid. It also does double-duty being the engine for torrents, endpoint for VPN, etc.

Ignoring the Mini and WD drives, the FreeNAS cost peanuts - the PC was an old Asus board with 4GB and a Q6600 (best value processor EVA!!!), the drives are 1TB 7200 RPM Seagates with decent caches (I forget the model and am away from the beast) which I bought as refurbs for under a hundred each and are running as 2x3 Raid 1.

I'm a big fan of mirrored drives - if the raid breaks, you still have a perfectly functional drive... disks are cheap - the shit on them quite often isn't.

eldog

23rd October 2015, 16:12

Save what you want to do to a text file.
Change the extension to CMD
Schedule it as a task, add any desired triggers.
IF that works you add a date to the file name.

I tried something similar to what you suggested - nothing so far has proved to be 100% reliable, I found I was having endless strange problems and although they seemed to work - there is a nagging feeling that a few things weren't right.
Today, I had errors that suggested that my filenames were longer than 256 char, as well as the backup stalling. I ran a little cmd line and saved the directory names and used Excel to determine the length of each one only took a few mins, only one was 256 all the rest were under. That was using File Explorer - there were fixes suggested by MS but none ran - wrong version.

went back and used ROBOCOPY - wasn't aware I needed to exit from File Explorer and any other programs that had files open as ROBOCOPY doesn't copy open files. Once I realised that then things improved somewhat.
Seemed to copy OK - still no definite feeling of I nailed the problem. Vague feedback.

I know its not RSYNC, or Linux based. I don't have time or energy to run around fixing these important things as my own job suffers. Its best for me to 'buy' the tech and set it up so someone else (non tech person) can activate it and do the same job if I am not present.

Downloaded Acronius True Image 2016 Trial - yeah I gave in, I just wanted a secure backup solution that gave me feedback as to IF it has backed up, and I can try to restore the computer without too much hassle as some later stage.

I will try and make a few comments on what I find in the future

Gremlin

23rd October 2015, 16:36

Acronis, StorageCraft ShadowProtect, Macrium Reflect. When backing up, if you're using two drives and rotating, I think all the software will have issues tracking backups. I prefer to have a repository of backups, then transfer those to drives that rotate.

Absolutely keep file path <256 char (this starts from drive letter). Any backup software will start having problems with files beyond that. It might back it up, but you'll struggle to restore etc.

Yes, Robocopy et al do not copy open files.

Check your backups periodically. All hard drives corrupt eventually... don't want to find out too late. Plus, the backup drive is under most strain when you're restoring the big backups... not a good time to fail, so any hint of issues and replace it... it's your last line of defence.

eldog

23rd October 2015, 17:54

When backing up, if you're using two drives and rotating, I think all the software will have issues tracking backups. I prefer to have a repository of backups, then transfer those to drives that rotate.

Absolutely keep file path <256 char.

I suppose I wanted to know if I had made the right decision, re backup method. I am the default tech person.
its been a few years since I really did anything PC tech wise apart from standard stuff and 'upgrading' from Vista to Windows 8.1 - because program requirements-so much fun that was.....
<256 I had forgotten about that but all are under.

I know its your job but can you elabourate on the repository/rotation.
At present set with 2 rotating drives and a File History drive as well.(File History is vague as well)

This is sorting out a big part of stuff i have been ignoring so I can move onto video editing which is the next project on the roundtoit list.

The NAS drive is really helpful I can set it up and it connects to multiple computers backing them up without much supervision, so will have to figure out the rotation/storage of the drives like you say.

Akzle

23rd October 2015, 18:03

I tried something similar to what you suggested - nothing so far has proved to be 100% reliable, I found I was having endless strange problems and although they seemed to work - there is a nagging feeling that a few things weren't right.
Today, I had errors that suggested that my filenames were longer than 256 char, as well as the backup stalling. I ran a little cmd line and saved the directory names and used Excel to determine the length of each one only took a few mins, only one was 256 all the rest were under. That was using File Explorer - there were fixes suggested by MS but none ran - wrong version.

went back and used ROBOCOPY - wasn't aware I needed to exit from File Explorer and any other programs that had files open as ROBOCOPY doesn't copy open files. Once I realised that then things improved somewhat.
Seemed to copy OK - still no definite feeling of I nailed the problem. Vague feedback.

I know its not RSYNC, or Linux based. I don't have time or energy to run around fixing these important things as my own job suffers. Its best for me to 'buy' the tech and set it up so someone else (non tech person) can activate it and do the same job if I am not present.

Downloaded Acronius True Image 2016 Trial - yeah I gave in, I just wanted a secure backup solution that gave me feedback as to IF it has backed up, and I can try to restore the computer without too much hassle as some later stage.

I will try and make a few comments on what I find in the future

nix. rsync. cron.

it's not 100% reliable, but it's close enough for fucken science.

super computers. atms and the computers that track jew gold shit, the large hadron collider....
but hey, who needs accuracy and reliability when you have microsoft?

Acronis, StorageCraft ShadowProtect, Macrium Reflect. When backing up, if you're using two drives and rotating, I think all the software will have issues tracking backups. I prefer to have a repository of backups, then transfer those to drives that rotate.

Absolutely keep file path <256 char (this starts from drive letter). Any backup software will start having problems with files beyond that. It might back it up, but you'll struggle to restore etc.

*255 you plebian. the joys of suffering a basically redundant (certainly retarded) file system (NTFS).

nix? 4096 all day long.

eldog

23rd October 2015, 18:10

nix. rsync. cron.

it's not 100% reliable

super computers. atms and the the large hadron collider....
but hey, who needs accuracy and reliability when you have microsoft?
*255 you plebian. the joys of suffering a basically redundant (certainly retarded) file system (NTFS). nix? 4096 all day long.

Unfortunately I am stuck with a MS based program, whose roots are firmly planted in the MS world and is a memory hog.
You may be right about the Linux stuff but I just dont have the time at the moment.
1 file was 256, 1 was 255, all the rest under 250, so I will sort it next week.

Akzle

23rd October 2015, 18:12

*255 you plebian. the joys of suffering a basically redundant (certainly retarded) file system (NTFS).

nix? 4096 all day long.

well, i'll just eat my hat right now. it's winblows horseshittery, not NTFS.

not to excuse anyone using NTFS. but apparently it supports up to 32 000 char paths.

Gremlin

23rd October 2015, 18:14

I know its your job but can you elabourate on the repository/rotation.
At present set with 2 rotating drives and a File History drive as well.(File History is vague as well)
Ideally, the core live data. Then a repository for backups. Then 2 drives you're rotating (you can certainly rotate more) but ideally not less than 2. Every week the 2 drive is rotated for each other, and while one is connected, the other is off site.

That way you have a repository for fast access of history, then other drive with possibly more history on it, and if the location goes kah-blewy (technical term) you have another off site for recovery...

Akzle

23rd October 2015, 18:15

Unfortunately I am stuck with a MS based program, whose roots are firmly planted in the MS world and is a memory hog.
You may be right about the Linux stuff but I just dont have the time at the moment.
1 file was 256, 1 was 255, all the rest under 250, so I will sort it next week.

apparently (since now you've got me interested and i'm relearning/reading/discovering this shyt) windows 8 generates it's longest path as 273, however you can sneak one out to 291 before the OS craps it's daks and cries to mum.

just because you run your crap on crap, doesn't mean you can't install nix on a quiet little box in the corner and have it save the world for you, frequently, and without fuckup nor headache. which, since you seem to be worried about time, will give you more to bang your wife. or ride your bike. whichever.

eldog

23rd October 2015, 18:57

Ideally, the core live data. Then a repository for backups. Then 2 drives you're rotating (you can certainly rotate more) but ideally not less than 2. Every week the 2 drive is rotated for each other, and while one is connected, the other is off site.

That way you have a repository for fast access of history, then other drive with possibly more history on it, and if the location goes kah-blewy (technical term) you have another off site for recovery...

Core is a raid 10 drive setup, its saved me a few times in the past. I havent bothered to upgrade the drivers for that (Original Intel ones) as I think 'if it works dont muck it around'
speed is reasonably good, its robust and doesnt drop out and rebuild itself, like other companies gear seems to do. It still keeps up with most other peoples brand new gear.

the second part above was how i envisioned it at the start.

I will have a look at the Acronis help files and google what I can about drive repsoitory etc
thanks Gremlin:2thumbsup

eldog

23rd October 2015, 19:16

now you've got me interested and i'm relearning/reading/discovering windows 8

just because you run your crap on crap, doesn't mean you can't install nix on a quiet little box in the corner and have it save the world for you, frequently, and without fuckup nor headache. which, since you seem to be worried about time, will give you more to ride your bike.

fixed yr first part

I had thought of doing something similar to what you suggested above (nix/box)in the past. I dont have the $/time/knowledge to set it at at present.

But I tried to shortcut/buy propietary gear- WD MyCloud Mirror using Smartware to back up multiple boxes - worked great on boxes with small hard disk footprints, kept track of changes and updated etc no worries. When it came to larger system I think it became overwhelmed with the amount of files. Writing this makes me wonder if I have hit a barrier (no of files in a directory/subdirectories) I am going to have to check it out

The WD NAS is great, easy to setup, runs nice and quiet (no fan-but i have a small personal one which i turn on just in case) would recommend it. The Smartware has improved from the previous versions which sucked the life out of all but the most powerful computer. For small personal use that would suit most people its great for the non technical use. Just doesnt cut it for this one task.
The WD service is good, but i am guessing its the shear size and quantity it was always on the back foot on this one task.

Erelyes

23rd October 2015, 19:59

RAID 6 is your best option

I'd have thought that Snapraid (http://www.snapraid.it/compare) is the go for the home user these days. You can start with already filled disks, add more disks later... and a bunch of other shit std RAID can't do (http://www.snapraid.it/faq#diffraid).

Gremlin

23rd October 2015, 20:10

I had thought of doing something similar to what you suggested above (nix/box)in the past. I dont have the $/time/knowledge to set it at at present.
I was running 2x WD MyBook Live Duo (earlier model to my cloud), until one fell over. Annoyingly enough, no alerts no nothing until the second drive was failing. Kinda defeated the purpose :facepalm:

Now got my old PC setup with FreeNAS, but god, I deal with tech all week, last thing I really want to do is my own tech in the weekend... got one drive failing and haven't got around to replacing for a month? shit... a month. s'ok, I've got dual parity :confused:

Perhaps tomorrow. Supposed to be raining. I know I should have used some disk tray setup. Just thinking of pulling the machine and I lose the will to live :bye:

eldog

23rd October 2015, 20:54

I was running 2x WD MyBook Live Duo (earlier model to my cloud), until one fell over. Annoyingly enough, no alerts no nothing until the second drive was failing. Kinda defeated the purpose :facepalm:

Now got my old PC setup with FreeNAS, but god, I deal with tech all week, last thing I really want to do is my own tech in the weekend... got one drive failing and haven't got around to replacing for a month? shit... a month. s'ok, I've got dual parity :confused:

Perhaps tomorrow. Supposed to be raining. I know I should have used some disk tray setup. Just thinking of pulling the machine and I lose the will to live :bye:

I know the feeling well.
My Boss supports my riding so i get a life away from work.
trouble is you know what the pitfalls are and how long it will take.
in the end its only a matter of keeping the smoke in the wires, no smoke no go.

I havent had a drive fail yet, seen plenty of others die a sudden death. Most of those were the way they were treated-hard drives are delicate, not frisbees.

sympathy about second drive going on the fritz, trouble is the user has no idea about the health unless they run chkdsk or similar often.
Used to be PCTOOLS/Norton you could run to get some idea..... the good old days

Got my practise run for the 1KC tomorrow/Sunday so will give it a think about then.

Akzle

23rd October 2015, 21:25

I know the feeling well.
My Boss supports my riding so i get a life away from work.
trouble is you know what the pitfalls are and how long it will take.
in the end its only a matter of keeping the smoke in the wires, no smoke no go.

I havent had a drive fail yet, seen plenty of others die a sudden death. Most of those were the way they were treated-hard drives are delicate, not frisbees.

sympathy about second drive going on the fritz, trouble is the user has no idea about the health unless they run chkdsk or similar often.
Used to be PCTOOLS/Norton you could run to get some idea..... the good old days

Got my practise run for the 1KC tomorrow/Sunday so will give it a think about then.

again. Nix. Smartmontools. Cron. Can be run at boot if our shit gets rebooted, or periodically if you're 100% uptime.
Hdparm to set maximum number of bad sectors tolerable.

Gremlin

24th October 2015, 01:52

I havent had a drive fail yet, seen plenty of others die a sudden death. Most of those were the way they were treated-hard drives are delicate, not frisbees.
I'm sure quality is going down hill. Tracking just under 700 drives for work, 8 have failed outside warranty, 43 have failed inside warranty... Doesn't include those retired from service (age etc). It used to be a real surprise when a drive died, now it's more ho-hum...

I bought 5 WD Red for my NAS, they've got a good reputation. Then I decided to increase the drives to 6 and bought 2 Seagate for variety, putting them in the raid and keeping 1 WD as a replacement.

1 Seagate started failing within a month. Fucken Seagate <_<

Akzle

24th October 2015, 06:02

Akzle

24th October 2015, 06:06

I had thought of doing something similar to what you suggested above (nix/box)in the past. I dont have the $/time/knowledge to set it at at present.

funny. Cos you seem to be throwing a lot of time and money at things that dont work.

But as ocean would tell ya, it's your time and money to do with what you will.

Gremlin

24th October 2015, 12:50

Had wd blues, blacks, lunched a half dozen of em. Got a 320gb that's survived but i wouldnt rank them.
Only got a handful of blues, I think every one failed. Actively avoided them.

eldog

25th October 2015, 08:44

Gremlin-Akzle

Thanks for the input.
Way back I used to use Seagate had no problems just got replaced - size/speed

I will have to check out reliabilty on my current system(s) thats why I am finally tackling backup etc - properly.

Re spending $ - I have far too much data which I have created myself to recreate any of it. Hence my thoughts about reliably resurrecting it on another computer/setup.

I dont trust MS File History or a Cloud solution except for something i can loose or as a means of easy file transfer.

I have been putting off quite a few projects over the years, now is my chance of getting rid of them. ATM Buying a solution is the easiest way of sorting this.

Big Dog

25th October 2015, 10:22

Do you perhaps have a 32 bit os? Or one updated from a 32 bit os?

The 256 character issue should be long gone with 64 bit unless you have residual 32 bit drivers or your file system was.

Once you have a good backup you should be able to rebuild your file system if you don't want to be limited.
You may need to locate a hotfix.

Sent via tapatalk.

Big Dog

25th October 2015, 10:23

pete376403

25th October 2015, 19:55

I'm sure quality is going down hill. Tracking just under 700 drives for work, 8 have failed outside warranty, 43 have failed inside warranty... Doesn't include those retired from service (age etc). It used to be a real surprise when a drive died, now it's more ho-hum...

I bought 5 WD Red for my NAS, they've got a good reputation. Then I decided to increase the drives to 6 and bought 2 Seagate for variety, putting them in the raid and keeping 1 WD as a replacement.

1 Seagate started failing within a month. Fucken Seagate <_<
WD reds are supposed to be NAS rated. I'll see if I can find where I read that NAS rated doesn't actually mean that much.
Found it at Spiceworks.com:

Scott Alan Miller Jan 21, 2013 at 6:16 AM
Niagara Technology Group (NTG) is an IT service provider.

WD Red drives are WD Green drives with TLER added, nothing more. They are aimed purely at small consumer usage, not for normal business RAID usage. The issues with parity rebuild failures due to URE (the key issues with RAID 5, but far from its only one) are not addressed by any consumer-range drive, WD Green being no exception and therefore Red not being an exception. TLER simply prevents the drives from unnecessarily dropping out of the array, not from failing to rebuild. So WD Red are perfect for low performance, low cost RAID 10 and, in smaller arrays, RAID 6 (in really large RAID 6 they are still too risky to use because of the URE issue.)

WD's enterprise drives that "could be" used in RAID 5 are the RE4 line which have the TLER of the Red drives, the URE rating of enterprise SAS drives, the spindle speed of Black drives and other advancements like bigger cache and CPU to boost performance over any other WD SATA drive.

The issue when you get a drive like the RE4 for RAID 5 is that RAID 5 remains slow (as all parity does) and risky for other reasons (drive failure rates, DAC, long rebuild times meaning loss of availability, etc.) but gets so expensive that RAID 6 and RAID 10 still beat it when actual dollars are applied. So RAID 5 is always ruled out, in every scenario, due to a large combination of factors.

So, no, Red does nothing for RAID 5.

eldog

25th October 2015, 20:04

Thanks Pete, I will now have to take the computer apart and see what is actually part of my raid 10 setup.
I have another group of smaller drives from an older system I will look into that as well.

The WD NAS MYCloud is easy to remove the cover and see what they are.

Its a bugger to know but its better to get it right before it all goes wrong i guess:niceone:

eldog

25th October 2015, 20:13

Acronis would be my choice for os backup. Mostly because it is also pretty good for data recovery.

Depends a lot on what you are backing up.
as I said I use robocopy a lot for high volume static files. Static files and most dynamic files should copy even when in use if you have your options and permissions right. To copy system files you need to be operating as a service. Note: you will not be able to restore your os directly from this backup but you will be able to recover altered files damaged during an operation like updates.

For dynamic files we use fsbackup for urgent one off backups of file trees or small volumes. This also has the advantage of breaking backups into files easily transferred with explorer or ftp over slow links.

Storage craft for managed system or volume backups.

Sent via tapatalk.

Acronis has a trial use period which i am making use of to see if i am happy to trust it.
I started it Friday arvo may go into tomorrow and check

The WD Smartware has been on for MONTHS and still has 50 GB to backup.

However it did work on much smaller systems with no problems at all.

Lots of the files would be static, when I am working a complete new design there is approx 250-500 files being modified at once (most are in the background and are referenced) I left the software take care of whats required to be updated
I am often surprised at the effects of opening 1 file has only lots of others - I am talking CAD files mostly

I got hauled over the coals recently about Internet useage - mbike stuff and a fair amount of work stuff, however it wasnt me it was someone else watching the RWC, while I was away. So I tend to keep away from Cloud based products.

eldog

26th October 2015, 13:46

Update: Acronius reports 777 GB backed up which sounds OK took approx. 6 hrs which seems legit.
I am looking to see if I can get it to output a summary report - at present I cant find one. Just so I know it hasn't had any problems and just stepped over them.
6 hours is rather good, much better than I expected for a full backup

yes it looks like a nice interface, a summary report after it finished would be nice too. still looking for it......

Then I will make a proper system restore, I remember that I can make one and then use it to test it... at last the NAS solution seems to be on the right track again.

Akzle

26th October 2015, 14:25

wow thats slow.

eldog

26th October 2015, 14:37

wow thats slow.

much better than the 12 months the other way I was using.

It can take all night, as long as I know it works and I have a backup that is OK WHEN I want to use it.
If it gives me a good report on what problems it had (if any) and what it managed to get backed up then at the moment I will be happy.

If this doesn't work then I may contact one of you EXPERTS for PAID help.

Last thing I need is a backup that wont be there when I need it.....

my bike is like that, slow and reliable - as long as I don't drain the battery..... again.

checked and found the backup setting is set to slow, there are slow, med and fast settings and user can set a speed limit as well
we default at slow, which is OK as I wont be using it during this time and if I am it wont drain the system while I am using it either.

still cant find a report type of output, set up for emailed output if there is a problem, I would rather it saved an text report to disk and I can look at it later, maybe a summary on the screen as well.
I am going to read the manual first before I toss my toys out of the cot.....

then I will contact them

Akzle

26th October 2015, 15:23

rsync --log-file=$.log

or ...2>&1>>$.log

Akzle

26th October 2015, 15:28

would be interesting to see both cpu and bandwidth usage, ergo time, if set to fast.

Paid software is not something i have much time for, so :scratch:

eldog

26th October 2015, 15:31

would be interesting to see both cpu and bandwidth usage, ergo time, if set to fast.

Paid software is not something i have much time for, so :scratch:

OK I will set it up for another full backup and see what happens... let you know

update 1: so far restriction is the original drive setup lagging way behind in drive access time....
update 2: Acroius True Image 2016 - is for the home user, nice simple interface etc.
update 3: Acronius Backup 11.5 is for the small business user who wants to have more control
Acronius rang me direct and answered my questions so now I am changing over to BAckup 11.5 and see whats up.

Thanks Akzle for getting me to look at the whole process (seems to me the major slow down is due to access time on the main system)

I suspected as much but wanted to backup everything at least 2 copies then I can proceed and update the Intel Raid Matrix drivers (they are called something else now, and I suppose there will be a W10 update out soon) for the RAID 10 system.

I have held off W10 till all the dust settles - recommended by the software company I use 90% of the time.

Big Dog

26th October 2015, 23:46

OK I will set it up for another full backup and see what happens... let you know

update 1: so far restriction is the original drive setup lagging way behind in drive access time....
update 2: Acroius True Image 2016 - is for the home user, nice simple interface etc.
update 3: Acronius Backup 11.5 is for the small business user who wants to have more control
Acronius rang me direct and answered my questions so now I am changing over to BAckup 11.5 and see whats up.

Thanks Akzle for getting me to look at the whole process (seems to me the major slow down is due to access time on the main system)

I suspected as much but wanted to backup everything at least 2 copies then I can proceed and update the Intel Raid Matrix drivers (they are called something else now, and I suppose there will be a W10 update out soon) for the RAID 10 system.

I have held off W10 till all the dust settles - recommended by the software company I use 90% of the time.

The only way to know the value of data is to lose it...

You should be able to load the 11.5 backups directly from VMwares viewer.
It will lag a little at first but give it ten minutes you'll know if your backup is worth doing bare metal restore testing on.

Well worth the 30 minutes to find out your overnight rebuild is not worth a try.

Sent via tapatalk.

eldog

27th October 2015, 06:24

The only way to know the value of data is to lose it...

Well worth the 30 minutes to find out your overnight rebuild is not worth a try.

Obsofreakinglutely....

Lost a few days work once through accidental mouse click. Luckily I had print outs of what info I needed.
Took me 12 hours to recover it by reentering all the info and creating the complete CAD model from scratch.
still met the deadline.

Akzle

27th October 2015, 09:18

OK I will set it up for another full backup and see what happens... let you know

update 1: so far restriction is the original drive setup lagging way behind in drive access time....
update 2: Acroius True Image 2016 - is for the home user, nice simple interface etc.
update 3: Acronius Backup 11.5 is for the small business user who wants to have more control
Acronius rang me direct and answered my questions so now I am changing over to BAckup 11.5 and see whats up.

Thanks Akzle for getting me to look at the whole process (seems to me the major slow down is due to access time on the main system)

I suspected as much but wanted to backup everything at least 2 copies then I can proceed and update the Intel Raid Matrix drivers (they are called something else now, and I suppose there will be a W10 update out soon) for the RAID 10 system.

I have held off W10 till all the dust settles - recommended by the software company I use 90% of the time.

using sas drives or 7200s? Either way i doubt it's a seek/read bottleneck.

On nix i'd renice the process (set higher priority) windows *may* have a way to.... But windows itself is a resource whore, so if you could boot into a terminal (dont thinkso any more) that would speed it up.

Alterntively, liveboot nix and win all over the place... Or if acronis has a bootable kernel...?

Big Dog

27th October 2015, 10:42

using sas drives or 7200s? Either way i doubt it's a seek/read bottleneck.

On nix i'd renice the process (set higher priority) windows *may* have a way to.... But windows itself is a resource whore, so if you could boot into a terminal (dont thinkso any more) that would speed it up.

Alterntively, liveboot nix and win all over the place... Or if acronis has a bootable kernel...?
Depends on the server purpose and version.
Core is awesome but puts a lot of handicaps on how a traditional user with administration rights does stuff.
Later server versions can be switched from PowerShell but it does require a restart. Word is next version is hot switchable.

But if no one ever logs on to the box core is the way to go.
Half my sql servers are core. The other half have full gui in case someone else needs to do stuff on an emergency.

Sent via tapatalk.

Akzle

27th October 2015, 11:27

But if no one ever logs on to the box core is the way to go.
Half my sql servers are core. The other half have full gui in case someone else needs to do stuff on an emergency.

Sent via tapatalk.

you need better someone else.

Nigga cant cli, aint worth shit.

Akzle

27th October 2015, 11:30

Depends on the server purpose and version.
Core is awesome but puts a lot of handicaps on how a traditional user with administration rights does stuff.

sysadmin or gtfo

Akzle

27th October 2015, 11:33

https://msdn.microsoft.com/en-us/library/dd184075.aspx

lol. Windows. Taking unitl 2k8 to do what linux has done since forever.

Gremlin

27th October 2015, 11:43

you need better someone else.

Nigga cant cli, aint worth shit.
Some of us wear too many hats to count. Big Dog does SQL, He'd do more in a day than I'd likely do in a year. Then again, I have whole networks to manage, firewalls to program, end user machines to deploy, email servers, websites and backups to watch. Oh, and troubleshoot all the above when it wants a holiday.

Powershell? Copy and paste my google learning more like it, as I probably won't use those commands in the next month...

Big Dog

27th October 2015, 11:55

you need better someone else.

Nigga cant cli, aint worth shit.
Pretty basic to connect from a command box to do anything that really does need a gui. But bugger expecting someone who only touches sql for backups to work that sit out so they can do a restore after a San failure.
On the other hand, if the San guy can get in the box and check the jobs are all working as expected after a system restore that means I don't have to get out of bed unless there are problems.

2015 and Linux still without a functional gui?

Sent via tapatalk.

Big Dog

27th October 2015, 11:59

sysadmin or gtfo
Organisations that divide the work up into teams big enough to cover leave I agree.

Here everyone has to wear many hats. We all have our specialties but if I am not here there are 5 others who can perform basic functions. Only difficult stuff waits for me.

Sent via tapatalk.

Big Dog

27th October 2015, 12:03

you need better someone else.

Nigga cant cli, aint worth shit.
Dude, I work with some pretty talented guys who all know their shit front to back and side to side but I can't expect then all to know my side. If they did I wouldn't be necessary.

sysadmin or gtfo

Sent via tapatalk.

Big Dog

27th October 2015, 12:05

I would normally rate myself a pretty smart guy... rarely would I consider someone my equal. In the team I work in I would rate most of them as brighter and do my best to learn from them.

Sent via tapatalk.

Akzle

27th October 2015, 12:36

2015 and Linux still without a functional gui?

Sent via tapatalk.

openbox ftw.

But if you want pretty (pretty gay) you have a choice of 4 for most distros (ugly ass gnome3, kde, xfce and lxde are much compatible) then theres ~ another 3 or 4 each distro.
asides from that, customising them to do exactly what you want, and nothing else, is a piece of ease compared to winblows' blow.

and then many many distros that were written around a particular gui. Macnix and the like.

If you cant find/make a gui work for you. Your not l33t enough. (possibly retarded, actually) :bleh:

Big Dog

27th October 2015, 13:52

The point of the gui isn't too make it work for the l33t so much as to work for the many. So that kind of misses the point of having a simplified interface.

Sent via tapatalk.

Akzle

27th October 2015, 14:51

The point of the gui isn't too make it work for the l33t so much as to work for the many. So that kind of misses the point of having a simplified interface.

Sent via tapatalk.

openbox ftw.

But if you want pretty (pretty gay) you have a choice of 4 for most distros (ugly ass gnome3, kde, xfce and lxde are much compatible) then theres ~ another 3 or 4 each distro.

. .

eldog

27th October 2015, 16:54

I don't do SQL, its not a server I am using just a older Vista upgraded box 8GB RAM Twin core Intel processor with 1.1 TB Hard drive made up of 2x hard drives in raid 10 config
Its an oldie but a solid worker, been very reliable apart from the usual Windows SNAFU

It wasn't tweaked much and I prefer it that way, simple solid and reliable.
Yes I like my outputs to be verbose so I know whats happening

I am not the usual pleb, I like to have a nice GUI so its simple to work out whats happening. I know it takes approx. 1/2 of whatever programming time is allowed to get that right, the other half is sorting out what the program does.

If there is a problem I can usually nut it out, just need the odd guiding hand to set me on the correct course, just like my riding.
Sometimes I need the big stick approach to make me actually tackle stuff and finish it on my own.....

just as I am doing currently with my riding, on my own, by myself, working out stuff and finally I will get pretty good at it, then I will move onto something else on the bucket list.

Acronius Backup 11.5 is what I am using, not bad so far.

Big Dog

27th October 2015, 17:27

I don't do SQL, its not a server I am using just a older Vista upgraded box 8GB RAM Twin core Intel processor with 1.1 TB Hard drive made up of 2x hard drives in raid 10 config
Its an oldie but a solid worker, been very reliable apart from the usual Windows SNAFU

Explains everything.

Not a server = needs a gui to do what you want.
older Vista = Robocopy won't do live copys of open files > Probably 32 bit which explains the file name length issue.
Twin core Intel processor = Mix that with Vista and that will be why so slow with the backup. Multiply your resulting processing time by a factor of 4 if you are using 32bit os to address greater than 512gb partitions (or 256gb if you don't have the large file name support patch).

IF it is fast enough for your needs no need to update but the file structures you describe really will perform better on 64bit.

eldog

27th October 2015, 17:56

Explains everything.

Not a server = needs a gui to do what you want.
older Vista = Robocopy won't do live copys of open files > Probably 32 bit which explains the file name length issue.
Twin core Intel processor = Mix that with Vista and that will be why so slow with the backup. Multiply your resulting processing time by a factor of 4 if you are using 32bit os to address greater than 512gb partitions (or 256gb if you don't have the large file name support patch).

IF it is fast enough for your needs no need to update but the file structures you describe really will perform better on 64bit.

Sorry Big Dog I put you a bit crook with that hasty reply.
Vista 64 bit upgraded to W8.1 64 bit
I suspect that there are quite a few remaining 32 bit drivers for all sorts of bits and pieces on the machine (1 is the original Intel Matrix Raid driver which has been upgraded and name changed to distance itself from all the problems of earlier drivers, which is why I am hesitant to 'upgrade' it without a very solid backup to rely on and have time to re install it if the worst happens)

The software I use is 64 bit but doesnt really use its capability much, if at all.
Only now is it trying to multithread itself but i think its just split the 2 parts of the drawing creation when it can.
Addressing 1.1 TB no problem, in fact I would have never considered the name length a problem until last week I was trying to backup and one of the suggested problems was the name length.

The age of the machine most likely puts it smack in the middle/end of the transistion between 32 to 64 with promises of 64 performance not really happening due to residual 32 drivers used else where

I liked Vista, never really had a problem...
W8.1 was a nightmare to install, didnt research it enough, didnt know about disappearing product keys blah, blah, blah but the CAD software upgrade meant I was in for Win7 or 8 either way and we didnt have the $ for a new system

Akzle

27th October 2015, 17:58

Multiply your resulting processing time by a factor of 4 if you are using 32bit os to address greater than 512gb partitions (or 256gb if you don't have the large file name support patch).

is that a fact?
i had an idea that is was to do with sector size.... (another thing windows is shit at) ie, 512 sectors... whereas good shit rocks out at 4096.
32bit limits you to individual file size of 2gb (?) the limit of fat partitions is 2T, i thought...

never mind that LFS has been implemented in winblows since 98...

all said. Vista is no use. for anything. burn it.

Gremlin

27th October 2015, 19:15

Ah... troubleshooting backup problems. Basically, unless you know otherwise you're pretty much pulling apart the entire network in your head and EVERYTHING has to be checked.

Now, obviously this can depend on whether there is something obvious that can be a bottleneck, otherwise the most obscure problems could be occuring. First up, the machine the backup is targetting, the network the data is transversing and the destination of the backup. You mention having WD NAS drives. I had the opportunity to directly test it against my FreeNAS a couple of days ago. Physically, they're both located next to each other, and plugged into an HP switch (my core switch). I copied an 8GB iso off my desktop onto both. WD could sustain a write around 32-33MB/s. FreeNAS sustained write was 100MB/s.

Bear in mind that small files vs large files will also have an impact on performance - and for details you're getting into the breakdown of hard drive config. Are you running multiple backups to the destination at once? Even a scheduling problem where one backup takes longer than it should then continues when the next backup starts... watch the disk queue length go through the roof...

Then you've got incompatibility of drivers/hardware with other hardware, resource allocation (jobs consuming too many resources or not enough) etc etc. Basically, unless you can easily locate the fault then you have to go through the entire chain with a fine tooth comb.

Just some random thoughts... :sunny:

Akzle

27th October 2015, 19:35

I copied an 8GB iso off my desktop onto both. WD could sustain a write around 32-33MB/s. FreeNAS sustained write was 100MB/s.
and freenas is........

also, and you do mention this later. .iso are contiguous files.

Basically, unless you can easily locate the fault then you have to go through the entire chain with a fine tooth comb.

or. nix.

Gremlin

27th October 2015, 19:51

and freenas is........
Well duh. However, your blanket idea of 'nixing everything is stupid. You're not actually solving anything (yes, I'm getting annoyed at the pointless contribution). If I kept telling my clients to replace everything they had with something else I wouldn't have a job. You have to assess needs and requirements then recommend something appropriate. For the record, we tried a flavour of Linux on a client (reasonably tech savvy) and practically, it just didn't work. Too much work getting addons for flash/java etc working and RDP to the local server didn't perform that well. Our remote access onto it didn't work either, as it's all Windows based.

Oooh aaah, we're trapped in it... well, that's how a business network works. As soon as I need to synchronise calendars, contacts, sync mobile devices into the network, manage them, share mailboxes and so forth... Linux isn't even on the page...

Further, I'd never be suggesting FreeNAS to any regular home user. There are a lot of pitfalls, it's definitely not plug and play (well if you go for the commercial side then it sort of is) and there are a lot of routes that have an end game of data gone.

Big Dog

27th October 2015, 22:20

is that a fact?
i had an idea that is was to do with sector size.... (another thing windows is shit at) ie, 512 sectors... whereas good shit rocks out at 4096.
32bit limits you to individual file size of 2gb (?) the limit of fat partitions is 2T, i thought...

never mind that LFS has been implemented in winblows since 98...

all said. Vista is no use. for anything. burn it.
More an experience tells me thing.

I have gone from a dual core 32 bit vista install to a 64. Found it better but not good enough. Went to a win764 followed by the same on a quad core.
Backing up from the same disk set to the same disk.
A few clean rebuilds in there as well.
Post clean rebuild vista 64 used both cores for processing. PRE it only used one. I don't know why, but it does convince me upgrades are not as good as a clean install.
Had developer windows license and holy well used it to find out for myself what works instead of blindly following a mantra provided by someone else. Another factor many people overlook... if your free ram is bigger than your largest contiguous file backups are faster and your throughput more consistent.

Sent via tapatalk.

Big Dog

27th October 2015, 23:09

Big Dog

27th October 2015, 23:18

Not exactly enterprise solution but similar to what happens at work... without the enterprise software.

It is also what i have done a few times while waiting on budget or licensing for said enterprise software on new servers.

So back up your entire system with acronis stull but a lovely get out of jail if the last backup was a lot of work ago.

Sent via tapatalk.

eldog

28th October 2015, 07:28

PS what .Net version are you running?
4/4.5?
If so consider attaching a USB drive and running a robocopy script in /mon mode on the folder your work in progress resides in. In the event of a catastrophic failure restore your acronis backup and copy in your current files. Then you only lose unsaved work and maybe work saved less your monitor internal.

Sent via tapatalk.

version 4
Gremlin was right transfer speed from Acronius data is 25-35 MB/s, tended to hang in the 32 for a greater part
Being a consumer product WDMyCloud I didn't expect blazing speed, just plain old mom and pop slow and reliable

Computer hard disk at 100% most of the time, as I was using it as well and keeping an eye on things, hence my earlier comment about computer rather than NAS been a bottleneck
I would also think a bare build would create a more stable/faster system.
A bit like building a house if you have the whole concept in mind from the start - all fleshed out the result is often better than tacking bits on as you go.
After the first build you often figure out what you missed and incorporate it on the next new one.

left it running 7-1/2 hrs finished total

Big Dog I will look into your suggestion and see what happens

just so you know, I am the token IT guy and dogs body/CAD user at work things take time

Big Dog

28th October 2015, 09:54

version 4
Gremlin was right transfer speed from Acronius data is 25-35 MB/s, tended to hang in the 32 for a greater part
Being a consumer product WDMyCloud I didn't expect blazing speed, just plain old mom and pop slow and reliable

Computer hard disk at 100% most of the time, as I was using it as well and keeping an eye on things, hence my earlier comment about computer rather than NAS been a bottleneck
I would also think a bare build would create a more stable/faster system.
A bit like building a house if you have the whole concept in mind from the start - all fleshed out the result is often better than tacking bits on as you go.
After the first build you often figure out what you missed and incorporate it on the next new one.

left it running 7-1/2 hrs finished total

Big Dog I will look into your suggestion and see what happens

just so you know, I am the token IT guy and dogs body/CAD user at work things take time
Never used cad myself but I have built a few systems for it.
They can be cranky. Demanding specific screen res etc. Not to mention requiring Windows.

Sent via tapatalk.

Akzle

28th October 2015, 10:04

Never used cad myself but I have built a few systems for it.
They can be cranky. Demanding specific screen res etc. Not to mention requiring Windows.

Sent via tapatalk.

except the nix breeds of :laugh:

native resolution dictated by the screen should work.
if you're serving it to varying spec thin type clients then shit gets heavy.

eldog

28th October 2015, 11:04

Never used cad myself but I have built a few systems for it.
They can be cranky. Demanding specific screen res etc. Not to mention requiring Windows.

Windows not through choice but by how initially these personal programs were developed for the masses.

I have seen in my distant past some really good
HP - electrical schematic
IBM - mechanical design software

but these were on what was then the top end of computing power......

its a pity Apple didn't go for the CAD market more for the graphics end IMO