RAID NAS backup or RAID NAS server? Or both?

Recent events have illustrated to us that recent growth needs to accommodate our computer ( IT ) resources as well.
( Tony's discussion about backing up your computer )

Originally, my thinking was to purchase a newer NAS and install a SSD for better performance. Then, I considered two SSDs, in the event that one should fail. This inevitably leads to a consideration of RAID level strategies, as well. During this, it also occurred to me that we now have several computers and I am often transferring data from one to another, or placing it in a special folder on the NAS for the purpose of doing so.


Now, key to remember is that we are still very small. I'm not talking about thousands of dollars, nor terabytes of data. Frankly, it looks like a few hundred dollars would do the trick. So, now we are looking at considering using a network server for most of the data .... which then brings up the question/possibility of using a Four bay NAS. Three drives for RAID enabled backups, and One drive for network serving. ( serving all CAD/CAM data storage, common program data, and a mail store for our mail client to pull from so that all computers have the same mails available )

So the questions are - Can they be used that way? ( 3x1 in the manner I've described ) And can anyone recommend some choices based upon actual experience? Maybe we should be looking at two separate solutions? ( one server/internal cloud and one RAID backup box ) I am not interested in theories, suppositions, conjecture, or stories garnered from a cousin's friend's barber's kid in school for IT. I'm interested in what others in our situation have done, experienced, and found to work well.

Oh, and I'm not interested in building anything, short of placing drives in box. Too much going on to be bothered right now.

And, if any of this is problematic for any reason, why.

Thanks.

I also have been thinking about and putting off a decision like this. I presently have all my data on a separate drive which is attached to my main computer with a USB cable. This way I can easily move it to another machine whenever I need to. I can also grab it quickly if the house is afire. Not exactly NOS, but I can access that drive over my network from other computers when I need to move a file.

But there is no back-up in this. I have been backing up to other computers in the house's network on an irregular basis. My wife doesn't even know I have a backup on her laptop - and she does not care as her primary usage for it is to get on Facebook.

Anyway, why would you need two different systems. I have been thinking about getting a single cloud device that can hold several drives, probably starting with 1 or 1.5 TB drives but I would want it to be upgrade-able with no sweat. My thought is that each of the drives would hold all of the data and they would be hot swappable. If a new drive or an out of date one is inserted, it should automatically be upgraded to the latest set of files. I would keep one drive in a bank vault and at least two in the box at all times. Or perhaps two in the box, one on a shelf here and a fourth one in the bank vault. They would be rotated on a regular basis. The bank vault one is in case of a house fire.

One device with multiple drives would provide both NAS and backups. And if carefully chosen it would be upgradeable to larger drives.

Is my thinking defective? If so, how?

I don't know what this would cost, but I think it is going to be my new year's resolution to get it in service.

Zahnrad Kopf said:

Can they be used that way? ( 3x1 in the manner I've described )

Click to expand...

Depends on the make/model. The more "user friendly" boxes autocreate one volume, end of story. Others will let you create a couple of volumes, and there should be no problem making one a 3-drive RAID set, and the other a single drive. Usually this is setup through a built-in Web UI over a local Ethernet connection; at least some CineRaid models have a tiny bank of DIP switches on the back to set RAID mode. Only practical recourse is select some likely models and then read their manuals to see what's possible.

Regardless of what NAS you have, periodically copy everything off to simple disk or SDD, and store that somewhere safe. Have several of these. Rotate them.

The point of this is that if you get hit with a virus that encrypts all of your files, or some physical disaster, having a disk stashed somewhere with a copy of everything important as of say 1 month ago isn't wonderful, but it's vastly better than not having that backup.

As for NAS - all of the ones I fiddled with had (at the time) proprietary formats, so if the NAS itself died (happened to me) you couldn't just read the data with your PC - because couldn't mount the volume.

So whatever you get, think through how you get the data back if the server, nas, workstation, laptop, dies. As in, you go buy another laptop, how do you copy all of your data onto it with the NAS got fried in the lightening storm with everything else? In my experience this is always the hardest thing.

Synology NAS would probably work well for you. Mine works great. Available in many sizes (number of drives).

Also supports cloud sync -- so you can store locally to the NAS and backup to any number of cloud storage providers. There are many built-in apps that do various things like sync / backup / streaming / surveillance camera recording / etc...


However, that said - I actually do it the other way around...

I have Dropbox installed on all my Macs / PCs. Then I use the Synology cloud sync to store backup of my Dropbox on-premises... I think performance is a little better this way as the first "hit" to the synced files are to a machine-local hard drive (or SSD). Then the stuff gets sync'd out to Dropbox as time permits, then back down to the Synology NAS, as well as the rest of the Macs / PCs on the account. This also has the added benefit of off-premises file access over the internet without needing to use a VPN to get back to the NAS. You can also use multiple accounts and share different folders with different folks - so you can very easily control access to files without having to get into the NAS admin interface.

Zahnrad Kopf said:

R and found to work well.

Click to expand...

We ran a Synology NAS for a couple of years ... my experience was, it was too limiting. Picked up an HP Microserver version 7 (8 is cuter but 7 is a little sturdier) and put Solaris 11.1 on it.

Most of the NAS'es these days run zfs but they won't let you do much. At least running Slow Larry there are actual tools for file management, file recovery, dealing with the file system, etc etc. And you can do ftp and other things on a little server, which you can't do on a NAS. Say for example, you have a humongous video that you want to let a customer see, how are you going to do it with a NAS ? Upload to a dropbox ? Eeeuw. Or if you want to run some kind of customer management or accounting program or even an inventory program, you can install php on a server and a database, then you can access it from any computer in the office or home, can't do that from a NAS. A server gives you a lot more capability for not much more work.

As of 11.1 SLarry zfs will speak natively to Windows and to NFS, so you can share with just about anything. 11.2 has a lot of cloud stuff in it - if that's what you want it's better but for me, no way Jose.

Otherwise, I'd run FreeNAS or similar. The commercial NAS'es are grossly overpriced. You are paying a lot so that the pretty box can look good sitting under the teevee set. Everyone is using zfs and the same hardware now so there's not really any advantage to a commercial offering.

The HP Microserver has been very good. Hate to recommend HP for anything these days because they are such shitheads but this particular box has worked well with no problems at all.

I have worked in IT for a number of years at a public University and I think you would be covered well if you had a server with a hardware RAID array in it and backing up to a NAS or other server. You mentioned not being into thousands of dollars so that probably puts this idea as being too expensive. I suggest you look at a Drobo unit (Drobo Data Storage Solutions, Network Attached Storage, Drobo Inc.). I have been running two 5N units for a number of years and they have worked well and are simple. Your Windows, Mac's and Linux machines can mount the network share and all can read/write data. You could then have a second hard drive in one of your computers and then nightly copy all the data from the NAS to computers hard drive as a backup. This would be a cost effective solution. Additionally you could also have a simple external USB drive that you plug in once a week and make another backup then unplug it to prevent any network virus etc.. I am a big fan of having multiple backups.

On the backup point, I am not backing up a server, but my solution is to do full images every few weeks, and to backup the data to a removable medium daily.

For a server, backing up to off site medium regularly is pretty essential. Having multiples rotating is also best.

It is only a matter of time until the ransomware starts having a delay after infection of up to several weeks so that it is certain to be on any backups. You need some strategy that will handle even that scenario.

At this moment, the plan is thus -

Laptop has two drives. 512Gb ( operating ) and 1Tb ( storage ). As soon as I am satisfied with everything I will pull the optical 1Tb and replace it with a 512TGb SSD that was in the old laptop. Then, daily back ups of the operating drive will be done so that if we lose a main drive again it will be an easier recovery.

Second, install a two or three bay NAS with Gigabit network ability and using 256Gb - 512Gb SSDs for serving the aforementioned files. ( while we actually don't need that much room, the lack of price differences make it the wise choice ) One drive will be for backing up the NAS itself.

Evaluating and researching software now.

You might be on it but let me ask a few questions.

Number of files?
Size of files?
Number of computers with access to the files?
Internet connection reliability, quality and bandwidth?

(Throwing an old drive in the midst of your new back up system might be false economy)

I have used RAID before and have fixed bugs on RAID systems in mini-computer installations. Very complicated. In linux there are options for using RAID, from mirroring to various levels of striping. I would not use anything more complicated than RAID level 1 mirroring. I don't use SELinux either. Keep you life simple.

henrya said:

You might be on it but let me ask a few questions.
Number of files?
Size of files?
Number of computers with access to the files?
Internet connection reliability, quality and bandwidth?
(Throwing an old drive in the midst of your new back up system might be false economy)

Click to expand...

Sure thing, Henrya. Happy to answer.
File count? I have not counted, but can say with some confidence that there are at least 500 CAD models. Additional to those would be all the CAM programs, notes, and other associated files that typically get created with a project. Estimates, Quotes, Time and Materials tracking, related tooling purchase notes, etc... This could easily be a few thousand files. But the are not necessarily large.

Number of computers? I just reduced it to 3, today.

Internet? Irrelevant to my task. All transfer is internal. And we will not store anything externally. This is partially my own dislike for the lack of actual ultimate security of internet based clouds, and partially a restriction of about 50% of the NDA's that we've agreed to. Files cannot leave our possession, nor the premises. ( this is one of the reasons we cannot use Fusion360 )

Drive - "Old" here is relative. The actual drive itself was in the "old" machine, but is less than a year old**. So, of no concern. **- I had upgraded the old 5400rpm optical drive in the laptop to a modern SSD.

rons said:

I have used RAID before and have fixed bugs on RAID systems in mini-computer installations. Very complicated. In linux there are options for using RAID, from mirroring to various levels of striping. I would not use anything more complicated than RAID level 1 mirroring. I don't use SELinux either. Keep you life simple.

Click to expand...

While it has been a while, I am very familiar ( and at home ) in 'nix's. Even so, as easy as that would be, re-read my earlier notes - I'm not willing to muck about with this. To the point, we agree. The simpler, the better, right now. I just plain don't have time or inclination for mucking about with it. I want to purchase a box/appliance/what-have-you, install it, power it up, enter some basic info, and... Walk. The. Eff. Away.

Hence, the leaning toward purchasing an appliance style NAS.

Thanks.

Sounds good.

You still need a backup to the appliance style NAS. It can fail and if a direct replacement is no longer available you will be SOL. I am still recovering from failure of a drive enclosure that supported a simple RAID 1 array. The enclosure had an interface issue to one of the drives and corrupted that RAID slice. Since all data on the other slice was still good. I could move that disk to any another enclosure add another disk in a second enclosure and rebuild the RAID. This left me only having to sort out if the problem was with the second disk or the enclosure. The enclosure supported a hardware RAID 1 which I had not used because if the same problem had happened I would have been scrambling to get a second exactly the same unit shipped in to rebuild the RAID.

You should have an offsite backup. A bank safety deposit box is a good place for it to live. You should rotate a couple of disks and rsync them to the array periodically. You would then swap the one on your shelf for the one in the lock box with a periodicity determined by your tolerance for loss of data.

I went through this last year or so. Ended up with a Qnap ts453 pro 8gb 4 bay with 2 hgst 6tb NAS drives in raid 1. The desktops back up daily automatically with Macrium Reflect to the NAS, which also serves the files that are not on the PCs. Also works as a web server if I want, and I can stream full HD video stored on it to anywhere over the web. Every other week I back up the NAS to a 3rd drive kept off site and off line. I'll soon add a 4th external backup drive so can alternate instead of overwriting the external backup each time. I also have a copy of all data files on another drive that only gets updated once in a while, from each PC directly. So far, so good, no cloud but the Qnap system allows for that if I ever want it. You'll need a UPS for the NAS as they will NOT be happy in the event of a power failure. I got a Tripplite Smartonline ON LINE type, which essentially is running on a charging battery pack at all times and makes its own AC so there is no chance of a sag, overvoltage or interruption, and uses an Ethernet SMTP trap so can shut down all the other equipment on the network before it runs out of battery. I'm very happy with qnap as they do very frequent firmware and app updates & support& their forum is great. Tripplite is fantastic about support as well, I think APC got bought by Schneider electric and I found it pretty hard to get answers from them in that big conglomerate. I just had to buy a better (and fan cooled) external drive enclosure because the external backup drive was getting so hot that I couldn't believe it, this one is working great:
https://smile.amazon.com/gp/product/B00N1Q2M6A/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1
I had just a thing that you plug the bare drive into, but besides the drive running hot the connectors on the drive are not rated for that kind of repeated use vs a USB 3 cable.

Zahnrad Kopf said:

Sure thing, Henrya. Happy to answer.

File count? I have not counted, but can say with some confidence that there are at least 500 CAD models. Additional to those would be all the CAM programs, notes, and other associated files that typically get created with a project. Estimates, Quotes, Time and Materials tracking, related tooling purchase notes, etc... This could easily be a few thousand files. But the are not necessarily large.

Click to expand...

Holy Fat Fumbles of Fabulous File Size, Batman!!!

Was doing some organizing while waiting for the new machine to arrive, and thought to find out about this beyond guessing. Bear in mind that these are all the work files. Not just CAD/CAM. But still...

< drum roll please >

688,116 Files. 562 Gigabytes in size.

We are a SMALL shop. Hard to wrap my head around the fact that there is actually this amount.

So maybe a little more volume than you thought. Can you sort some out that are not active files and save them in an archive of their own? Maybe break them out by client? By file type?

Just saving without sorting and indexing what you have is easiest right now, probably not long term though.

Many moons ago I worked in the publishing business, at the beginning of the computer era. It was a huge deal keeping everything together, secure and organized. The only way to make it efficient (or to happen at all) was to be super organized on the front end. If the job was supposed to run on Thursday but you didn’t have all the parts present and in order it would not happen. And they still wanted the job on time. You were ¥^£]#@* if you were unprepared.

If you want to figure out where those large files are and potentially cleanup/archive unused stuff, there is a very helpful free program called windirstat that will graphically break down the location/type of large files. It is much faster than checking through individual folders and will quickly help you identify if, for example, someone backed up their 400GB music collection to company servers.

carbonbl said:

If you want to figure out where those large files are and potentially cleanup/archive unused stuff, there is a very helpful free program called windirstat that will graphically break down the location/type of large files. It is much faster than checking through individual folders and will quickly help you identify if, for example, someone backed up their 400GB music collection to company servers.

Click to expand...

I appreciate the thought, but I know EXACTLY where and what they are. As I wrote before, these are our working CAD/CAM files. The larger files will be the CAD solid models and CAM files, and the smaller files will be the things like documents and spreadsheets along with some PDFs. Nothing mysterious about any of it because I created all of it.

Zahnrad Kopf said:

Holy Fat Fumbles of Fabulous File Size, Batman!!!

Was doing some organizing while waiting for the new machine to arrive, and thought to find out about this beyond guessing. Bear in mind that these are all the work files. Not just CAD/CAM. But still...

< drum roll please >

688,116 Files. 562 Gigabytes in size.

We are a SMALL shop. Hard to wrap my head around the fact that there is actually this amount.

Click to expand...

That's an easy one, then. The killer with many file systems in "common" use is running out of file handles long before running out of storage space.

The cheap and effective solution is multiple pairs of not TOO DAMNED big storage units.
"Software" RAID1 is good enough. It doesn't WRITE as fast as "hardware" RAID, but it READS rather well, thanks.

Storage is purchased for the purchase-era economic "sweet spot" for the week.

Read: last year's tech, high market volume, and Oh gee whiz, already proven in brazillions of installs, not new and risky. EG: At a time others are buying 1 TB & 2 TB and up, I'm still buying 500 GB, 1 TB, etc..