SUN SPARCstorage Array Survey Responses
SUN SPARCstorage Array Survey Responses
These are the responses to my survey. I have edited them slightly to
remove some quoted passages (e.g., the description of my problem, or my
signature). You can view the text of the survey here.
Included in the responses, there are a few other answers from folks who
didn't answer my survey, but had useful suggestions nevertheless. The
numbered responses listed below directly addressed my survey, and they
are the ones I included in my summary.
Jump to the first response.
- doug@Eng.Auburn.EDU (Doug Hughes)
- w288@g0014 (Marty Boos)
- ake@cs.umu.se (Ake Sandgren)
- (Anonymous)
- greg.kranz@amp.com (Greg Kranz)
- john@oncology.uthscsa.edu (John Justin Hough)
- saieva@ppllc.com (Salvatore Saieva)
- ottenber@mr.med.ge.com (Paul A. Ottenberg 4-6166 MR)
- lopez@abqato.scs.philips.com (Robert Lopez)
- pug@arlut.utexas.edu (Pug)
- bf566z9@is000913.BELL-ATL.COM (Kuldell)
- ebumfr@ebu.ericsson.se (Mike Rembis)
- diekema@linus.si.com (Jon Diekema)
- fletcherc@postoffice.ttmc.com (Fletcher Cocquyt)
- trumbull@unx.sas.com (Ed Trumbull)
- dtomo@scg8a.es.hac.com (Dave T)
- andrew@crls54.srm.crl.gov.bc.ca (Andrew Gallagher)
- gedau@mim.com.au (George Dau)
- sys013@abdn.ac.uk (G.ROBERTSON)
- ben@larbs5.sbi.com (Ben Evans)
- Scott.Briand@NSPower.NS.Ca (Scott P. Briand 428-6140)
- job@xinit.se (Joakim B. Berglund)
- craig@jetcity.com (Craig Nelson)
From: jdd@cdf.toronto.edu (John DiMarco)
Date: Tue, 19 Mar 1996 13:19:42 -0500
In comp.sys.sun.admin you write:
>If you manage (or used to manage) Sun SPARCstorage Arrays, please read this.
We've just been RAID-shopping. We steered away from the SPARCstorage
array because it doesn't appear to have a particularly robust RAID5
implementation. It's a strange hybrid of hardware and software, with many
of the drawbacks of both. I'm not terribly surprised to hear that you've
been having problems with them, although I do think Sun will get them
straightened out eventually.
We ordered a CMD 5500; it's not in place yet, so it remains to be seen
how well it works.
A cheaper alternative to an Auspex might be a hardware RAID box like a CMD
5500 or a Digital Storageworks. Auspexen are nice but quite pricy, especially
as you add disk.
Regards,
John
--
John DiMarco Office: EA201B
Computing Disciplines Facility Systems Manager Phone: 416-978-1928
University of Toronto Fax: 416-978-1931
http://www.cdf.toronto.edu/~jdd
From: doug@Eng.Auburn.EDU (Doug Hughes)
Date: Tue, 19 Mar 1996 15:25:13 -0600
You are not alone. RAID 5 has been very unstable in the past 6 months. It
doesn't matter if its Veritas or ODS. I suspect if we had to do it over
again, despite the price differences, we probably would have gone with
a hardware plug-it-in solution (like from Land-5, Clariion, DEC, or RaidTech)
|>
|> What I want to know from you is:
|>
|> 1) How many arrays are you running? For how long? What models? How many
|> disks per array? What size?
2 arrays, 102 (2G disks) 15 disks per array soon to be 18 (3 per controller)
|> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
yes, dual-ported, but the second port is just for redundancy and not in
active use. currently using supersparcs. Sun told us our quad hypersparcs
were an unsupported configuration.
|> 3) Do you use uninterruptible power supplies?
Yes, absolutely!
|> 4) Have you had hardware problems? If so, what kinds?
2 bad disks, one bad SSA controller card.
|> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
|> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
|> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
|> (/dev/vx/dsk/dg/rawvol)?
RAID5 file systems (which has lead to our problems). It seems most of
Sun's customers use RAID0+1 on these things on raw file systems for
databases which has yielded VERY few problems. If you have prestoserve
turned on (NVSIMMS or cards) turn it off until you get a patch! We have
recently been givin a supported configuration from Sun. We're installing
it tonight. It is supposed to fix all of our problems. We have been
running for the last 8 weeks without prestoserve turned on with
only one problem. There is also a bug in several version of the disk
firmware for the Seagate disks we're using, we we're going to upgrade
those tonight too, as well as the array firmware itself.
|> 6) What version of Solaris OS and Array software are you running?
Volume Manager 2.1 on Solaris 2.4
|> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Yes, the internal NVRAM is reliable and robust. We use RAID5 and DRL.
|> 8) Have you had any trouble with the array software (managing disks, hot
|> sparing, RAID "protection")?
We have no hot spares. the software has been very nice.
|> 9) How do you use your SSA's (database, NFS service, other)? How is
|> performance?
Performance is fairly good. It was better when we had presto-serve
on as a second level write cache. about 20-40% better. Unfortunately,
the machines were also crashing almost weekly with it on.
|> 10) How satisfied are you with your SSA's? Will you keep them? Will you
|> buy more? Or will you buy an Auspex?
|>
So far they've been stable without prestoserve. We'll be keeping them.
We're going with sun's recommendation to see how it works. If it works
with prestoserve on, we'll be placated, but still burned. If it doesn't,
we'll likely be very peeved, but still keep them.
We've been through every kind of hurdle with sun support that you
can imagine. So far there is a site in Germany (the only one we know
of) that is configured like us, though it sounds like you may be
configured similarly. If the patching/updates tonight work out, and you
wish, we can email you the exact level of software, firmware, and
patches that you need for a stable configuration as recommended by
Sun. We're still pushing for quad Ross CPU modules, since they give
better NFS performance than dual supersparcs. (and leave the machine
free to do other things, like snooping our FDDI ring, which is IMPOSSIBLE
with the supersparcs.. the machine goes into lala land.)
From: kupec@agouron.com (John W. Kupec)
Date: Tue, 19 Mar 1996 15:38:34 -0800
I bought an Auspex!
Actually, I never tried the SsA step- I looked at buying
a 1000E with a SsA (among other things) or buying an Auspex.
Sounds like you've got a lot more disk than we do- currently
we have about 60GB online. Since we brought the NS7000/500
online in about October of 94 we've had: zero disk failures,
zero hardware failures. In other words, we frequently forget
we've got the thing. Life is good...
Just gloating,
John
p.s.
I'll be glad to help answer any Auspex questions you may
have.
--
__________________________________________________________________________
John Galt Kupec (kupec@agouron.com) | Agouron Pharmaceuticals, Inc.
Voice: (619) 622-3040 | 10350 North Torrey Pines Rd.
FAX: (619) 622-3297 ! La Jolla, CA 92037
From: w288@g0014 (Marty Boos)
Date: 19 Mar 1996 19:03:56 GMT
: What I want to know from you is:
: 1) How many arrays are you running? For how long? What models? How many
: disks per array? What size?
We have 65 SSAs model 102s, they have been in productin for just under
1 year. Most of the arrars are fully populated with 1gb drives (mix of
Conner and Seagate). Four of the arrarys have 2GB drives in them.
: 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
42 of the arrarys are single ported to Sun SS2000Es, 20 are dual ported to
a Sparc CLuster containing 2 2000es. The rest are hooked to 1000e machines.
: 3) Do you use uninterruptible power supplies?
Yes, our entire building is protected by a UPS system.
: 4) Have you had hardware problems? If so, what kinds?
We have about 1 drive failure per week. Some weeks we might have 2 or 3
failures and some weeks no failures at all. Since we have almost 2000 drives
1 per week is really not as bad as it sounds and real close to the MTBF that
Sun and Seagate publish. We have also had a couple of power supplies go bad
in the past year.
: 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
: Veritas Volume Manger or Solstice DiskSuite? What RAID levels
: (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
: (/dev/vx/dsk/dg/rawvol)?
All of the arrays are under Veritas volume manager control. We are running
with stripping and mirroring. All stripes in an array are mirrored to another
array on the system. Most of the activity to the arrays is using raw devices
/dev/vx/rdsk/diskgroup/*, we only have a couple of filesystems that we
are using for Oracle archived logs.
: 6) What version of Solaris OS and Array software are you running?
We are running Solaris 2.4 patch level 37 and Volume manager 1.1
(CVM 1.1 on the PDB cluster)
: 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
I just turned on fast writes for testing and did not see any improvement
to write times. I am using DRL on all drives that are not on the PDB cluster
machine. I will be turning on DRL on the PDB machine very soon because the
latest version of the PDB software now supports it.
: 8) Have you had any trouble with the array software (managing disks, hot
: sparing, RAID "protection")?
We gave up on the GUI a long time ago and now do everything from the
command line and with scripts that we put together. The GUI is just not
ready to support large numbers of volumes like we have. The Veritas
volume manager is by far the weakest product that Sun sells. Sun and
Veritas still don't consider it a problem that when a plex detaches
>From a volume that a hot spare does not kick in. They keep telling us
that a hot spare will only work if veritas can not read the private
region of the disk.
: 9) How do you use your SSA's (database, NFS service, other)? How is
: performance?
Almost all of the SSA's are used for Oracle database files on RAW devices.
Performance is real good considering that there is no cache in front of
the drives. Since most of our database is used for reads the round robin
reads that volume manager supports helps spread out the load.
: 10) How satisfied are you with your SSA's? Will you keep them? Will you
: buy more? Or will you buy an Auspex?
As far as disks go I don't think that they are any better or worse than any
body elses on the market. The biggest problem that I have with them is
that I have to spin down 10 drives to change a single disk. Other disk
arrays on the market (Clariion) you can swap out just the failed disk and
not affect the others. It's also too bad that we are forced to use Veritas
volume manager on the PDB systems. Like I said earlier it really is the
weak link in the product.
marty
--
--------------------------------------------------------------------------------
Marty R. Boos Phone: (612) 519-4123
Database Administrator Fax : (612) 519-4711
Fingerhut Companies, Inc. Email: marty.boos@seag.fingerhut.com
Pager: 1-800-SKY-PAGE PIN 70699
--------------------------------------------------------------------------------
From: ake@cs.umu.se (Ake Sandgren)
Date: Wed, 20 Mar 1996 07:49:18 +0100
> What I want to know from you is:
>
> 1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
One array for 1 1/2 year. Model 101 with 12 the first year and now 18 disks
the first 12 are 1G and the last ones 2G
> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Single port SS20/50 64MB
> 3) Do you use uninterruptible power supplies?
Our whole machine room is powered by a big UPA.
> 4) Have you had hardware problems? If so, what kinds?
The only problem we have had was one disk that were dead on arrival.
> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
Concatenated, mirrors and striped mirrors
> 6) What version of Solaris OS and Array software are you running?
SunOS 5.4 hw 11/94 Veritas 2.1
> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
NVRAM on for all disks, no DRL or Raid5
> 8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
We don't use hot sparing (can't afford to have enough disks free), but that
aside the only problem we had were that vxva used to dump core sometimes with
rev 2.0 and below.
> 9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
The main part of the SSA is used for home-dirs for our +150 machines
and performance is VERY good. (will be even better when we start using
100MB ethernet :-).
The 2 bottlenecks we have are outside the SSA (ethernet and CPU)
> 10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
We are very satisfied with it and will definitely keep it. With the current
rate of increased diskuse we will probably need to buy another SSA within the
next 2 years (this is rather slow isn't it :-) unless something else comes out
on the market wich beats the hell out of the SSA :-)
From: amy.hollander@amp.com (Amy Hollander)
Date: Wed, 20 Mar 1996 07:26:10 -0500
Our SSA problems have been mostly failed 1.05GB disks. The 2.1s
seem to be more stable.
We have lost fibre channel connection with one of our three
SSA's repeatedly and various hardware has been replaced...the
module, the card, the card in the SSA.
Our raid5 has worked very well. We had no hot spares at the time,
but when we replaced a bad disk, we lost no data.
We now have hot spares, but have not used them yet.
amy.hollander@amp.com
From: (Anonymous)
Date: Wed, 20 Mar 1996 06:53:23 -0600
In response to your posting to sun-managers concerning SparcStorage Arrays:
>
> What I want to know from you is:
>
> 1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
1 Array, 6 1.05gb Connor disks
> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Single, off of SparcServer 1000
> 3) Do you use uninterruptible power supplies?
We do now - didn't when we first got the array and had too many problems.
> 4) Have you had hardware problems? If so, what kinds?
YES! Disk failures mainly.
> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
Veritas - was running at RAID 5, but went to striped w/tape backup
because RAID 5 DOES NOT WORK. Twice, I lost all the data on the array
when it was set up as RAID 5 because a single disk failed. So why bother?
> 6) What version of Solaris OS and Array software are you running?
Solaris 2.4 now, were using 2.3
> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
fast_writes: yes did use RAID 5 logging before (see above)
> 8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
The RAID "protection" is a joke (see above)
> 9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
NFS service. I don't get any noticible performance increase from the RAID
vs. a local FastSCSI disk - was certainly expecting some.
> 10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
>
I'd drop this sucker like a hot rock if I had the money to buy something else.
I think Sun should buy me a different one ;)
As you can tell, I am very dissatisfied with the performance of this device.
From: ge@mcnc.org (George Entenman)
Date: Wed, 20 Mar 1996 08:50:37 -0500
Margarita,
I've forwarded this article to the sysadmin of our SSA but I would
like to point out that Auspex isn't the only alternative. It's been a while
since I spoke with Auspex, but 9 months or so ago they didn't have
RAID at all. They gave me a story about using military-spec drives,
as if it were possible to predict how long drives will last.
I recommended the purchase of two 7-bay Clariion disk arrays from
Invincible Technologies (the local rep is Ken Koocher,
koocher@notes1.invincible.com)
and have been very impressed. Here are some reasons:
- The Clariion doesn't need special firmware for different versions
of Solaris. The SSA does because it's doing its processing
in the
server's CPU, not in the array.
- Volume Manager is a great mystery. Why is the number of sectors
it sees different from the number on the physical disk? Why
doesn't it use tracks instead of sectors (isn't it faster to
read a whole track?). I find that with Volume Manager I had
to just click and pray that it was right. The Clariion is a
breeze to configure.
- You really can't remove a disk from the SSA without stopping the
tray that it's in. You really can from the Clariion.
The Clariion is apparently the base for HP's arrays, and others. Each of
ours looks like a big happy 15GB filesystem to Solaris 2.5! You can buy
models with dual controllers, etc., that will make them bullet-proof for
your database needs (we didn't because we're not using them for such
a critical application).
I hope that the other sysadmin answers this survey for you, but I wanted
to vent my frustrations with the SSA to you myself.
Sincerely,
George Entenman
From: greg.kranz@amp.com (Greg Kranz)
Date: Wed, 20 Mar 1996 09:43:34 -0500
1. Two model 102s with 6 drives each, increased in Jan 96 to 12 drives each. All
disks are 2gig seagate barracudas.
2. Single attached to a SC2000 with four 50 Mhz, 2 MB cache cpus.
3. No battery backup (in the works) but fully conditioned power.
4. problems:
- same as your number 1 but patches, and replacing one FOM fixed it in
separate instances.
- same as your number 3 BUT we have never had a failed disk. All of our
damage occurred after the system hung (froze). The only common thread
was Legato Networker. I don't believe the hangs were caused by the
SSAs. The "fsck'ed -- thousands of inodes are misplaced" matches
exactly. Has happened on different arrays on different controllers.
- Have encountered one failed disk ... bad from the factory.
5. Veritas Volume Manger
striped, RAID 5, and "managed" raw disks. The raw disks are not in the SSAs.
They are external boot disks.
6. Solaris 2.4 kernel 101945-29. Just moved up to 101945-34 hoping to address
the hangs.
SPARCstorage Volume Manager - VERSION: 2.1,PATCH=2
SPARCstorage Volume Manager Visual Administrator - VERSION: 2.1.2
7. fast_writes - Yes
RAID 5 logging - Yes - watch out where you put it!
8. Not using hot spares.
Biggest complaint with Veratas, after the reliability issue, is the
"grow filesystem" feature. it is a wonderful feature except it gives
you very little (if any) control over where and how the it grows. Disk
groups help some but it still does stupid stuff and then on top of it
the "undo" process is nasty. At the very least, it come back and say
this is what I'm going to do to your subdisks, do you want to proceed.
9. All NFS services, in a read intensive environment.
Mixed returns on the performance side. The big problem with performance
is writes. Clients frequently have nfs write times ranging from 400 -
1000 ms. I believe the arrays give good write performance until the
cache is overloaded (ie concurrent writes of large files (10 MB) from
fast sparcs with multiple net interfaces to the server).
I am upgrading to the faster controller with the 16 MB cache.
I'm suspicious of the fiber channel link speed and the single sbus card
every thing has to go thru. Faster fiber channel speeds was talked
about when the arrays were first released. It might be useful for Sun
to support two sbus cards in the same server to the same array. This
would seem to spread the load especially in servers like the 2000 and
2000E.
10. Not completely satisfied and not ready to buy more (yet). Switching
to a 3rd party like Auspex opens up a large number of additional issues,
service, compatibility, .....
From: heas@nexen.com (heas)
Date: Wed, 20 Mar 1996 10:35:03 -0500 (EST)
i've never had a SSA...but if you are considering an auspex be sure to
look into a Network appliance FAServer first (415.428.5100).
-heas
From: john@oncology.uthscsa.edu (John Justin Hough)
Date: Wed, 20 Mar 1996 09:37:04 -0600 (CST)
Margarita,
1) How many arrays are you running? For how long? What models? How many
disks per array? What size?
a) 1
b) 1.5years
c) 100
d) 12
e) 6 1G & 6 2G
It is my understanding that you can't load the 100's down too much. I would
have to upgrade the power supply and should upgrade the controller to be a
102.
2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Single now though I just got a second interface and second soc.
3) Do you use uninterruptible power supplies?
We have very bad power at this institution, so we have 3KVA unit that is
pretty efficient and all the hardware on it is about half of rated load.
4) Have you had hardware problems? If so, what kinds?
a) yes, one problem.
b) upon moving the systems around the fiber channel SBus board became
loose in our SPARCserver 1000. It gave a lot of bogus errors that
aren't listed in SunSolve databases nor in Sunmanagers.
5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
Veritas Volume Manger or Solstice DiskSuite? What RAID levels
(concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
(/dev/vx/dsk/dg/rawvol)?
a&b) as disks and raw disks for databases.
c) neither though I have been thinking about DiskSuite. But I have
been leary about adding a layer of complexity. I have never thought
that Sun is a committed to help Veritas make their product stronger
since they have had a competing product, and until recently DiskSuite
seemed neither stable enough or robust enough. But DiskSuite appears
to be licensed. What happens if you license management scheme fails -
are you dead in the water? I am waiting until I feel it is safe to
move in some direction.
d&e) na
6) What version of Solaris OS and Array software are you running?
SunOS 5.4, 1.33 soc firmware and 1.24 array firmware, and neither DiskSuite
Veritas
7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Yes, no.
8) Have you had any trouble with the array software (managing disks, hot
sparing, RAID "protection")?
na
9) How do you use your SSA's (database, NFS service, other)? How is
performance?
a) database, and nfs file service.
b) Performance is good, but is drive dependent. The drives are all
synchronous, wide SCSI, tag queue, but don't have write cache.
10) How satisfied are you with your SSA's? Will you keep them? Will you
buy more? Or will you buy an Auspex?
a) yes
b) yes
c) yes
d) If I could afford an Auspex, I would like one instead.
john
From: saieva@ppllc.com (Salvatore Saieva)
Date: Wed, 20 Mar 1996 10:45:50 -0500 (EST)
Margarita,
I've been using a SPARC Storage Array for two years now. I haven't
experienced any problems at all.
> What I want to know from you is:
>
> 1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
One Array. Two years. Model 101. 18 disks. 18GB.
> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Single ported. SPARCstation 20 model 712.
> 3) Do you use uninterruptible power supplies?
Yes. My servers and SSA are in the same rack. This rack has an APC 1400
power supply.
> 4) Have you had hardware problems? If so, what kinds?
No hardware problems related to the SSA.
> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
I use Veritas, right now V2.0 w/o the SSA hardware upgrade that included a
battery fix, etc. (I have the upgrade just haven't installed it yet.) I
have both volumes used as raw partitions (for Informix databases) and
volumes for ufs. I do concatenation and stripping, no level 5 or
mirroring.
> 6) What version of Solaris OS and Array software are you running?
Solaris 2.5 and Volume manager v2.0.
> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Not sure about the fast writes, I'm running the defaults. No logging.
> 8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
No problems managing disks. However, I never use the vxva GUI to create or
manage SSA volumes. I strictly work from the command line using the
Veritas command line tools. I use the GUI exclusively for monitoring the
SSA (from the Analyze->Analyze All menu).
> 9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
Informix database, ufs filesystems used for NFS.
> 10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
Very happy with the SSA. I'm planning on buying another.
I should say that I don't trust the Volume Manager GUI. I don't like the
way it will sometimes allocate space for subdisks and volumes. I prefer,
rather, to use the Veritas command line utils to create subdisks, plexes,
and volumes specifically on SSA disks being careful to have subdisks begin
and end of cylinder boundaries. In this way I have total control of the
size and placement of subdisks and volumes taking into consideration the
function of the volume. (For instance, I'm careful not to have Informix
database volumes on the same SSA SCSI bus with volumes used for the Notes
database, since these are high I/O volumes.)
Sal.
---
Salvatore Saieva
PRINCIPIA Partners LLC E-Mail: saieva@ppllc.com
Harborside Financial Center Phone: (201) 946-0300
902 Plaza II, 9th Floor Fax: (201) 946-0320
Jersey City, NJ 07311
Exhortation: ``Aesthetics are more important than efficiency.'' -DEK
From: ottenber@mr.med.ge.com (Paul A. Ottenberg 4-6166 MR)
Date: Wed, 20 Mar 1996 10:03:47 +0600
Marg:
no need for me to repeat the story that you know so well. We have experienced
the same issues/problems since day one (Sept/Oct 1995). Sun contributed the
problems to firmware (both SSA & disk). Since then we have upgraded to the
latest recommended firmware versions. We are still experiencing too many disk
failures (see comments below).
little twist in our configuration is that we currently have (2) SSA101 on loan
>From Sun and 90G of conventional disk. We had migrated completely off of one
of our SSA102s. approx 90% of our data is under RAID5, the remainder is under
RAID 0+1. Currently we are migrating back onto our SSA102 & attempting to
return to our original configuration.
Please: I too am in the same situation that you are - little help from Sun. I
strongly urge you to forward any/all comments you receive on this topic to my
attention.
Thank you very much.
paul.
> What I want to know from you is:
>
> 1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
(2) SSA102
- 30 2G drives
- 24 2G drives
- system brought online Sept 1995
(2) SSA101 (currently on loan from SunMicrosystems)
- both have 30 1G drives (total 60 drives)
- system brought online Oct 1995
> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
> 3) Do you use uninterruptible power supplies?
- no
> 4) Have you had hardware problems? If so, what kinds?
- yes: many.
- multiple failed volumes
- Fiber Channel on arrays stop functioning for no apparent reason
- Seagate ST32550N and ST32550W fireware issues
- we upgraded to current rev 0416
- controller failure
- disk failures at rate of 0.5 per week (seagate/conner, 1 & 2G)
- oh by the way, I walked in the door this morning find another
disk failed overnight .... :(
- soc failure: online/offline
> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
- Veritas
- RAID 5
> 6) What version of Solaris OS and Array software are you running?
- SS1000e OS5.4
- SSA 3.4
> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
- NVRAM cache
- RAID5 logging
> 8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
- yes
- Exactly same as you described above in Software Problems (does not
fail over to hot spare)
- Exactly as you described in RAID5 Problems (lost entire RAID5 vols
with fsck failing)
> 9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
- NFS server
- performance: acceptable
- down time/failures: not acceptable
> 10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
- not too happy with amount of down time
- currently shopping around for alternatives
- Auspex (and Cray) is $$
- Artecon - max 25G
- Network Appliance RAID4 & "Write Anywhere File Layout"
(WAFL) file system
--
'''
(o o)
--------------------------o00--(_)--00o-------------------------------
Paul A. Ottenberg | email : ottenbergp@med.ge.com
EIS Admin Team Leader | voice : 414.521.6166
GE Medical Systems | fax : 414.521.6800
PO Box 414; Mail Stop: W826 |
Milwaukee, WI 53201-0414 |
--
Date: Wed, 20 Mar 1996 11:50:05 -0700
From: lopez@abqato.scs.philips.com (Robert Lopez)
> Problems 1) and 3) may have been caused by dirty power. We have been
> putting our arrays on UPS's to clean the power. We haven't lost a disk
> in a whole week :-)
VERY LIKELY !!!
>
> What I want to know from you is:
>
> 1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
One. 2 years. 101. 30. 1G.
> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Single. SS-20.
> 3) Do you use uninterruptible power supplies?
Run all computer and network equipment on clean power system provided by a
data center conditioning system.
> 4) Have you had hardware problems? If so, what kinds?
1st battery board was not available. Later provided.
1st controller board died in about 1 month. Replaced.
1 drive crashed and was replaced.
> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
Started w/ Veritas; ran some raw disks for Oracle; some concat and mirror.
Now running Solstice DiskSuite; no raw disks; all RAID 5.
> 6) What version of Solaris OS and Array software are you running?
Solaris 2.5. DiskSuite 4.0.
> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Yes. RAID 5 logging.
> 8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
No.
> 9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
Oracle db. Very high NFS load (for SS-20).
Acceptable 90% of time; some peak use problems.
> 10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
Acceptable. Yes. Yes. No.
From: pug@arlut.utexas.edu (Pug)
Date: 20 Mar 1996 08:05:42 -0600
>What I want to know from you is:
[ Damn, my setup seems puny compared to the other answer already out there. ]
>1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
3 SSA 102 and 2 SSA 200. For just about a year. The SSA 200's are
currently using 2.1 gig drives, soon to have 9 gig drives as well. The
102's have 72 2.1 gig drives total.
>2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Dual ported to SS2000 and SS2000E.
>3) Do you use uninterruptible power supplies?
Of course.
>4) Have you had hardware problems? If so, what kinds?
About a dozen drive failures total. (Within the SSA 102s that is.) 2
failed prom updates rendering the cards useless until they brought new
ones out. (Both on the same day. I've never done them in mass since.)
>5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
Using Veritas Volume Manager. We are using a mixture of concat, stripped
and/or mirrored. Mostly concat that might be mirrored.
>6) What version of Solaris OS and Array software are you running?
2.4 kernel patch -37 with all the "latest" patches. VxVM is 2.1 or 2.1.1.
(We still need to update one of the systems.)
>7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
fast_writes, but no DRL or RAID 5. We don't particularly care how long
it takes to rebuild a mirror.
>8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
Hot sparing? They have it? We've never had it kick in despite disk
failures. This is apparently a political battle between Sun and Veritas
(as our local reps told us) on when a disk is in failure mode. We'll have
to see if it gets resolved correctly.
>9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
NFS service. Performance is wonderful when they aren't in a failure mode
due to a disk.
>10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
We are pretty satisfied *now*. THe growth pains in the beginning before
the software/prom caught up with the technology hurt a lot! We'll keep
them, but more, and won't buy an Auspex since my understanding is you
*have* to buy all their products. (Unless that has changed recently.) At
least we can buy any disk to shove in the SSA 200 systems. (We've never
had a failure in an SSA 200 yet.)
Ciao,
--
Richard Bainter Mundanely | OS Specialist - OMG/CSD
Pug Generally | Applied Research Labs - U.Texas
pug@arlut.utexas.edu | pug@eden.com | {any user}@pug.net
Note: The views may not reflect my employers, or even my own for that matter.
From: bf566z9@is000913.BELL-ATL.COM (Kuldell)
Date: Wed, 20 Mar 96 18:31:12 EST
Margaret:
Oh yes...we've hit on many of these items as well.
See below.
In article <4immqj$6iq@apakabar.cc.columbia.edu> you write:
>Our problems have been of the following sorts:
>1) flaky hardware - we have had to replace 2 controllers, one suspect
> power supply, and at least 3 disk trays. our first 3 arrays were
> plagued with the ONLINE/OFFLINE problems which were fixed by
> replacing optical fibre hardware (FOM's, SOC cards, etc.) we have more
> than 150 disks and we've seen more than 10% fail in the last year.
> the problems have occurred on all types of Sun SSA disks:
> 1gig (conner and seagate) and 2gig (seagate barracuda).
Been there. Sun has advised us to install a recommended
patch which HELPS TO RELIEVE the OFFLINE/ONLINE problem, and
that the problem is scheduled to actually be FIXED in the next
few months (oh joy). We also replace disks regularly, and are
tracking them now to harrass Sun with false claims of MTBF
numbers.
>2) software problems - the Veritas volume manager often does not employ
> hot spares, based on a fuzzy interpretation of "disk failure." Sun tells
> us the next release of volume manager is supposed to fix this.
We have had a Sun guy write a script to combat this, we're testing
it now. He's changing the vxnotify to send mail AND run a script
to replace the disk with a hot spare anyway. I assume you're talking
about the condition where one subdisk on a disk fails but it can still
write to the private region, so it just detaches the plex containing
the bad subdisk but doesn't kick in a hot spare? Been there too.
>3) RAID5 problems - we have lost entire RAID5 volumes mysteriously with
> a single disk failure, or with no disk failure at all. The symptom
> is that the filesystem cannot be fsck'ed -- thousands of inodes are
> misplaced. it looks like something just scribbled all over the filesystem.
> dunno if it was the controller, RAID 5 driver, kernel, or what.
>
Don't do any RAID 5 (yet). We're at RAID 1 on all disks.
>Problems 1) and 3) may have been caused by dirty power. We have been
>putting our arrays on UPS's to clean the power. We haven't lost a disk
>in a whole week :-)
>
>What I want to know from you is:
>
>1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
OK: 40 SSA 100's, 32 let's say since Jan 95, 8 since Oct 96. We're
up to about 24 disks per array (all 1GB) but will be filling empty
slots and adding 2GB disks and swapping some 1GB disks for 2GB disks
as well.
We also have 21 more SSA's (102`s I suppose, 18 2GB disks each, plus
60-some odd extra 2GB disks in boxes) unboxed but not currently in use.
>2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
The 40 currently running are all Dual-Ported. The 21 new ones were
ordered Single-Ported, and I don't plan on installing them until we
purchase enough fiber channel adapters to dual-port them all.
They are connected to SS2000's (8) and SS2000E's (2) in a strange pair
formation which I will explain if you request.
>3) Do you use uninterruptible power supplies?
I am quite fortuitous in this regard, as they are in a data center
with rather robust building UPS (which is probably conditioned as well).
>4) Have you had hardware problems? If so, what kinds?
Mostly SSA disks, one or two SS2000 internal disks, several system
board failures, and many, many, fiber channel adapter failures. We
finally got word from Sun that there was a problem with the Rev of
fiber channel adapters we had, so we had Sun come in and check all
the Rev's and replace those that were not Rev 3. Total number of
fiber channel adapters checked was 160. The 2000E`s had the new Rev's,
most of the 2000's were old Rev's (unless it had already failed and
been replaced recently).
>5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
Managed raw disks for use mostly by sybase; Volume Manager (version
1.3 -- aack! Upgrade planned for next weekend to 2.1.1) under RAID 1
(mirrored). Some disks are used for filesystem space (for the data
loaders, not any system paritions like /, /opt or the like).
>6) What version of Solaris OS and Array software are you running?
THIS is a matter of great consternation between us and Sun. We
are currently at:
Solaris 2.3
Volume Manager 1.3
SSA version 1.0
Firmware version 1.11
ODS 2.0 (used only on SS2000 internal disks)
>7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Nope. WAY to far ahead of our versions' capability (the NVRAM cache
anyway, we haven't gotten around to trying to implement DRL yet).
>8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
Been there. See above for commiseration for your "not quite failed
enough" problem. Also had a DREADFUL time tracking down a hardware
problem which manifested itself with: 1. vxva dumps core; 2. sybase
gets traceback errors. Took Sun over 6 weeks to diagnose and fix
the problem (had to pull it out of production, but production is a
long story...).
>9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
Database. Unfortunately we've spent so much time trying to get the
systems right, we haven't been able to really let the users at it. It's
for a new application (well, it was new in '94, now it's old but still
untried).
>10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
Hmmm. Not sure we'll keep them. Auspex was not brought up...but
HP was. ;-)
Please compile and post/reply with your results. I'd be VERY interested
in further commiseration with other sufferers. I wrote WE above everywhere,
but truth be known, we = me. There has s'posed to have been another admin
person since late '94. We may get one in next month (finally found one
that was fairly qualified, now we just need to get him past our "less than
cooperative" HR department.
Thanks,
Chipp
--
Chipp Kuldell, Analyst | chipp@bell-atl.com
Bell Atlantic | (301) 236-2238
13101 Columbia Pike, Room 209B | Why isn't phonetic spelled
Silver Spring, MD 20904 | the way it sounds?
From: ebumfr@ebu.ericsson.se (Mike Rembis)
Date: Wed, 20 Mar 1996 16:22:08 -0800
Hi.
Suggestion. If you want to sleep at night - buy the a Auspex. That's what we
are doing. We've had a SSA for 1.5 years and it's been nothing but trouble
>From day 1. We have experienced *ALL* the problems you have mentioned.
We were completely down from Christmas until New Year's Day because SUN couldn't
fix the problems with the SSA/Veritas software. Did I neglect the fact that
we have a GOLD maintenance contract??? And that it took SUN **DAYS** not hours
to even make a guess what was wrong????!?!?!?!
I'm not sure how much the cleaner power will help. Our has been on a UPS
since day 1. Didn't help us.
Our config.
SS1000 - 4 CPU's, 256 MB RAM, 4x535 MB disk, 2xQEC (quad ethernet), TGX
SSA 100 - 30x1 GB disks - all configured as RAID-5 thru Veritas.
Fast writes were recently enabled.
Most of our troubles started when one disk died. When we replaced it and
attempted RAID-5 rebuild - that was the end of life as we knew it. Nothing
but core dumping fsck's after that. Oh, and much corrupt data.
During it's life the SS1000 has had all guts swapped out 2.5 times.
" " " " SSA " " " " " 1.5 times.
SS1000/SSA spend its life running as NFS/NIS/DNS server.
Satisfied? no
Buy SSA again? Never
Buy Auspex? Yes
---
Mike Rembis | Internet: mike.rembis@ebu.ericsson.com
Ericsson, Inc. (EUS/BT) | MEMO: ERI.EUS.EUSBMFR
````````````````Reality is in the Eye of the Beholder'''''''''''''''
From: diekema@linus.si.com (Jon Diekema)
Date: Thu, 21 Mar 1996 10:05:32 -0500 (EST)
A. SPARC 1000 with single ported Storage Array model 100.
We are using the array as JABOD, Just A Bunch Of Disks,
/dev/dsk/cNtNdNsN under Solaris 2.3. The volume manager
is NOT managing the disks in the array. The write caching
is disabled on Solaris 2.3, you need Solaris 2.4 or greater to have
it enabled.
Volume Manager 2.1 patch 3
Array firmware 1.12
We are using the Volume Manager to mirror the root partition and the
swap space in the internal four disks. We do have DRL enabled
for all the mirrors except for the root disk.
Both the SPARC 1000 and the SA 100 are on a UPS.
This disk storage in the SA 100 is used to support several Oracle database.
The disk performance has been adequate (i.e. the users are happy).
We haven't had any hardware problems with the SA 100, and we have had
it since the fall of 94.
27 1 gig disks
B. SPARC 2000 with two single ported Storage Array model 100's.
We are mirroring the data from one array against another
under Solaris 2.4. The data looks like one big volume.
The write caching is enabled along with DRL on all mirrors except for
the mirror that contains the root filesystem.
Volume Manager 2.1 patch 3
Array firmware 2.4
We are using the Volume Manager to mirror the root partition and the
swap space in the internal six disks. We do have DRL enabled
for all the mirrors except for the root disk.
Both the SPARC 2000 and the two SA 100's are on a UPS.
This disk storage in the SA 100 is used to support the Pro III MRP
application. The disk performance has been adequate (i.e. the users
are happy).
We had one disk failure in one of the SA 100, and we have had
it since the sping of 95.
12 1 gig disks in each array.
I would consider buying more SPARC Storage arrays.
From: fletcherc@postoffice.ttmc.com (Fletcher Cocquyt)
Date: Thu, 21 Mar 1996 12:02:50 -0600
Hi, I just took over (about 1 month ago) at a site running Solaris 2.5,
with 4 Storage Arrays 3 running DIskSuite 4.0 the other running Veritas,
we have had problems, and since I have had limited experience with the arrays,
I may not be able to answer all the questions, but I'll try:
> 1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
4, 6months, model 102, 30disks/array, 2Gb Seagate Barracuda
> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Dual ported, only on port being used. Sparc 20's
> 3) Do you use uninterruptible power supplies?
Yes
> 4) Have you had hardware problems? If so, what kinds?
We had one array going offline and hard locking the Sparc 20 it was attached
to. I just replaced the SSA controller last night at midnight (had to do
some interesting device/path_to_inst manipulation), but so far no lockups.
The array was locking up randomly about 5 times/week, then one day it was
locking up every 15 minutes for an hour...that's when we got on Sun's back.
> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
3 DiskSuites Raid 1 (mirroring only), 1 Veritas
> 6) What version of Solaris OS and Array software are you running?
Solaris 2.5 (with latest SSA patch) and firmware 3.4 (latest for the SSA
controller)
> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
No, and we use transaction logging.
> 8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
nope
> 9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
NFS and Sybase
> 10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
definately, NOT satisifed
----------------------------------------------------------------------
Fletcher Cocquyt fletch@ttmc.com (809) 299-2900
System Administrator Trout Trading Hamilton, Bermuda
----------------------------------------------------------------------
From: trumbull@unx.sas.com (Ed Trumbull)
Date: Thu, 21 Mar 1996 16:19:30 -0500
In article <4immqj$6iq@apakabar.cc.columbia.edu>, you write:
|> Hi Sun Managers,
|>
|> If you manage (or used to manage) Sun SPARCstorage Arrays, please read this.
|>
|> What I want to know from you is:
|>
|> 1) How many arrays are you running? For how long? What models? How many
|> disks per array? What size?
3 SSA's, each with 30 2Gb barracudas on a SS1000
1 SSA ditto
1 SSA with 18 1Gb drives on another SS1000
|> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Single
|> 3) Do you use uninterruptible power supplies?
Yes
|> 4) Have you had hardware problems? If so, what kinds?
We've lost 3 of the barracudas, that I can remember
|> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
|> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
|> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
|> (/dev/vx/dsk/dg/rawvol)?
the 3 SSA system is all veritas vxfs raid 5
the 1 SSA 60Gb system is a mix (and a mess)
not sure about the other system.
|> 6) What version of Solaris OS and Array software are you running?
solaris 2.4 Generic_101945-34
veritas 2.1
|> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
^^^^^^^^^^ ^^^^^^^^^^^^^^
I think so. yes.
|> 8) Have you had any trouble with the array software (managing disks, hot
|> sparing, RAID "protection")?
We did encounter a bug, which Veritas promises to work on. Don't remember
the details.
|> 9) How do you use your SSA's (database, NFS service, other)? How is
|> performance?
NFS service. The users (software developers) don't complain too much
about performance.
|> 10) How satisfied are you with your SSA's? Will you keep them? Will you
|> buy more? Or will you buy an Auspex?
We just bought another 60Mb SSA at the beginning of this year (making the
3rd in the largest system). If we need more space, we'll probably buy another
server, not because we aren't happy with the SSA, but because I feel that
180G is enough for any single system.
From: dtomo@scg8a.es.hac.com (Dave T)
Date: Thu, 21 Mar 1996 21:01:29 -0800 (PST)
Hi Margarita!
> What I want to know from you is:
>
> 1) How many arrays are you running? For how long? What models? How many
> disks per array? What size?
a. 4 (that I am directly responsible for, there are others)
b. one year
c. 101
d. 30
e. 1GB
> 2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
a. single
b. SC1000
> 3) Do you use uninterruptible power supplies?
a. yes (I would say a UPS is essential for using SSA's with DB's)
> 4) Have you had hardware problems? If so, what kinds?
a. yes
b. So far (knock wood) sick disks (usual SCSI errors).
> 5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
> Veritas Volume Manger or Solstice DiskSuite? What RAID levels
> (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
> (/dev/vx/dsk/dg/rawvol)?
a. Not sure what you are asking, but each is configured as
one big volume.
b. No.
c. Veritas VM
d. RAID 0 (this was the Sun SE's advice)
> 6) What version of Solaris OS and Array software are you running?
a. Solaris 2.4 with recommended and required patches. VM 2.1
> 7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
a. Not yet (haven't sent for them yet).
b. No.
> 8) Have you had any trouble with the array software (managing disks, hot
> sparing, RAID "protection")?
a. No
> 9) How do you use your SSA's (database, NFS service, other)? How is
> performance?
a. Relational DB data.
b. Twice what we saw with the disk towers (12 SCSI buses with 4
535MB narrow SCSI drives on each bus). The DBA's were happy.
> 10) How satisfied are you with your SSA's? Will you keep them? Will you
> buy more? Or will you buy an Auspex?
a. We were dis-satisfied initially due to the high failure rate
of disks until we discovered that our failures were to due to
heat, additional cooling has lowered the failure rate.
b. At this point we are planning on keeping them due to the cost
of replacing them and the experience base we would have to
rebuild if we used another RAID vendor.
c. We are planning to buy additional drives to allow us to run
RAID 0+1 in hopes of increasing our reliability.
d. We have some Auspex's for NFS, I use the SSA's on our DB engines.
BTW, some of my friends have highly recommended Network Appliance NFS servers
with extreme praise. They claimed they were inexpensive (relatively speaking),
extremely reliable, easy to service, required little maintenance, and were very
easy to expand. If you are interested in NFS servers you might want to take a
look at the Network Appliance NFS servers.
Thanks, I look forward to reading your summary. I'm always curious to see
what other sites are using and I am very interested in hearing what sites
running RAID 0+1 are experiencing.
Bye!
----------------------------------------------------------------
DISCLAIMER: I am responsible for any opinions expressed above,
which are my own and not those of the Hughes Aircraft Company
----------------------------------------------------------------
From: andrew@crls54.srm.crl.gov.bc.ca (Andrew Gallagher)
Date: Fri, 22 Mar 1996 22:22:20 GMT
Hello Margarita,
1) 1 Array. 11 Months. SSA-100. Now we are running 30x2Gb disks
(full capacity). We started with 6x1Gb disks and have worked up from
there.
2) Single Ported. It was originally hosted by a sparc 20/712 and is
now hosted by an Ultra170.
3) No. We are getting one for it this year.
4) Lots of hardware problems. Sun replaced the fibre-channel cards 3
times, and was going to do it a fourth time, when I insisted they look
at the problem more closely. We were getting the ONLINE/OFFLINE
messages appearing every two or three months (causing me to phone Sun,
who would just send us new fibre channel cards). The fourth time,
they replaced the controller on the array, the Sbus card in the ultra,
both fibre channel cards and the fibre cable between the host and the
SSA. When we had upgraded to 30x2Gb disks, they had replaced the
power supply so we pretty much have had the whole array replaced,
except for the plastic case, ribbon cables, connectors and trays. Two
disks have gone bad in the array over the last 11 months. We have had
50 disks in-and-out of the array (originally 18x1Gb, now 30x2Gb, 2
bad), so our failure rate is roughly 4%.
5) I am running the disks under the Veritas Volume Manager (2.1)
6) I have run the array under Solaris 2.3, 2.4 and 2.5
7) I have fast_writes enabled. I have 7 disks in a RAID-5 array and I
am using the DRL.
8) I have not had trouble with the array software except when I
upgraded to Veritas version 2.1 under Solaris 2.3. This caused an
enormous amount of problems including kernel panics that was only
solved by upgrading the OS to Solaris 2.4.
9) We use the array as a centralized data storage area for our 3 sun
boxes. It is running a few small (under 3Gb) Oracle databases from
the Ultra, and projects on the other boxes NFS mount volumes. I have
no complaints about performance but, I haven't done any benchmarking
on the array. We have the three boxes hooked up on an ATM network,
over which we do the NFS sharing, and users on the non-SSA-host
machines haven't indicated to me that there are any performance
problems.
10) I am not very satisfied with the SSA over the past year due to
all the hardware problems but, I haven't had a problem since all the
stuff was replaced about 5 weeks ago. I would not recommend them to
anyone. I would prefer to go to StorageWorks cabinets or some other
solution for our future needs. I don't know what an Auspex is, but
now I'm going to look it up!
Andrew Gallagher
From: gedau@mim.com.au (George Dau)
Date: Sat, 23 Mar 1996 23:05:46 GMT
Organization: Mount Isa Mines (Isa Operations)
]What I want to know from you is:
]
]1) How many arrays are you running? For how long? What models? How many
] disks per array? What size?
Two. one has 12 x 2Gig, the other 24 X 2Gig Both 102s, one running for about a
year, the other for 6 months.
]2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Both single ported to 1000e machines.
]3) Do you use uninterruptible power supplies?
Yes.
]4) Have you had hardware problems? If so, what kinds?
2 disks failed last week in the newer array.
]5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
] Veritas Volume Manger or Solstice DiskSuite? What RAID levels
] (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
] (/dev/vx/dsk/dg/rawvol)?
Veritas VM with a mixture of all types of volumes except Raw disks.
]6) What version of Solaris OS and Array software are you running?
Solaris 2.4, Veritas 2.1 with patches and a firmware upgrade.
]7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Yes, Yes.
]8) Have you had any trouble with the array software (managing disks, hot
] sparing, RAID "protection")?
The hot sparing didn't work both times. This is a major concern.
]9) How do you use your SSA's (database, NFS service, other)? How is
] performance?
Oracle database + files. Oracle 10Gig database with 50/50 read write on
RAID5 gives 60ms write times. This fast enough for us.
]10) How satisfied are you with your SSA's? Will you keep them? Will you
] buy more? Or will you buy an Auspex?
The hot sparing not working has caused us great concern. We will keep them
because the money has run out, but we bought them for the hot spare
capability and it doesn't work.
]
]Please copy your replies to me.
OK.
]
]Thanks for your time,
15Mins @ $70/hr = $17:50. Australian currency only.
]
]Margarita Suarez
]Columbia University
]Academic Information Systems
]UNIX Systems Group
From: sys013@abdn.ac.uk (G.ROBERTSON)
Date: Mon, 25 Mar 1996 16:25:39 GMT
What I want to know from you is:
1) How many arrays are you running? For how long? What models? How many
disks per array? What size?
One array Model 100 w. 9 Seagate Barracudas
2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
1 array shared by 2 SS1000
3) Do you use uninterruptible power supplies?
no
4) Have you had hardware problems? If so, what kinds?
Lots.
Replaced most FC/S cards. Replaced controller board twice. replaced 1 disk.
Most errors were
ON LINE/OFF LINE or Transport Rejected (Error 2)
5) How are you running your arrays?
Raw disks (/dev/dsk/cNtNdNsN)?
6) What version of Solaris OS and Array software are you running?
Software 1.0, Firmware 3.4
7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Neither
8) Have you had any trouble with the array software (managing disks, hot
sparing, RAID "protection")?
No
9) How do you use your SSA's (database, NFS service, other)? How is
performance?
general Unix filesystems with NFS.
Performance is great when it works.
10) How satisfied are you with your SSA's? Will you keep them? Will you
buy more? Or will you buy an Auspex?
Unhappy. After the 1st controlled board replacement it worked for about 4 weeks.
Then it started failing. We've just replaced it again, and again its OK FOR THE
MOMENT!!!
We use a Sun stacking unit with built-in power distribution (not UPS). This has
tripped out during supply fluctuations when other Sun H/W stayed up.
Hope this helps,
Gordon.
-------------------------------------------------------------------------
Gordon Robertson, Head of Systems, Aberdeen University Computing Centre
Tel +44(0)224 273340
E-Mail : g.robertson@abdn.ac.uk
--------------------------------------------------------------------------
From: ben@larbs5.sbi.com (Ben Evans)
Date: Tue, 26 Mar 1996 14:35:32 GMT
Hi There, sorry to hear about your SSA problems. WRT
to your survey:
1) How many arrays are you running? For how long? What models? How many
disks per array? What size?
I have two SSAs fully populated with 2.1G Barracudas,
originally they were both 102s, and I have been running
them for about five months.
2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Both are single-ported onto SS20s in either a 512 or 712 config,
3) Do you use uninterruptible power supplies?
Everything here runs on UPS.
4) Have you had hardware problems? If so, what kinds?
Arrrrg! One of my storage arrays has performed with no problems
at all so far (touch wood!). The other was a total bastard, and
eventually I insisted Sun take the whole thing away lock, stock
and barrel and provide a replacement. The following sad story records
the details.
When I rolled the second array into production, it was only
a few days before I had a disk failure with a "track following" error.
Irritatingly Veritas did not kick in the hot spare. I replaced the disk
and rebuilt the plex and re-attached it. Within a few days, another
disk on the same controller failed with the same problem. Endless
patches, firmware upgrades, new trays, disk firmware upgrades, new
disks etc etc etc were applied. All to no avail, the SSA disks kept
having "track following" errors, and even though eventually Veritas
could be persuaded to cut in the hot spare when it happened, I
was still not happy that the failures were taking place.
Eventually the worst happened, I had a "track following" error, and
the system cut in the hot spare and started rebuilding the plex.
While this was happening I had a track following error on the
remaining good plex at 9 PM on a Saturday night. Naturally this
completely shafted the file system on the SSA, and I had to recover
my system from tape, against the clock before we opened for business
on a monday morning. It took seventeen hours to recover the data
on some spare space on the other SSA and we were fully operational
just in time for start of business on monday morning.
Needless to say I flamed everyone at Sun until they were nice and
crispy. I insisted that they replace the whole unit and they supplied
a replacement SSA 112 at no charge. Sadly it took them three
weeks to get one for me.
Apart from that I have had one actual, genuine, honest, regular disk
failure which seems about par for the course, given the MTBF on Barracuda
drives.
5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
Veritas Volume Manger or Solstice DiskSuite? What RAID levels
(concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
(/dev/vx/dsk/dg/rawvol)?
The arrays are configured as RAID 1+0 striped mirrors, with
14 disks in each mirror, one hot spare and one DRL
disk using Veritas. They each are used to provide storage for
NFS servers.
6) What version of Solaris OS and Array software are you running?
I now have a Solaris 24 server running Veritas 2.1 and a 2.5 server
running Veritas 2.1.1
7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
I have fast_writes enabled and use DRL for the 1+0 RAID volumes
9) How do you use your SSA's (database, NFS service, other)? How is
performance?
u
I use them for NFS services, and coupled with fast systems and
100Mb networks, they perform well. I have measured more than 4 Mbytes
per second sustained writes onto an NFS filesystem, which is not bad
IMHO.
10) How satisfied are you with your SSA's? Will you keep them? Will you
buy more? Or will you buy an Auspex?
I am now satisfied, but I *really* suffered, and I had to flame
Sun from top to bottom to get what I wanted. I will probably buy more
SSA in future, because I now have the configuration I require
fully de-bugged.
CHeers
Ben
################################################################
# Ben Evans, #
# Salomon Brothers International #
# Victoria Plaza, 111 Buckingham Palace Road, London, England. #
################################################################
From: Scott.Briand@NSPower.NS.Ca (Scott P. Briand 428-6140)
Date: Tue, 26 Mar 1996 13:55:10 -0400 (AST)
Margarita;
--------> We have recently awarded our Oracle Financials hardware
contract to SUN. Although we
--------> have had SUN workstations around for years, this is our first
production application on SUN.
--------> I am new to this SUN/SSA environment and am concearned about
my/your results to-date.
--------> I am responding with the appropriate info for you.
--------> COULD YOU PLEASE COPY ALL RESPONSES TO ME FOR MY INFO.
1) How many arrays are you running? For how long? What models? How
many disks per array? What size?
--------> Production System - "Duraflame" with one SSA and 19 x 2gb
Seagate Barracuda disks.
--------> Development System - "Duraflame" with one SSA and 10 x 2gb
Seagate Barracuda disks.
2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
--------> All single ported.
--------> Currently connected to Ultra-1 but will be hooked to
"Duraflame" boxes when shipped/released.
3) Do you use uninterruptible power supplies?
--------> Not now but will be when staging/setup is complete
4) Have you had hardware problems? If so, what kinds?
--------> One disk failure already before the development system was
online. Replaced disk.
--------> Hot Spare did not kick in.
5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
Veritas Volume Manger or Solstice DiskSuite? What RAID levels
(concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
(/dev/vx/dsk/dg/rawvol)?
--------> 8 x 2gb striped and mirrored for data
--------> 1 x 2gb system disk mirrored
--------> 1 x 2gb Hot Spare
--------> All disks under VxVm control
6) What version of Solaris OS and Array software are you running?
--------> Solaris 2.5 VxVm 2.1
7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
--------> Fast_writes are enabled. NO DRL or RAID5 logging.
8) Have you had any trouble with the array software (managing disks, hot
sparing, RAID "protection")?
--------> HotSpare not kicking in when lost disk drive.
9) How do you use your SSA's (database, NFS service, other)? How is
performance?
--------> Oracle Financials Database/Application.
--------> Performance - Still setting up development box. No idea yet.
10) How satisfied are you with your SSA's? Will you keep them? Will
you buy more? Or will you buy an Auspex?
--------> OK. But very concerned with bad disk and Hot Spare not functioning.
Scott P. Briand
Software Analyst
Nova Scotia Power Inc.
PO Box 910 B05
Halifax, NS Canada B3J 2W5
(902) 428-6140
Scott.Briand@NSPower.ns.ca
From: job@xinit.se (Joakim B. Berglund)
Date: 26 Mar 1996 16:06:12 GMT
>]What I want to know from you is:
>]
>]1) How many arrays are you running? For how long? What models? How many
>] disks per array? What size?
6, 5 month, 102, aprox 24, 2.1 GB
>]2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
Single ported to SS1000e
>]3) Do you use uninterruptible power supplies?
Yes, dual power feeds. One SSA on each per SS1000e.
>]4) Have you had hardware problems? If so, what kinds?
Totaly 8 disks has failed.
>]5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
>] Veritas Volume Manger or Solstice DiskSuite? What RAID levels
>] (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
>] (/dev/vx/dsk/dg/rawvol)?
SDS with raw disks and filesystems (Raid0/1)
Veritas with raw disks and filesystems (Raid0/1)
>]6) What version of Solaris OS and Array software are you running?
2.4-37 and 2.5 (3.4)
>]7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Yes, no
>]8) Have you had any trouble with the array software (managing disks, hot
>] sparing, RAID "protection")?
No.
>]9) How do you use your SSA's (database, NFS service, other)? How is
>] performance?
Sybase, Informix raw partions plus filesystems. Perfomance isn't that great,
lets wait for the 112 upgrade.
>]10) How satisfied are you with your SSA's? Will you keep them? Will you
>] buy more? Or will you buy an Auspex?
Yes, yes, yes, no.
Joakim B. Berglund
From: craig@jetcity.com (Craig Nelson)
Date: Fri, 29 Mar 1996 10:39:44 GMT
>]What I want to know from you is:
>]
>]1) How many arrays are you running? For how long? What models? How many
>] disks per array? What size?
Six model 100 30gb SSA's (30x1gb disks)
One model 102 60gb SSA (30x2gb disks)
One model 102 (8x2gb disks)
>]2) Are your SSA's Single or Dual-Ported? What kinds of host processors?
All are single ported to either a SC1000, SC1000E, SC2000, or SC2000E
>]3) Do you use uninterruptible power supplies?
Yes
>]4) Have you had hardware problems? If so, what kinds?
Major problems with a bad batch of the 2gb drives in the 60gb array.
We lost 6 2gb drives in the course of 3 months.
Otherwise a few failures that aren't above the background noise when
you're dealing with over 200 separate drives...
>]5) How are you running your arrays? Raw disks (/dev/dsk/cNtNdNsN)?
>] Veritas Volume Manger or Solstice DiskSuite? What RAID levels
>] (concatenated, mirrored, striped, RAID 5), or as "managed" raw disks
>] (/dev/vx/dsk/dg/rawvol)?
Veritas VM, striped and mirrored.
>]6) What version of Solaris OS and Array software are you running?
Solaris 2.3, 2.4 and 2.5. Veritas 1.3 and 2.1
>]7) Do you use fast_writes (NVRAM cache)? Do you use DRL or RAID 5 logging?
Yes. No logging
>
>]8) Have you had any trouble with the array software (managing disks, hot
>] sparing, RAID "protection")?
Sparing doesn't work consistently. Veritas once decided a 'PLEX' was
bad after a system crach. We never could get it back and had to delete
and restore.
>]9) How do you use your SSA's (database, NFS service, other)? How is
>] performance?
Sybase raw databases and regular file systems (one is 35gb).
>]10) How satisfied are you with your SSA's? Will you keep them? Will you
>] buy more? Or will you buy an Auspex?
We have a major investment in them, but I don't feel good about buying
more. I'm keeping an open mind and waiting for the new disk technology
from Sun later this year. The throughput on the fiber channel can't be
beat (yet) and I will need to be VERY convinced of another array
technology before I start mixing the types we use...
Back to marg's Sun SPARCstorage Array Page