State Database and State Database Replicas problem?

State Database and State Database Replicas problem?

Post by Pavlos Parissi » Wed, 19 Mar 2003 19:11:37



Hello all,
I am trying to figure out why when I created 4 extra state databases on
the second disk the metadb is saying that they are marked with "r".Which
means r - replica does not have device relocation information.

Here it is my senario. I have a V480 server with 2 disks and solaris 9
12/02 installed in the first disk.
I created 2 partitions in each disks slice6 and slice 7.
I create the meta database like this
metadb -a -f -c2 c1t0d0s6 c1t0d0s7
metadb -a -c2 c1t1d0s6 c1t1d0s7
and when I run metadb -i I see that in the state databases for the
second disk has the flug "r",all states has the flug "u".

Is this normal?

Regards,
Pavlos

 
 
 

State Database and State Database Replicas problem?

Post by Edmond van A » Wed, 19 Mar 2003 22:04:52


Quote:> I am trying to figure out why when I created 4 extra state databases on
> the second disk the metadb is saying that they are marked with "r".Which
> means r - replica does not have device relocation information.

Try restricting yourself to only one state database slice on every disk.
Please note that having only two disks will render a system un-bootable
automatically because Disksuite does not have a majority in consistent state
databases. This is fixable and you still have a mirror, the system just
doesn't boot by itself.

Quote:> Here it is my senario. I have a V480 server with 2 disks and solaris 9
> 12/02 installed in the first disk.
> I created 2 partitions in each disks slice6 and slice 7.
> I create the meta database like this
> metadb -a -f -c2 c1t0d0s6 c1t0d0s7
> metadb -a -c2 c1t1d0s6 c1t1d0s7
> and when I run metadb -i I see that in the state databases for the
> second disk has the flug "r",all states has the flug "u".

Hope this helps.

Cheers,

Edmond

 
 
 

State Database and State Database Replicas problem?

Post by Edmond van A » Wed, 19 Mar 2003 22:09:17




Quote:

> > I am trying to figure out why when I created 4 extra state databases on
> > the second disk the metadb is saying that they are marked with "r".Which
> > means r - replica does not have device relocation information.

> Try restricting yourself to only one state database slice on every disk.
> Please note that having only two disks will render a system un-bootable
> automatically because Disksuite does not have a majority in consistent
state
> databases. This is fixable and you still have a mirror, the system just
> doesn't boot by itself.

Ofcourse, only in case one of the disks breaks.
 
 
 

State Database and State Database Replicas problem?

Post by Darren Dunha » Thu, 20 Mar 2003 01:41:36



Quote:>> I am trying to figure out why when I created 4 extra state databases on
>> the second disk the metadb is saying that they are marked with "r".Which
>> means r - replica does not have device relocation information.
> Try restricting yourself to only one state database slice on every
> disk.

That's counter to Sun's recommendations.  For only 2 disks in a system,
they recommend 2 replicas on each disk.  This follows from their
recommendation of a minimum of 3 replicas, plus a requirement that
neither drive have a greater number of replicas.

Quote:> Please note that having only two disks will render a system un-bootable
> automatically because Disksuite does not have a majority in consistent state
> databases. This is fixable and you still have a mirror, the system just
> doesn't boot by itself.

Only if you're not running 4.2.1 or you haven't added
set md:mirrored_root_flag=1 in /etc/system.

--

Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

 
 
 

State Database and State Database Replicas problem?

Post by Philip Bro » Thu, 20 Mar 2003 11:00:53




>>> I am trying to figure out why when I created 4 extra state databases on
>>> the second disk the metadb is saying that they are marked with "r".Which
>>> means r - replica does not have device relocation information.

>> Try restricting yourself to only one state database slice on every
>> disk.

>That's counter to Sun's recommendations.

Now try to give some justification for sun's documented recommendations.

Quote:>  For only 2 disks in a system,
>they recommend 2 replicas on each disk.  This follows from their
>recommendation of a minimum of 3 replicas, plus a requirement that
>neither drive have a greater number of replicas.

The "three replicas" really was from the requirement of having
a 50%+1 quorum of state db's. Which usually comes down to,
"3 disks, on 3 independant controllers".
I think some bonehead tech writer then oversimplified it to
"you need 3 state databases", and then took a REALLY bad logical leap
to, "you should always use '-c3' when creating metadbs on a disk,
  because 'you always need 3."

When reduced to that level, the 'you always need 3' mantra is fundamentally
flawed, and should have been removed from the SDS documentation in its
current incarnation a long time ago.

The sun documentation is usually, but NOT always, right.
Another example that came up recently at the office, was when sun initially
released the documentation on how to set up IP Multi-pathing, they
recommended setting it up BACKWARDS:  having the virtual, floating IP
address on qfe0, and the "fixed", aka non-floating IP on qfe0:1

They have since revised the documentation to have the virtual IP,
on the virtual interface. (what a concept!)

--
  http://www.blastwave.org/ for solaris pre-packaged binaries with pkg-get
    Organized by the author of pkg-get
[Trim the no-bots from my address to reply to me by email!]

                            http://www.spamlaws.com/state/ca1.html

 
 
 

State Database and State Database Replicas problem?

Post by Pavlos Parissi » Thu, 20 Mar 2003 15:26:38




>>>I am trying to figure out why when I created 4 extra state databases on
>>>the second disk the metadb is saying that they are marked with "r".Which
>>>means r - replica does not have device relocation information.

>>Try restricting yourself to only one state database slice on every
>>disk.

> That's counter to Sun's recommendations.  For only 2 disks in a system,
> they recommend 2 replicas on each disk.  This follows from their
> recommendation of a minimum of 3 replicas, plus a requirement that
> neither drive have a greater number of replicas.

>>Please note that having only two disks will render a system un-bootable
>>automatically because Disksuite does not have a majority in consistent state
>>databases. This is fixable and you still have a mirror, the system just
>>doesn't boot by itself.

it boots but not int runlevel 3 only in 1.
When you are booting you get a message to delete "metadb -d " the
databases you had in the bad disk and reset in run level 3.
I did this and it worked.

Quote:

> Only if you're not running 4.2.1 or you haven't added
> set md:mirrored_root_flag=1 in /etc/system.

what is the meaning of this ?

Pavlos

 
 
 

State Database and State Database Replicas problem?

Post by Darren Dunha » Thu, 20 Mar 2003 16:27:48



>> set md:mirrored_root_flag=1 in /etc/system.
> what is the meaning of this ?

If you are running a 2 disk system, and cannot place replicas on at
least 3 disks, then this line allows the system to boot without a true
majority.  

Note that this is potentially dangerous, because you can get a
"split-brain" condition.  If a temporary failure causes half the
replicas to go offline, a change is made, then the offline condition
switches to the other replicas, you can lose those changes, causing
corruption.

Fortunately, that situation is rare.  By using that flag, you're
allowing for the possibility, but you can also boot unattended with one
disk dead.

--

Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

 
 
 

State Database and State Database Replicas problem?

Post by Pavlos Parissi » Thu, 20 Mar 2003 19:42:23




>>>set md:mirrored_root_flag=1 in /etc/system.

>>what is the meaning of this ?

> If you are running a 2 disk system, and cannot place replicas on at
> least 3 disks, then this line allows the system to boot without a true
> majority.  

you mean to boot in run level 3 and not in 1,correct?

Quote:> Note that this is potentially dangerous, because you can get a
> "split-brain" condition.  If a temporary failure causes half the
> replicas to go offline, a change is made, then the offline condition
> switches to the other replicas, you can lose those changes, causing
> corruption.

> Fortunately, that situation is rare.  By using that flag, you're
> allowing for the possibility, but you can also boot unattended with one
> disk dead.

OK let see if i get right.
We have 2 disks and  I have under RAID-1 the  / , /opt , /var and swap.
I have created 2 partitions on each disk to put 2 replicans on each
partition.As a result I have totally 8 replicas.

Now I am going to simulate a hard failed of the SECOND disk.
metaoffice the submirrors of the failed disk ....
luxadm remove_device /dev/rdsk/c1t1d0s2
reboot
and If I used "set md:mirrored_root_flag=1 in /etc/system" the system
will boot in run level 3.
correct?

Pavlos

 
 
 

State Database and State Database Replicas problem?

Post by Darren Dunha » Fri, 21 Mar 2003 01:47:47



> you mean to boot in run level 3 and not in 1,correct?

Yes.

Quote:>> Note that this is potentially dangerous, because you can get a
>> "split-brain" condition.  If a temporary failure causes half the
>> replicas to go offline, a change is made, then the offline condition
>> switches to the other replicas, you can lose those changes, causing
>> corruption.

>> Fortunately, that situation is rare.  By using that flag, you're
>> allowing for the possibility, but you can also boot unattended with one
>> disk dead.

> OK let see if i get right.
> We have 2 disks and  I have under RAID-1 the  / , /opt , /var and swap.
> I have created 2 partitions on each disk to put 2 replicans on each
> partition.As a result I have totally 8 replicas.

Any particular reason you're putting replicas on each of those slices?
Why not dedicate a small slice and put replicas there?

Quote:> Now I am going to simulate a hard failed of the SECOND disk.
> metaoffice the submirrors of the failed disk ....
> luxadm remove_device /dev/rdsk/c1t1d0s2
> reboot
> and If I used "set md:mirrored_root_flag=1 in /etc/system" the system
> will boot in run level 3.
> correct?

If you have exatly 50% of your metadevice replicas online, then it
should allow SDS to start and go to multiuser

--

Unix System Administrator                    Taos - The SysAdmin Company
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

 
 
 

State Database and State Database Replicas problem?

Post by Pavlos Parissi » Fri, 21 Mar 2003 17:05:31




>>you mean to boot in run level 3 and not in 1,correct?

> Yes.

>>>Note that this is potentially dangerous, because you can get a
>>>"split-brain" condition.  If a temporary failure causes half the
>>>replicas to go offline, a change is made, then the offline condition

>>>switches to the other replicas, you can lose those changes, causing
>>>corruption.

>>>Fortunately, that situation is rare.  By using that flag, you're
>>>allowing for the possibility, but you can also boot unattended with one
>>>disk dead.

>>OK let see if i get right.
>>We have 2 disks and  I have under RAID-1 the  / , /opt , /var and swap.
>>I have created 2 partitions on each disk to put 2 replicans on each
>>partition.As a result I have totally 8 replicas.

> Any particular reason you're putting replicas on each of those slices?

No

Quote:> Why not dedicate a small slice and put replicas there?

I can also do that

Quote:

>>Now I am going to simulate a hard failed of the SECOND disk.
>>metaoffice the submirrors of the failed disk ....
>>luxadm remove_device /dev/rdsk/c1t1d0s2
>>reboot
>>and If I used "set md:mirrored_root_flag=1 in /etc/system" the system
>>will boot in run level 3.
>>correct?

> If you have exatly 50% of your metadevice replicas online, then it
> should allow SDS to start and go to multiuser

OKey , I am going to use this trick in /etc/system file.
When I did this simulation I noticed that after the reboot with one disk
I got a messages saying to delete the metadb which was on the failed
disk,without having the disk plugin, and reboot.The machine reboot in
runlevel 3.

As a result there are two solution for the problem rebooting in run
level 3 with having the half of metadb.

1-> Use the option you wrote in /etc/system file

2-> When the disk failed ,removed and reboot in run level 1 and delete
the metadb which was on the failed disk then reboot in 3 run level.

Am I right?

Of course if we have hot spare disk, there is no reason doing all this:)

Regards,
Pavlos

 
 
 

State Database and State Database Replicas problem?

Post by Edmond van A » Fri, 21 Mar 2003 22:18:53




> >> I am trying to figure out why when I created 4 extra state databases on
> >> the second disk the metadb is saying that they are marked with
"r".Which
> >> means r - replica does not have device relocation information.

> > Try restricting yourself to only one state database slice on every
> > disk.

> That's counter to Sun's recommendations.  For only 2 disks in a system,
> they recommend 2 replicas on each disk.  This follows from their
> recommendation of a minimum of 3 replicas, plus a requirement that
> neither drive have a greater number of replicas.

Try reading better. I mentioned one state database SLICE on every disk. The
original poster had more than one slice on the same disk.
 
 
 

1. State Database Replicas

hi ,
    I have following Harddrives ,
               c0t0d0 , c0t1d0 , c0t2d0 and c0t3d0 .

First two are 73 GB and others are 148 GB .

First disk it boot disk , i wanted to mirror this boot disk to c0t1d0 ,

 and third disk (c0t2d0) will contain data , which i wanted to mirror
to c0t3d0.

Please sugegst me how i should design State Database Replica number

Thanks
Bhushan

2. DNS Server software?

3. Solaris 9: cannot create state database replicas

4. 128MB RAM

5. SDS 4.1 Trans Metadevice question, related to state database replicas

6. Solaris 8 release names

7. Questions about DiskSuite state database replica and slices config.

8. PHP: /usr/local/lib/php.ini file ignored

9. State-Database replica goes missing

10. Solaris Volume Manager - State Database Replicas

11. state databases in shared disk set

12. run metadb against previously configured state database slices

13. pl/sql cusror state using Oracle web agent, web server & database server