OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by Sue Skonets » Thu, 03 Jul 2003 03:45:56



There was a word schedule attached but I know how you folks feel about
that.

sue

----------------------------------------------------

Host-Based MiniMerge Schedule Now Available

The OpenVMS Engineering and Business Management group would like to
announce the schedule for Host-Based MiniMerge capability.  As most of
you know, the teams have been spending virtually all of their time
over the past 3 months doing design, "proof of concept" and schedule
development for this important feature.  We are now ready to make this
information available to the OpenVMS user community.

Overview - The purpose of the Host-Based MiniMerge feature is to allow
a "minimum" MERGE of only shadow set changes (rather than the
currently-required "full" MERGE) when a member of an OpenVMS cluster
is removed and re-introduced into a cluster.  A similar capability
exists today for CI-based storage, and this new "host-based"
implementation will allow a MiniMerge to occur on any type of storage
that is utilized in a Host-Based Volume Shadowing shadow set.  This
implementation utilizes the "write bitmap" technology in use with
MiniCopy today, in order to keep track of shadow set data changes
while the cluster member is out of the cluster.  When the member is
brought back in, shadow sets will be able to MERGE only the changed
areas of the disk, thus resulting in a much faster restoration of
shadow sets into the cluster.

 
 
 

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by Rob You » Thu, 03 Jul 2003 04:25:07



Quote:

> Overview - The purpose of the Host-Based MiniMerge feature is to allow
> a "minimum" MERGE of only shadow set changes (rather than the
> currently-required "full" MERGE) when a member of an OpenVMS cluster
> is removed and re-introduced into a cluster.  A similar capability
> exists today for CI-based storage, and this new "host-based"
> implementation will allow a MiniMerge to occur on any type of storage
> that is utilized in a Host-Based Volume Shadowing shadow set.  This
> implementation utilizes the "write bitmap" technology in use with
> MiniCopy today, in order to keep track of shadow set data changes
> while the cluster member is out of the cluster.  When the member is
> brought back in, shadow sets will be able to MERGE only the changed
> areas of the disk, thus resulting in a much faster restoration of
> shadow sets into the cluster.

        This paragraph reads that the merge is dependent on the
        crashed member returning to do the merge.  What if the
        crashed node is gone for several hours/days?  Maybe I'm
        misreading that paragraph.  

        A scenario would be a node crashes month-end Monday (hardware failure)
        and isn't back until mid-afternoon...
        A host-based mini-merge could be a high priority in that case,
        and hopefully isn't dependent on a returning node.

                                Rob

 
 
 

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by Rob Broo » Thu, 03 Jul 2003 05:31:40




>> When the member is
>> brought back in, shadow sets will be able to MERGE only the changed
>> areas of the disk, thus resulting in a much faster restoration of
>> shadow sets into the cluster.

>    This paragraph reads that the merge is dependent on the
>    crashed member returning to do the merge.  What if the
>    crashed node is gone for several hours/days?  Maybe I'm
>    misreading that paragraph.  

>    A scenario would be a node crashes month-end Monday (hardware failure)
>    and isn't back until mid-afternoon...
>    A host-based mini-merge could be a high priority in that case,
>    and hopefully isn't dependent on a returning node.

The minimerge is not at all dependent upon the return of the crashing node(s).
It *is* dependent upon at least one node that was mastering the bitmap to
survive.  If all the nodes that had bitmaps for a given shadow set crash,
then a full merge is required.

It appears that there is some confusion between mini-merge and mini-copy,
given that last sentence.

--

Rob Brooks    VMS Engineering -- I/O Exec Group     brooks!cuebid.zko.dec.com

 
 
 

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by Rob You » Thu, 03 Jul 2003 05:46:48





>>> When the member is
>>> brought back in, shadow sets will be able to MERGE only the changed
>>> areas of the disk, thus resulting in a much faster restoration of
>>> shadow sets into the cluster.

>>        This paragraph reads that the merge is dependent on the
>>        crashed member returning to do the merge.  What if the
>>        crashed node is gone for several hours/days?  Maybe I'm
>>        misreading that paragraph.  

>>        A scenario would be a node crashes month-end Monday (hardware failure)
>>        and isn't back until mid-afternoon...
>>        A host-based mini-merge could be a high priority in that case,
>>        and hopefully isn't dependent on a returning node.

> The minimerge is not at all dependent upon the return of the crashing node(s).
> It *is* dependent upon at least one node that was mastering the bitmap to
> survive.  If all the nodes that had bitmaps for a given shadow set crash,
> then a full merge is required.

> It appears that there is some confusion between mini-merge and mini-copy,
> given that last sentence.

        No.  Part of it is -my- trouble with the paragraph.  Maybe I'm
        alone:

 This
implementation utilizes the "write bitmap" technology in use with
MiniCopy today, in order to keep track of shadow set data changes
while the cluster member is out of the cluster.  When the member is
brought back in, shadow sets will be able to MERGE only the changed    
areas of the disk, thus resulting in a much faster restoration of  
shadow sets into the cluster.

        "Member" refers to two different things, real close to each
        other lending itself to *appear* as if things work on
        return of a node:

        "keep track of shadow set data changes while the -cluster- member is
        out of the cluster. When the [-shadow-] member is brought back in,
        shadow sets will be able to MERGE only the changed"

        As noted, maybe better would be a slight tweek:

        "When the shadow member is brought back in"

        The ambiguity threw me.  I thought maybe a new method required
        a tweek of how mini-merge would be taking place.  Yes , after this
        long I know the difference between minicopy/minimerge having been
        admittedly confused at one time.  And Keith Parris pointed out my
        confusion a few years back (IIRC).  And I certainly don't want to get
        lost again.

        And to get really nitpicky, shadow sets don't merge anything.
        Nodes do the merging of shadow sets.

                                Rob

 
 
 

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by Keith Parr » Thu, 03 Jul 2003 06:22:40



> There was a word schedule attached but I know how you folks feel about
> that.

I often have to 'copy' the text from a Word window and 'paste' it into
a Notepad window to make it suitable for use for character-cell
oriented applications.

The important dates from the schedule (subject to change, of course),
are:
   External Field Test        12/31/03 to 3/24/04
   Production Kit available    3/25/04
Initial availability is slated to be on 7.3-1, with 7.3-2 support
shortly thereafter.

 
 
 

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by Keith Parr » Thu, 03 Jul 2003 07:10:15



> This
> implementation utilizes the "write bitmap" technology in use with
> MiniCopy today, in order to keep track of shadow set data changes
> while the cluster member is out of the cluster.  When the member is
> brought back in, shadow sets will be able to MERGE only the changed
> areas of the disk, thus resulting in a much faster restoration of
> shadow sets into the cluster.

This paragraph is broken. (And Rob, it was very kind of you to give us
the benefit of the doubt, but I think it's impossible to parse this in
any way that makes it correct.)   I think what they meant to say was
something like this:

"This implementation utilizes the "write bitmap" technology used for
MiniCopy functionality today, to keep track of shadow set data changes
while a shadowset member is temporarily removed from a shadowset.
With MiniCopy, when a shadowset member is brought back in, Shadowing
is able to copy only the changed areas of the disk, thus resulting in
a much faster restoration of the shadow set member into the shadowset.
 By using similar "write bitmap" technology to track the changes being
made by each node in a cluster to the members of a shadowset, then at
the time a node fails and leaves the cluster, it is possible to
identify those specific areas on the shadowset that were being written
to by the node at the time it failed, and thus where the failure might
have resulted in data ending up different on different members of the
shadowset, and as a result Shadowing can merge just those areas which
were in the process of being modified at the time the node failed (in
a MiniMerge operation), rather than having to scan the entire disk
looking for potential discrepancies caused by the node's failure (in a
Full Merge operation)."

 
 
 

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by David J. Dachter » Thu, 03 Jul 2003 10:51:57



> There was a word schedule attached but I know how you folks feel about
> that.

Your consideration is appreciated. VMS management could take an example
from you.

--
David J. Dachtera
dba DJE Systems
http://www.djesys.com/

Unofficial Affordable OpenVMS Home Page:
http://www.djesys.com/vms/soho/

 
 
 

OpenVMS Host-Based MiniMerge Statement and Schedule for Customers

Post by norm.raph.. » Thu, 03 Jul 2003 22:16:38


And your point is...? [Now that wasn't so hard.]

Host-Based Minimerge Schedule as of 6/30/03

Task Name                               Start           Completion
---------                               -----           ----------
HBMM Features (code and debug)
  Minimerge from a bitmap (essentials)  06/20/2003      08/15/2003
  Bitmap Reset                          06/19/2003      09/25/2003
  Merge Priority                        06/24/2003      09/25/2003
  Master Bitmap Policy                  06/19/2003      10/01/2003
HBMM code and debug phase complete      06/17/2003      10/20/2003
Performance characterization            06/17/2003      10/09/2003
User documentation                      07/14/2003      11/20/2003
Internal testing and Qualification      10/10/2003      12/29/2003
External Field Test                     12/31/2003      03/24/2004
Production Kit available                03/25/2004




cc:

Subject:    OpenVMS Host-Based MiniMerge Statement and Schedule for
       Customers

There was a word schedule attached but I know how you folks feel about
that.

sue

----------------------------------------------------

Host-Based MiniMerge Schedule Now Available

The OpenVMS Engineering and Business Management group would like to
announce the schedule for Host-Based MiniMerge capability.  As most of
you know, the teams have been spending virtually all of their time
over the past 3 months doing design, "proof of concept" and schedule
development for this important feature.  We are now ready to make this
information available to the OpenVMS user community.

Overview - The purpose of the Host-Based MiniMerge feature is to allow
a "minimum" MERGE of only shadow set changes (rather than the
currently-required "full" MERGE) when a member of an OpenVMS cluster
is removed and re-introduced into a cluster.  A similar capability
exists today for CI-based storage, and this new "host-based"
implementation will allow a MiniMerge to occur on any type of storage
that is utilized in a Host-Based Volume Shadowing shadow set.  This
implementation utilizes the "write bitmap" technology in use with
MiniCopy today, in order to keep track of shadow set data changes
while the cluster member is out of the cluster.  When the member is
brought back in, shadow sets will be able to MERGE only the changed
areas of the disk, thus resulting in a much faster restoration of
shadow sets into the cluster.