Listing Steps in Datagrid in order of Execution

Listing Steps in Datagrid in order of Execution

Post by Micha » Wed, 25 Jun 2003 00:21:15



Hello, I am building a program that will run a series of packages for
our Data Warehouse project.  The reason I am doing this, is we wanted
to add some additional functionality that we just couldn't get out of
DTS, such as sending a 'net send' if the package fails (we're not
using jobs), and being able to run multiple packages in sequence and
still being able to see the "datagrid" interface of each step as in EM
(something ExecutePackage task does not do).  At any rate, I have the
program going and it works fine, however, in order to fill the
datagrid, I'm just using the step descriptions as returned by the
package.steps collection, which unfortunately lists them in order of
creation, and not of workflow.  These packages are rather large and
have some complex workflow in them, and I haven't been able to figure
out a way of listing them according to there workflow.  If anyone has
done this or has some insight into it that could get me on the right
track, I would surely appreciate it.  I'm developing the program in
VB.NET.
 
 
 

Listing Steps in Datagrid in order of Execution

Post by Allan Mitchel » Wed, 25 Jun 2003 01:06:45


The only way I can see that you do it is to

1.  To find the first step look for the step with no precendence constraints
2.  Each Step has a PrecedenceConstraints collection which contains 0+
PrecedenceConstraint objects.  A PrecedenceConstraint object has a Property
of StepName which is the Step previous that is before it.

HTH

--

Allan Mitchell (Microsoft SQL Server MVP)
MCSE,MCDBA
www.SQLDTS.com
I support PASS - the definitive, global community
for SQL Server professionals - http://www.sqlpass.org


Quote:> Hello, I am building a program that will run a series of packages for
> our Data Warehouse project.  The reason I am doing this, is we wanted
> to add some additional functionality that we just couldn't get out of
> DTS, such as sending a 'net send' if the package fails (we're not
> using jobs), and being able to run multiple packages in sequence and
> still being able to see the "datagrid" interface of each step as in EM
> (something ExecutePackage task does not do).  At any rate, I have the
> program going and it works fine, however, in order to fill the
> datagrid, I'm just using the step descriptions as returned by the
> package.steps collection, which unfortunately lists them in order of
> creation, and not of workflow.  These packages are rather large and
> have some complex workflow in them, and I haven't been able to figure
> out a way of listing them according to there workflow.  If anyone has
> done this or has some insight into it that could get me on the right
> track, I would surely appreciate it.  I'm developing the program in
> VB.NET.


 
 
 

Listing Steps in Datagrid in order of Execution

Post by Darren Gree » Wed, 25 Jun 2003 01:20:37


If you wish to pre-populate the grid, then you will need to evaluate all
workflow to determine the execution order, in effect what goers on behind
the scenes by the dts execution engine.

An alternative is to use events, and just add rows to the grid for an
OnStart and update based on the other events as required.

--
Darren Green
http://www.sqldts.com


Quote:> Hello, I am building a program that will run a series of packages for
> our Data Warehouse project.  The reason I am doing this, is we wanted
> to add some additional functionality that we just couldn't get out of
> DTS, such as sending a 'net send' if the package fails (we're not
> using jobs), and being able to run multiple packages in sequence and
> still being able to see the "datagrid" interface of each step as in EM
> (something ExecutePackage task does not do).  At any rate, I have the
> program going and it works fine, however, in order to fill the
> datagrid, I'm just using the step descriptions as returned by the
> package.steps collection, which unfortunately lists them in order of
> creation, and not of workflow.  These packages are rather large and
> have some complex workflow in them, and I haven't been able to figure
> out a way of listing them according to there workflow.  If anyone has
> done this or has some insight into it that could get me on the right
> track, I would surely appreciate it.  I'm developing the program in
> VB.NET.

 
 
 

Listing Steps in Datagrid in order of Execution

Post by Micha » Wed, 25 Jun 2003 21:48:43


Yeah, I was afraid of that.  I am trying to emulate running a DTS job
in EM as much as possible (in fact the progress dialog looks exactly
like the one in EM only I've increased it's size so that you don't
have to scroll to see all the columns), so listing all the steps in
order of workflow is needed.  I was hoping maybe someone had done
something like this and may have had some code that could start me
off, but I guess I'll have to start from scratch.  I'll be sure to
post it if anyone is interested.
Thanks for your help guys.

Michael Morrison
SE2 Developer/Consultant
Daugherty Business Solutions
www.daugherty.com


> If you wish to pre-populate the grid, then you will need to evaluate all
> workflow to determine the execution order, in effect what goers on behind
> the scenes by the dts execution engine.

> An alternative is to use events, and just add rows to the grid for an
> OnStart and update based on the other events as required.

> --
> Darren Green
> http://www.sqldts.com



> > Hello, I am building a program that will run a series of packages for
> > our Data Warehouse project.  The reason I am doing this, is we wanted
> > to add some additional functionality that we just couldn't get out of
> > DTS, such as sending a 'net send' if the package fails (we're not
> > using jobs), and being able to run multiple packages in sequence and
> > still being able to see the "datagrid" interface of each step as in EM
> > (something ExecutePackage task does not do).  At any rate, I have the
> > program going and it works fine, however, in order to fill the
> > datagrid, I'm just using the step descriptions as returned by the
> > package.steps collection, which unfortunately lists them in order of
> > creation, and not of workflow.  These packages are rather large and
> > have some complex workflow in them, and I haven't been able to figure
> > out a way of listing them according to there workflow.  If anyone has
> > done this or has some insight into it that could get me on the right
> > track, I would surely appreciate it.  I'm developing the program in
> > VB.NET.

 
 
 

Listing Steps in Datagrid in order of Execution

Post by Micha » Fri, 27 Jun 2003 03:00:56


Ok, after some trial and error, I have figured out how to do it, well
sort of.  In EM, when a package is run, it lists the steps in order of
execution and also by grouping.  For example, if Task1 was linked to
Task2 and Task3 and they were linked to Task4 and Task5 respectively,
which in turn were linked to Task6 and Task7 respectively, which were
both linked to Task8, the listing of steps would look something like
the following:
Task1
Task2
Task4
Task6
Task3
Task5
Task7
Task8

Well, I couldn't get it to do exactly that, but what I've come up with
actually looks better to us as it minimizes the scrolling needed to
see the current running tasks (we have very large packages).  So my
solution lists the steps in workflow order and then by order of
creation.  This may not necessarily mean that each subsequent task
will run one after the other, DTS still will optimize execution as it
sees fit, and there is really no way to arrange this according to true
execution unless you only list the steps as they are run as Darren
noted.  So with the above example, my code will list the steps as
follows:
Task1
Task2
Task3
Task4
Task5
Task6
Task7
Task8

So here goes.  This may not be the best way to implement this, so if
anyone can take this and further refine it to be more concise or
quicker, let me know please
(michael.morrison@(removethis)daugherty.com).

'Assuming you have already created an instance of a package object
called pkg
            pkg.LoadFromStorageFile(pkgPath, "", "", "", pkgName,
Nothing)

            Dim stp As DTS.Step2
            Dim prc As DTS.PrecedenceConstraint
            Dim prcName As String
            Dim stpsCol As New Collection
            Dim stpSort As New Collection
            Dim stpsArr As ArrayList
            Dim stpSorArr As ArrayList
            Dim InstpSort As Boolean
            Dim x As Integer = 0

            'Fill the collection with all the steps
            'This will enter them in order of creation
            For Each stp In pkg.Steps
                stpsCol.Add(stp, stp.Name)
            Next
            'Use ArrayLists Adapter method and send it collection
            'This will give us a copy of the collection
            'Not sure if this has to be an ArrayList, but my earlier
attempts
            'centered on using them, so I kept them and it all works,
so....
            stpsArr = stpsArr.Adapter(stpsCol)
            stpsArr.TrimToSize()

            'Loop through the ArrayList and for each stp that doesn't
have
            'a PrecedenceConstraint add it to the second collection
(the one
            'that will be sorted) and remove from first collection
            Do
                stp = stpsArr.Item(x)
                If stp.PrecedenceConstraints.Count = 0 Then
                    stpSort.Add(stp, stp.Name)
                    stpsCol.Remove(stp.Name)
                End If
                x += 1
            Loop Until x > stpsCol.Count - 1
            stpSorArr = stpSorArr.Adapter(stpSort)
            stpSorArr.TrimToSize()

            'Now loop through first collection again and for each step
check
            'that each of it's PrecedenceConstraints StepName steps is
already
            'in collection two.  If it is it will set variable
InstpSort to
            'true, otherwise it will set it to false and exit that For
loop
            x = 0
            Do
                If x >= stpsCol.Count Then
                    x = 0
                End If
                stp = stpsArr.Item(x)
                For Each prc In stp.PrecedenceConstraints
                    prcName = prc.StepName
                    InstpSort = stpSorArr.Contains(pkg.Steps.Item(prcName))
                    If InstpSort = False Then
                        Exit For
                    End If
                Next

                'For each stp, after all the PrecedenceConstraints
have been
                'checked (or it returned a false), if InstpSort is
true, it
                'will go ahead and move it into second collection and
remove
                'it from the first collection
                If InstpSort = True Then
                    stpSort.Add(stp, stp.Name)
                    stpsCol.Remove(stp.Name)
                End If

                'If InstpSort was false, it will skip that step and go
on with
                'the loop testing the next step.  This will ensure
that a step
                'is not added to the second collection before any and
all of
                'its parent steps have been added
                x += 1
                stpSorArr = stpSorArr.Adapter(stpSort)
                stpSorArr.TrimToSize()
            Loop Until stpsCol.Count = 0

            'Now that the sorted collection exists, we fill the
datagrid with
            'the collection item by item. The Tag property here is
used for
            'identification later so that I can set the icon to what
the task
            'current status is you may or may not need it.  Again, my
grid has
            'has the same columns as the EM version, so column 1 is
the icon
            'followed by Step Description, Status, StartTime,
FinishTime, and
            'ExecutionTime.
            x = 0
            For Each stp In stpSort
                lstVwSteps.Items.Add("", 0)
                lstVwSteps.Items(x).Tag = stp.Name
                lstVwSteps.Items(x).SubItems.Add(stp.Description)
                lstVwSteps.Items(x).SubItems.Add("Waiting")
                lstVwSteps.Items(x).SubItems.Add("")
                lstVwSteps.Items(x).SubItems.Add("")
                lstVwSteps.Items(x).SubItems.Add("")
                x += 1
            Next

'The following commented code will list the steps just as pkg.steps
collection
'lists them in order of creation, regardless of workflow order
'This is not needed, but thought I'd include it in case somebody
wanted it
            'For Each stp In pkg.Steps
            '    lstVwSteps.Items.Add("", 0)
            '    lstVwSteps.Items(x).Tag = stp.Name
            '    lstVwSteps.Items(x).SubItems.Add(stp.Description)
            '    lstVwSteps.Items(x).SubItems.Add("Waiting")
            '    lstVwSteps.Items(x).SubItems.Add("")
            '    lstVwSteps.Items(x).SubItems.Add("")
            '    lstVwSteps.Items(x).SubItems.Add("")
            '    x += 1
            'Next

Hope this helps the next guy having the same problem.
Michael

shaggy13...@hotmail.com (Michael) wrote in message <news:c78a6350.0306230721.14ff9f44@posting.google.com>...
> Hello, I am building a program that will run a series of packages for
> our Data Warehouse project.  The reason I am doing this, is we wanted
> to add some additional functionality that we just couldn't get out of
> DTS, such as sending a 'net send' if the package fails (we're not
> using jobs), and being able to run multiple packages in sequence and
> still being able to see the "datagrid" interface of each step as in EM
> (something ExecutePackage task does not do).  At any rate, I have the
> program going and it works fine, however, in order to fill the
> datagrid, I'm just using the step descriptions as returned by the
> package.steps collection, which unfortunately lists them in order of
> creation, and not of workflow.  These packages are rather large and
> have some complex workflow in them, and I haven't been able to figure
> out a way of listing them according to there workflow.  If anyone has
> done this or has some insight into it that could get me on the right
> track, I would surely appreciate it.  I'm developing the program in
> VB.NET.

 
 
 

1. VB and listing package steps in order of execution (workflow)

Sorry for the repost, but after posting I realized the subject may
have been a little narrowing:

Hello, I am building a program that will run a series of packages for
our Data Warehouse project.  The reason I am doing this, is we wanted
to add some additional functionality that we just couldn't get out of
DTS, such as sending a 'net send' if the package fails (we're not
using jobs), and being able to run multiple packages in sequence and
still being able to see the "datagrid" interface of each step as in EM
(something ExecutePackage task does not do).  At any rate, I have the
program going and it works fine, however, in order to fill the
datagrid, I'm just using the step descriptions as returned by the
package.steps collection, which unfortunately lists them in order of
creation, and not of workflow.  These packages are rather large and
have some complex workflow in them, and I haven't been able to figure
out a way of listing them according to there workflow.  If anyone has
done this or has some insight into it that could get me on the right
track, I would surely appreciate it.  I'm developing the program in
VB.NET.

2. building a data warehouse

3. Step in an ordered list of rows

4. Long Island - Powerbuilder development w/Sybase or Oracle

5. DTS Execution Sequence, check for existence of step by name

6. HELP - Paradox 5 changeValue

7. DTS Workflow and Conditional Step Execution

8. ODBC error 2103 with ASP should be 3126

9. Conditional step execution

10. Scheduled Run and Manual Step Execution problems

11. Parallel execution of different steps in DTS

12. Workflow - step execution / synchronization

13. Where to look for the log to find the execution times of steps in dts packages