Background processing after incremental dimension processing

Background processing after incremental dimension processing

Post by <da.. » Wed, 22 May 2002 15:52:35



We too have been seeing this behaviour. Is there any way
to prevent this?

Quote:>-----Original Message-----
>After Incrementally Processing a dimension we have

noticed that the AS2K
Quote:>service seems to go off and do some sort of back ground
processing of
>indexes and aggregations on any existing partitions even
though we have
>the lazy aggregation option turned off for all the cubes
in this
>database.

>Can anyone shed any light on why this may be happening
and how we can
>stop it?

>BACKGROUND
>==========
>The cube is rather large - 25 partitions, and takes up
about 15Gb when
>fully processed. When we come to process the cube each
month we do an
>incremental process on any of the dimensions that have
acquired new
>members and then we process the incoming data into a new
partition. I
>can't see why the index and aggregation files get updated
after the
>dimension processing, because at that time none of the
new members exist
>in the current partitions.

>We would like to stop this background processing from
occurring as it
>takes longer than some of our partition loads and

prevents us from being

- Show quoted text -

>able to move the new data from our staging server to our
production
>server in a timely manner.

>--
>Regards
>Darren Gosbell  MCSD MCP

>Senior Consultant
>Datum Group  (http://www.datum.com.au)
>.

 
 
 

Background processing after incremental dimension processing

Post by [MS] Hai-Wei » Wed, 22 May 2002 17:13:02


Hi ,

Do you have any changing dimensions? If it's true, the following article
might be helpful.

INF: The Impact of Using Changing Dimensions                  [sqlserver]
ID: Q303408
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q303408

This posting is provided 'AS IS' with no warranties, and confers no rights.

Regards,
Haiwei Xu

 
 
 

Background processing after incremental dimension processing

Post by Darren Gosbel » Thu, 23 May 2002 08:38:11


Thanks for you response, I think we are on to something, but it smells
like a bug to me.

We only have one dimension we have that is flagged as changing, but this
is because it is a virtual dimension. We implemented it as a virtual
instead of a normal changing dimension to avoid this type of behaviour.

This raises 2 issues.

1) As I understand it Virtual Dimensions are not supposed to have any
aggregations stored against them and hence any changes should not result
in recalculation of aggregations.

2) Last time this happened we did not process either the virtual
dimension or the dimension it depends on and the aggregations were
_still_ updated.

You can reproduce this behaviour in FoodMart 2000. The HR cube contains
a virtual dimension called "Position" which is derived from the
"Employee" dimension. If you do an incremental process on the Department
dimension (which is unrelated to the virtual dimension) the last
processed date of the cube is updated even though NOTHING has changed in
the base tables. (the underlying .pdr and some of the .map files get
updated)

In FoodMart 2000 this takes no time at all, but our database takes about
40 hours to fully process and so this back group task can run for 5-6
hours!

Quote:[MS] Hai-Wei Xu wrote...
> Hi ,

> Do you have any changing dimensions? If it's true, the following article
> might be helpful.

> INF: The Impact of Using Changing Dimensions                  [sqlserver]
> ID: Q303408
> http://support.microsoft.com/default.aspx?scid=kb;en-us;Q303408

> This posting is provided 'AS IS' with no warranties, and confers no rights.

> Regards,
> Haiwei Xu

--
Regards
Darren Gosbell  MCSD MCP

Senior Consultant
Datum Group  (http://www.datum.com.au)
 
 
 

Background processing after incremental dimension processing

Post by [MS] Hai-Wei » Sat, 25 May 2002 15:58:31


Hi ,

Are there any parent-child dimensions or ROLAP dimensions in your cube?
They are also changing dimensions.

This posting is provided 'AS IS' with no warranties, and confers no rights.

Regards,
Haiwei Xu

 
 
 

Background processing after incremental dimension processing

Post by Darren Gosbel » Tue, 28 May 2002 15:51:05


No, we do not have any parent-child or ROLAP dimensions in our cube.

We used to have a few parent-child dimensions, but removed them for the
very fact that they are inherently treated as changing dimensions.

Quote:[MS] Hai-Wei Xu wrote...
> Hi ,

> Are there any parent-child dimensions or ROLAP dimensions in your cube?
> They are also changing dimensions.

> This posting is provided 'AS IS' with no warranties, and confers no rights.

> Regards,
> Haiwei Xu

--
Regards
Darren Gosbell  MCSD MCP

Senior Consultant
Datum Group  (http://www.datum.com.au)
 
 
 

1. Incremental Processing and Virtual Dimensions

I am trying to find the cause of a strange problem involving incremental
processing and virtual dimensions.

We have a real dimension "Product by Category", with levels Category,
Subcategory, Group and Item.
Member keys are unique throughout the dimension, separate from the member
names - e.g. CategoryKey, CategoryName.
In addition, each level has an associated property with an alternate
language for the member name - e.g. CategoryNameFrench.

We also defined a virtual dimension "Product by Category French", based on
the above "Product by Category" dimension,  using the same keys (e.g.
CategoryKey), plus the member-properties (CategoryNameFrench) to define each
level.

Full processing of the dimension and cube works fine.  However when I do
incremental process on the the "Product by Category" dimension (which also
completes successfully), I get incorrect meaure values when querying the
cube on the "Product by Category French" dimension.

Please let me know if you've seen anything like this ASAP!

2. Back up and Restore Operations thru VB Front End

3. Incremental process dimension and cube

4. Keeping project on server - read only message

5. using DSO to process/incremental update a cube/dimension

6. SQL 7.0 Performance slower than 6.5

7. Problem in incremental processing large dimension

8. XSD mapping schema or XML EXPLICIT?

9. Locking problem with dimension during incremental processing -- need help

10. Incremental Processing of a dimension

11. Incremental Processing a Dimension / Deadlock ?

12. Incremental Processing Dimensions - Weird!!!!!!!!!!!!!!

13. Help: Processing partition does not process private dimension