Data Warehousing

Data Warehousing

Post by Craig Chantl » Mon, 18 Mar 1996 04:00:00



Does anybody have any opinions on the subject of data warehousing?.
Do you think it is another fad that will stay fashionable for a while
and then become redundant or do you think it is a necessary part of
corporate integration within the knowldege society.  Does anyone have
any examples of companies carrying out this process and the resultant
advantages/ disadvantages that it has brought.  

 
 
 

Data Warehousing

Post by Andy Dingl » Tue, 26 Mar 1996 04:00:00



>Does anybody have any opinions on the subject of data warehousing?.

Yes, I've worked on a few of these projects over the last few years.
Many are now tied into architectural notions of "business objects" or
"three layer architectures" as well.

Quote:>Do you think it is another fad that will stay fashionable for a while
>and then become redundant

Of course. They always do. It doesn't mean that it's any more or less
of a useful technique.

Quote:> or do you think it is a necessary part of
>corporate integration within the knowldege society.  

Nothing is _necessary_. You make anything work if you're prepared to
foot the bills.

On an omniscient scale, it's a nice idea. It's also technically
feasible to do it. The biggest problem is usually one of office
politics; "data warehousing" is basically a shorthand for "the return
of the faceless IT department" and this will suffer considerable user
resistance from those who've become fond of the freedom that
in-department development can bring.

An ideal solution would build well-managed corporate data resources
that were still flexible to user demands.

There would be a presentation layer (OLE servers, business objects, or
a well-defined SQL Server interface of published queries and stored
procedures). These business objects would offer data services in a
business-oriented manner, hiding the underlying structure and
implementation details in textbook manner.

On top of this presentation layer would be built applications,
possibly built by small teams within departments. These would use RAD
techniques to produce end products quickly and flexibly. RAD is a
particularly appropriate technology, because the amount of processing
done within the business objects reduces the implementation complexity
of the application layer. This is particularly useful with tools like
VB, which offer good development speed but have poor implemnetation
features for sophisticated data manipulation.

There are many reasons why we don't see this ideal:

Conflict of interest between teams. Each team is responsible for
producing one layer of the solution and it's in their interests to
produce the simplest and most expedient solution to their own layer,
not to produce the best overall system. Current management techniques
are adversarial and empire-building, with no-one taking an overall
interest in the _project_.

Poor quality product. Building a project end-to-end is simpler than
building an architectural project, as you can keep any corners cut
within the team. If you're writing layers OTOH, your published
interfaces must be good, and they must be implemented well. You can't
take a "short cut" between the layers without comproising the
architecture, and thus the work of several teams.

RAD. No-one knows how to do it. RAD (in practice) means "just bang it
together - we don't have to do it right any more". This is a
particular soapbox of mine, but there are some pretty awfully managed
projects out there.

Inflexible business objects. A good business object requires some
particularly careful business analysis beforehand. The prevailing
atmosphere of "We can now build huge projects overnight" means that
any analysis and design phase is impossibly squeezed for time and
effort. This leads to poorly thought out objects which are
inappropriate (bad analysis) and inflexible (bad design)

Slow business objects. There is a big temptation to make objects do
all things for all men, simply because no-one takes the time and
trouble to work out what really needs doing. This leads to over-blown
objects that are too slow in operation. Although there are many
techniques (e.g. caching & access-on-demand)  that could be
implemented inside the objects to alleviate this, it's not in the
object team's short-term interests to implement this, and higher level
management is too blinkered to see this as an advantage to the overall
project, beyond being yet another non-essential feature that can be
cut to meet deadlines.

Quote:> Does anyone have
>any examples of companies carrying out this process and the resultant
>advantages/ disadvantages that it has brought.  

Advantages:

Clean machine rooms. The first one I worked on was for a 10 year old
business that had (for political reasons) collected a ridiculous
number of disparate and uncommunicative mini systems. The IT rework
allowed all of these systems to be replaced by SQL Servers on a number
of commodity PC platforms. We killed off the expensive mini, we killed
off the _variety_ of minis, and we killed off the need to maintain a
raised-floor machine room environment.

Centralisation of business practices and rules, eliminating
inconsistencies between different parts of the business. Although
these incidents are often publically embarassing when they arise,
they're rarely serious problems for the business (maybe a posting to
comp.risks about airline duble-booking errors).

Sharing of corporate data, encouraging trendy management practices of
"Open Business" and data sharing.

Tight security to data that does need it, as the access is constricted
to one easily managed bottleneck.

Done well, it combines good housekeeping practices for the core data,
with the flexibility that ad hoc small-scale development projects can
offer.

Quick, easy and reliable (i.e. unlikely to go over-budget) development
of front-end applications.

Good facilities for data mining. Mind you, when did you ever see
someone so high up that they could usefully benefit from smart data
mining, yet had the development effort available to implement them.

On the last three advantages, these are all theoretical. I've not yet
seen _one_ project that pulled them off.

Disadvantages:

This is a large business-wide project and much of it needs to be
implemented simultaneously. The size of this project often exceeds the
management's capacity to cope and the entire project crashes and
burns.

All the company's IT eggs are in one basket, so any problems in
implementing it are likely to have repercussions at board level.
There's a tendency to hide problems for as long as possible, then
over-react when they become too big to hide.

Particular implementation techniques (particularly remote OLE) may be
tied into one vendor's product line.

The whole notion of "RAD" and its kin has been used (wrongly) as an
excuse for shoddy workmanship, bad management and incompetent
development work. This whole industry deserves a good kicking.

Combining three layer & two layer architectural models in the same
environment is an absolute recipe for disaster.

 
 
 

1. Direct client requirement for Data Architect/Data Warehouse Architect - Netezza

Hi ,

Hope you are doing good !!
Kindly send me the updated resume along with contact details and expected
rate for the below position ASAP.

Job Description:
Job Title: Data Architect or Data Warehouse Architect - Netezza
Location - Wilmington, DE
Duration - 3 Months +

At least 5 years experience working as a data architect in DW appliance
environments
Currently hands on in data modeling design for both transactional and data
warehouse/data mart system
5 years hands on data modeling experience, 3 years hands on in a DW
appliance environment
At least 3 years experience supporting a production appliance environment as
any kind of architect
At least 2 to 3 years experience doing ETL coding using Informatica
Able to create architectural diagrams and data architecture diagrams in
great detail
Deep understanding of Ralph Kimball's Bus architecture, Constellation, data
warehouse toolkit.
Deep understanding of slow changing dimensions, type I, II, IIPrior
experience converting from Oracle to DW appliance
Able to mentor Domain architects on developing roadmap and vision for data
warehouse appliance
At least 2 years, prior experience developing Data architecture framework
for a data warehouse appliance environment, prefer someone with a starter
kit of leverageable data architecture designs from prior engagements

*Thanks & Regards,
*
*Jai Shukla

International Business Solutions
Phone- 732-981-0450*428
Fax- 732-909-2178

Web- http://www.IBS-NJ.com
255-Old New Burnswick Road
Suite N230, Piscataway NJ-08854

IM: jai_shukla1*

Linkedin: http://www.linkedin.com/in/jaishukla

MBE Certified - State of New Jersey

***************************************************************************************************
Today's Quote:- "Yas tu indriyani manasa niyamya 'rabhate 'rjuna
karmendriyaih karmayogam asaktah sa visisyate"-- Bhagavadgita
****************************************************************************************************

Disclaimer: We respect your Online Privacy. This is not an unsolicited mail.
Under Bills.1618 Title III passed by the 105th U.S. Congress this mail
cannot be considered Spam as long as we include Contact information and a
method to be removed from our mailing list. If you are not interested in
receiving our e-mails then please reply with a "REMOVE" in the subject line
and mention all the e-mail addresses to be removed with any e-mail
addresses, which might be diverting the e-mails to you. We are sorry for the
inconvenience.

We reserve the right to work with the parent company, if the candidate is
not on your H1/W2. If the candidate is in the process of transferring H1s,
we require WAC numbers.

2. oops

3. Data Quality and Data Warehouse Testing

4. Old IGS Unit Password Recovery - Wyse Terminal

5. SAS/SPSS data warehouse, data mining, CRM consultant available in Malta

6. How do you run a bat file?

7. Data Warehouse: Turning Data Into Lead?

8. Access speed and registry tweek

9. MAL-0103101: REKMT: opening for the position of "Data Warehouse Analyst (Professional) in MN".

10. Req: Data Warehouse DataArchitect Immediately

11. immediate requirement for Data Warehouse Test Manager

12. immediate requirement for Oracle PL/SQL Developer with Data Warehouse Department EXP

13. Req;Business Analysis - Data Warehouse