Browsing large datasets.

Browsing large datasets.

Post by Carl Hilto » Mon, 15 Sep 1997 04:00:00



I have a 4 million plus dataset.  The main dataset is normalized with 9
lookup tables.  

What is the best way to display groups of these records?

I guess I am having two problems.  I thought that creating a local view
would work but I keep getting errors.  If I limit myself to 1000 records
and only try to look at a total of 3 of the tables, selecting all fields
and having two inner joins, then I do not get a correct display.  Even
though one of the  lookup values are changing the actual value from the
lookup table does not change.  If I exit the view designer and reenter it,
I get errors about a specific column not being found and then all my data
is lost.

That is my problem with views.

Then there is the original question about what is the best way to display
records........

Actually I will never want to display ALL the records just some smaller
subset.

Help please

Thanks
Carl

 
 
 

Browsing large datasets.

Post by José Constan » Mon, 15 Sep 1997 04:00:00


Parameterized views are the way to go.  The View Designer is somewhat
limited.  You might
achieve better results in code.  For the lookup tables you might use
calculated fields in your grid.

--
Jos Constant
Constant Software Systems

http://www.club.innet.be/~year2396/



Quote:> I have a 4 million plus dataset.  The main dataset is normalized with 9
> lookup tables.  

> What is the best way to display groups of these records?

> I guess I am having two problems.  I thought that creating a local view
> would work but I keep getting errors.  If I limit myself to 1000 records
> and only try to look at a total of 3 of the tables, selecting all fields
> and having two inner joins, then I do not get a correct display.  Even
> though one of the  lookup values are changing the actual value from the
> lookup table does not change.  If I exit the view designer and reenter
it,
> I get errors about a specific column not being found and then all my data
> is lost.

> That is my problem with views.

> Then there is the original question about what is the best way to display
> records........

> Actually I will never want to display ALL the records just some smaller
> subset.

> Help please

> Thanks
> Carl


 
 
 

Browsing large datasets.

Post by Mark S. Fran » Mon, 15 Sep 1997 04:00:00


Carl, where is this 4 million record dataset?  Native VFP tables on your
workstation?...on a file server?  Or in a data warehouse managed by a
separate server such as SQL Server or Oracle?



Quote:> I have a 4 million plus dataset.  The main dataset is normalized with 9
> lookup tables.  

 
 
 

Browsing large datasets.

Post by Carl Hilto » Tue, 16 Sep 1997 04:00:00


On my workstation in Native VFP dbc



> Carl, where is this 4 million record dataset?  Native VFP tables on your
> workstation?...on a file server?  Or in a data warehouse managed by a
> separate server such as SQL Server or Oracle?



> > I have a 4 million plus dataset.  The main dataset is normalized with 9
> > lookup tables.  

 
 
 

1. How to delete large datasets

We have an application that created a 'web trens' type
report for a coldfusion application. The developer never
put an archive or delete old records function in the code
and so now we have 2 milion records (small) in a table.

The structure has a self incrementing key field of int,
and so the log is sequentially nmbered and that number has
a relation to the age of the record.

The problem is that when we try T-SQL comands like:

delete from log where id < 100000

we eventually get an timeout error from SQL - I am
interpreting this to mean there were too many records to
process and the command took too long to get a response.

Any suggestions on how to kill larger numbers of records
when you have a low powered machine in a non stop
environment?

Thanks in advance.
Marty

2. I need to send fax

3. SELECTing large datasets (50+ MB)

4. Activex Scripting Transform

5. Best Practice for Updating Large Datasets

6. network error

7. transparent use of pre-aggregated value for very large datasets

8. (PDXJOBS) Software Applications Analyst

9. Good components for use with large datasets.

10. Q: Advanced record count for large datasets

11. Large datasets >2GB

12. Information on working with very large datasets

13. Graphing large? datasets