BLOBS (was: 500'000 records)

BLOBS (was: 500'000 records)

Post by Mischa Sandbe » Sat, 02 Jan 1993 13:10:11



Quote:> Hugh LaMaster writes:
>                                    With 30K records, you are dealing
> |> with blobs outside the normal record structure; you can run greater
> |> risks.

> Is there a "correct" way to deal with this problem yet?  It can show
> up in surprisingly simple DBMS applications, such as where you want to
> search through a bunch of trouble calls for matches to a particular
> problem.  1-2 years ago, when I inquired to this newsgroup, there was
> no RDBMS that could handle text searches efficiently.  (Efficiently,
> that is, compared to a text processing/retrieval system.)  Is this still
> the case?

Seems to be still the case, among the mainstream database products.
You might have some luck with middle-users (as opposed to end-users)
such as those providing db's for, say, the publishing industry ---
a wonderful source of solutions to problems that I, thank goodness,
seldom see (think about how you store the articles and pictures in
a magazine; this ad must NOT be near *that* ad, etc.) And, if you
REALLY want to see the leading edge in OODB technology and really
awesome record size and relationship problems, take a peek at
where office automation systems are going (no, I'm being serious).

I agree, though, that really usable blobs would find instant use.
We started off trying to move our E-Mail/CIM system into a standard
SQL database. Turned out to be more hassle than the old system, which
created (compressed) files in a standard unix directory and stored
the filenames in the database; directories could be nested within
directories; links could be made for multiple copies; etc.

The standard database, with 1970's technology for locking, journalling
and space management isn't a very promising engine for such applications.

Pardon me: one exception: Revelation. With 1-64kb records as part of
the basic file system,and extremely flexible dictionaries, AREV
can probably handle just about anything you'd like to put on a PC
or a LAN. But that's a rather limited domain.
--

                 or uunet!van-bc!rsoft!mindlink!Mischa_Sandberg
*-*-*-*-*-*-*-*-*-*-*
Engineers think equations are an approximation of reality.
Physicists think reality is an approximation of the equations.
Mathematicians never make the connection.

 
 
 

BLOBS (was: 500'000 records)

Post by Hugh LaMast » Wed, 06 Jan 1993 10:42:13


|> > Hugh LaMaster writes:
|> >                                    With 30K records, you are dealing
|> > |> with blobs outside the normal record structure; you can run greater
|> > |> risks.
|> >
|> > Is there a "correct" way to deal with this problem yet?  It can show
|> > up in surprisingly simple DBMS applications, such as where you want to
|> > search through a bunch of trouble calls for matches to a particular
|> > problem.  1-2 years ago, when I inquired to this newsgroup, there was
|> > no RDBMS that could handle text searches efficiently.  (Efficiently,
|> > that is, compared to a text processing/retrieval system.)  Is this still
|> > the case?
|>
|> Seems to be still the case, among the mainstream database products.
|> You might have some luck with middle-users (as opposed to end-users)
|> such as those providing db's for, say, the publishing industry ---
|> a wonderful source of solutions to problems that I, thank goodness,
|> seldom see

Who are the vendors of this "middle-user" software?  

What RDBMS's do they use as the database part (or, do they do their own?)?  

What are the capabilities on the text side?

Is there a newsgroup on this, or a place where technical people and users
of such products post?

--
  Hugh LaMaster, M/S 233-9,     UUCP:      ames!lamaster


  Phone:  415/604-1056                     #include <usenet/std_disclaimer.h>

 
 
 

BLOBS (was: 500'000 records)

Post by Ivan - Goda » Thu, 07 Jan 1993 03:22:04


Hugh McMaster writes:

b
>Hugh McMaster writes:

>erg) writes:
>|>
>|> Seems to be still the case, among the mainstream database products.
>|> You might have some luck with middle-users (as opposed to end-users)
>|> such as those providing db's for, say, the publishing industry ---
>|> a wonderful source of solutions to problems that I, thank goodness,
>|> seldom see

>Who are the vendors of this "middle-user" software?  

Also considered 'middle users' are the vendors of components, c.f. the
Kala persistence server written up in the December Byte.

Quote:>What RDBMS's do they use as the database part (or, do they do their own?)?  

For Kala, self hosting (requires block read, write and seek from the OS)

Quote:

>What are the capabilities on the text side?

Anything you want - see the article.

Quote:

>Is there a newsgroup on this, or a place where technical people and users
>of such products post?

For Kala there's a moderated forum, you can get on by posting to

Ivan

 
 
 

BLOBS (was: 500'000 records)

Post by Vija » Thu, 07 Jan 1993 19:17:14




> |> > Hugh LaMaster writes:
> |> >                                    With 30K records, you are dealing
> |> > |> with blobs outside the normal record structure; you can run greater
> |> > |> risks.
> |> >
> |> > Is there a "correct" way to deal with this problem yet?  It can show
> |> > up in surprisingly simple DBMS applications, such as where you want to
> |> > search through a bunch of trouble calls for matches to a particular
> |> > problem.  1-2 years ago, when I inquired to this newsgroup, there was
> |> > no RDBMS that could handle text searches efficiently.  (Efficiently,
> |> > that is, compared to a text processing/retrieval system.)  Is this still
> |> > the case?
> |>
> |> Seems to be still the case, among the mainstream database products.
> |> You might have some luck with middle-users (as opposed to end-users)
> |> such as those providing db's for, say, the publishing industry ---
> |> a wonderful source of solutions to problems that I, thank goodness,
> |> seldom see

> Who are the vendors of this "middle-user" software?  

> What RDBMS's do they use as the database part (or, do they do their own?)?  

> What are the capabilities on the text side?

> Is there a newsgroup on this, or a place where technical people and users
> of such products post?

> --
>   Hugh LaMaster, M/S 233-9,     UUCP:      ames!lamaster


>   Phone:  415/604-1056                     #include <usenet/std_disclaimer.h>

Please also see Note 13063.

BASISplus Document Database is designed to manipulate text from ground up.
BASISplus has the capability of searching on both "Structured" and
"Unstructured" fields simultaneously.

e.g. Search for all documents by Vijay Sharma that contain the word -
     "BASISplus".

Additionally the database handles SGML documents hence maintaining the
document structure as well as giving the user the capability of searching
at the component level of the documents.

--
Regards,

Vijay

**************************************************************************
*                                                                        *
* Vijay Sharma    Mail ID : VSHARMA                                      *

*                                                                        *
*                 Bus     : 416-368-7724                                 *
*                                                                        *
*                 "Do the unexpected"                                    *
*                                                                        *
**************************************************************************

 
 
 

BLOBS (was: 500'000 records)

Post by Michael Friedm » Thu, 11 Feb 1993 05:58:19





>> |> > Is there a "correct" way to deal with this problem yet?  It can show
>> |> > up in surprisingly simple DBMS applications, such as where you want to
>> |> > search through a bunch of trouble calls for matches to a particular
>> |> > problem.  1-2 years ago, when I inquired to this newsgroup, there was
>> |> > no RDBMS that could handle text searches efficiently.  (Efficiently,
>> |> > that is, compared to a text processing/retrieval system.)  Is this still
>> |> > the case?
>> |> Seems to be still the case, among the mainstream database products.
>> |> You might have some luck with middle-users (as opposed to end-users)
>> |> such as those providing db's for, say, the publishing industry ---
>> |> a wonderful source of solutions to problems that I, thank goodness,
>> |> seldom see

Oracle has a solution for this problem.  We have a product,
SQL*TextRetrieval which integrates with the Oracle RDBMS and lets you
do queries like

Select CANDIDATE_ID from RESUME_TABLE where SALARY_REQUEST < 60000 and
       SKILLS contains 'C++'

I could give you all a big spiel on the product, but I'm not a
salesrep and it would be kind of inappropriate anyway.  If you want to
know more, call a salesrep.  They'll talk your ears off about it.

 
 
 

BLOBS (was: 500'000 records)

Post by Ra » Fri, 19 Feb 1993 07:45:53


I know of a 4GL that deals with alternate-key intersection very well (I don't
work for the folks, just know people who use it).  It's called TOADS
(something something something Developement System) and is availble from
USC Software Systems, 213-743-0091.

Ray

--

Ray (DJ) DeGennaro II

 
 
 

1. SQL Server and 500 000 000 records ?

One simple question : is SQL Server 7 possible to work with five hundred
millions of records (500 000 000). Record has cca 10 fields and most of them
are integer. Server is strong enough (2G RAM, dual processor P3, lot of HDD
space).

Thanks,
Drago. :-)

2. What does this error mean and how do I get rid of it?

3. 500'000 records - who does best?

4. Populating BLOB with a DIctionary Object

5. Some inquiries regarding Filemaker Pro 6.0 (trial)

6. problems Opening 500 000 + recordset

7. Decompiler

8. How to improve speed of a 1.000.000 records grid population

9. Access Vs. FoxPro (1.000.000 records DB)

10. 1.000.000 records

11. 100 000 000 records...SQL