Inverted index

Inverted index

Post by Larry Rappapo » Wed, 25 May 1994 16:06:03




Quote:> Hi All,

> Any idea how to create a so called "inverted index".

> dBASE/FoxPro code or pseudo code would be great. Also a
> shareware/PD/freeware program will do.

Don't know if it will help, but in Clipper you do it by using the descend()
function.  I don't believe the function exists in dBase III+; not sure about
Fox.

Larry

-----------------------------------------------------------------------------

Colebrook, NH  03576-0158             CIS 72427,2567    fax   +1 603 237 8430

 
 
 

Inverted index

Post by Bill Lapwor » Thu, 26 May 1994 06:12:51





>>> Hi All,

>>> Any idea how to create a so called "inverted index".

.. stuff deleted about Clipper descend() ...

Quote:>I believe that you can create an inverted index by replacing each character
>of the index expression with CHR(255-ASC(m.the_character)).  This will
>'invert' the expression.  Numbers, Dates, Logicals, etc will have to be
>handled differently...  The following function should do the trick.  

.. function deleted ...

Either I have no idea what ya'll mean by an 'inverted' index OR there
is a much easier way to do this.  The INDEX command in FoxPro 2.5
contains the ASCENDING | DESCENDING clause for changing the order
of the index.  If that's not what you meant, please explain 'inverted'
to me.

Bill

>Any suggestions or comments are welcome...
>--
>Mark T. Boyer              


>Oak Ridge, Tennessee


 
 
 

Inverted index

Post by Philip L Mil » Thu, 26 May 1994 10:32:34



: > Hi All,
: >
: >
: > Any idea how to create a so called "inverted index".
: >
: > dBASE/FoxPro code or pseudo code would be great. Also a
: > shareware/PD/freeware program will do.

: Don't know if it will help, but in Clipper you do it by using the descend()
: function.  I don't believe the function exists in dBase III+; not sure about
: Fox.

: Larry

: -----------------------------------------------------------------------------

: Colebrook, NH  03576-0158             CIS 72427,2567    fax   +1 603 237 8430

Yes, DESCEND will work in foxpro aswell. Just add 'DESCENDING' to the end of
your index statement. ie:
                     SET ORDER TO TAG index_name DESCENDING

*******************************************************************
*  Philip Miles              *                                    *
*  Dalhousie University      *  Write a piece of code that does   *
*  Computing Science         *  what ?!?                          *

*******************************************************************

 
 
 

Inverted index

Post by A. Gonzal » Thu, 26 May 1994 12:35:49





>: > Hi All,
>: >
>: >
>: > Any idea how to create a so called "inverted index".
>: >
>: > dBASE/FoxPro code or pseudo code would be great. Also a
>: > shareware/PD/freeware program will do.
>: Don't know if it will help, but in Clipper you do it by using the descend()
>: function.  I don't believe the function exists in dBase III+; not sure about
>: Fox.
>: Larry
>: -----------------------------------------------------------------------------
>: Colebrook, NH  03576-0158             CIS 72427,2567    fax   +1 603 237 8430
>Yes, DESCEND will work in foxpro aswell. Just add 'DESCENDING' to the end of
>your index statement. ie:
>                 SET ORDER TO TAG index_name DESCENDING
>*******************************************************************
>*  Philip Miles              *                                    *
>*  Dalhousie University      *  Write a piece of code that does   *
>*  Computing Science         *  what ?!?                          *

>*******************************************************************

******************************************************************

In dBase III+ you can always try something like

USE DFILE
INDEX ON -RECNO() TO REVERSED
USE DFILE INDEX REVERSED
LIST ALL

This should display the DBF file in reverse order.

Armando R. Gonzalez
The University of Iowa

******************************************************************

 
 
 

Inverted index

Post by Mark T. Boye » Thu, 26 May 1994 04:26:44




>> Hi All,

>> Any idea how to create a so called "inverted index".

>> dBASE/FoxPro code or pseudo code would be great. Also a
>> shareware/PD/freeware program will do.

>Don't know if it will help, but in Clipper you do it by using the descend()
>function.  I don't believe the function exists in dBase III+; not sure about
>Fox.

>Larry

>-----------------------------------------------------------------------------

>Colebrook, NH  03576-0158             CIS 72427,2567    fax   +1 603 237 8430

I believe that you can create an inverted index by replacing each character
of the index expression with CHR(255-ASC(m.the_character)).  This will
'invert' the expression.  Numbers, Dates, Logicals, etc will have to be
handled differently...  The following function should do the trick.  

FUNCTION DESCEND
PARAMETERS cOldKey
PRIVATE cNewKey
cNewKey = ""
FOR X = 1 TO LEN(cOldKey)
        cNewKey = cNewKey + CHR(255-ASC(SUBSTR(cOldKey,X,1)))
ENDFOR
RETURN cNewKey

You could also use a CHRTRAN() function in FoxPro, something like...

FUNCTION Descend
PARAMETERS ctheExpr
RETURN CHRTRAN(ctheExpr,'ABCDEFGHI','IHGFEDCBA')

The second and third parameters of CHRTRAN may need to be 256 characters
long with the entire set inverted in one of the two expressions.  (I used a
simple set for purposes of illustration).

One benefit of using the second example over the first is that it can be
incorporated as part of an index without having to call an external
function.  It will operate significantly faster as a result, however, I'm
not sure how the expression will work if you include the entire ASCII set
within the string.  You will need to consider that the string delimiters
are also part of the ASCII set and you will probably have to split the
expression and add to get a working function.

Any suggestions or comments are welcome...

--
Mark T. Boyer              


Oak Ridge, Tennessee

 
 
 

Inverted index

Post by David Church » Thu, 26 May 1994 23:14:07



>Any idea how to create a so called "inverted index".

 There was an article about this in FoxTalk magazine, December 1991.
It came with FoxPro source and seemed to work OK, but understandably
it wasn't very fast.

 You can order back issues of FoxTalk from:

 Pinnacle Publishing INC,
 P.O. Box 888,
 Kent, WA 98035-0888.
 Phone: (206) 251-1900
 Fax: (206) 251-5057

 Cost is $US15.00 outside the US, and source code disks are an extra
$US10.

 Also Xitech do a text indexer for memo files that seems to be very fast.

5515 Southwyck Blvd
Toledo OH 43614

Tel: 419 867 3622
Fax: 419 865 9365

 or Xitech Europe:

 Phone +44 707 276 637
 Fax   +44 707 267 777

 Regards,
 Dave.

 P.S. for those posting methods of making descending indexes, an
"inverted" index actually indexes every word in a text database. In a
FoxPro context you would usually use it to index every word in every
memo field in a file.

--

 
 
 

Inverted index

Post by Keith A. Cochr » Sat, 28 May 1994 00:45:21






>>> Any idea how to create a so called "inverted index".

>>> dBASE/FoxPro code or pseudo code would be great. Also a
>>> shareware/PD/freeware program will do.

>>Don't know if it will help, but in Clipper you do it by using the descend()
>>function.  I don't believe the function exists in dBase III+; not sure about
>>Fox.

>I believe that you can create an inverted index by replacing each character
>of the index expression with CHR(255-ASC(m.the_character)).  This will
>'invert' the expression.  Numbers, Dates, Logicals, etc will have to be
>handled differently...  The following function should do the trick.  

>FUNCTION DESCEND
>PARAMETERS cOldKey
>PRIVATE cNewKey
>cNewKey = ""
>FOR X = 1 TO LEN(cOldKey)
>    cNewKey = cNewKey + CHR(255-ASC(SUBSTR(cOldKey,X,1)))
>ENDFOR
>RETURN cNewKey

>You could also use a CHRTRAN() function in FoxPro, something like...

>FUNCTION Descend
>PARAMETERS ctheExpr
>RETURN CHRTRAN(ctheExpr,'ABCDEFGHI','IHGFEDCBA')

>The second and third parameters of CHRTRAN may need to be 256 characters
>long with the entire set inverted in one of the two expressions.  (I used a
>simple set for purposes of illustration).

>One benefit of using the second example over the first is that it can be
>incorporated as part of an index without having to call an external
>function.  It will operate significantly faster as a result, however, I'm
>not sure how the expression will work if you include the entire ASCII set
>within the string.  You will need to consider that the string delimiters
>are also part of the ASCII set and you will probably have to split the
>expression and add to get a working function.

Um, how about

INDEX ON tag NAME tag DESCENDING

Actually, I think what they're looking for is a routine where you can
put in

"Keith Cochran"

But when you sort it, it sorts like you had put in:

"Cochran Keith"

The simplest way I can think to do that is to have two input fields, and
then:

INDEX ON lname+fname TAG invert DESCENDING
--

=My thoughts, my posts, my ideas, my responsibility, my beer, my pizza.  OK???=
=        "I had to wait 300 years for a * to light the candle..."        =

 
 
 

Inverted index

Post by Paul Russel » Thu, 02 Jun 1994 14:36:18


In message Tue, 24 May 94 07:06:03 EDT,

Quote:>>   Any idea how to create a so called "inverted index".

> Don't know if it will help, but in Clipper you do it by using the
> descend() function.  I don't believe the function exists in dBase III+;
> not sure about Fox.

FoxPro can, indeed index on something 'DESCENDING'.  Or, you can index
normally and then: SET ORDER TO TAG <tag> DESCENDING.

/Paul
--
+--------------------------+----------------------------------------------+
| Paul Russell             | Phone:     (902) 481-3680                    |
| Software Kinetics        | Fax:       (902) 468-3679                    |

| Dartmouth                | "Use technology because it's appropriate...  |
| Nova Scotia, Canada      |     not just because it's cool!"             |
| B3B 1W2                  |             - E. Davidson, Software Kinetics |
+--------------------------+----------------------------------------------+

 
 
 

Inverted index

Post by Daiv Ston » Sat, 04 Jun 1994 06:39:59



>In message Tue, 24 May 94 07:06:03 EDT,

>>>   Any idea how to create a so called "inverted index".

>> Don't know if it will help, but in Clipper you do it by using the
>> descend() function.  I don't believe the function exists in dBase III+;
>> not sure about Fox.
>FoxPro can, indeed index on something 'DESCENDING'.  Or, you can index
>normally and then: SET ORDER TO TAG <tag> DESCENDING.
>/Paul

Of course this only works if you're using CDXs, not IDXs.

--
Daiv Stoner                   ||  Busy life / Not enough time to look in

 
 
 

Inverted index

Post by leo g. divinagracia i » Sat, 04 Jun 1994 10:50:38





>>In message Tue, 24 May 94 07:06:03 EDT,

>>>>   Any idea how to create a so called "inverted index".

>>> Don't know if it will help, but in Clipper you do it by using the
>>> descend() function.  I don't believe the function exists in dBase III+;
>>> not sure about Fox.

 try this to do a descending (or inverted) index, even those using dbiii+

index on -(asc (substr (x_field, 1, 1))) to x_ndx

the trick is to use the unary negative operator when the ascii value has
been returned.  of course if the field you wish to index on is numeric,
then the asc() and substr() functions aren't even used.  

of course the limitation on this is that only one character of the field
is used.  but then again, you can concatenate the expression:

index on -(expr1) + -(expr2) + ... + -(expr n) to x_ndx
   where  expr n is the thing above.

later days...

leo d.

--
------------------------------------------------------------------------------
 \ The above does not represent OIT, UNC-CH, laUNChpad, or its other users. /
   ------------------------------------------------------------------------

 
 
 

Inverted index

Post by A. Gonzal » Sat, 04 Jun 1994 12:20:45


In dBase III+  I think the way to do this is to index on
a negative record number -RECNO(). You can sort/copy the DBF
the way you want it and then index it backwards on a negative
record number.

USE DFILE
INDEX ON -RECNO() TO REVERSED
USE DFILE INDEX REVERSED

This also works fine If the field is numeric. If PRICE is a numeric
field you can always do something like:

INDEX ON -PRICE TO REV_PRICE

I'm sure something can be done for dates with a function
like CTOD(). But the best way to solve this problem is to
use dbest xbase language   C l i p p e r.

Armando

 
 
 

Inverted index

Post by Steve Koters » Sat, 04 Jun 1994 22:26:16


: In dBase III+  I think the way to do this is to index on

: I'm sure something can be done for dates with a function
: like CTOD(). But the best way to solve this problem is to
: use dbest xbase language   C l i p p e r.

In dBASE III PLUS, you can create a descending order index based on a date
field by, in the index expression, subtracting the date field from a very
large date -- preferrably one that would never appear in the table. For
example:

INDEX ON CTOD('12/31/3099') - DateFld TO DateDesc

--
_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
_/ Steve Koterski               _/   The opinions expressed here are    _/

_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

 
 
 

Inverted index

Post by Tony Li » Tue, 07 Jun 1994 02:26:00


AG>USE DFILE
  >INDEX ON -RECNO() TO REVERSED
  >USE DFILE INDEX REVERSED

A quite good technique for reversing record order.

AG>I'm sure something can be done for dates with a function
  >like CTOD(). But the best way to solve this problem is to
  >use dbest xbase language   C l i p p e r.

Use the Big M method:  subtract the date from a very large
date, e.g.,

INDEX ON {12/31/2999} - <date> TO <etc>

Of course, that will create a numeric type index rather than
date type, but it will reverse the dates.  You can do
something similar for indexing character type fields, but it
involves a UDF in the index.  Hope this gives you some ideas.

---
 * SLMR 2.0 #1193 * My other car is a modem!

 
 
 

1. Inverted Indexes

 Hello,

      I would like to have your comments and suggestion about using
inverted indexes in the DWH. Is it possible in the future that this index,
which is used in
Information Retrival System, will be replaced the indexing techniques in
DWH (i.e., bitmap, B-tree)? I read white paper from a company. The
performance of using the inverted index is unbelivable. It's very fast.
      The different between the inverted index and the bit-map index is
the former stores the keyword(term) with the record number while the later
creates a vector and stores only one bit (0 or 1) for a record. Both came
from the same ideal which stores keyword with the way to reach the record.
      I think the bit-map index, overall, is over than the inverted index:
use less space (since it stores a bit), fast, and etc. The disadvantage of
both are the same (i.e., not good for high cardinality of data).
      From the performance of the paper that I read, which showed that
inverted index is suitable and the BEST choice to use answering the query
in DWH,  does this implied that we can use the techniqe that introduced in
Information Retrieval System in the DWH? In the paper, they also said that
if we use the inverted index we don't need to store summary data to
support the DSS. The index perform well both ad-hoc query and aggregation
query without touching the underlying database (only index level is
reached).
     Any comment and suggestion would appreciate.

Sincerely Yours,

-sirirut-

2. Timeouts to remote SQL Server from QA but not (some) EM

3. Full Text Inverted Indexes

4. ORACLE FINANCIALS FUNCTIONAL CONSULTANT

5. Inverted Indexes with volatile data?

6. Buffer Latch Timeout.

7. Full Text Inverted Indexes

8. dBase IV for Unix

9. What is a inverted index?

10. Inverted index

11. indexed sequential, indexed, inverted, hash or direct file support

12. Insight into indexes? (or inverting then externally)

13. Inverted commas in SQL 2000