Re: Fast Indexing....?

 BBS: Inland Empire Archive
Date: 04-13-92 (23:24)             Number: 175
From: DONN BLY                     Refer#: 88
  To: ASA ROSSOFF                   Recvd: NO  
Subj: Re: Fast Indexing....?         Conf: (2) Quik_Bas

 > I wrote a program to display quotes from a file.... But
 > it's very slow the first time it's run, because it
 > creates an index file... I see programs written in other
 > languages creating similar index files in seconds, while
 > my  program takes about 20 minutes.... (for a 360k quote
 > file)
 >
 > The index file is about 14k for a 360k file...
 >
 > Here's my code to create the index file... any ideas or
 > suggestions would be GREATLY appreciated...

Well, I suppose there are three things I want to say here.  First, I want to
tell you that your code was very hard to follow because of
all of the nesting of IF/THEN statements, however, I think
I followed your logic. The reasons it is so slow are
numerious, but mostly it is because you are trying to (a)
read the file one byte at a time and (b) trying to use
RANDOM files where they are inappropriate, abd (c) using a
fixed length string in a manner which really slows things
down (it is constantly converted to a variable-length
string so that it can be acted upon by the QB functions).

Basically, what you are seeing is an implementation
problem, not a language problem,

Following is some sample code.  The first program will
build your index file in substantually less time, the
second will show you how to pull messages out of the file
on a rotating basis without building and rebuilding an
index file.

_ _ _ O_/_ _C_U_T_ _H_E_R_E_ _ _ _ _ _ _
      O \
   QuoteFile% = FREEFILE                           ' Open The Files
   OPEN "Cookie.txt" FOR INPUT AS #QuoteFile%      '
   QuoteIndex% = FREEFILE                          '
   OPEN "Cookie.idx" FOR BINARY AS #QuoteIndex%    '
   IF LOF(QuoteIndex%) = 0 THEN                    ' Re-Index if Needed
      QuoteCount& = 0
      PUT #QuoteIndex%, 1, QuoteCount&
      IDXPointer& = 1
      PRINT "Quote(s) Indexed : ";
      WHILE NOT EOF(QuoteFile%)
         LINE INPUT #QuoteFile%, LineBuff$
         IF LEN(LineBuff$) = 0 THEN
            IDXPointer& = SEEK(QuoteFile%)
         ELSE
            IF IDXPointer& THEN
                QuoteCount& = QuoteCount& + 1
                PUT #QuoteIndex%, , IDXPointer&
            END IF
            IDXPointer& = 0
         END IF
         LOCATE , 19                               '  Tell User where we are
         Pct% = (SEEK(QuoteFile%) - 1) * 100& \ LOF(QuoteFile%)
         PRINT QuoteCount&; "("; LTRIM$(STR$(Pct%)); "%)";
      WEND
      PRINT
      PUT #QuoteIndex%, 1, QuoteCount&
   END IF

   ' Spit Them Back Out
   FOR Quote% = 1 TO LOF(QuoteIndex%) \ 4 - 1
      PrintNextMessage QuoteFile%, QuoteIndex%
   NEXT
   CLOSE #QuoteFile%, QuoteIndex%


SUB PrintNextMessage (QuoteFile%, QuoteIndex%)
   ' Get Next Message # in series and save counter
   GET #QuoteIndex%, 1, MsgPointer&
   MsgPointer& = MsgPointer& + 1
   IF MsgPointer& * 4 = LOF(QuoteIndex%) THEN MsgPointer& = 1
   PUT #QuoteIndex%, 1, MsgPointer&
   ' Now Print Message
   GET #QuoteIndex%, MsgPointer& * 4 + 1, IDXPointer&
   PRINT "*** Quote Number"; MsgPointer&; "****"
   SEEK #QuoteFile%, IDXPointer&
   LINE INPUT #QuoteFile%, LineBuff$
   DO UNTIL EOF(QuoteFile%) OR LEN(LineBuff$) = 0
      PRINT LineBuff$
      LINE INPUT #QuoteFile%, LineBuff$
   LOOP
END SUB
_ _ _ O_/_ _C_U_T_ _H_E_R_E_ _ _ _ _ _ _
      O \

Now, for that alternative method:

_ _ _ O_/_ _C_U_T_ _H_E_R_E_ _ _ _ _ _ _
      O \
   ' Alternative with no need to build an index
   FOR I% = 1 TO 6
      PRINT I%; "****"
      PrintNextMessage2
   NEXT

SUB PrintNextMessage2
   QuoteFile% = FREEFILE
   OPEN "COOKIE.TXT" FOR INPUT AS #QuoteFile%
   QuotePointer% = FREEFILE
   OPEN "COOKIE.ID2" FOR BINARY AS #QuotePointer%
   IF LOF(QuotePointer%) = 0 THEN
      IDXPointer& = 1
   ELSE
      GET #QuotePointer%, 1, IDXPointer&
   END IF
   IF LOF(QuoteFile%) <= IDXPointer& THEN IDXPointer& = 1
   SEEK #QuoteFile%, IDXPointer&
   LINE INPUT #QuoteFile%, LineBuff$
   DO UNTIL EOF(QuoteFile%) OR LEN(LineBuff$) = 0
      PRINT LineBuff$
      LINE INPUT #QuoteFile%, LineBuff$
   LOOP
   IDXPointer& = SEEK(QuoteFile%)
   PUT #QuotePointer%, 1, IDXPointer&
   CLOSE QuoteFile%, QuotePointer%
END SUB
_ _ _ O_/_ _C_U_T_ _H_E_R_E_ _ _ _ _ _ _
      O \

--- OPMED 3.00
 * Origin: The Loft *HST/DS/V42bis* [NC, SDS] (1:236/7)