FOLDER 40 bug - New TOS 1.4/1.6 Memory Management Problems

FOLDER 40 bug - New TOS 1.4/1.6 Memory Management Problems

Post by GWDGV1::WHUEBNE » Tue, 20 Mar 1990 20:40:29

Hello 'Netters',

Because of the second 'offical' Patch for the TOS 1.4 version I have
investigated some time to analyse the new/old problems with the memory
management routines within the new TOS releases.

The so called '40 folder bug' is really fixed but...:

        ...there are some NEW errors/problems within these routines!

The biggest one was the totaly wrong code for the 'post garbage collection' of
memory descriptors (Allan Pratt fix this with 'poolfix3' patch).

Some background information:

In the new version of TOS (1.4,1.6,x.x?) the handling of the system internal
memory (pool) has completly changed. Now there can be up to four memory
descriptors (MDs) placed within one pool block. Those special pool blocks
are not more available for other purposes like folder descriptions etc.
until all four MD's within this MD container are released (I have named those
special pool blocks 'MD container').

A MD is need by GEMDOS to describe the usage (allocated,free), location, size
and ownership of (user) memory chunks.
MD's are registered (with regard of there usage state) in two memory management
lists (the ALLOCATED and the FREE list). The roots of these lists are registered
within a structure called 'memory parameter block'(MPB). In this MPB is a third
information located, the ROVING pointer. This pointer is used to avoid the
concentration of small blocks at the front of the memory space after repeatedly
using of allocation/free user memory. For more information of the theorie of
memory management implementation see: Donald E. Knuth, The Art of Computer
Programming, Volume 1, Chapter 2.5 with exercise 6 (First fit method with a
roving ptr). A MD becomes unused if two free (user) memory chunks (with its
own MD's) are consecutive so that its memory could be described by one common

After some alloc/free operations there could be some MD containers exist, wich
contains one,two,three or four placeholders for unused MD's.

at any time, one will open a new file, this needs at least one WHOLE pool block,
but there are only some (partionally) filled MD containers available, what
could we do ? Right, we could try to compact MD containers so that we get
one pool block with four unused MD's.

.. and this should be done by the erronous implemented routine. Some
errors of this routine are:
        a) the implemented algorithms locks itself to find MD container
           with unused MD's with a smaller unused count as at there first hit,
        b) if they found only one MD container with unused MD's in it, they
           treat the next pool block (REGARDLESS of its usage eg. file,folder
           etc. descriptor!!) as MD container and destroyes its contents,
        c) after moving MD's between MD container they forgot to update the
           roving pointer, so that this pointer will point into the desert.


The usage of Allans POOLFIX3 patch it strongly recomended! By the way, Allan,
your patch should save all registers like the ROM GEMDOS handler do it.

If somewhat is interested in the replacement (within (ep)rom) code, please let
me know. I think, I could publishing this code here without problems because
I am using a totaly different algorithms. Any comments, ATARI?

Now the other errors/problems (not complete!) I have found through the journey
of the TOS 1.4 memory management :
        - mshrink: call of mshrink with newsize=0, the MD of this memory
          space will be released within pool but forgotten to be unlinked from
          allocated list.
        - ptermres: the information of the memory allocated by this process
          will be released from the allocated chain but not within pool.
        - the message: *** OUT OF INTERNAL MEMORY also displayed, if the
          memory management encounters errors within pool management, then
          the usage of FOLDRxxx is not the right way to correct this problem.
        - some 'dead code' within routines wich never be reached.
        - some 'optimizing' modifications made by atari forces that the new
          code really becomes 'un-optimized'.
          (In german: der Schuss ging nach hinter los - I do not know the
          english equivalent, sorry). In some routines they change register
          usage against 'in-direct' access of routine parameters, this will
          be one processor cmd less, but on the other side each reference
          of a parameter will we longer (size&time)! See malloc, pool
          allocation code and so on.
        - malloc: there is no need to look for the largest memory block
          available if we use this function for really allocating memory.
        - malloc: if you got via malloc(..-1..) the information, that there
          is stil memory available, for example 500000 bytes, and you request
          400000 bytes of it, you should get it (I mean). This needs one
          additional MD for the remaining memory space. If the pool is full,
          GEMDOS gives you NO memory in this situation! Why? I think, GEMDOS
          should give at this point the totaly remaining memory to the user.
          If the user request more after this, the information call of
          malloc(..-1..) should return zero and the allocation request also

To correct all these problems one have to publish the total memory management
code. If I do it here, (I assume) I get problems with ATARI's copyrights.

Winfried Huebner

| BITNET:  U0179 at DGoGWDG5 (W.Huebner)     |
| VAXMAIL: PSI%45551032808::GWDGV1::WHUEBNER |
| West Germany                               |
| All opinions are my own, and not necessarily always correct ... |
| You know  >>> NOBODY IS PERFECT <<<                             |


1. 40 Folder Bug *BACK* in TOS 1.4?!

The following message appeared on the San Diego Atari Computer Enthusiast's
BBS.  The originator of the message is quite a competent
programmer/developer for the ST, so I gave his message more ``weight'' than
if just anyone had posted it.  Can anyone confirm or dismiss this bug

Message : 974 [Open] 12-10-89 12:01am
From    : John Sudikatus
To      : All
Subject : TOS 1.4 bugs
Sig(s)  : 1 (General news)


Despite what Atari has said about the 40 folder bug being fixed in TOS
1.4, IT ISN'T!  Actually, the "40 folder" problem in previous
versions of TOS may have indeed been fixed, but in reducing the amount
of low memory used by TOS (in attempt to be more compatible with TOS
1.0) another bug was introduced. It seems that the directory buffers
now overlap slightly with the dynamic memory pointer list. This means
that when you have accessed more than about 35 folders and then try to
allocate memory, there is a good chance that the program will bomb
horribly. This is most likely to occur in programs that search all
subdirectories on your disk (like a utility I wrote does).

folder. FOLDRXXX.PRG seems to work since the added buffers are put at
the head of the list, meaning that they will be used before the
original 40 (80 actually, since each folder requires 2 buffers). But
one thing to note is that the size of disk buffers in TOS 1.4 has
increased slightly over that of TOS 1.0 & 1.2. To get the requested
number of buffers you must multiply the number of extra folders you
want by about 1.06 (70/66 to be exact) and rename FOLDRXXX with that

Fix a bug, create a bug. That's Atari for you. Oh, well....
Diane Barlow Close
      {nosc, ucsd}!crash!canada

      Free Canada -- Trade Mulroney

2. Web Hosting


4. Can't get 56K PCMCIA modem to work properly.

5. STE TOS / TOS 1.4/1.6 etc.

6. FF678 now avaible via FTP

7. POOLFIX3.ARC - fix program for TOS 1.4, TOS 1.6

8. Copies of Copies - what is the real scoop?

9. TOS 1.6 && TOS 1.4

10. Developer's gripes (was STE TOS / TOS 1.4/1.6 etc.)

11. poolfix, tos bugs and the 40 folder limit

12. TOS 1.4/1.6 Autobooting

13. OOPS - BAD PATCH FOR TOS 1.4/1.6