realloc bug in MSL?

realloc bug in MSL?

Post by jaxoj.. » Wed, 24 Apr 2002 05:57:23



I've been having a problem with realloc(). I traced it to the
following MSL routine. It turns out that MemPtrResize is returning
memErrChunkLocked. Shouldn't you be checking for this and returning
NULL?
Joe

__inline void *realloc(void *ptr, size_t size)
{
        if (ptr == NULL) { return MemPtrNew(size); }
        if (MemPtrResize(ptr, size) == memErrNotEnoughSpace)
        {
                void *newPtr = MemPtrNew(size);
                if (newPtr) { MemMove(newPtr, ptr, (Int32)MemPtrSize(ptr)); }
                MemPtrFree(ptr);
                ptr = newPtr;
        }

        return ptr;

Quote:}

 
 
 

realloc bug in MSL?

Post by Ben Combe » Wed, 24 Apr 2002 11:30:04




Quote:> I've been having a problem with realloc(). I traced it to the
> following MSL routine. It turns out that MemPtrResize is returning
> memErrChunkLocked. Shouldn't you be checking for this and returning
> NULL?

Actually, we should be checking for any failure of MemPtrResize.  This
is a good catch, the bug is my fault, and I'll make sure this change is
in the next version of MSL for Palm OS.

Thanks!
--

CodeWarrior for Palm OS technical lead
Get help at http://palmoswerks.com/

 
 
 

realloc bug in MSL?

Post by jaxoj.. » Wed, 24 Apr 2002 11:34:39


I tried fixing it this way, which seems to work.  The Unlock doesn't
seem to do any good, so it ends up being freed (I hope) and re-newed.
Does this look ok?

void *reallocFix4Mwerks(void *ptr, size_t size)
{
    // Just do a new if it's a nil pointer.
    if (ptr == 0) { return MemPtrNew(size); }

    // Try to resize it, in-place.
    Err err = MemPtrResize(ptr, size);
    if (err == 0) { return ptr; }

    // If it cannot be resized in-place, then renew it.
    if (err == memErrNotEnoughSpace) {
        void *newPtr = MemPtrNew(size);
        if (newPtr) { MemMove(newPtr, ptr, (Int32)MemPtrSize(ptr)); }
        MemPtrFree(ptr);
        return newPtr;
    }

    // If it's locked (the case MSL did not handle)
    // then try to unlock and resize it,
    // otherwise renew it.
    if (err == memErrChunkLocked) {
        err = MemPtrUnlock(ptr);
        err = MemPtrResize(ptr, size);
        if (err == 0) { return 0; }

        void *newPtr = MemPtrNew(size);
        if (newPtr) { MemMove(newPtr, ptr, (Int32)MemPtrSize(ptr)); }
        MemPtrFree(ptr);
        return newPtr;
    }

    return 0;

Quote:}

 
 
 

realloc bug in MSL?

Post by Ben Combe » Wed, 24 Apr 2002 23:20:56




Quote:> I tried fixing it this way, which seems to work.  The Unlock doesn't
> seem to do any good, so it ends up being freed (I hope) and re-newed.
> Does this look ok?

No... I've got a problem with the last section:

Quote:>     // If it's locked (the case MSL did not handle)
>     // then try to unlock and resize it,
>     // otherwise renew it.
>     if (err == memErrChunkLocked) {
>         err = MemPtrUnlock(ptr);
>         err = MemPtrResize(ptr, size);
>         if (err == 0) { return 0; }

If MemPtrUnlock succeeded, then ptr is no longer valid to be used, so
the following call to MemPtrResize is invalid.  My suggestion is to just
do the "new block/copy/free old" pattern for memErrChunkLocked.  On
inspection of the Palm OS source, you will only get the
memErrNotEnoughSpace error if the chunk is unlocked and there isn't
enough space to expand, which means that the later MemPtrNew would also
fail.

There's one other change needed: if size entering the routine is 0, we
should just free the pointer.  This is a valid behavior according to the
ISO/ANSI C documentation, and on inspection of the Palm OS source, doing
a MemPtrNew of size 0 will just return NULL, so this keeps it
consistent.

Here's my new code:

__inline void *realloc(void *ptr, size_t size)
{
    // input ptr NULL is same as malloc
    if (ptr == NULL) { return MemPtrNew(size); }

    // MemPtrNew(0) always returns NULL
    if (size == 0) { MemPtrFree(ptr); return NULL; }

    // if we can't resize in place, alloc new block
    if (MemPtrResize(ptr, size) == memErrChunkLocked)
    {
        void *newPtr = MemPtrNew(size);
        if (newPtr) { MemMove(newPtr, ptr, (Int32)MemPtrSize(ptr)); }
        MemPtrFree(ptr);
        return newPtr;
    }

    // amy other error condition requires freeing
    // original pointer and returning NULL
    MemPtrFree(ptr); return NULL;

Quote:}

--

CodeWarrior for Palm OS technical lead
Get help at http://palmoswerks.com/
 
 
 

realloc bug in MSL?

Post by rexxrexxr.. » Thu, 25 Apr 2002 03:36:44


Thanks, Ben, I incorporated your new realloc.
 
 
 

realloc bug in MSL?

Post by rexxrexxr.. » Thu, 25 Apr 2002 12:32:04


Oops...if MemPtrResize succeeds in your code, then that's trouble (-:
I'm sure you will fix it the same way:

    Err err = MemPtrResize(ptr, size);
    if (err == 0) { return ptr; }
    if (err == memErrChunkLocked) { ...

Quote:>     // if we can't resize in place, alloc new block
>     if (MemPtrResize(ptr, size) == memErrChunkLocked)
>     {
>         void *newPtr = MemPtrNew(size);
>         if (newPtr) { MemMove(newPtr, ptr, (Int32)MemPtrSize(ptr)); }
>         MemPtrFree(ptr);
>         return newPtr;
>     }

>     // amy other error condition requires freeing
>     // original pointer and returning NULL
>     MemPtrFree(ptr); return NULL;
> }

 
 
 

realloc bug in MSL?

Post by Ben Combe » Thu, 25 Apr 2002 14:32:32





> Oops...if MemPtrResize succeeds in your code, then that's trouble (-:
> I'm sure you will fix it the same way:

>     Err err = MemPtrResize(ptr, size);
>     if (err == 0) { return ptr; }
>     if (err == memErrChunkLocked) { ...

Yes, you're right... I've not committed the change, so I'll be sure to
fix up that logic too.  I think now we both have a correct realloc.

--

CodeWarrior for Palm OS technical lead
Get help at http://palmoswerks.com/

 
 
 

realloc bug in MSL?

Post by r.. » Mon, 29 Apr 2002 01:02:26



> Yes, you're right... I've not committed the change, so I'll be sure to
> fix up that logic too.  I think now we both have a correct realloc.

I did one other thing to this, Ben, to help out on Palm OS 3.0, where
the memory constraints are more strict.  If MemPtrNew returns 0, then
I try to help it out a bit by copying the contents of 'ptr' to a
database record, then freeing it, and then trying MemPtrNew again.

        // Copy the old data to it.
        MemHandle hMemory = DmGetRecord(myDB, myRecNo);
        char *pMemory = (char *)MemHandleLock(hMemory);
        DmWrite(pMemory, 0, ptr, MemPtrSize(ptr));
        MemHandleUnlock(hMemory);
        DmReleaseRecord(myDB, myRecNo, true);

        // Free the old chunk.
        err = MemPtrFree(ptr);

        // Try to get a new chunk.
        newPtr = MemPtrNew(size);
        if (!newPtr) { return 0; }

        // Copy the old data to the new chunk.
        hMemory = DmGetRecord(s_dbMemory, s_recNumber);
        pMemory = (char *)MemHandleLock(hMemory);
        MemMove(newPtr, pMemory, oldSize);
        MemHandleUnlock(hMemory);
        DmReleaseRecord(s_dbMemory, s_recNumber, false);
        return newPtr;

Joe

 
 
 

realloc bug in MSL?

Post by Ben Combe » Mon, 29 Apr 2002 03:41:48





> > Yes, you're right... I've not committed the change, so I'll be sure to
> > fix up that logic too.  I think now we both have a correct realloc.

> I did one other thing to this, Ben, to help out on Palm OS 3.0, where
> the memory constraints are more strict.  If MemPtrNew returns 0, then
> I try to help it out a bit by copying the contents of 'ptr' to a
> database record, then freeing it, and then trying MemPtrNew again.

That's not a bad technique, but if you're having to go through these
hoops with memory, you probably should be using handles anyway.  Also,
since realloc() is implemented as an inline function, I don't want to
make it any larger than it has to be.

Perhaps if we get to the point where we implement parts of MSL using a
static library we can can offer an extended solution like you suggest.

One note: you could just use DmNewHandle and give is a database ID of
NULL.  That will allocate RAM on the storage heap without associating it
with any database.  That's the technique we use in our expanded mode
runtime library to get the extra storage space on the storage heap.

--

CodeWarrior for Palm OS technical lead
Get help at http://palmoswerks.com/

 
 
 

realloc bug in MSL?

Post by r.. » Mon, 29 Apr 2002 07:52:48



> One note: you could just use DmNewHandle and give is a database ID of
> NULL.  That will allocate RAM on the storage heap without associating it
> with any database.  That's the technique we use in our expanded mode
> runtime library to get the extra storage space on the storage heap.

Thanks!

I also let you know about the technique because I thought you
might have a comment on it, or a better way of doing it...and you did.

Yes, we should be using memory handles, but we wanted to get
the port done ASAP, and this was the best way to go for now.
We may do a memory manager.  But, we do have our own nice
little C++ streams library, now, and we have
cin, cout, and cerr implemented using it.

 
 
 

realloc bug in MSL?

Post by r.. » Mon, 29 Apr 2002 07:56:02



> That's not a bad technique, but if you're having to go through these
> hoops with memory, you probably should be using handles anyway.  

Oh, by the way, we only have this memory issue with Palm OS 3.0.
I guess later versions of Palm OS don't restrict the
new heap size as much?  I think I read this somewhere
in the Palm doc.

I do wish there was also a big warning message, too,
for people using memcmp.  I didn't know about the
Palm OS 3.0 limitation of that function.

 
 
 

realloc bug in MSL?

Post by Ben Combe » Mon, 29 Apr 2002 14:26:02




Quote:> I do wish there was also a big warning message, too,
> for people using memcmp.  I didn't know about the
> Palm OS 3.0 limitation of that function.

Ouch... I'd never remembered reading that part of the Palm OS
documentation.  For the unaware, MemCompare (the API we use to implement
memcmp), is only useful for testing equality on Palm OS 3.5 and later.

To quote the API manual:

MemCmp can be used to test the equality of blocks in memory on all
versions that support MemCmp; however, testing the sort ordering of
blocks in memory works reliably only on Palm OS? versions 3.5 and
higher. On versions earlier than 3.2, MemCmp always returns a positive
value if the blocks are unequal. On versions 3.2 and 3.3, MemCmp
reliably returns positive to indicate s1 > s2 and negative to indicate
s1 < s2 only if the characters that differ are less than 128 apart. If
the difference is greater than that, MemCmp may return positive when it
should return negative and vice versa.

We'll put a different implemention of memcmp in place for the next MSL
release that doesn't use the MemCmp API call.  Thanks for the heads up.

--

CodeWarrior for Palm OS technical lead
Get help at http://palmoswerks.com/

 
 
 

1. Weird bug in MSL implementation of alloc.c

Hi,

General Description :
---------------------
I have a weird bug in MSL implementation of var pools alloc (CW 7.2).
I use a std::deque <SharedPtr::SomeInterface>
where SharedPtr is an implementation of a kind of shared ptr type class
and SomeInterface is an abstract class.
The bug appears while trying to make a push_back on that deque.

I use that in our private libs, that has always worked with various
product on various platforms (92 products on mac, 3 platforms). For 2
products from our products range, the bug appears on carbon version (MSL_
All_Carbon.lib), running MacOSX.
Note that I'm not saying that this is an MSL bug.

Detailled Description :
-----------------------
while making a push_back on that deque, it tries to alloc some memory
from the pool using placement new, and the call finally ends up in Block_
subBlock (alloc.c revision 1.37.4.1).

for some reason (this is the bug in fact), after the call st = Block_
start (ths) (line 803), st->next_ is equal to 0 so line 815 fails (see
macro of SubBlock_size).

I would really be glad to have some help, since I'm not rich enough to
handle codewarrior support plan.

Thanks,

Raphael


2. 6400 printer configuration problem

3. Possible Bug in PGP 6.0 and Outlook/OE Plug-ins

4. cluster computing

5. about realloc memo in TornadoII

6. Tunnelling PPP via telnet or ssh

7. Memory allocation and realloc

8. Updates, reviews and more at DaveNet

9. A problem with realloc

10. Memory fragmentation with realloc()

11. realloc down

12. realloc freed memory

13. realloc problem