Problem with fwrite()

Problem with fwrite()

Post by Hemant Sha » Sun, 06 Oct 2002 02:31:41



Folks,

  I am having problem with fwrite() function. I have written a utility to
  convert file from one format to another and I use fwrite() to write to the
  file. I get EFBIG error when file reaches 2GB limit. The file system on
  which I am writing the file is large file enabled, and ulimit is set to
  unlimited.

  As further test I tried to create a large file using dd command:

  dd if=/dev/zero of=test bs=32k

  and I was able to create file larger then 2GB, so the problem is not with
  the file system.

  Is there any limitation with fwrite()?

  Here is the fragment of code:

void SeqWriteRecord(struct WriteInfo *WriteInfo, enum RecordType RecordType,
                    char *Buffer, int DataSize)
{
   int SaveErrno;

   if (WriteInfo == NULL)
   {
      return;
   }
   WriteHeader(DataSize + 4, RecordType, WriteInfo->FilePtr);

   if (fwrite(Buffer, (size_t)1, (size_t)DataSize, WriteInfo->FilePtr)
       != DataSize)
   {
      SaveErrno = errno;
      eFatal(10828, "SeqWriteRecord: Could not write full record of size %d  errno = %d", DataSize, SaveErrno);
   }

Quote:}  

  The errno is 27 which ie EFBIG.

  I need to fix is A.S.A.P.

  Thanks is advance.

--
Hemant Shah                           /"\  ASCII ribbon campaign

                                       X     against HTML mail
TO REPLY, REMOVE NoJunkMail           / \      and postings      
FROM MY E-MAIL ADDRESS.          
-----------------[DO NOT SEND UNSOLICITED BULK E-MAIL]------------------
I haven't lost my mind,                Above opinions are mine only.
it's backed up on tape somewhere.      Others can have their own.

 
 
 

Problem with fwrite()

Post by Dragan Cvetkovi » Sun, 06 Oct 2002 02:37:13



> Folks,

>   I am having problem with fwrite() function. I have written a utility to
>   convert file from one format to another and I use fwrite() to write to the
>   file. I get EFBIG error when file reaches 2GB limit. The file system on
>   which I am writing the file is large file enabled, and ulimit is set to
>   unlimited.

>   As further test I tried to create a large file using dd command:

>   dd if=/dev/zero of=test bs=32k

>   and I was able to create file larger then 2GB, so the problem is not with
>   the file system.

>   Is there any limitation with fwrite()?

Does write() works for you? Did you compile your program with large file
enabled?

Bye, Dragan

--
Dragan Cvetkovic,

To be or not to be is true. G. Boole      No it isn't.  L. E. J. Brouwer

 
 
 

Problem with fwrite()

Post by Ming X » Sun, 06 Oct 2002 03:38:45


Hi,
The explanation for EFBIG from the doc is,
EFBIG Indicates that an attempt was made to write a file that exceeds the
file size of the process limit or the systemwide maximum file size.
So, please check the limit for the both.

Good luck,
Ming


Quote:

> Folks,

>   I am having problem with fwrite() function. I have written a utility to
>   convert file from one format to another and I use fwrite() to write to
the
>   file. I get EFBIG error when file reaches 2GB limit. The file system on
>   which I am writing the file is large file enabled, and ulimit is set to
>   unlimited.

>   As further test I tried to create a large file using dd command:

>   dd if=/dev/zero of=test bs=32k

>   and I was able to create file larger then 2GB, so the problem is not
with
>   the file system.

>   Is there any limitation with fwrite()?

>   Here is the fragment of code:

> void SeqWriteRecord(struct WriteInfo *WriteInfo, enum RecordType
RecordType,
>                     char *Buffer, int DataSize)
> {
>    int SaveErrno;

>    if (WriteInfo == NULL)
>    {
>       return;
>    }
>    WriteHeader(DataSize + 4, RecordType, WriteInfo->FilePtr);

>    if (fwrite(Buffer, (size_t)1, (size_t)DataSize, WriteInfo->FilePtr)
>        != DataSize)
>    {
>       SaveErrno = errno;
>       eFatal(10828, "SeqWriteRecord: Could not write full record of size

%d  errno = %d", DataSize, SaveErrno);

- Show quoted text -

>    }
> }

>   The errno is 27 which ie EFBIG.

>   I need to fix is A.S.A.P.

>   Thanks is advance.

> --
> Hemant Shah                           /"\  ASCII ribbon campaign

>                                        X     against HTML mail
> TO REPLY, REMOVE NoJunkMail           / \      and postings
> FROM MY E-MAIL ADDRESS.
> -----------------[DO NOT SEND UNSOLICITED BULK E-MAIL]------------------
> I haven't lost my mind,                Above opinions are mine only.
> it's backed up on tape somewhere.      Others can have their own.

 
 
 

Problem with fwrite()

Post by Hemant Sha » Sun, 06 Oct 2002 04:33:58



:)
:)> Folks,
:)>
:)>   I am having problem with fwrite() function. I have written a utility to
:)>   convert file from one format to another and I use fwrite() to write to the
:)>   file. I get EFBIG error when file reaches 2GB limit. The file system on
:)>   which I am writing the file is large file enabled, and ulimit is set to
:)>   unlimited.
:)>
:)>   As further test I tried to create a large file using dd command:
:)>
:)>   dd if=/dev/zero of=test bs=32k
:)>
:)>   and I was able to create file larger then 2GB, so the problem is not with
:)>   the file system.
:)>  
:)>   Is there any limitation with fwrite()?
:)>
:)
:)Does write() works for you? Did you compile your program with large file
:)enabled?
:)
:)Bye, Dragan

  I am not sure if my first followup made it so here it is again.

  In order to change to write, will have to change several other files,
  since this is a utility function used by many other programs.

  How do I compile code as large file enabled? I could not find any option in
  man page for xlc.

  Thanks.

:)
:)
:)--
:)Dragan Cvetkovic,
:)
:)To be or not to be is true. G. Boole      No it isn't.  L. E. J. Brouwer

--
Hemant Shah                           /"\  ASCII ribbon campaign

                                       X     against HTML mail
TO REPLY, REMOVE NoJunkMail           / \      and postings      
FROM MY E-MAIL ADDRESS.          
-----------------[DO NOT SEND UNSOLICITED BULK E-MAIL]------------------
I haven't lost my mind,                Above opinions are mine only.
it's backed up on tape somewhere.      Others can have their own.

 
 
 

Problem with fwrite()

Post by Hemant Sha » Sun, 06 Oct 2002 04:35:47



:)Hi,
:)The explanation for EFBIG from the doc is,
:)EFBIG Indicates that an attempt was made to write a file that exceeds the
:)file size of the process limit or the systemwide maximum file size.
:)So, please check the limit for the both.
:)
:)Good luck,
:)Ming
:)

  I checked the limit, it is set to unlimited for all users. I was able to
  create file larger than 2GB using GB using dd.


:)>
:)> Folks,
:)>
:)>   I am having problem with fwrite() function. I have written a utility to
:)>   convert file from one format to another and I use fwrite() to write to
:)the
:)>   file. I get EFBIG error when file reaches 2GB limit. The file system on
:)>   which I am writing the file is large file enabled, and ulimit is set to
:)>   unlimited.
:)>
:)>   As further test I tried to create a large file using dd command:
:)>
:)>   dd if=/dev/zero of=test bs=32k
:)>
:)>   and I was able to create file larger then 2GB, so the problem is not
:)with
:)>   the file system.
:)>
:)>   Is there any limitation with fwrite()?
:)>
:)>   Here is the fragment of code:
:)>
:)> void SeqWriteRecord(struct WriteInfo *WriteInfo, enum RecordType
:)RecordType,
:)>                     char *Buffer, int DataSize)
:)> {
:)>    int SaveErrno;
:)>
:)>    if (WriteInfo == NULL)
:)>    {
:)>       return;
:)>    }
:)>    WriteHeader(DataSize + 4, RecordType, WriteInfo->FilePtr);
:)>
:)>    if (fwrite(Buffer, (size_t)1, (size_t)DataSize, WriteInfo->FilePtr)
:)>        != DataSize)
:)>    {
:)>       SaveErrno = errno;
:)>       eFatal(10828, "SeqWriteRecord: Could not write full record of size
:)%d  errno = %d", DataSize, SaveErrno);
:)>    }
:)> }
:)>
:)>   The errno is 27 which ie EFBIG.
:)>
:)>
:)>   I need to fix is A.S.A.P.
:)>
:)>   Thanks is advance.
:)>
:)>
:)>
:)> --
:)> Hemant Shah                           /"\  ASCII ribbon campaign

:)>                                        X     against HTML mail
:)> TO REPLY, REMOVE NoJunkMail           / \      and postings
:)> FROM MY E-MAIL ADDRESS.
:)> -----------------[DO NOT SEND UNSOLICITED BULK E-MAIL]------------------
:)> I haven't lost my mind,                Above opinions are mine only.
:)> it's backed up on tape somewhere.      Others can have their own.
:)
:)

--
Hemant Shah                           /"\  ASCII ribbon campaign

                                       X     against HTML mail
TO REPLY, REMOVE NoJunkMail           / \      and postings      
FROM MY E-MAIL ADDRESS.          
-----------------[DO NOT SEND UNSOLICITED BULK E-MAIL]------------------
I haven't lost my mind,                Above opinions are mine only.
it's backed up on tape somewhere.      Others can have their own.

 
 
 

Problem with fwrite()

Post by Dragan Cvetkovi » Sun, 06 Oct 2002 04:50:06




> :)Does write() works for you? Did you compile your program with large file
> :)enabled?
> :)
> :)Bye, Dragan

>   I am not sure if my first followup made it so here it is again.

>   In order to change to write, will have to change several other files,
>   since this is a utility function used by many other programs.

>   How do I compile code as large file enabled? I could not find any option in
>   man page for xlc.

I think you need to define _LARGE_FILES. Also, in the case of open/write
usage, you have to use O_LARGEFILE flag, not sure what's the equivalent for
streams. See man page for open.

I used to have some web links pointing to AIX documentation but it seems
that IBM has reorganised it and none of my links work any more. :-(

Bye, Dragan

--
Dragan Cvetkovic,

To be or not to be is true. G. Boole      No it isn't.  L. E. J. Brouwer

 
 
 

Problem with fwrite()

Post by Dragan Cvetkovi » Sun, 06 Oct 2002 04:58:03



> I think you need to define _LARGE_FILES. Also, in the case of open/write
> usage, you have to use O_LARGEFILE flag, not sure what's the equivalent for
> streams. See man page for open.

> I used to have some web links pointing to AIX documentation but it seems
> that IBM has reorganised it and none of my links work any more. :-(

> Bye, Dragan

OK, I have found the link. For AIX4.3, it's at

http://publib.boulder.ibm.com/doc_link/en_US/a_doc_lib/aixprggd/genpr...

Have fun. Dragan

--
Dragan Cvetkovic,

To be or not to be is true. G. Boole      No it isn't.  L. E. J. Brouwer

 
 
 

Problem with fwrite()

Post by Ming X » Sun, 06 Oct 2002 05:48:46


How about check the file /etc/security/limites. The file defines the limit
for file size of each user can create. But you said you have created a file
over 2GB. Just give a check.

I don't see any limit on the fwrite() itself. It looks like from File
System. I also check the xlc (I have C for AIX 5.0) and don't find any
options for large file.

Ming



> :)Hi,
> :)The explanation for EFBIG from the doc is,
> :)EFBIG Indicates that an attempt was made to write a file that exceeds
the
> :)file size of the process limit or the systemwide maximum file size.
> :)So, please check the limit for the both.
> :)
> :)Good luck,
> :)Ming
> :)

>   I checked the limit, it is set to unlimited for all users. I was able to
>   create file larger than 2GB using GB using dd.



> :)>
> :)> Folks,
> :)>
> :)>   I am having problem with fwrite() function. I have written a utility
to
> :)>   convert file from one format to another and I use fwrite() to write
to
> :)the
> :)>   file. I get EFBIG error when file reaches 2GB limit. The file system
on
> :)>   which I am writing the file is large file enabled, and ulimit is set
to
> :)>   unlimited.
> :)>
> :)>   As further test I tried to create a large file using dd command:
> :)>
> :)>   dd if=/dev/zero of=test bs=32k
> :)>
> :)>   and I was able to create file larger then 2GB, so the problem is not
> :)with
> :)>   the file system.
> :)>
> :)>   Is there any limitation with fwrite()?
> :)>
> :)>   Here is the fragment of code:
> :)>
> :)> void SeqWriteRecord(struct WriteInfo *WriteInfo, enum RecordType
> :)RecordType,
> :)>                     char *Buffer, int DataSize)
> :)> {
> :)>    int SaveErrno;
> :)>
> :)>    if (WriteInfo == NULL)
> :)>    {
> :)>       return;
> :)>    }
> :)>    WriteHeader(DataSize + 4, RecordType, WriteInfo->FilePtr);
> :)>
> :)>    if (fwrite(Buffer, (size_t)1, (size_t)DataSize, WriteInfo->FilePtr)
> :)>        != DataSize)
> :)>    {
> :)>       SaveErrno = errno;
> :)>       eFatal(10828, "SeqWriteRecord: Could not write full record of
size
> :)%d  errno = %d", DataSize, SaveErrno);
> :)>    }
> :)> }
> :)>
> :)>   The errno is 27 which ie EFBIG.
> :)>
> :)>
> :)>   I need to fix is A.S.A.P.
> :)>
> :)>   Thanks is advance.
> :)>
> :)>
> :)>
> :)> --
> :)> Hemant Shah                           /"\  ASCII ribbon campaign

> :)>                                        X     against HTML mail
> :)> TO REPLY, REMOVE NoJunkMail           / \      and postings
> :)> FROM MY E-MAIL ADDRESS.
> :)> -----------------[DO NOT SEND UNSOLICITED BULK

E-MAIL]------------------

- Show quoted text -

> :)> I haven't lost my mind,                Above opinions are mine only.
> :)> it's backed up on tape somewhere.      Others can have their own.
> :)
> :)

> --
> Hemant Shah                           /"\  ASCII ribbon campaign

>                                        X     against HTML mail
> TO REPLY, REMOVE NoJunkMail           / \      and postings
> FROM MY E-MAIL ADDRESS.
> -----------------[DO NOT SEND UNSOLICITED BULK E-MAIL]------------------
> I haven't lost my mind,                Above opinions are mine only.
> it's backed up on tape somewhere.      Others can have their own.

 
 
 

Problem with fwrite()

Post by Hemant Sha » Sun, 06 Oct 2002 06:07:59



:)> I think you need to define _LARGE_FILES. Also, in the case of open/write
:)> usage, you have to use O_LARGEFILE flag, not sure what's the equivalent for
:)> streams. See man page for open.
:)>
:)> I used to have some web links pointing to AIX documentation but it seems
:)> that IBM has reorganised it and none of my links work any more. :-(
:)>
:)> Bye, Dragan
:)>
:)
:)OK, I have found the link. For AIX4.3, it's at
:)
:)http://publib.boulder.ibm.com/doc_link/en_US/a_doc_lib/aixprggd/genpr...
:)
:)Have fun. Dragan

  Thanks. This is what I was looking for.
:)
:)--
:)Dragan Cvetkovic,
:)
:)To be or not to be is true. G. Boole      No it isn't.  L. E. J. Brouwer

--
Hemant Shah                           /"\  ASCII ribbon campaign

                                       X     against HTML mail
TO REPLY, REMOVE NoJunkMail           / \      and postings      
FROM MY E-MAIL ADDRESS.          
-----------------[DO NOT SEND UNSOLICITED BULK E-MAIL]------------------
I haven't lost my mind,                Above opinions are mine only.
it's backed up on tape somewhere.      Others can have their own.

 
 
 

1. Problem fread/fwriting sockets

I'm doing a lot of client-server stuff here, using both Inet and UNIX
sockets.  For portability reasons, I'm fdopen()ing connected sockets,
and then using fread() and fwrite() to do the i/o.  I'm having a
problem where a client writes 512 bytes to the server, and, rather than
receiving that same data back (which is what should happen), it
receives it with the last byte it sent prepended.  That is, it's like
the last byte doesn't quite make it out the other end (although the
server reads it just fine), and is pushed back to the client in front
of the data being written by the server.  Everything looks like it's
working fine on the server end.  All these socket connections have a
setvbuf(FILE *,(char *) NULL,_IONBF,(size_t) 0) call immediately after
the fdopen(), so the i/o should be completely unbuffered.  The
fread/fwrite's are of the form fread(void *, 1, 512, FILE *).  I've
found a workaround by calling fseek(FILE *, 0, SEEK_CUR) after each
fread and fwrite, but this is obviously a hack masking a problem which
I don't understand.  Any suggestions as to what the problem might be
would be greatly appreciated.  I love a mystery, but this one's gone on
too long, and I'm out of suspects.
--
Jim Tomlinson                         (206)865-6578  \  "falling snow

Boeing Computer Services   ...uunet!bcstec!voodoo!jdt  \  - Anderson/Gabriel

2. RR Cable and Linux configuration

3. fwrite problem

4. Upgrading: Utilities

5. problem in C using fwrite

6. LAN connection to Morningstar PPP

7. weird problem with (fseek/ftell/fwrite/putc)

8. is 3.1 basic config misleading?

9. Problem: fwrite and fseek.

10. How do you use fread and fwrite in c with unix

11. fwrite error 32 - what to do ?

12. How to fwrite/write file exceed 2gig on solaris 2.6?

13. fwrite doesn't seem to work right