End-of-file & Viewing Binary Files

End-of-file & Viewing Binary Files

Post by Andoma » Mon, 02 Aug 1999 04:00:00



If you write a text file it normally ends with an EOF character.
Does anyone know what this is under Linux?
(I want to add it to a partly binary file with an ASCII header,
so commands like cat and head will stop at the end of the
ASCII section.)

Secondly, is there a way to view files in hex format?
That would be very useful in debugging.

 
 
 

End-of-file & Viewing Binary Files

Post by Kaz Kylhe » Mon, 02 Aug 1999 04:00:00



>If you write a text file it normally ends with an EOF character.

Nonsense. This is only a kludge found in MS-DOS and its descendants.

Quote:>Does anyone know what this is under Linux?

There is no such thing under Linux. A file ends at the point where no more
characters are available.

Quote:>(I want to add it to a partly binary file with an ASCII header,
>so commands like cat and head will stop at the end of the
>ASCII section.)

You are out of luck. Commands like cat will happily work on binary files.  They
won't stop at any character. You could use cat to play sound files (cat meow.au
Quote:> /dev/audio), to files or disks, or concatenate binary files

that have been split (cat part.001 part.002 part.003 > whole) etc.

You need to reevaluate your Microsoft-specific understanding of files
when you upgrade to a different operating system.

 
 
 

End-of-file & Viewing Binary Files

Post by Mattias Engdeg? » Mon, 02 Aug 1999 04:00:00




>>If you write a text file it normally ends with an EOF character.
>Nonsense. This is only a kludge found in MS-DOS and its descendants.

In fact, MS-DOS has it from CP/M, where it actually served a purpose: Files
were not streams of bytes but streams of 128 byte (I think) records, so
a ^Z character in a text file by convention meant that the rest of that
(usually last) record should be disregarded by the application.

Since MS-DOS FAT actually recorded file sizes with byte resolution, it
was obsolete even then.

Quote:>>(I want to add it to a partly binary file with an ASCII header,
>>so commands like cat and head will stop at the end of the
>>ASCII section.)

If you are designing a binary file format, better write a magic(5) entry
for it. Or better, rethink what you are doing and use a text format instead.
 
 
 

End-of-file & Viewing Binary Files

Post by Ron Hous » Tue, 03 Aug 1999 04:00:00



> If you are designing a binary file format, better write a magic(5) entry
> for it. Or better, rethink what you are doing and use a text format instead.

I'll second that! Many programmers jump for binary files without
thinking the issues through properly, and usually the system is the
worse for it. I recall an application I inherited with a binary file
format. My first job was to upgrde it and add a few new fields, and I
stupidly didn't dump the binary format then and there, but struggled
through and got out a version that could read both the first and second
binary format (by dint of having structure declarations for both
formats). Then I wanted to put it on Linux, and discovered that the byte
alignment of fields differed across OSes. Then I did the snesible thing
and dumped the mess, creating an ascii format that started with the
number of the release and the number of fields. If I add extra
information to the format, I increase the field number; the new release
simply says something like:

if (fields > whatever) { ...read next bit... }

and also each release tests that fields is no greater than the number it
expects (so newer data can't be read into older versions of the program
with possibly bad mistakes). If I change the format enough to require a
whole new system of input I bump up the version number. But I've never
yet had to do that because the ascii format works so well - and I can
read it by hand if need be.

--

The evils of each age always seem self-evidently right at the time.

 
 
 

End-of-file & Viewing Binary Files

Post by Christopher B. Brow » Tue, 03 Aug 1999 04:00:00




>> If you are designing a binary file format, better write a magic(5) entry
>> for it. Or better, rethink what you are doing and use a text format instead.

>I'll second that! Many programmers jump for binary files without
>thinking the issues through properly, and usually the system is the
>worse for it.

The usual assumption is that a binary format will be:
a) Terser, or
b) More efficiently read.

These are rarely valid assumptions, and this decidedly goes against
the UNIX Philosophy principle that portable data is Very Valuable.
See URL below...
--
/* I'd just like to take this moment to point out that C has all
   the expressive power of two dixie cups and a string.
 */
-- Jamie Zawinski in the XKeyCaps source

 
 
 

End-of-file & Viewing Binary Files

Post by Michel Bardiau » Tue, 03 Aug 1999 04:00:00





> >> If you are designing a binary file format, better write a magic(5) entry
> >> for it. Or better, rethink what you are doing and use a text format instead.

> >I'll second that! Many programmers jump for binary files without
> >thinking the issues through properly, and usually the system is the
> >worse for it.

> The usual assumption is that a binary format will be:
> a) Terser, or
> b) More efficiently read.

> These are rarely valid assumptions, and this decidedly goes against
> the UNIX Philosophy principle that portable data is Very Valuable.
> See URL below...

Besides, compactness and portability need not be incompatible. Writing
out
data as highly readable text *then* compressing it with ZLIB often gives
as good results (and often better!) then a quick-and-dirty-designed
binary format.

> --
> /* I'd just like to take this moment to point out that C has all
>    the expressive power of two dixie cups and a string.
>  */
> -- Jamie Zawinski in the XKeyCaps source


--
Michel Bardiaux
UsrConsult S.P.R.L.  Rue Margot, 37  B-1457 Nil St Vincent
Tel : +32 10 65.44.15  Fax : +32 10 65.44.10
 
 
 

End-of-file & Viewing Binary Files

Post by Eric Hegstro » Tue, 03 Aug 1999 04:00:00


Being on old Mac hand, this begs a question from me. Is there some sort
of standard resource editor for Linux (much like ResEdit or Resourcerer
on the Mac)? I know the resource/data fork seperation is not a paradigm
that Linux supports.

I am thinking of a configurable binary editor that allows you to define
different binary fields. The files would have to be developed with this
is mind with an index with type identifiers and offsets for
compatibility.

-Eric


> Besides, compactness and portability need not be incompatible. Writing
> out
> data as highly readable text *then* compressing it with ZLIB often gives
> as good results (and often better!) then a quick-and-dirty-designed
> binary format.

> > --
> > /* I'd just like to take this moment to point out that C has all
> >    the expressive power of two dixie cups and a string.
> >  */
> > -- Jamie Zawinski in the XKeyCaps source

> --
> Michel Bardiaux
> UsrConsult S.P.R.L.  Rue Margot, 37  B-1457 Nil St Vincent
> Tel : +32 10 65.44.15  Fax : +32 10 65.44.10

--
Eric Hegstrom                          .~.
Senior Software Engineer               /V\  
Sonoran Scanners, Inc.                // \\          L I N U X

520-617-0072 x402                     ^^-^^
 
 
 

End-of-file & Viewing Binary Files

Post by Christopher Brow » Wed, 04 Aug 1999 04:00:00


On Mon, 02 Aug 1999 10:05:28 +0200, Michel Bardiaux





>> >> If you are designing a binary file format, better write a magic(5) entry
>> >> for it. Or better, rethink what you are doing and use a text format instead.

>> >I'll second that! Many programmers jump for binary files without
>> >thinking the issues through properly, and usually the system is the
>> >worse for it.

>> The usual assumption is that a binary format will be:
>> a) Terser, or
>> b) More efficiently read.

>> These are rarely valid assumptions, and this decidedly goes against
>> the UNIX Philosophy principle that portable data is Very Valuable.
>> See URL below...

>Besides, compactness and portability need not be incompatible. Writing
>out
>data as highly readable text *then* compressing it with ZLIB often gives
>as good results (and often better!) then a quick-and-dirty-designed
>binary format.

Agreed.

Spreadsheets created using Gnumeric use an XML-based format as the
native format.  XML is noted for being fairly verbose, but also for,
by the same token, being Highly Compressible.

Conversely, Microsoft Word and Excel use binary formats, and are
difficult to accuse of efficiency.  :-)
--
To iterate is human; to recurse, divine.

 
 
 

End-of-file & Viewing Binary Files

Post by Christopher Brow » Wed, 04 Aug 1999 04:00:00


On Mon, 02 Aug 1999 10:33:58 -0700, Eric Hegstrom


>Being on old Mac hand, this begs a question from me. Is there some sort
>of standard resource editor for Linux (much like ResEdit or Resourcerer
>on the Mac)? I know the resource/data fork seperation is not a paradigm
>that Linux supports.

>I am thinking of a configurable binary editor that allows you to define
>different binary fields. The files would have to be developed with this
>is mind with an index with type identifiers and offsets for
>compatibility.

There are somewhat analagous tools for working with binary objects
e.g. - as in the output of a language compiler.

It is considered *highly* unusual to edit them by hand, particularly
when a "make" is usually a simpler way to handle this.

Traditionally, it has been more usual to use text files with simple
languages for this purpose.

The much-hyped XML language represents a standardized way of managing
this sort of thing that would represent a more appropriate direction
to go.
--
How do I type "for i in *.dvi do xdvi $i done" in a GUI?
(Discussion in comp.os.linux.misc on the intuitiveness of interfaces.)

 
 
 

End-of-file & Viewing Binary Files

Post by Eric Hegstro » Wed, 04 Aug 1999 04:00:00


While compiling particular text files to create resources is a
necessity. Being able to view and edit those resources (at least
identify them) was a great virtue of the Macintosh resource/data fork
paradigm.

Loosely it means logically each file had a data and resource part
(called forks on the Mac). The data fork contains raw data. That is what
we would see in a fread or read. The resource fork is accessed by using
a set of APIs for finding and working with a resource by name ID, or
type or by indexing through them.

Each resource fork has a directory (sort of like a flat file system)
that has for each contained resource: An ID number (16bit int) a
Resource Type (32bit int usually represented by 4 chars for human
readability). A Name (string) and it's offset.

The great feature is that you can then open any files resource fork with
ResEdit or Resourcerer and it will display a list of all the resources
in that file. If you have a template for that resource it will let you
view/edit/replace/copy it if possible. It was a great compatible way for
an application to store generic configurable data like lists of strings,
pictures, sounds, pallates, as well as other user defined arrays of
values and strings in a binary format. Obviously there are some major
problems with aligment and word size issues if something like this was
used across multiple platforms.

I have always missed these beasts. Maybe XML will help me in the future.

-Eric


> There are somewhat analagous tools for working with binary objects
> e.g. - as in the output of a language compiler.

> It is considered *highly* unusual to edit them by hand, particularly
> when a "make" is usually a simpler way to handle this.

> Traditionally, it has been more usual to use text files with simple
> languages for this purpose.

> The much-hyped XML language represents a standardized way of managing
> this sort of thing that would represent a more appropriate direction
> to go.
> --
> How do I type "for i in *.dvi do xdvi $i done" in a GUI?
> (Discussion in comp.os.linux.misc on the intuitiveness of interfaces.)


--
Eric Hegstrom                          .~.
Senior Software Engineer               /V\  
Sonoran Scanners, Inc.                // \\          L I N U X

520-617-0072 x402                     ^^-^^
 
 
 

End-of-file & Viewing Binary Files

Post by a.. » Wed, 04 Aug 1999 04:00:00



Quote:>I am thinking of a configurable binary editor that allows you to define
>different binary fields. The files would have to be developed with this
>is mind with an index with type identifiers and offsets for compatibility.

Well, there is BE, but I'm not sure this is quite what you want.
See http://www.interalpha.net/customer/nyangau/

{{{ Andy

 
 
 

End-of-file & Viewing Binary Files

Post by Christopher Brow » Thu, 05 Aug 1999 04:00:00


On Mon, 02 Aug 1999 10:05:28 +0200, Michel Bardiaux





>> >> If you are designing a binary file format, better write a magic(5) entry
>> >> for it. Or better, rethink what you are doing and use a text format instead.

>> >I'll second that! Many programmers jump for binary files without
>> >thinking the issues through properly, and usually the system is the
>> >worse for it.

>> The usual assumption is that a binary format will be:
>> a) Terser, or
>> b) More efficiently read.

>> These are rarely valid assumptions, and this decidedly goes against
>> the UNIX Philosophy principle that portable data is Very Valuable.
>> See URL below...

>Besides, compactness and portability need not be incompatible. Writing
>out
>data as highly readable text *then* compressing it with ZLIB often gives
>as good results (and often better!) then a quick-and-dirty-designed
>binary format.

Agreed.

Spreadsheets created using Gnumeric use an XML-based format as the
native format.  XML is noted for being fairly verbose, but also for,
by the same token, being Highly Compressible.

Conversely, Microsoft Word and Excel use binary formats, and are
difficult to accuse of efficiency.  :-)
--
To iterate is human; to recurse, divine.

 
 
 

End-of-file & Viewing Binary Files

Post by Christopher Brow » Thu, 05 Aug 1999 04:00:00


On Mon, 02 Aug 1999 10:33:58 -0700, Eric Hegstrom


>Being on old Mac hand, this begs a question from me. Is there some sort
>of standard resource editor for Linux (much like ResEdit or Resourcerer
>on the Mac)? I know the resource/data fork seperation is not a paradigm
>that Linux supports.

>I am thinking of a configurable binary editor that allows you to define
>different binary fields. The files would have to be developed with this
>is mind with an index with type identifiers and offsets for
>compatibility.

There are somewhat analagous tools for working with binary objects
e.g. - as in the output of a language compiler.

It is considered *highly* unusual to edit them by hand, particularly
when a "make" is usually a simpler way to handle this.

Traditionally, it has been more usual to use text files with simple
languages for this purpose.

The much-hyped XML language represents a standardized way of managing
this sort of thing that would represent a more appropriate direction
to go.
--
How do I type "for i in *.dvi do xdvi $i done" in a GUI?
(Discussion in comp.os.linux.misc on the intuitiveness of interfaces.)

 
 
 

End-of-file & Viewing Binary Files

Post by Mattias Engdeg? » Thu, 05 Aug 1999 04:00:00



>While compiling particular text files to create resources is a
>necessity. Being able to view and edit those resources (at least
>identify them) was a great virtue of the Macintosh resource/data fork
>paradigm.

The chief advantages of the Mac resource system are:

        a. Data is stored in common, well-defined formats
and
        b. There is a good standard tool to create/edit these pieces of data.

The faintly controversial dual-forked file system is not really
relevant, since the advantages remain with separate resource files or
resources embedded in sections of binaries (ELF, for instance).

The closest Unix equivalent is perhaps a directory tree with lots of
small files, each in a standard format, that can be manipulated with
standard tools. I recently had to choose a resource system for a game,
and by using a directory tree with images, sounds and texts in
standard formats, I didn't have to invent any custom tools at all.
And ordinary commands (find, grep, shell/perl/awk scripts) can be used to
traverse the "resource database". I think it is the Unix Way.

If someone is concerned about the increased disk usage, put the tree
in a tar/ar/xml file and write a wrapper lib for all resource access.
This way the tree can be compacted to one file for delivery, while
keeping the flexibility during development. Or use Reiserfs.

 
 
 

1. Does end-of-file (end-of-page) key work in Netscape 4.03 (LinuxDoes end-of-file (end-of-page) key work in Netscape 4.03 (Linux

For Netscape 4.03 (stand-alone) under Linux:

I have tried Ctrl-PageDown, Alt-PageDown, Ctrl-End,
Alt-End, Meta->  (Esc->) and many other keystroke combinations
to go the end of web page.

Is there a keystroke to do this that perhaps I haven't tried?

If not, is there a way to fix it perhaps with the addition
of a line in preferences.js or .Xdefaults ?

(Of course, I would like to be able to jump to the top of
the page, also.)

If it's not fixable in 4.03, do the latest versions fix this?

Thanks

P.S. If there's a more appropriate group to post this
message in, please let me know.

NOTE-- Junk mail trap:
To e-mail me, please remove the following word from the
address above: system

2. ISAPNP error when gamepad is plugged in?

3. Corrupted fonts after viewing binary file

4. moc ??

5. viewing binary files

6. don't understand route table!

7. Viewing a binary file

8. Vi Editor that shows comments in different colors

9. Write End Of File in current file?

10. Problem with mksysb:tar:Reached end-of-file before expected.

11. Reading past end-of-file on DAT tape

12. tar-question: reached end-of-file before expected ???

13. End-of-file marker using dd