64M buffer passed from user to kernel

64M buffer passed from user to kernel

Post by lsnove » Tue, 19 Sep 2000 04:00:00



I'm a newbie and need help.
I need to allocate a 64MB of contiguous memory which needs to be mapped
into user memory.  Then the buffer will be passed to a kernel module
for DMAing to a custom PCI board.  I've heard one approach is to use
kmalloc() and copy_to_user().  Is this my only option?  Does
copy_to_user() create a second copy of the buffer, hence creating 2 64MB
buffers?

Any other suggestions?

Lisa

Sent via Deja.com http://www.deja.com/
Before you buy.

 
 
 

64M buffer passed from user to kernel

Post by Ed Car » Tue, 19 Sep 2000 04:00:00



> I'm a newbie and need help.
> I need to allocate a 64MB of contiguous memory which needs to be mapped
> into user memory.  Then the buffer will be passed to a kernel module
> for DMAing to a custom PCI board.  I've heard one approach is to use
> kmalloc() and copy_to_user().  Is this my only option?  Does
> copy_to_user() create a second copy of the buffer, hence creating 2 64MB
> buffers?

Don't you mean 64KB?

 
 
 

64M buffer passed from user to kernel

Post by Pete Zaitc » Tue, 19 Sep 2000 04:00:00



> I need to allocate a 64MB of contiguous memory which needs to be mapped
> into user memory.  Then the buffer will be passed to a kernel module
> for DMAing to a custom PCI board.  I've heard one approach is to use
> kmalloc() and copy_to_user().  Is this my only option?  Does
> copy_to_user() create a second copy of the buffer, hence creating 2 64MB
> buffers?

I guess you deal with an MPEG encoder/decoder or video capture board.
The only way to get 64MB is to use Pauline's Bigphisarea, allocate
the needed memory in the driver with Bigphisarea calls, then mmap
this into the user space. Check how Zoran and BTTV drivers do it.

The copy_to_user() is a copy, which is not necesserily a bad idea,
but it is a copy nonetheless. The kmalloc() cannot give you large
enough chunks. Neither can get_free_pages().

Quote:> Any other suggestions?

Rape the board designers, preferably with a screwdriver.
For instance, over time C-cube provided alternative protocol
that does not require insane amounts of contiguous memory.
For most boards it can be done with changes in the microcode.

--Pete

 
 
 

64M buffer passed from user to kernel

Post by Mathias Waac » Wed, 20 Sep 2000 04:00:00




> > I need to allocate a 64MB of contiguous memory which needs to be
> >mapped into user memory.  Then the buffer will be passed to a kernel
> >module for DMAing to a custom PCI board.  I've heard one approach is
> >to use kmalloc() and copy_to_user().  Is this my only option?  Does
> >copy_to_user() create a second copy of the buffer, hence creating 2
> >64MB buffers?

> I guess you deal with an MPEG encoder/decoder or video capture board.
> The only way to get 64MB is to use Pauline's Bigphisarea,

No, there are other possible solutions. I prefer reserving memory at
boot time by passing a mem switch to kernel. Its IMHO a cleaner solution then
patching a kernel.

Another really dirty solution would be a mechanism known as "aggressive
allocation": try to allocate as much memory as possible, check for
physical continuous pages, release the other pages. Continue as long
as necessary. But I wouldn't do that.

Mathias

 
 
 

64M buffer passed from user to kernel

Post by lsnove » Wed, 20 Sep 2000 04:00:00







> > > I need to allocate a 64MB of contiguous memory which needs to be
> > >mapped into user memory.  Then the buffer will be passed to a
kernel
> > >module for DMAing to a custom PCI board.  I've heard one approach
is
> > >to use kmalloc() and copy_to_user().  Is this my only option?  Does
> > >copy_to_user() create a second copy of the buffer, hence creating 2
> > >64MB buffers?

> > I guess you deal with an MPEG encoder/decoder or video capture
board.
> > The only way to get 64MB is to use Pauline's Bigphisarea,

> No, there are other possible solutions. I prefer reserving memory at
> boot time by passing a mem switch to kernel. Its IMHO a cleaner
solution then
> patching a kernel.

> Another really dirty solution would be a mechanism known as
"aggressive
> allocation": try to allocate as much memory as possible, check for
> physical continuous pages, release the other pages. Continue as long
> as necessary. But I wouldn't do that.

> Mathias

I like the idea of reserving memory at boot time.  Could you give me
more information on how to do this.  Again, I am a newbie coming from
the vxWorks world.
Lisa

Sent via Deja.com http://www.deja.com/
Before you buy.

 
 
 

64M buffer passed from user to kernel

Post by Norm Dresne » Wed, 20 Sep 2000 04:00:00


Here's an excerpt from the beginning of my /etc/lilo.conf from my RH 5.2
system (k2.0.36):
    boot=/dev/hda
    map=/boot/map
    install=/boot/boot.b
    prompt
    timeout=500
    image=/boot/vmlinuz-2.0.36-0.7
         label=linux
         root=/dev/hda3
         append="mem=122m  shm=0x3FFC00"
         initrd=/boot/initrd-2.0.36-0.7.img
         read-only
    ...

The "mem=122m" tells the kernel that there are only 122 MB of RAM;  there
are more, but it won't even try to use it.  In this system that leaves 127 -
122 = 5 MB for "other" things.

Another way to do this, assuming (1) that you use lilo and (2) that your
kernel-image is called "linux" is to respond to the lilo prompt with
    linux    append="mem=122m"

The quotes are necessary exactly as shown.  If there are multiple
append-strings, they are separated by one or more spaces as shown in the
excerpt from lilo.conf

    Norm



> I like the idea of reserving memory at boot time.  Could you give me
> more information on how to do this.  Again, I am a newbie coming from
> the vxWorks world.
> Lisa

 
 
 

64M buffer passed from user to kernel

Post by Pete Zaitc » Wed, 20 Sep 2000 04:00:00


Quote:> > No, there are other possible solutions. I prefer reserving memory at
> > boot time by passing a mem switch to kernel.
> > Its IMHO a cleaner solution then patching a kernel.

> > Mathias

> I like the idea of reserving memory at boot time.  Could you give me
> more information on how to do this.  Again, I am a newbie coming from
> the vxWorks world.
> Lisa

Don't be stupid, Lisa. In what respect bigphisarea is "dirtier"
than using "mem="? Mathias advocates to pull the subtle work of
picking at mem_start/memory_end inside your driver. If you do that,
you have to deal with any bugs, have intricate understanding how
the memory is allocated, and support multiply kernel revisions.
Certainly a well defined interface of bigphisarea, tested by
numerous developers, provides you a better option.

Just go to the following URL and get the bigphisarea:
 http://www.polyware.nl/~middelin/En/hob-v4l.html

--Pete

 
 
 

64M buffer passed from user to kernel

Post by Mario Klebs » Wed, 20 Sep 2000 04:00:00



>The "mem=122m" tells the kernel that there are only 122 MB of RAM;  there
>are more, but it won't even try to use it.  In this system that leaves 127 -
>122 = 5 MB for "other" things.

Loock like a really ugly trick :-(

There still are some people in this world, who are not using lilo or
any other booter. E.g Linux boots fine from floppy disk. Any driver,
which requires the user to routinely pass parameters to the kernel
IMHO is badly broken. The kernel has to work when no parameters are
passed!

73, Mario
--

PGP-Key available at http://www.klebsch.de/public.key
Fingerprint DSS: EE7C DBCC D9C8 5DC1 D4DB  1483 30CE 9FB2 A047 9CE0
 Diffie-Hellman: D447 4ED6 8A10 2C65 C5E5  8B98 9464 53FF 9382 F518

 
 
 

64M buffer passed from user to kernel

Post by Mathias Waac » Thu, 21 Sep 2000 04:00:00



> There still are some people in this world, who are not using lilo or
> any other booter. E.g Linux boots fine from floppy disk. Any driver,
> which requires the user to routinely pass parameters to the kernel
> IMHO is badly broken. The kernel has to work when no parameters are
> passed!

The kernel will work (eventually). But the driver doesn't. Passing
parameters to the kernel is a well documented feature of Linux, which
can and should be used. There is only one other solution: patching the
kernel. What kind of solution would you prefer? If you think that
kernel parameters are ugly, what do think about kernel patches?

Mathias

 
 
 

64M buffer passed from user to kernel

Post by Mathias Waac » Thu, 21 Sep 2000 04:00:00



> Don't be stupid, Lisa. In what respect bigphisarea is "dirtier" than
> using "mem="? Mathias advocates to pull the subtle work of picking at
> mem_start/memory_end inside your driver.

I answered a posting saying you _must_ use bigphysarea just to show
other solutions. I didn't say you _must_ use the mem switch.

Quote:> If you do that, you have to
> deal with any bugs, have intricate understanding how the memory is
> allocated, and support multiply kernel revisions.  Certainly a well
> defined interface of bigphisarea, tested by numerous developers,
> provides you a better option.

Fine, thats the drawback of the mem switch and the advantages of
the bigphysarea patch. It isn't really serious. Please remember
you can't (eventually) patch already patched kernels, you need a
patch for your exact kernel version and you depend on the
goodwill of the developers. The mem switch is a well documented feature
of the kernel. That's just the opposite point of view. Please don't tell
someone how to do something. Tell her all possibilities and let her
decide. That's the way. I think we agree on that?

Mathias

 
 
 

64M buffer passed from user to kernel

Post by lsnove » Thu, 21 Sep 2000 04:00:00





> > Don't be stupid, Lisa. In what respect bigphisarea is "dirtier" than
> > using "mem="? Mathias advocates to pull the subtle work of picking
at
> > mem_start/memory_end inside your driver.

> I answered a posting saying you _must_ use bigphysarea just to show
> other solutions. I didn't say you _must_ use the mem switch.

> > If you do that, you have to
> > deal with any bugs, have intricate understanding how the memory is
> > allocated, and support multiply kernel revisions.  Certainly a well
> > defined interface of bigphisarea, tested by numerous developers,
> > provides you a better option.

> Fine, thats the drawback of the mem switch and the advantages of
> the bigphysarea patch. It isn't really serious. Please remember
> you can't (eventually) patch already patched kernels, you need a
> patch for your exact kernel version and you depend on the
> goodwill of the developers. The mem switch is a well documented
feature
> of the kernel. That's just the opposite point of view. Please don't
tell
> someone how to do something. Tell her all possibilities and let her
> decide. That's the way. I think we agree on that?

> Mathias

Thanks for all the suggestions, eventhough some of them could have been
presented a little friendlier.  I certainly prefer to get several
solutions so I can pick the best for my situation.  My requirements in
the embedded world may not be the same for those working in the desktop
world.

I am using lilo on a 2.2.12 kernel.  I will probably end up using the
boot parameter solution for short term and then go to the bigphysarea
patch in the near future.  (I need a quick and dirtly for prototype
demo.)
Lisa

Sent via Deja.com http://www.deja.com/
Before you buy.

 
 
 

64M buffer passed from user to kernel

Post by Mario Klebs » Thu, 21 Sep 2000 04:00:00



>The kernel will work (eventually). But the driver doesn't. Passing
>parameters to the kernel is a well documented feature of Linux, which
>can and should be used. There is only one other solution: patching the
>kernel. What kind of solution would you prefer? If you think that
>kernel parameters are ugly, what do think about kernel patches?

What about patching the parameters into the kernel. I can imagine
having a kind of default command line, which is used, when no
parameters are specified. If any driver requires some parameter on the
command line, they could be added to this default command line during
the build process.

73, Mario
--

PGP-Key available at http://www.klebsch.de/public.key
Fingerprint DSS: EE7C DBCC D9C8 5DC1 D4DB  1483 30CE 9FB2 A047 9CE0
 Diffie-Hellman: D447 4ED6 8A10 2C65 C5E5  8B98 9464 53FF 9382 F518