[RFC] new module interface

[RFC] new module interface

Post by Roman Zippe » Fri, 26 Jul 2002 05:10:09



Hi,

The patch below is for 2.4 but it's easily ported to 2.5, beside of this I
think the core is stable and will allow a more flexible module handling
in the future. After updating to 2.5 and updating some more archs I will
submit the patch officially, so any feedback now would be very welcome.
(The patch requires no new modutils, although a new version could avoid
some workarounds, but that can wait.)

This patch implements a new module interface. The most important change is
probably that the usercount is not part of the module structure anymore.
If the module code needs it, it asks the module for it. That count doesn't
has to be accurate, only the return value of the exit functions decides
about the unloading of the module.
This also means no kernel structures have to be polluted with module
pointers, the module structure is basically now private to the module
code, no normal module needs to mess with it. This also means
MOD_INC_USE_COUNT/MOD_DEC_USE_COUNT/MOD_IN_USE are now finally really
obsolete. Modules have to maintain their own usecount, but this usually
less painful than using a module pointer (see fs example below). In other
situation such a count already exists, e.g. a proc like interface could
use the i_count field of the inode (stop() unlinks the entry, exit()
would test if i_count became 1).
Below I converted affs and the fs code to demonstrate the new interface.
So a module is defined like this:

DEFINE_MODULE
        .start =        start_affs_fs,
        .stop =         stop_affs_fs,
        .exit =         exit_affs_fs,
        .usecount =     usecount_affs_fs,
DEFINE_MODULE_END

This defines a structure, which is automatically registered at module
init. A structure is more flexible than lots of function entries. insmod
has to know about all these functions, where as above structure is
automatically initialized when the module is linked. New fields can be
added easily, e.g. new fixup sections can be added without bothering
insmod.
On the other hand the old interfaces still work (even a direct
init_module() call), it's just not possible to unload such modules.

One user visible difference is that /proc/modules now includes builtin
modules. It requires a small kbuild change, as it needs unique module
names. I'm not completely happy with this part yet, but it shows what is
basically possible. More important is the difference between builtin
modules and loaded modules becomes less.

Some more boring implementation details:
- get_kernel_syms syscall is gone, it's not needed by modutils anymore,
  only klogd is still using it, but it can live without it and should
  rather be updated.
- new macros mod_for_each_locked/mod_for_done_locked to iterate over the
  modules list
- __MODULE_STRING is replaced with __stringify
- BKL is replaced with a semaphore
- get_mod_name/put_mod_name is replaced with getname/putname

bye, Roman

Index: Rules.make
===================================================================
RCS file: /home/other/cvs/linux/linux-2.4/Rules.make,v
retrieving revision 1.1.1.8
diff -u -p -r1.1.1.8 Rules.make
--- Rules.make  2002/03/12 13:09:42     1.1.1.8
+++ Rules.make  2002/07/24 10:06:19
@@ -44,6 +44,7 @@ SUB_DIRS      := $(subdir-y)
 MOD_SUB_DIRS   := $(sort $(subdir-m) $(both-m))
 ALL_SUB_DIRS   := $(sort $(subdir-y) $(subdir-m) $(subdir-n) $(subdir-))

+KBUILD_BASEDIR := $(subst /,_,$(subst -,_,$(subst $(TOPDIR)/,,$(shell /bin/pwd))))

 #
 # Common rules
@@ -56,7 +57,7 @@ ALL_SUB_DIRS  := $(sort $(subdir-y) $(sub
        $(CPP) $(CFLAGS) $(EXTRA_CFLAGS) $(CFLAGS_$@) $< > $@

 %.o: %.c
-       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F))) $(CFLAGS_$@) -c -o $@ $<
+       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) -DKBUILD_BASENAME=$(KBUILD_BASEDIR)_$(subst $(comma),_,$(subst -,_,$(*F))) $(CFLAGS_$@) -c -o $@ $<
        @ ( \
            echo 'ifeq ($(strip $(subst $(comma),:,$(CFLAGS) $(EXTRA_CFLAGS) $(CFLAGS_$@))),$$(strip $$(subst $$(comma),:,$$(CFLAGS) $$(EXTRA_CFLAGS) $$(CFLAGS_$@))))' ; \
            echo 'FILES_FLAGS_UP_TO_DATE += $@' ; \
@@ -272,7 +273,7 @@ endif # CONFIG_MODVERSIONS

 ifneq "$(strip $(export-objs))" ""
 $(export-objs): $(export-objs:.o=.c) $(TOPDIR)/include/linux/modversions.h
-       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F))) $(CFLAGS_$@) -DEXPORT_SYMTAB -c $(@:.o=.c)
+       $(CC) $(CFLAGS) $(EXTRA_CFLAGS) -DKBUILD_BASENAME=$(KBUILD_BASEDIR)_$(subst $(comma),_,$(subst -,_,$(*F))) $(CFLAGS_$@) -DEXPORT_SYMTAB -c $(@:.o=.c)
        @ ( \
            echo 'ifeq ($(strip $(subst $(comma),:,$(CFLAGS) $(EXTRA_CFLAGS) $(CFLAGS_$@) -DEXPORT_SYMTAB)),$$(strip $$(subst $$(comma),:,$$(CFLAGS) $$(EXTRA_CFLAGS) $$(CFLAGS_$@) -DEXPORT_SYMTAB)))' ; \
            echo 'FILES_FLAGS_UP_TO_DATE += $@' ; \
Index: arch/i386/kernel/entry.S
===================================================================
RCS file: /home/other/cvs/linux/linux-2.4/arch/i386/kernel/entry.S,v
retrieving revision 1.1.1.14
diff -u -p -r1.1.1.14 entry.S
--- arch/i386/kernel/entry.S    2002/03/12 13:09:49     1.1.1.14
+++ arch/i386/kernel/entry.S    2002/07/24 10:06:19
@@ -526,7 +526,7 @@ ENTRY(sys_call_table)
        .long SYMBOL_NAME(sys_create_module)
        .long SYMBOL_NAME(sys_init_module)
        .long SYMBOL_NAME(sys_delete_module)
-       .long SYMBOL_NAME(sys_get_kernel_syms)  /* 130 */
+       .long SYMBOL_NAME(sys_ni_syscall)       /* 130 */               /* old get_kernel_syms syscall */
        .long SYMBOL_NAME(sys_quotactl)
        .long SYMBOL_NAME(sys_getpgid)
        .long SYMBOL_NAME(sys_fchdir)
Index: arch/i386/kernel/traps.c
===================================================================
RCS file: /home/other/cvs/linux/linux-2.4/arch/i386/kernel/traps.c,v
retrieving revision 1.1.1.19
diff -u -p -r1.1.1.19 traps.c
--- arch/i386/kernel/traps.c    2001/10/18 14:09:04     1.1.1.19
+++ arch/i386/kernel/traps.c    2002/07/24 10:06:19
@@ -94,21 +94,19 @@ int kstack_depth_to_print = 24;
  * be the address of a calling routine
  */

-#ifdef CONFIG_MODULES
-
-extern struct module *module_list;
-extern struct module kernel_module;
-
 static inline int kernel_text_address(unsigned long addr)
 {
        int retval = 0;
+#ifdef CONFIG_MODULES
        struct module *mod;
+#endif

        if (addr >= (unsigned long) &_stext &&
            addr <= (unsigned long) &_etext)
                return 1;

-       for (mod = module_list; mod != &kernel_module; mod = mod->next) {
+#ifdef CONFIG_MODULES
+       mod_for_each_locked(mod) {
                /* mod_bound tests for addr being inside the vmalloc'ed
                 * module area. Of course it'd be better to test only
                 * for the .text subset... */
@@ -116,20 +114,11 @@ static inline int kernel_text_address(un
                        retval = 1;
                        break;
                }
-       }
+       } mod_for_done_locked
+#endif

        return retval;
 }
-
-#else
-
-static inline int kernel_text_address(unsigned long addr)
-{
-       return (addr >= (unsigned long) &_stext &&
-               addr <= (unsigned long) &_etext);
-}
-
-#endif

 void show_trace(unsigned long * stack)
 {
Index: arch/i386/mm/extable.c
===================================================================
RCS file: /home/other/cvs/linux/linux-2.4/arch/i386/mm/extable.c,v
retrieving revision 1.1.1.4
diff -u -p -r1.1.1.4 extable.c
--- arch/i386/mm/extable.c      2001/10/18 13:38:54     1.1.1.4
+++ arch/i386/mm/extable.c      2002/07/24 10:06:19
@@ -15,7 +15,7 @@ search_one_table(const struct exception_
                 const struct exception_table_entry *last,
                 unsigned long value)
 {
-        while (first <= last) {
+        while (first < last) {
                const struct exception_table_entry *mid;
                long diff;

@@ -31,32 +31,24 @@ search_one_table(const struct exception_
         return 0;
 }

-extern spinlock_t modlist_lock;
-
 unsigned long
 search_exception_table(unsigned long addr)
 {
-       unsigned long ret = 0;

 #ifndef CONFIG_MODULES
        /* There is only the kernel to search.  */
-       ret = search_one_table(__start___ex_table, __stop___ex_table-1, addr);
-       return ret;
+       return search_one_table(__start___ex_table, __stop___ex_table, addr);
 #else
-       unsigned long flags;
-       /* The kernel is the last "module" -- no need to treat it special.  */
+       unsigned long ret = 0;
        struct module *mp;

-       spin_lock_irqsave(&modlist_lock, flags);
-       for (mp = module_list; mp != NULL; mp = mp->next) {
-               if (mp->ex_table_start == NULL || !(mp->flags&(MOD_RUNNING|MOD_INITIALIZING)))
-                       continue;
+       mod_for_each_locked(mp) {
                ret = search_one_table(mp->ex_table_start,
-                                      mp->ex_table_end - 1, addr);
+                                      mp->ex_table_end, addr);
                if (ret)
                        break;
-       }
-       spin_unlock_irqrestore(&modlist_lock, flags);
+       } mod_for_done_locked
+
        return ret;
 #endif
 }
Index: fs/dnotify.c
===================================================================
RCS file: /home/other/cvs/linux/linux-2.4/fs/dnotify.c,v
retrieving revision 1.1.1.2
diff -u -p -r1.1.1.2 dnotify.c
--- fs/dnotify.c        2000/12/05 11:46:49     1.1.1.2
+++ fs/dnotify.c        2002/07/24 10:06:19
@@ -13,6 +13,7 @@
  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
  * General Public License for more details.
  */
+#include <linux/module.h>
 #include <linux/fs.h>
 #include <linux/sched.h>
 #include <linux/dnotify.h>
Index: fs/fcntl.c
===================================================================
RCS file: /home/other/cvs/linux/linux-2.4/fs/fcntl.c,v
retrieving revision 1.1.1.10
diff -u -p -r1.1.1.10 fcntl.c
--- fs/fcntl.c  2001/10/18 13:40:46     1.1.1.10
+++ fs/fcntl.c  2002/07/24 10:06:19
@@ -4,6 +4,7 @@
  *  Copyright (C) 1991, 1992  Linus Torvalds
  */

+#include <linux/module.h>
 #include <linux/init.h>
 #include <linux/mm.h>
 #include <linux/file.h>
Index: fs/locks.c
===================================================================
RCS file: /home/other/cvs/linux/linux-2.4/fs/locks.c,v
retrieving revision 1.1.1.13
diff -u -p -r1.1.1.13 locks.c
--- fs/locks.c  2001/10/24 09:03:18     1.1.1.13
+++ fs/locks.c  2002/07/24 10:06:19
@@ -115,6 +115,7 @@
  *  Stephen Rothwell <s...@canb.auug.org.au>, June, 2000.
  */

+#include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/file.h>
 #include <linux/smp_lock.h>
Index: fs/super.c
===================================================================
RCS file: ...

read more »

 
 
 

[RFC] new module interface

Post by Rusty Russel » Fri, 26 Jul 2002 18:10:10


On Wed, 24 Jul 2002 22:02:36 +0200 (CEST)


> The patch below is for 2.4 but it's easily ported to 2.5, beside of this I
> think the core is stable and will allow a more flexible module handling
> in the future. After updating to 2.5 and updating some more archs I will
> submit the patch officially, so any feedback now would be very welcome.
> (The patch requires no new modutils, although a new version could avoid
> some workarounds, but that can wait.)

Hi Roman!

        Firstly, I give up: what kernel is this patch against?  It's
hard to read a patch this big which doesn't apply to any kernel I can find 8(

Quote:> DEFINE_MODULE
>    .start =        start_affs_fs,
>    .stop =         stop_affs_fs,
>    .exit =         exit_affs_fs,
>    .usecount =     usecount_affs_fs,
> DEFINE_MODULE_END

Interesting approach.  Splitting init and start and stop and exit is
normal, but encapsulating the usecount is different.  I made start
and exit return void, though.

Hmmm... you sidestepped the "rmmod -f" problem, by running module->start()
again if module->exit() fails.  I decided against this because module
authors have to make sure this works.

I chose the more standard "INIT(init, start)" & "EXIT(stop, exit)" which
makes it easier to drop the exit part if it's built-in.

My favorite part is including the builtin-modules.  I assume this means
that "request_module("foo")" returns success if CONFIG_DRIVER_FOO=y now?

Sorry I've been slack in posting my patch: will do tonight I promise 8)

Cheers!
Rusty.
--
   there are those who do and those who hang on and you don't see too
   many doers quoting their contemporaries.  -- Larry McVoy
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Roman Zippe » Fri, 26 Jul 2002 19:00:17


Hi,


>    Firstly, I give up: what kernel is this patch against?  It's
> hard to read a patch this big which doesn't apply to any kernel I can find 8(

2.4.18. Maybe pine garbled the patch... Here is a copy of the patch:
http://www.xs4all.nl/~zippel/mod.diff

Quote:> Interesting approach.  Splitting init and start and stop and exit is
> normal, but encapsulating the usecount is different.  I made start
> and exit return void, though.

I introduced usecount() to gain more flexibility, currently one is forced
to pass the module pointer everywhere.
Allowing exit to fail simplifies the interface. Normal code doesn't has
to bother about the current state of the module, instead exit() is now the
synchronization point. This also means the unload_lock via
try_inc_mod_count is not needed anymore.

Quote:> Hmmm... you sidestepped the "rmmod -f" problem, by running module->start()
> again if module->exit() fails.  I decided against this because module
> authors have to make sure this works.

In the future it rather should work. The problem is that the current
module interface is very limited, later we can introduce a interface to
just stop a module. Right now I just emulate the old behaviour to avoid
having a module in half initailized state.

Quote:> I chose the more standard "INIT(init, start)" & "EXIT(stop, exit)" which
> makes it easier to drop the exit part if it's built-in.

I was thinking about it, but couldn't we just put these function in a
seperate section and discard them during init (maybe depending on some
hotplug switch)?

Quote:> My favorite part is including the builtin-modules.  I assume this means
> that "request_module("foo")" returns success if CONFIG_DRIVER_FOO=y now?

Not yet. The problem is the module name, e.g. ext2 is called
fs_ext2_super, it will need some kbuild changes to get the right module
name.

bye, Roman

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Rusty Russel » Sat, 27 Jul 2002 12:50:06



> Hi,


> >       Firstly, I give up: what kernel is this patch against?  It's
> > hard to read a patch this big which doesn't apply to any kernel I can find
8(

> 2.4.18. Maybe pine garbled the patch... Here is a copy of the patch:
> http://www.xs4all.nl/~zippel/mod.diff

Much better: thanks!

Quote:> > Interesting approach.  Splitting init and start and stop and exit is
> > normal, but encapsulating the usecount is different.  I made start
> > and exit return void, though.

> I introduced usecount() to gain more flexibility, currently one is forced
> to pass the module pointer everywhere.

Well, you substituted the module pointer for an atomic counter.  Bit
of a wash, really.

Quote:> Allowing exit to fail simplifies the interface. Normal code doesn't has
> to bother about the current state of the module, instead exit() is now the
> synchronization point. This also means the unload_lock via
> try_inc_mod_count is not needed anymore.

Except that rmmod fails rather frequently on busy modules.  Which
might be ok.

Quote:> > I chose the more standard "INIT(init, start)" & "EXIT(stop, exit)" which
> > makes it easier to drop the exit part if it's built-in.

> I was thinking about it, but couldn't we just put these function in a
> seperate section and discard them during init (maybe depending on some
> hotplug switch)?

No, if you drop them newer binutils notices the link problem, hence
the __devexit_p(x) macro.

Quote:> > My favorite part is including the builtin-modules.  I assume this means
> > that "request_module("foo")" returns success if CONFIG_DRIVER_FOO=y now?

> Not yet. The problem is the module name, e.g. ext2 is called
> fs_ext2_super, it will need some kbuild changes to get the right module
> name.

I need that too: the mythical "KBUILD_MODNAME".  Both Keith and Kai
promised it to me...

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Keith Owen » Sat, 27 Jul 2002 13:30:08


On Fri, 26 Jul 2002 13:43:39 +1000,


>I need that too: the mythical "KBUILD_MODNAME".  Both Keith and Kai
>promised it to me...

KBUILD_OBJECT has been in kbuild 2.5 since April 5 (kbuild-2.5-core-1).
It is not my fault if Linus won't take it and Kai will not implement
it.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Roman Zippe » Sat, 27 Jul 2002 19:20:10


Hi,


> > I introduced usecount() to gain more flexibility, currently one is forced
> > to pass the module pointer everywhere.

> Well, you substituted the module pointer for an atomic counter.  Bit
> of a wash, really.

Converting the module pointer into a counter is the easiest way to convert
to the new interface. Behind that is a very important concept - complete
seperation of module state management (done in kernel/module.c) and module
usage management (done by the module). Both are independent in my patch,
so the module has complete freedom how to do the later. This means it
doesn't has to use a counter, the usecount function could be as well
something like "return busy ? 1 : 0;" and the module won't be bothered
with unloading. On the other hand if a module needs something more
complex, it can do so without bothering the remaining the module code
(e.g. if I look at the LSM hooks, I'm really not sure how to sanely unload
a module from that).
We get this flexibility only by removing the usecount from the module
structure. As long as that usecount is used for synchronisation, we have
to pass around the module pointer everywhere to get to that damned
usecount.

Quote:> > I was thinking about it, but couldn't we just put these function in a
> > seperate section and discard them during init (maybe depending on some
> > hotplug switch)?

> No, if you drop them newer binutils notices the link problem, hence
> the __devexit_p(x) macro.

AFAIK we have that problem if we discard the sections immediately at link
time, but we could also discard them at kernel init. On the other hand I'm
not completely against wrapping the field initialization in a macro.

Quote:> > Not yet. The problem is the module name, e.g. ext2 is called
> > fs_ext2_super, it will need some kbuild changes to get the right module
> > name.

> I need that too: the mythical "KBUILD_MODNAME".  Both Keith and Kai
> promised it to me...

I found a solution for that yesterday. :)

I looked at your patch and some interesting parts are missing, so it's
difficult to comment. It's really small, but it also has lots of FIXMEs. :)
My module pointer comment above applies to your patch as well. Since
module unloading isn't implemented yet, it's difficult to say how you want
to avoid the races.
One thing I mentioned earlier is that I still think that module linkage
is better done in user space, if we also keep all the symbol and
dependency information in user space. Insmod just had to relocate the
module and the kernel only needs the pointer to the module structure and
finds the rest through it, so no adding of new sections/symbols or
initialization of the module structure would be required, so insmod hadn'

BTW I also looked at the automatic initcall generation script, I think I
have a solution for a big part of the problem, that would at least allow
to order the init calls of all normal modules. (Patches are coming
hopefully soon. :) )

bye, Roman

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Kai Germaschewsk » Sun, 28 Jul 2002 03:20:11



> > Not yet. The problem is the module name, e.g. ext2 is called
> > fs_ext2_super, it will need some kbuild changes to get the right module
> > name.

> I need that too: the mythical "KBUILD_MODNAME".  Both Keith and Kai
> promised it to me...

I told you that I have the code, and that I can provide it to you as soon
as you need it. You did never ask for it, though. Do you need it now?

--Kai

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Rusty Russel » Sun, 28 Jul 2002 16:10:06



> Hi,


> > Well, you substituted the module pointer for an atomic counter.  Bit
> > of a wash, really.

> Converting the module pointer into a counter is the easiest way to convert
> to the new interface. Behind that is a very important concept - complete
> seperation of module state management (done in kernel/module.c) and module
> usage management (done by the module). Both are independent in my patch,
> so the module has complete freedom how to do the later. This means it
> doesn't has to use a counter, the usecount function could be as well
> something like "return busy ? 1 : 0;" and the module won't be bothered
> with unloading.

But your added flexibility doesn't actually buy you any more power.
It does mean (as implemented) that even without module support in your
kernel, filesystems have to be reference counted.

Quote:> On the other hand if a module needs something more
> complex, it can do so without bothering the remaining the module code
> (e.g. if I look at the LSM hooks, I'm really not sure how to sanely unload
> a module from that).

Exactly.  I don't see how it is a definitive win 8(

Quote:> > No, if you drop them newer binutils notices the link problem, hence
> > the __devexit_p(x) macro.

> AFAIK we have that problem if we discard the sections immediately at link
> time, but we could also discard them at kernel init. On the other hand I'm
> not completely against wrapping the field initialization in a macro.

Yes.  I hate the macro wrapping myself though 8(

Quote:> > > Not yet. The problem is the module name, e.g. ext2 is called
> > > fs_ext2_super, it will need some kbuild changes to get the right module
> > > name.

> > I need that too: the mythical "KBUILD_MODNAME".  Both Keith and Kai
> > promised it to me...

> I found a solution for that yesterday. :)
> I looked at your patch and some interesting parts are missing, so it's
> difficult to comment. It's really small, but it also has lots of FIXMEs. :)
> My module pointer comment above applies to your patch as well. Since
> module unloading isn't implemented yet, it's difficult to say how you want
> to avoid the races.

I removed module unloading from the current patch, because of ongoing
debate (and as you know, this is a whole issue by itself 8).

The implementation in the older patches did:

        module->waiting = current;
        err = module->stop();
        if (err)
                return err;
        synchronize_kernel();
        module_put(module);
        while (atomic_read(&module->count)) {
                set_current_state(TASK_INTERRUPTIBLE);
                schedule();
        }

synchronize_kernel() waits for every CPU to schedule (with preemption,
it means wait for every preempted thread to voluntarily schedule).
This prevents the "MOD_DEC_USE_COUNT; return" race.

module_put() was a simple:
        if (atomic_dec_and_test(&module->count))
                wake_up_process(module->waiting);

Quote:> One thing I mentioned earlier is that I still think that module linkage
> is better done in user space, if we also keep all the symbol and
> dependency information in user space. Insmod just had to relocate the
> module and the kernel only needs the pointer to the module structure and
> finds the rest through it, so no adding of new sections/symbols or
> initialization of the module structure would be required, so insmod hadn'

I think the in-kernel linker is much neater, and much smaller than the
combined one.  One big advantage is that I don't have to do 64-bit
links on a 32-bit userspace.

Quote:> BTW I also looked at the automatic initcall generation script, I think I
> have a solution for a big part of the problem, that would at least allow
> to order the init calls of all normal modules. (Patches are coming
> hopefully soon. :) )

Yes.  I thought of treating each module as one "unit" for the purposes
of initialization order, and resolving the ordering within each module
as a separate problem, but I ran out of energy, and I'm not convinced
that some core code loops are not fundamental.  The problem is that if
there are still loops, they may well depend on .config so some poor
user with a different config can't compile the kernel 8(

And the explicit initcalls are pretty nice, too!

Thanks!
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Rusty Russel » Sun, 28 Jul 2002 16:10:06



 write:


> > > Not yet. The problem is the module name, e.g. ext2 is called
> > > fs_ext2_super, it will need some kbuild changes to get the right module
> > > name.

> > I need that too: the mythical "KBUILD_MODNAME".  Both Keith and Kai
> > promised it to me...

> I told you that I have the code, and that I can provide it to you as soon
> as you need it. You did never ask for it, though. Do you need it now?

Sorry for the misunderstanding, I expected it to appear in the build
system one day magically 8)

You can just send me the patch though, and I'll throw it on my page
and make the module replacement patch Depend: on it.  I'll also give
it some good testing.

Thanks!
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Kai Germaschewsk » Mon, 29 Jul 2002 02:20:06



> You can just send me the patch though, and I'll throw it on my page
> and make the module replacement patch Depend: on it.  I'll also give
> it some good testing.

Here we go.

--Kai

===== Rules.make 1.67 vs edited =====
--- 1.67/Rules.make     Thu Jun 20 12:50:27 2002

 multi-used-y := $(filter-out $(list-multi),$(__multi-used-y))
 multi-used-m := $(filter-out $(list-multi),$(__multi-used-m))

+multi-used   := $(multi-used-y) $(multi-used-m)
+
 # Build list of the parts of our composite objects, our composite
 # objects depend on those (obviously)
 multi-objs-y := $(foreach m, $(multi-used-y), $($(m:.o=-objs)))
 multi-objs-m := $(foreach m, $(multi-used-m), $($(m:.o=-objs)))

+multi-objs   := $(multi-objs-y) $(multi-objs-m)
+
 # $(subdir-obj-y) is the list of objects in $(obj-y) which do not live
 # in the local directory

 # contain a comma

+# These flags are needed for modversions and compiling, so we define them here
+# already
+# $(modname_flags) #defines KBUILD_OBJECT as the name of the module it will
+# end up in (or would, if it gets compiled in)
+# Note: It's possible that one object gets potentially linked into more
+#       than one module. In that case KBUILD_OBJECT will be set to foo_bar,
+#       where foo and bar are the name of the modules.
+basename_flags = -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F)))
+modname_flags  = -DKBUILD_OBJECT=$(subst $(space),_,$(strip $(if $(filter $(*F).o,$(multi-objs)),\
+                $(foreach m,$(multi-used),\
+                   $(if $(filter $(*F).o,$($(m:.o=-objs))),$(m:.o=))),\
+                $(*F))))
+c_flags        = -Wp,-MD,$(depfile) $(CFLAGS) $(NOSTDINC_FLAGS) \
+                $(modkern_cflags) $(EXTRA_CFLAGS) $(CFLAGS_$(*F).o) \
+                $(basename_flags) $(modname_flags) $(export_flags)
+
+
 # We're called for one of three purposes:
 # o fastdep: build module version files (.ver) for $(export-objs) in

 $(addprefix $(MODVERDIR)/,$(real-objs-m:.o=.ver)): modkern_cflags := $(CFLAGS_MODULE)
 $(addprefix $(MODVERDIR)/,$(export-objs:.o=.ver)): export_flags   := -D__GENKSYMS__

-c_flags = -Wp,-MD,$(depfile) $(CFLAGS) $(NOSTDINC_FLAGS) \
-         $(modkern_cflags) $(EXTRA_CFLAGS) $(CFLAGS_$(*F).o) \
-         -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F))) \
-         $(export_flags)
-
 # Our objects only depend on modversions.h, not on the individual .ver
 # files (fix-dep filters them), so touch modversions.h if any of the .ver

 $(export-objs:.o=.i)  : export_flags   := $(EXPORT_FLAGS)
 $(export-objs:.o=.s)  : export_flags   := $(EXPORT_FLAGS)
 $(export-objs:.o=.lst): export_flags   := $(EXPORT_FLAGS)
-
-c_flags = -Wp,-MD,$(depfile) $(CFLAGS) $(NOSTDINC_FLAGS) \
-         $(modkern_cflags) $(EXTRA_CFLAGS) $(CFLAGS_$(*F).o) \
-         -DKBUILD_BASENAME=$(subst $(comma),_,$(subst -,_,$(*F))) \
-         $(export_flags)

 quiet_cmd_cc_s_c = CC     $(echo_target)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

[RFC] new module interface

Post by Roman Zippe » Mon, 29 Jul 2002 21:00:11


Hi,


> > Converting the module pointer into a counter is the easiest way to convert
> > to the new interface. Behind that is a very important concept - complete
> > seperation of module state management (done in kernel/module.c) and module
> > usage management (done by the module). Both are independent in my patch,
> > so the module has complete freedom how to do the later. This means it
> > doesn't has to use a counter, the usecount function could be as well
> > something like "return busy ? 1 : 0;" and the module won't be bothered
> > with unloading.

> But your added flexibility doesn't actually buy you any more power.
> It does mean (as implemented) that even without module support in your
> kernel, filesystems have to be reference counted.

Somehow you to keep the filesystem busy, while it's mounted, how else do
you want to do this?

Quote:> > On the other hand if a module needs something more
> > complex, it can do so without bothering the remaining the module code
> > (e.g. if I look at the LSM hooks, I'm really not sure how to sanely unload
> > a module from that).

> Exactly.  I don't see how it is a definitive win 8(

The important point is seperation of concepts and consequently making them
independent. This means I can exchange parts of them without breaking
anything else. Reference counting is simple and portable, whereas
scheduling tricks are not. If LSM needs a synchronize_kernel() to unload,
it's not difficult to add it to my patch, but if someone changes the
scheduler and breaks synchronize_kernel(), it will only break LSM
unloading and not every single module. My module interface is more
resistant to changes in other parts of the kernel.

Quote:> The implementation in the older patches did:

>    module->waiting = current;
>    err = module->stop();
>    if (err)
>            return err;
>    synchronize_kernel();
>    module_put(module);
>    while (atomic_read(&module->count)) {
>            set_current_state(TASK_INTERRUPTIBLE);
>            schedule();
>    }

What is stop() supposed to do? If it just removes interfaces, it still
possible someone starts using the module and you possibly wait forever in
the loop.

Quote:> > Insmod just had to relocate the
> > module and the kernel only needs the pointer to the module structure and
> > finds the rest through it, so no adding of new sections/symbols or
> > initialization of the module structure would be required, so insmod hadn'

> I think the in-kernel linker is much neater, and much smaller than the
> combined one.  One big advantage is that I don't have to do 64-bit
> links on a 32-bit userspace.

Linking a 64-bit module in 32-bit userspace shouldn't be that big
problem? Initializing some kernel structures would be a problem, but I
want to get rid of this too and keep them in userspace. The userspace
linker could be as simple as your kernel linker.

bye, Roman

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/