Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Noell » Sat, 03 Dec 2005 00:46:56



Hi:

Does anyone know or have a the soultion to the following.

========
Given:
========
 - Jumpstart Solaris 10 onto an !!NON-Sun!! manufactured x86 systems
   (where there is a BIOS, and not OpenBoot).

 - Completely hands-off since we are building racks of these x86
systems.

========
Question
========
How does one get around the fact that the BIOS, once set to boot off
the network
via PXE, cannot "programmatically" be reset to boot off the disk? (as
far as I know
it can only be done manually, which means no hands-off jumpstart). :-(

Side Note: Having to deal with this issue early on, Linux does so via a
system program
called pivot_root, which avoids rebooting altogether by essentially
unwinding the in-memory
O/S used at build time to a certain point, and then rebuilding the O/S
but from information
contained on the disk to create the run-time version (... it never
reboots).

Anyone see any work arounds for Solaris x86 (in any fashion) for this
problem?

TIA
Regards,
Noelle

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Casper H.S. Di » Sat, 03 Dec 2005 01:10:47



> - Jumpstart Solaris 10 onto an !!NON-Sun!! manufactured x86 systems
>   (where there is a BIOS, and not OpenBoot).

Sun manufactured x86 systems also lack a OpenBoot PROM; but some
may have a form of lights out management.

Quote:> - Completely hands-off since we are building racks of these x86
>systems.

Is it allowable to tell them to ethernet boot once?  In that
case it's all possible using PXE boot and the appropriate
jumpstart configurations.

What configuration is acceptable on the platforms?

With Solaris Express and S10 update 1 you can have a grubmenu booted
over the net which first defaults to booting an install image using
PXE and later resets it after the install is complete to booting from
disk.

Quote:>How does one get around the fact that the BIOS, once set to boot off
>the network
>via PXE, cannot "programmatically" be reset to boot off the disk? (as
>far as I know
>it can only be done manually, which means no hands-off jumpstart). :-(

Get different hardware which allows this.

E.g., if you have hardware with a BMC (baseboard management console)
which can be controlled using ipmitool, you can run:

        ipmitool chassis bootdev disk

after completing the network install using PXE.

Quote:>Side Note: Having to deal with this issue early on, Linux does so via a
>system program
>called pivot_root, which avoids rebooting altogether by essentially
>unwinding the in-memory
>O/S used at build time to a certain point, and then rebuilding the O/S
>but from information
>contained on the disk to create the run-time version (... it never
>reboots).

So if it reboots it reinstalls?

Quote:>Anyone see any work arounds for Solaris x86 (in any fashion) for this
>problem?

Depends on the hardware.  I'm sure you can also craft something with
the device configuration assistent where it will always boot from the
net boot then load the OS from the disk.

In order to do so you'd have to have some post install script which
registers the system and creates a custom 01MACADDRESS.bootenv.rc or
01MACADDRESS.menu.lst file in the server's /tftpboot server after the
install is done.  This file should redirect the netboot to the appropriate
device.

But if you x86 hardware is half-way decent it may have sufficient support
to change the boot device in flight like SPARCs do.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Mike » Sat, 03 Dec 2005 03:17:34



> How does one get around the fact that the BIOS, once set to boot off
> the network
> via PXE, cannot "programmatically" be reset to boot off the disk? (as
> far as I know
> it can only be done manually, which means no hands-off jumpstart). :-(

Boot order?  1st disk then DVD then network?  I usually have to hit F-12
to get a pxe boot to kick in once I've set the boot order correctly.

If I dont' hit F-12 it just follows the bios boot order.

Quote:> Anyone see any work arounds for Solaris x86 (in any fashion) for this
> problem?

I've had no problems on Sun or HP proliant systems doing what you're
talking about.

I even hacked up the /etc/bootrc file so it forces it into a custom jumpstart
the 1st time around.

But on completion of jumpstart it just boots off of disk.

Now - I played around with some IBM X series systems and they looped exactly
like you're talking about.  I didn't bother to look at BIOS settings to see
if there was somethign I could change though.

BTW, Update1 has GRUB & easier DHCP configuration for PXE boot.

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Noell » Sat, 03 Dec 2005 03:30:06


Quote:>Casper Dik wrote...

> With Solaris Express and S10 update 1 you can have a grubmenu booted
> over the net which first defaults to booting an install image using
> PXE and later resets it after the install is complete to booting from
> disk.

Thank you. the Grubmenu option sounds interesting to me. A few followup
questions
on the part:

 (1) How could that boot from-net-the first time and then switch to
boot-from-disk
      the next time mechanism work? Correct me if I am wrong here...

      -- PXE over a NIC (configured in BIOS)
      -- tftp over the Grub boot loader/menu
      -- That loader is configured to start the boot and install of an
O/S over the net
         (i.e. initiate Custom Jumpstart)
      -- JumpStart finishes, and now you have a permanent Grub loader
on disk.
      -- but ... the BIOS is still configured to PXE boot off the net
first before going
         to disk -- thus it would start again.

  I'm probably missing something.

  (2) I'm using Standard Solaris 10 (not the Express version) as the
JumpStart server,
  and also for the install image.

  Do you think its possible to download the latest Solaris 10 Express
and extract
  just the Grubmenu piece from it and integrate it into my Standard
  Solaris 10 environment (for example -- if it comes in packages, just
take the
  packages that implement this Grubmenu feature -- and what are the
packages)?

BTW... Im checking the ipmitool now, btw

Regards,
Noelle

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Noell » Sat, 03 Dec 2005 03:39:42


Thanks Mike...

Alot of these systems (which create a grid engine) are remote, so F-12
is not available. :-(

Between you and Casper it looks like Solaris 10 Express GRUB and DHCP
is the way to go,
although Im still wondering how to tell it the boot/install (i.e.
initiate jumpstart) the first time,
and then boot to disk the next, since to my knowledge, it cannot
interact with the BIOS
to change the order (unless it does something ipmitool(1m)-ish).

But, as I asked Casper, Im wondering if I can just extract the GRUB and
DHCP packages
Solaris 10 Express and integrate into my Standard Solaris 10
Environment -- and what
those packages are. Since this would be easier than starting from
scratch.

Finally, (lots of followup questions TIA), is the Sol10Express DHCP
easier (because it's
a pain in Standard Solaris 10, and also has limitations in the number
of characters you
can have in your MACROS -- so if you have a long paths, it breaks).

Regards,
Noelle

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Mike » Sat, 03 Dec 2005 04:41:56



> Alot of these systems (which create a grid engine) are remote, so F-12
> is not available. :-(

But even GRID systems have to some sort of iLo or remote management
to them, right?

I've played around with the x4200, v20, v40 & DL-585 and no problem
with hands off installation (other than hitting F-12) the first
time around (ctrl-N on the x4200).

Quote:> Solaris 10 Express and integrate into my Standard Solaris 10
> Environment -- and what
> those packages are. Since this would be easier than starting from
> scratch.

Hmm...that's interesting.  I put my hardware release on my jumpstart
directory like this:

        /jumpstart/sol10/i386pc-hw10305/
        /jumpstart/sol10/sparc-hw10305/
        /jumpstart/sol10/scripts        < -- finish/start etc..etc...
        /jumpstart/sol10/config         < -- sysidcfg and other config file
        /jumpstart/sol10/sparcpackages
        /jumpstart/sol10/x86packages    < -- all my custom packages

So if I do a new hardware release or new build I just have to replace or
add another /jumpstart/sol10/i386pc-hw0107/ dir and everything else just
stays the same.

Why is it so hard to put a new release on your jumpstart server?

Quote:> Finally, (lots of followup questions TIA), is the Sol10Express DHCP
> easier (because it's
> a pain in Standard Solaris 10, and also has limitations in the number
> of characters you
> can have in your MACROS -- so if you have a long paths, it breaks).

I had that problem!  Going over 255bytes and you can't get the vendor
options down to the client without some include statements and other
work.

With the newboot you don't need vendor options.  Still need PXE but
it's easy as it is with linux.

-Mike

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Noell » Sat, 03 Dec 2005 05:34:04


Thanks. I have a similar scalable jumpstart infrastructure that can
accommodate
simultaneous Solaris versions and sub-versions (5.6, 5.7, 5.8, 5.9,
5.10). So that's
not the problem.

The problem is that the jumpstart server itself run is running Solaris
10 Standard
(and not Solaris 10 Express), so I don't have the packages that
implement the
easier DHCP server that you speak of.

So I was hoping to download Solaris 10 Express and rip off those
packages and
pkgadd them onto my Standard Solaris 10 O/S (assuming that that
approach would
not cause conflicts -- not that I would run both simultaneously; but
them overwriting
each others conf files or whatever during pkgadd). Do you know the
package names
for the DHCP software on Sol10Express (grep -i some-dhcp-binary
/var/sadm/install/contents).

As far as the Grub components go, I assume that part is appropriately
installed
in /tftpboot when you run add_install_client from within the Solaris 10
Express
distribution directory (I guess I'll download Sol10Exp and start my
setup_install_server's
to see).

Thanks again!
Noelle

Thanks

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Mike » Sat, 03 Dec 2005 05:44:22



> The problem is that the jumpstart server itself run is running Solaris
> 10 Standard
> (and not Solaris 10 Express), so I don't have the packages that
> implement the
> easier DHCP server that you speak of.

All client side.  You'll just reduce the complexity of your config.
You don't need to change the release of the dhcp server you are
running.

Quote:> As far as the Grub components go, I assume that part is appropriately
> installed
> in /tftpboot when you run add_install_client from within the Solaris 10
> Express
> distribution directory (I guess I'll download Sol10Exp and start my
> setup_install_server's
> to see).

That should take care of it.  No need to upgrade your dhcp server.  The
boot process changes with Update 1.
 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Noell » Sat, 03 Dec 2005 06:03:45


Got it... thats good news. Downloading it now. Appreciate your keeping
up with
this thread.

BTW... Earlier in the thread the following was mentioned as a
possibility to
implementing complete hands-off jumpstarting (which I assume also
means no F-12 since my initial post specified complete hands off
jumpstarting).
Not sure if/how this works, but it's worth a re-mention here (... I'll
be looking into
it myself).

Quote:

>> With Solaris Express and S10 update 1 you can have a grubmenu booted
>> over the net which first defaults to booting an install image using
>> PXE and later resets it after the install is complete to booting from
>> disk.

Best Regards Mike,
Noelle
 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Mike Delane » Sat, 03 Dec 2005 07:25:56


On 1 Dec 2005 13:03:45 -0800, Noelle said something similar to:
:
:  BTW... Earlier in the thread the following was mentioned as a
:  possibility to
:  implementing complete hands-off jumpstarting (which I assume also
:  means no F-12 since my initial post specified complete hands off
:  jumpstarting).
:  Not sure if/how this works, but it's worth a re-mention here (... I'll
:  be looking into
:  it myself).
: >>
: >> With Solaris Express and S10 update 1 you can have a grubmenu booted
: >> over the net which first defaults to booting an install image using
: >> PXE and later resets it after the install is complete to booting from
: >> disk.

I'm not sure, but what Casper may be refering to here is having the grub
menu file on the boot server change post-install to boot from the hard drive
rather than doing another install.

I've got a setup like that running for one of my clients' Linux
clusters.  Here's how we do it:

There are two grub menus - one that boots from the network to perform the
install, and another that boots from the hard disk.  For each client, there
is a symlink which in normal operation points at the grub menu for booting
from the local disk.  That symlink is the path that the DHCP server hands to
the client.

To reinstall a system, we point the symlink for that client at the install
menu, and reboot the client machine.  During postinstall, the client logs
into the boot server via ssh with a restricted key, causing the symlink to
be pointed back at the menu file that boots from disk.

This of course does require that the DHCP server be configured to hand
a different grubmenu path to each client.  Since I'm already configuring
it to hand out fixed addresses to the cluster nodes, I don't find this
to be a big issue for the site I'm describing.  You might feel differently.

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Mike » Sat, 03 Dec 2005 10:14:36



> BTW... Earlier in the thread the following was mentioned as a
> possibility to
> implementing complete hands-off jumpstarting (which I assume also
> means no F-12 since my initial post specified complete hands off
> jumpstarting).

Is this technically possible?  You have to interact with the system
at some point to say "Go build yourself off the network and then
boot off your disk".

I know you can issue a reboot command and specify '-install' to
have it jumpstart off the 'Net but even that would assume the
box is running an OS & that isn't hands off either.

Or maybe there's a solution to this - I just don't see it's usefulness
if it does exist :-)

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Casper H.S. Di » Sat, 03 Dec 2005 19:56:22



>I'm not sure, but what Casper may be refering to here is having the grub
>menu file on the boot server change post-install to boot from the hard drive
>rather than doing another install.

Correct.

Quote:>This of course does require that the DHCP server be configured to hand
>a different grubmenu path to each client.  Since I'm already configuring
>it to hand out fixed addresses to the cluster nodes, I don't find this
>to be a big issue for the site I'm describing.  You might feel differently.

The Solaris grub loader always tries a machine specific grub menu
before it tries the generic one.

The only mechanism needed is some way to tell the server that the
menu needs to change.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Noell » Sun, 04 Dec 2005 07:09:14


Thanks for all that...

Mike S., here is the usefulness of hands off...

We have a very tight Solaris kernel 25mb (67mb when java
is included). Really! Its that small. And its extremely secure,
and screams because we absolutely need it to. We
compile it from scratch.

Anyway, when we reboot machines, they automatically
just rebuild themselves from scratch -- it takes less than
two minutes (start to finish). We reboot for various reasons
(testing a new build for a trading core matching engine;
switching from one build to another because processing
needs change from daytime trade matching, to nigh time
computation intensive work on a grid with hundreds of
machines, etc).

In the SPARC world, the reboots are initiated remotely as follows:

  root# reboot -- "net - install"

And everything takes care of itself. Hands off. No F-12
In two minutes the machines are all back up and running
a new application.

The x86 world simply needs the same thing. Especially when
you are rebuilding 600 machines that participate in a grid
trading engine in one shot.

:-)

Incidentally, the grub version is much easier so far (the DHCP string
is much smaller). Glad you guys pointed me in that direction.

I'll setup an ssh feedback similar to the one Mike D spoke of
(using pre setup ssh keys).

Thanks all.

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Dan Foste » Sun, 04 Dec 2005 07:20:58



Quote:

> In the SPARC world, the reboots are initiated remotely as follows:

>   root# reboot -- "net - install"

> And everything takes care of itself. Hands off. No F-12
> In two minutes the machines are all back up and running
> a new application.

Yup.

Quote:> The x86 world simply needs the same thing. Especially when

Agreed! The x86 BIOS is really showing its age and architectural
cruddiness now. This is a big reason why Microsoft and others has pushed
for a consortium to develop a next generation firmware.

This already exists, and is called EFI (an Intel standard) -- Extensible
Firmware Interface. It's pretty slick stuff, and is already running on
HP Itanium servers. EFI is a lot like a mini-OS and handles devices,
future extensions, and initial hardware bring-up in a sane way.

We sure wish our x86 Suns had OF or something decent, just like our
SPARCs, since it does indeed make it significantly easier to automate.

Kinda a PITA to have to connect to a console, F12 it, repeat for 10-20
machines that just arrived. I'd cry if I had to do that for 600. :-)

Quote:> Incidentally, the grub version is much easier so far (the DHCP string
> is much smaller). Glad you guys pointed me in that direction.

Hadn't known about that one. Sounds intriguing.

-Dan

 
 
 

Jumpstart x86: How not to PXE after jumpstart finishes and reboots ...

Post by Mike » Sun, 04 Dec 2005 23:17:59



> In the SPARC world, the reboots are initiated remotely as follows:

>  root# reboot -- "net - install"

Update your bootenv.rc to specify the network as the boot device?
And the '-install' in bootenv.rc to do custom jumpstart?
 
 
 

1. Jumpstart not jumpstarting a Sparcstation2

Hello folks,

I am trying to jumpstart a few sparcstation 2's and for some reason after
the
packages have been installed and the relevant 2.6 patches loaded the system
reboots and give the following message:


bootblk: can't find the boot program
Program terminated

The SCSI disk is target3 and I assume it should install the necessary eeprom
setting to allow this to work. disk is aliased to


The jumpstart server is a Ultra1 running Solaris 8 and am trying to install
2.6 5/98 on the Sparcstatio2.

Thanks for any feedback/info,

Ryan

2. 586 CPU?

3. Jumpstart does not execute script after pkgpatch finished

4. LOCAL NYC - UNIGROUP Meeting 20-NOV-2003: IPv6 & ifch

5. Solaris 10 x86 Jumpstart: No auto-reboot?

6. Process hiding

7. Why is hostname getting set on 2nd reboot, not 1st after jumpstart?

8. Getting PCnet ISA+ to work with Linux

9. Does not reboot from cron after Jumpstart install of Sol 2.6

10. Jumpstart 2.5.1 Client from a 2.6 Jumpstart Server

11. jumpstart pxe problem

12. yet another pxe & jumpstart question

13. jumpstarting x86; /a/etc/.UNCONFIGURED not removed