RHEL2.1 kernel rebuild procedure

To utilize crash with RHEL2.1 kernels, a vmlinux kernel must be built with the -g CFLAG. The newly-built kernel does not have to be run, but the vmlinux file with the extra debug data is required to run crash on a live system or with a dumpfile.

This is the suggested RHEL2.1 kernel build procedure:

  1. Install the kernel src.rpm:
      # rpm -ivh kernel-2.4.9-e.xx.src.rpm
    where "xx" is the kernel version in use.

  2. Go to /usr/src/redhat/SPECS, or wherever the builder's .rpmmacros file points to; /usr/src/redhat is the default build area.

  3. Edit the kernel-2.4.spec file; the top few lines of which are shown below. At the top of the file, note that there are "%define buildxxx" settings, one for each kernel "flavor":
      Summary: The Linux kernel (the core of the Linux operating system)
      # What parts do we want to build?  We must build at least one kernel.
      # These are the kernels that are built IF the architecture allows and
      # no contrary --with/--without arguments are given on the command line.
      %define buildup 1
      %define buildsmp 1
      %define buildBOOT 1
      %define buildenterprise 1
      %define buildsummit 1
      %define builddebug 1
      %define buildjensen 0
      %define buildtape 1
      %define buildBOOTtape 1

    To build only one kernel flavor, change all of the kernel "buildxxx" values to 0 -- except for the one that is to be debugged. So, for example, if you are debugging the "smp" kernel, leave buildsmp as 1, but change all of the others to be 0. Make sure that only a single instance of "1" exists, that one for the kernel type being debugged. For example:

      %define buildup 0
      %define buildsmp 1
      %define buildBOOT 0
      %define buildenterprise 0
      %define buildsummit 0
      %define builddebug 0
      %define buildjensen 0
      %define buildtape 0
      %define buildBOOTtape 0

  4. Then build the binary RPM:
      # rpmbuild -bb --target i686 kernel-2.4.spec
    This will build only one kernel RPM, that one for the kernel type specified by leaving its build value to 1.

  5. Go into /usr/src/redhat/BUILD/kernel-2.4/linux (or wherever the builder's .rpmmacros file specifies), and edit the top-level Makefile there. Find the CFLAGS definition line that looks like this:
      CFLAGS := $(CPPFLAGS) -Wall -Wstrict-prototypes -Wno-trigraphs -O2 \
                -fno-strict-aliasing -fno-common -Wno-unused
    Change it by adding -g, as in:
      CFLAGS := $(CPPFLAGS) -Wall -Wstrict-prototypes -Wno-trigraphs -O2 \
                -fno-strict-aliasing -fno-common -Wno-unused -g
  6. Rebuild the kernel:
      # make oldconfig dep bzImage

    This will build the debug version of vmlinux, located in the top-level from where the above command was invoked. Copy it someplace safe, or re-name it so it won't get over-written if another build is done in this directory. For the purposes of the examples below, the debug vmlinux file has been re-named as vmlinux.dbg.

  7. This debug vmlinux.dbg file can be use with crash, the vmcore file, and the /boot/System.map file to debug the kernel that crashed, as in:
      # crash vmlinux.dbg vmcore /boot/System.map
    Or, to run crash on the live system, enter:
      # crash vmlinux.dbg /boot/System.map

    In both cases above, the debug data is gathered from vmlinux.dbg file, but the actual symbol addresses are taken from the System.map file associated with the "real" kernel. Again, also note that you don't have to actually run the debug kernel; it is only necessary that it matches the running/crashed kernel flavor, i.e., such as the "up", "smp" or "enterprise" flavors.

  8. To debug a specific module with full debug data, rebuild it with -g. If the running version of the module can be unloaded with rmmod, do so, and then insmod the new debug module object file. If that is not possible, replace the module found in the appropriate /lib/modules tree with the new debug version, and reboot. Alternatively, all modules in the kernel source tree can be rebuilt with -g by running:
      # make modules
    after vmlinux.dbg debug kernel has been rebuilt in step 6 above. To copy them all to the appropriate /lib/modules tree, enter:
      # make modules_install
    Then you can load the new module's debug data while in a crash session with the mod command. Note that there is no crash utility support for loading the debug data from a module built with -g against a non-debug "stock" version of the module running in a live kernel or vmcore; the mod command will typically fail due to conflicting symbol values. However, it won't hurt to try it first.