crash-whitepaper-1.0/0000755000175000017500000000000010357527750015057 5ustar troyhebetroyhebecrash-whitepaper-1.0/.originals/0000755000175000017500000000000010357524144017116 5ustar troyhebetroyhebecrash-whitepaper-1.0/.originals/problems.html0000644000175000017500000004021610357524144021632 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access
  Contents Next >

The Problems

Inconvenient and Unreasonable Console Constraints
Any user with physical access to a computer generally has the ability to trivially override any system security. This is certainly true on most Linux systems by default; unless BIOS passwords are set and the case is securely locked, anyone with physical access to the machine can break in regardless of the operating system in use. Proper use of passwords just makes entry a little harder.

Linux (by default) places some limitations on what a user with physical access to the machine can do, despite the general lack of absolute security. This is good, in that it protects users from their mistakes. However, it is annoying when it makes it difficult for users with physical access to do things that they expect to be able to do, such as read from and write to floppy drives, access sound hardware, and so forth.

Users have surmounted this problem in the past in a few ways:

  • Use the su command to assume superuser status in order to temporarily override system permission limitations (su -c 'tar xf /dev/fd0').
  • Allow one particular user or group to access system devices regardless of whether that user or group is physically at the system console at the time (chown jim /dev/fd0).
  • Allow everyone on the system to use system devices (chmod 666 /dev/fd0).

These are not really solutions; they are essentially work-arounds. The need to use any of the work-arounds makes Linux a less desirable operating system for nearly any user, but especially for users who look at that list and do not have the foggiest idea how they would go about doing any one of the three work-arounds, or what the example command lines given in parentheses mean.

An additional problem has been running commands that only the superuser can run. One problem in particular is the shutdown command which allows the user to shut the computer down correctly. While shutdown is invoked by pressing the control-alt-delete key combination at a text console, it does not work from within X, and so users who have no way of knowing better may simply turn their systems off without a proper shutdown.

Inconvenient X Security Mechanisms
The other end of the spectrum has been X access. X has often been set up in a rather insecure mode where anyone on an accessible network can quietly "sniff" keystrokes and thus learn users' passwords without their knowledge. Unfortunately, protecting against this has caused inconvenience. Red Hat Linux has, in the past, come configured with a compromise that makes machines relatively secure but occasionally difficult to use.

The difficult-to-use part occurs when you use the su command to assume the privileges of another user account—and discover that as that user you are unable to run X-based applications. You have just fallen victim to security. Unless you have heard of (and know how to use) the xauth or xhost commands, you will have difficulty getting around the problem.

On the other hand, that problem is a smaller price to pay than the alternative, which is having untrustworthy users stealing passwords and other secrets as you type them. Not everyone is on a network with untrustworthy users, but those who are appreciate the security.

Many users frustrated by X inaccessibility have just blindly typed the command

xhost +

that some kind friend told them solves the problem. Unfortunately, this command entirely disables all X security. It will not hurt on a stand-alone, single-user, non-networked computer; but such setups are becoming increasingly rare. If, for example, you have a PPP connection, or an ADSL connection, or a cable modem connection, you want X security mechanisms in place.
  Contents Next >
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/.originals/index.html0000644000175000017500000003156010357524144021120 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access

by Michael K. Johnson
<johnsonm@redhat.com>

Copyright © 1999 by Red Hat Software, Inc.

Content

  1. The Problems
  2. The Solutions
  3. Tinkering

Abstract

Red Hat Linux 6.0 solves two problems that are small but annoying for experienced users, but puzzling and even debilitating for users new to Linux or UNIX. The solutions operate without user intervention, but curious users may wish to know how they work—and perennial tinkerers and conscientious systems administrators can manipulate the new features that provide the solutions.

Next >
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/.originals/solutions.html0000644000175000017500000003516310357524144022053 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access
< Prev Contents Next >

The Solutions

Red Hat Linux 6.0 includes solutions to both the device and X access problems, and the solutions are automatic. You do not need to know how to set up anything; by default, users at the physical console are able to use devices like floppy and sound, and to cleanly shut down the machine. X applications just work when you use the su command to assume the privileges of the superuser. If you would like to know how this works, or would like to customize or even disable these solutions, read on.

The solution to each of the two access problems was to implement a PAM module. PAM (Pluggable Authentication Modules) is a way of generalizing authentication so that it can be changed on a per-application basis without recompiling applications, but it also deals with some other things that happen at the same time as authentication. In particular, it manages sessions.

When you log in, a session is started, and when you log out, that session is terminated. Likewise, a session exists for the lifetime of any interactive invocation of the su command.

Red Hat Software wrote two new PAM modules, pam_console and pam_xauth, both of which manage sessions. The first checks every login to see if it is a console session—that is, if it is originating from the physical console—and modifies system file permissions appropriately, and the second module is invoked from su to pass around the keys that are used to keep X secure.

The pam_console module also includes an authentication component. Non-root users are now allowed to run the shutdown command if they are at the console and provide their own password (not the root password). This is done by providing a /usr/bin/shutdown link to the new consolehelper program which uses PAM to authenticate that the user is on the console and then invokes the real shutdown program with full root privileges.

Another excellent use of the pam_console module is that X can now only be started from the system console. In the past, it has been possible for remote users to start X on the system console. In Red Hat Linux 6.0, the xwrapper program which starts X uses pam_console to ensure that users are physically located at the system console. (This check is, of course, not applied to the superuser.)
< Prev Contents Next >
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/.originals/tinkering.html0000644000175000017500000005722310357524144022007 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access
< Prev Contents  

Tinkering

These facilities are extensively customizable. For the pam_console module, the system administrator can define what is considered to be a local system console, which devices are attached to the console, and what the device permissions should be. The pam_xauth module can be told on a per-user basis whether to forward keys.

Disabling program console access
One of the simplest and commonest customizations is to entirely disable all console-equivalent access to programs like shutdown and halt. To do this, run

# rm /etc/security/console.apps/servicename

where servicename is the name of the program to which you wish to disable console-equivalent access. Unless you use xdm, however, be careful not to remove the xserver file, or else no one but root will be able to start the X server. (If you always use xdm to start the X server, root is the only user that needs to start X, in which case you might actually want to remove the xserver file.)

Disabling device mangling
Some users and administrators do not want devices to change ownership or permission at login time. In order to prevent this, you can comment out (or remove) lines like

session    optional     /lib/security/pam_console.so

in files in the /etc/pam.d/ directory.

Defining new device classes
Perhaps a more interesting customization to pam_console is to define a new class of "console devices." Let's say that you use an IDE tape drive and you want to be able to access it from the console without assuming superuser privileges. That means that you want access to the /dev/ht0 and /dev/nht0 files. Here are the steps.

  1. Find the current permissions on the devices: 
    $ ls -l /dev/*ht0
crw-rw---- 1 root disk  37,   0 May 5 1998 /dev/ht0
crw-rw---- 1 root disk  37, 128 May 5 1998 /dev/nht0
  2. Open the /etc/security/console.perms file in your favorite text editor. The file looks something like this: 
    # /etc/security/console.perms
# ...
# file classes -- these are regular expressions
<console>=tty[0-9][0-9]* :[0-9]\.[0-9] :[0-9]

# device classes -- these are shell-style globs
<floppy>=/dev/fd[0-1]*
<sound>=/dev/dsp* /dev/audio* /dev/midi* \
/dev/mixer* /dev/sequencer
<cdrom>=/dev/cdrom
<pilot>=/dev/pilot
<jaz>=/dev/jaz
<zip>=/dev/zip
<fb>=/dev/fb /dev/fb[0-9]*
<kbd>=/dev/kbd
<joystick>=/dev/js*
<v4l>=/dev/video* /dev/radio* /dev/winradio* \
/dev/vtx* /dev/vbi*

# permission definitions
<console> 0600 <floppy>    0600 root
<console> 0600 <sound>     0644 root
<console> 0600 <cdrom>     0600 root
<console> 0600 <pilot>     0660 root
<console> 0600 <jaz>       0660 root
<console> 0600 <zip>       0660 root
<console> 0600 <fb>        0600 root
<console> 0600 <kbd>       0600 root
<console> 0600 <joystick>  0600 root
<console> 0600 <v4l>       0600 root
  3. At the end of the "device classes" section, add a line like this: 
    <idetape>=/dev/*ht0
  4. At the end of the "permission definitions" section, add a line like this: 
    <console> 0660 <idetape>   0660 root
  5. Here, 0660 is a file permissions mode; it means readable and writable by both owner and group. The first 0660 specifies the permissions applied to the device when a console user logs in, and the second 0660 specifies the permissions applied when the user logs out. The "root" at the end specifies the user that will own the device when the console user logs out.
  6. Note that these permissions will not take effect until the current console user logs out of all current login sessions and then logs in again.

Redefining the console
You may also redefine the notion of "console." Let's say that you have two machines in close physical proximity, and log into one (which we'll call desktop.example.com) directly and into the other (which we'll call second.example.com) over the network via xdm from the first, so that you have access to two machines on one screen. pam_console will not consider the network login to be a console login, but you might want to change that. On second.example.com, in the file /etc/security/console.perms, you can change the line 

<console>=tty[0-9][0-9]* :[0-9]\.[0-9] :[0-9]

to 

<console>=tty[0-9][0-9]* :[0-9]\.[0-9] :[0-9] \
desktop.example.com:1\.0

(change the 1\.0 to whatever X screen on desktop you normally log in from).

Next, you need to set up your "local" machine (desktop.example.com) to have an X session that queries the "remote" machine via XDMCP. There are two main ways to do this. The first is to use the Xnest X server, and the second is to start a separate X server on the console. The Xnest option is generally a bit nicer. Here is a sample script, which defaults to creating an Xnest window that just barely fits into an empty GNOME desktop with the GNOME panel showing below it. All the customization that you need to do should be to the two variables at the top.

#!/bin/sh

HOST=second.example.com
GEOM=1016x696

while [ -n "$*" ] ; do
case $1 in
-host) shift; HOST=$1; shift; ;;
-geometry) shift; GEOM=$1; shift; ;;
*) echo unknown option "$1" 1>&2; shift; ;;
esac
done

FP=$(echo $(
for i in misc 75dpi Speedo Type1 freefont ; do
[ -f /usr/X11R6/lib/X11/fonts/$i ] && \
echo /usr/X11R6/lib/X11/fonts/$i ;
done) | sed 's/ /,/g')

Xnest :1 -fp $FP -query $HOST -geometry $GEOM &

Alternatively, you can simply run startx -- -query second.example.com :1 causing the next free virtual console (likely virtual console 8, accessed with control-alt-F8; return to your main X session with control-alt-F7) to have a separate X server showing a display from second.example.com.

Allowing console access to new programs
Let's assume that you want to allow anyone at the physical console to be able to reprogram the keyboard, to change the keyboard repeat rate and delay. The /sbin/kbdrate command does precisely that. The steps (which must be performed as root) are as follows:

  1. Create a link in /usr/bin to the consolehelper program. 
    # cd /usr/bin
# ln -s consolehelper kbdrate

    Optionally, if you prefer, you can put the link somewhere else, such as /usr/local/bin: 

    # cd /usr/local/bin
# ln -s /usr/bin/consolehelper kbdrate
  2. Of course, in any case, the directory containing the link must be in the path of any user that wants to use the program. Perhaps less obviously, the directory containing the link must be before /sbin in those users' paths—otherwise the users will run /sbin/kbdrate instead of /usr/bin/kbdrate if they just call kbdrate.
  3. Prepare the PAM configuration. The easiest thing to do is to copy an existing working configuration. If you want the users to be prompted for their own password before allowing them to run kbdrate, do 
    # cd /etc/pam.d/
# cp shutdown kbdrate

    otherwise, do 

    # cd /etc/pam.d/
# cp xserver kbdrate
  4. Enable pam_console authentication for the kbdrate program 
    # touch /etc/security/console.apps/kbdrate

Changing pam_xauth settings
Here is a list of changes that users are likely to be interested in making to pam_xauth configuration:

  • A user does not want to export any xauth keys, even to the root account: 
    $ touch ~/.xauth/export
  • A system administrator often needs to su from other people's accounts to his/her own account while working within the other users' accounts to fix their problems. The administrator does not want his/her xauth database polluted with extra keys: 
    $ touch ~/.xauth/import
  • Bill often helps his friends with their accounts and wants X programs to work when he has used su to assume their accounts, no matter what friend he is helping: 
    $ echo '*' > ~/.xauth/export
  • Frank trusts Bill and the root user, but no one else, with his xauth keys: 
    $ echo 'root
bill' > ~/.xauth/export

Further documentation
For more details, there are several man pages you can read:

$ man pam_xauth
$ man 8 pam_console
$ man 5 console.perms
$ man 5 console.apps
< Prev Contents  
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/doc/0000755000175000017500000000000010357525535015623 5ustar troyhebetroyhebecrash-whitepaper-1.0/doc/.originals/0000755000175000017500000000000010357525152017663 5ustar troyhebetroyhebecrash-whitepaper-1.0/doc/.originals/problems.html0000644000175000017500000004021610357525152022377 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access
  Contents Next >

The Problems

Inconvenient and Unreasonable Console Constraints
Any user with physical access to a computer generally has the ability to trivially override any system security. This is certainly true on most Linux systems by default; unless BIOS passwords are set and the case is securely locked, anyone with physical access to the machine can break in regardless of the operating system in use. Proper use of passwords just makes entry a little harder.

Linux (by default) places some limitations on what a user with physical access to the machine can do, despite the general lack of absolute security. This is good, in that it protects users from their mistakes. However, it is annoying when it makes it difficult for users with physical access to do things that they expect to be able to do, such as read from and write to floppy drives, access sound hardware, and so forth.

Users have surmounted this problem in the past in a few ways:

  • Use the su command to assume superuser status in order to temporarily override system permission limitations (su -c 'tar xf /dev/fd0').
  • Allow one particular user or group to access system devices regardless of whether that user or group is physically at the system console at the time (chown jim /dev/fd0).
  • Allow everyone on the system to use system devices (chmod 666 /dev/fd0).

These are not really solutions; they are essentially work-arounds. The need to use any of the work-arounds makes Linux a less desirable operating system for nearly any user, but especially for users who look at that list and do not have the foggiest idea how they would go about doing any one of the three work-arounds, or what the example command lines given in parentheses mean.

An additional problem has been running commands that only the superuser can run. One problem in particular is the shutdown command which allows the user to shut the computer down correctly. While shutdown is invoked by pressing the control-alt-delete key combination at a text console, it does not work from within X, and so users who have no way of knowing better may simply turn their systems off without a proper shutdown.

Inconvenient X Security Mechanisms
The other end of the spectrum has been X access. X has often been set up in a rather insecure mode where anyone on an accessible network can quietly "sniff" keystrokes and thus learn users' passwords without their knowledge. Unfortunately, protecting against this has caused inconvenience. Red Hat Linux has, in the past, come configured with a compromise that makes machines relatively secure but occasionally difficult to use.

The difficult-to-use part occurs when you use the su command to assume the privileges of another user account—and discover that as that user you are unable to run X-based applications. You have just fallen victim to security. Unless you have heard of (and know how to use) the xauth or xhost commands, you will have difficulty getting around the problem.

On the other hand, that problem is a smaller price to pay than the alternative, which is having untrustworthy users stealing passwords and other secrets as you type them. Not everyone is on a network with untrustworthy users, but those who are appreciate the security.

Many users frustrated by X inaccessibility have just blindly typed the command

xhost +

that some kind friend told them solves the problem. Unfortunately, this command entirely disables all X security. It will not hurt on a stand-alone, single-user, non-networked computer; but such setups are becoming increasingly rare. If, for example, you have a PPP connection, or an ADSL connection, or a cable modem connection, you want X security mechanisms in place.
  Contents Next >
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/doc/.originals/index.html0000644000175000017500000003156010357525152021665 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access

by Michael K. Johnson
<johnsonm@redhat.com>

Copyright © 1999 by Red Hat Software, Inc.

Content

  1. The Problems
  2. The Solutions
  3. Tinkering

Abstract

Red Hat Linux 6.0 solves two problems that are small but annoying for experienced users, but puzzling and even debilitating for users new to Linux or UNIX. The solutions operate without user intervention, but curious users may wish to know how they work—and perennial tinkerers and conscientious systems administrators can manipulate the new features that provide the solutions.

Next >
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/doc/.originals/solutions.html0000644000175000017500000003516310357525152022620 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access
< Prev Contents Next >

The Solutions

Red Hat Linux 6.0 includes solutions to both the device and X access problems, and the solutions are automatic. You do not need to know how to set up anything; by default, users at the physical console are able to use devices like floppy and sound, and to cleanly shut down the machine. X applications just work when you use the su command to assume the privileges of the superuser. If you would like to know how this works, or would like to customize or even disable these solutions, read on.

The solution to each of the two access problems was to implement a PAM module. PAM (Pluggable Authentication Modules) is a way of generalizing authentication so that it can be changed on a per-application basis without recompiling applications, but it also deals with some other things that happen at the same time as authentication. In particular, it manages sessions.

When you log in, a session is started, and when you log out, that session is terminated. Likewise, a session exists for the lifetime of any interactive invocation of the su command.

Red Hat Software wrote two new PAM modules, pam_console and pam_xauth, both of which manage sessions. The first checks every login to see if it is a console session—that is, if it is originating from the physical console—and modifies system file permissions appropriately, and the second module is invoked from su to pass around the keys that are used to keep X secure.

The pam_console module also includes an authentication component. Non-root users are now allowed to run the shutdown command if they are at the console and provide their own password (not the root password). This is done by providing a /usr/bin/shutdown link to the new consolehelper program which uses PAM to authenticate that the user is on the console and then invokes the real shutdown program with full root privileges.

Another excellent use of the pam_console module is that X can now only be started from the system console. In the past, it has been possible for remote users to start X on the system console. In Red Hat Linux 6.0, the xwrapper program which starts X uses pam_console to ensure that users are physically located at the system console. (This check is, of course, not applied to the superuser.)
< Prev Contents Next >
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/doc/.originals/tinkering.html0000644000175000017500000005722310357525152022554 0ustar troyhebetroyhebe redhat.com | Red Hat Support Software Subscriptions Professional Services Solutions Support and Docs Training About Red Hat
login/logoutbecome a member downloadstorecart
red hat logo
redhat.comredhat network
Red Hat Docs  >  Whitepapers  > 
 

White Paper: Enhanced Console Access
< Prev Contents  

Tinkering

These facilities are extensively customizable. For the pam_console module, the system administrator can define what is considered to be a local system console, which devices are attached to the console, and what the device permissions should be. The pam_xauth module can be told on a per-user basis whether to forward keys.

Disabling program console access
One of the simplest and commonest customizations is to entirely disable all console-equivalent access to programs like shutdown and halt. To do this, run

# rm /etc/security/console.apps/servicename

where servicename is the name of the program to which you wish to disable console-equivalent access. Unless you use xdm, however, be careful not to remove the xserver file, or else no one but root will be able to start the X server. (If you always use xdm to start the X server, root is the only user that needs to start X, in which case you might actually want to remove the xserver file.)

Disabling device mangling
Some users and administrators do not want devices to change ownership or permission at login time. In order to prevent this, you can comment out (or remove) lines like

session    optional     /lib/security/pam_console.so

in files in the /etc/pam.d/ directory.

Defining new device classes
Perhaps a more interesting customization to pam_console is to define a new class of "console devices." Let's say that you use an IDE tape drive and you want to be able to access it from the console without assuming superuser privileges. That means that you want access to the /dev/ht0 and /dev/nht0 files. Here are the steps.

  1. Find the current permissions on the devices: 
    $ ls -l /dev/*ht0
crw-rw---- 1 root disk  37,   0 May 5 1998 /dev/ht0
crw-rw---- 1 root disk  37, 128 May 5 1998 /dev/nht0
  2. Open the /etc/security/console.perms file in your favorite text editor. The file looks something like this: 
    # /etc/security/console.perms
# ...
# file classes -- these are regular expressions
<console>=tty[0-9][0-9]* :[0-9]\.[0-9] :[0-9]

# device classes -- these are shell-style globs
<floppy>=/dev/fd[0-1]*
<sound>=/dev/dsp* /dev/audio* /dev/midi* \
/dev/mixer* /dev/sequencer
<cdrom>=/dev/cdrom
<pilot>=/dev/pilot
<jaz>=/dev/jaz
<zip>=/dev/zip
<fb>=/dev/fb /dev/fb[0-9]*
<kbd>=/dev/kbd
<joystick>=/dev/js*
<v4l>=/dev/video* /dev/radio* /dev/winradio* \
/dev/vtx* /dev/vbi*

# permission definitions
<console> 0600 <floppy>    0600 root
<console> 0600 <sound>     0644 root
<console> 0600 <cdrom>     0600 root
<console> 0600 <pilot>     0660 root
<console> 0600 <jaz>       0660 root
<console> 0600 <zip>       0660 root
<console> 0600 <fb>        0600 root
<console> 0600 <kbd>       0600 root
<console> 0600 <joystick>  0600 root
<console> 0600 <v4l>       0600 root
  3. At the end of the "device classes" section, add a line like this: 
    <idetape>=/dev/*ht0
  4. At the end of the "permission definitions" section, add a line like this: 
    <console> 0660 <idetape>   0660 root
  5. Here, 0660 is a file permissions mode; it means readable and writable by both owner and group. The first 0660 specifies the permissions applied to the device when a console user logs in, and the second 0660 specifies the permissions applied when the user logs out. The "root" at the end specifies the user that will own the device when the console user logs out.
  6. Note that these permissions will not take effect until the current console user logs out of all current login sessions and then logs in again.

Redefining the console
You may also redefine the notion of "console." Let's say that you have two machines in close physical proximity, and log into one (which we'll call desktop.example.com) directly and into the other (which we'll call second.example.com) over the network via xdm from the first, so that you have access to two machines on one screen. pam_console will not consider the network login to be a console login, but you might want to change that. On second.example.com, in the file /etc/security/console.perms, you can change the line 

<console>=tty[0-9][0-9]* :[0-9]\.[0-9] :[0-9]

to 

<console>=tty[0-9][0-9]* :[0-9]\.[0-9] :[0-9] \
desktop.example.com:1\.0

(change the 1\.0 to whatever X screen on desktop you normally log in from).

Next, you need to set up your "local" machine (desktop.example.com) to have an X session that queries the "remote" machine via XDMCP. There are two main ways to do this. The first is to use the Xnest X server, and the second is to start a separate X server on the console. The Xnest option is generally a bit nicer. Here is a sample script, which defaults to creating an Xnest window that just barely fits into an empty GNOME desktop with the GNOME panel showing below it. All the customization that you need to do should be to the two variables at the top.

#!/bin/sh

HOST=second.example.com
GEOM=1016x696

while [ -n "$*" ] ; do
case $1 in
-host) shift; HOST=$1; shift; ;;
-geometry) shift; GEOM=$1; shift; ;;
*) echo unknown option "$1" 1>&2; shift; ;;
esac
done

FP=$(echo $(
for i in misc 75dpi Speedo Type1 freefont ; do
[ -f /usr/X11R6/lib/X11/fonts/$i ] && \
echo /usr/X11R6/lib/X11/fonts/$i ;
done) | sed 's/ /,/g')

Xnest :1 -fp $FP -query $HOST -geometry $GEOM &

Alternatively, you can simply run startx -- -query second.example.com :1 causing the next free virtual console (likely virtual console 8, accessed with control-alt-F8; return to your main X session with control-alt-F7) to have a separate X server showing a display from second.example.com.

Allowing console access to new programs
Let's assume that you want to allow anyone at the physical console to be able to reprogram the keyboard, to change the keyboard repeat rate and delay. The /sbin/kbdrate command does precisely that. The steps (which must be performed as root) are as follows:

  1. Create a link in /usr/bin to the consolehelper program. 
    # cd /usr/bin
# ln -s consolehelper kbdrate

    Optionally, if you prefer, you can put the link somewhere else, such as /usr/local/bin: 

    # cd /usr/local/bin
# ln -s /usr/bin/consolehelper kbdrate
  2. Of course, in any case, the directory containing the link must be in the path of any user that wants to use the program. Perhaps less obviously, the directory containing the link must be before /sbin in those users' paths—otherwise the users will run /sbin/kbdrate instead of /usr/bin/kbdrate if they just call kbdrate.
  3. Prepare the PAM configuration. The easiest thing to do is to copy an existing working configuration. If you want the users to be prompted for their own password before allowing them to run kbdrate, do 
    # cd /etc/pam.d/
# cp shutdown kbdrate

    otherwise, do 

    # cd /etc/pam.d/
# cp xserver kbdrate
  4. Enable pam_console authentication for the kbdrate program 
    # touch /etc/security/console.apps/kbdrate

Changing pam_xauth settings
Here is a list of changes that users are likely to be interested in making to pam_xauth configuration:

  • A user does not want to export any xauth keys, even to the root account: 
    $ touch ~/.xauth/export
  • A system administrator often needs to su from other people's accounts to his/her own account while working within the other users' accounts to fix their problems. The administrator does not want his/her xauth database polluted with extra keys: 
    $ touch ~/.xauth/import
  • Bill often helps his friends with their accounts and wants X programs to work when he has used su to assume their accounts, no matter what friend he is helping: 
    $ echo '*' > ~/.xauth/export
  • Frank trusts Bill and the root user, but no one else, with his xauth keys: 
    $ echo 'root
bill' > ~/.xauth/export

Further documentation
For more details, there are several man pages you can read:

$ man pam_xauth
$ man 8 pam_console
$ man 5 console.perms
$ man 5 console.apps
< Prev Contents  
 
Product Highlight
CCVS runs on selected POSIX compliant platforms. If your Unix appears on the following list, you can download CCVS and start looking at the demo configuration.
 

 
 


 
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google
Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat

 
 
crash-whitepaper-1.0/doc/help_pages/0000755000175000017500000000000010357525152017725 5ustar troyhebetroyhebecrash-whitepaper-1.0/doc/help_pages/kernel_rebuild.html0000644000175000017500000001707110357525152023607 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

Red Hat Advanced Server 2.1 kernel rebuild procedure

To utilize crash with Red Hat Advance Server S2.1 kernels, a vmlinux kernel must be built with the -g CFLAG. The newly-built kernel does not have to be run, but the vmlinux file with the extra debug data is required to run crash on a live system or with a dumpfile.

This is the suggested Red Hat Advanced Server 2.1 kernel build procedure:

  1. Install the kernel src.rpm: 
      # rpm -ivh kernel-2.4.9-e.xx.src.rpm
    where "xx" is the kernel version in use.

  2. Go to /usr/src/redhat/SPECS, or wherever the builder's .rpmmacros file points to; /usr/src/redhat is the default build area.

  3. Edit the kernel-2.4.spec file; the top few lines of which are shown below. At the top of the file, note that there are "%define buildxxx" settings, one for each kernel "flavor": 
      Summary: The Linux kernel (the core of the Linux operating system)
  
  # What parts do we want to build?  We must build at least one kernel.
  # These are the kernels that are built IF the architecture allows and
  # no contrary --with/--without arguments are given on the command line.
  %define buildup 1
  %define buildsmp 1
  %define buildBOOT 1
  %define buildenterprise 1
  %define buildsummit 1
  %define builddebug 1
  %define buildjensen 0
  %define buildtape 1
  %define buildBOOTtape 1

    To build only one kernel flavor, change all of the kernel "buildxxx" values to 0 -- except for the one that is to be debugged. So, for example, if you are debugging the "smp" kernel, leave buildsmp as 1, but change all of the others to be 0. Make sure that only a single instance of "1" exists, that one for the kernel type being debugged. For example: 

      %define buildup 0
  %define buildsmp 1
  %define buildBOOT 0
  %define buildenterprise 0
  %define buildsummit 0
  %define builddebug 0
  %define buildjensen 0
  %define buildtape 0
  %define buildBOOTtape 0

  4. Then build the binary RPM: 
      # rpmbuild -bb --target i686 kernel-2.4.spec
    This will build only one kernel RPM, that one for the kernel type specified by leaving its build value to 1.

  5. Go into /usr/src/redhat/BUILD/kernel-2.4/linux (or wherever the builder's .rpmmacros file specifies), and edit the top-level Makefile there. Find the CFLAGS definition line that looks like this: 
      CFLAGS := $(CPPFLAGS) -Wall -Wstrict-prototypes -Wno-trigraphs -O2 \
            -fno-strict-aliasing -fno-common -Wno-unused
    Change it by adding -g, as in: 
      CFLAGS := $(CPPFLAGS) -Wall -Wstrict-prototypes -Wno-trigraphs -O2 \
            -fno-strict-aliasing -fno-common -Wno-unused -g
  6. Rebuild the kernel: 
      # make oldconfig dep bzImage

    This will build the debug version of vmlinux, located in the top-level from where the above command was invoked. Copy it someplace safe, or re-name it so it won't get over-written if another build is done in this directory. For the purposes of the examples below, the debug vmlinux file has been re-named as vmlinux.dbg.

  7. This debug vmlinux.dbg file can be use with crash, the vmcore file, and the /boot/System.map file to debug the kernel that crashed, as in: 
      # crash vmlinux.dbg vmcore /boot/System.map
    Or, to run crash on the live system, enter: 
      # crash vmlinux.dbg /boot/System.map

    In both cases above, the debug data is gathered from vmlinux.dbg file, but the actual symbol addresses are taken from the System.map file associated with the "real" kernel. Again, also note that you don't have to actually run the debug kernel; it is only necessary that it matches the running/crashed kernel flavor, i.e., such as the "up", "smp" or "enterprise" flavors.

  8. To debug a specific module with full debug data, rebuild it with -g. If the running version of the module can be unloaded with rmmod, do so, and then insmod the new debug module object file. If that is not possible, replace the module found in the appropriate /lib/modules tree with the new debug version, and reboot. Alternatively, all modules in the kernel source tree can be rebuilt with -g by running: 
      # make modules
    after vmlinux.dbg debug kernel has been rebuilt in step 6 above. To copy them all to the appropriate /lib/modules tree, enter: 
      # make modules_install
    Then you can load the new module's debug data while in a crash session with the mod command. Note that there is no crash utility support for loading the debug data from a module built with -g against a non-debug "stock" version of the module running in a live kernel or vmcore; the mod command will typically fail due to conflicting symbol values. However, it won't hurt to try it first.


Contents
crash-whitepaper-1.0/doc/help_pages/alias.html0000644000175000017500000001127510357525152021712 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  alias


NAME
  alias - command aliases

SYNOPSIS
  alias [alias] [command string]

DESCRIPTION
  This command creates an alias for a given command string.  If no arguments
  are entered, the current list of aliases are displayed.  If one argument is
  entered, the command string for that alias, if any, is displayed.

           alias  the single word to be used as an alias
  command string  the word(s) that will be substituted for the alias

  Aliases may be created in four manners:

    1. entering the alias in $HOME/.crashrc.
    2. entering the alias in .crashrc in the current directory.
    3. executing an input file containing the alias command.
    4. during runtime with this command.

  During initialization, $HOME/.crashrc is read first, followed by the
  .crashrc file in the current directory.  Aliases in the .crashrc file
  in the current directory override those in $HOME/.crashrc.  Aliases 
  entered with this command or by runtime input file override those
  defined in either .crashrc file.  Aliases may be deleted by entering an
  empty string for the second argument.  If redirection characters are to
  be part of the command string, the command string must be enclosed by
  quotation marks.

  Note that there are a number of helpful built-in aliases -- see the 
  first example below.

EXAMPLES
  Display the currently-defined aliases, which in this example, only
  consist of the built-in aliases:

    crash> alias
    ORIGIN   ALIAS    COMMAND
    builtin  man      help 
    builtin  ?        help 
    builtin  quit     q
    builtin  sf       set scroll off
    builtin  sn       set scroll on
    builtin  hex      set radix 16
    builtin  dec      set radix 10
    builtin  g        gdb
    builtin  px       p -x
    builtin  pd       p -d
    builtin  for      foreach
    builtin  size     *
    builtin  dmesg    log
 
  Create a new alias to be added to the list:

    crash> alias kp kmem -p
    ORIGIN   ALIAS    COMMAND
    runtime  kp       kmem -p

  Create an alias with redirection characters:

    crash> alias ksd "kmem -p | grep slab | grep DMA"
    ORIGIN   ALIAS    COMMAND
    runtime  ksd      kmem -p | grep slab | grep DMA

  Remove an alias:

    crash> alias kp ""
    alias deleted: kp
Contents
crash-whitepaper-1.0/doc/help_pages/ascii.html0000644000175000017500000000716010357525152021707 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  ascii


NAME
  ascii - translate a hexadecimal string to ASCII

SYNOPSIS
  ascii value ...

DESCRIPTION
  Translates 32-bit or 64-bit hexadecimal values to ASCII.  If no argument
  is entered, an ASCII chart is displayed.

EXAMPLES
  Translate the hexadecimal value of 0x62696c2f7273752f to ASCII:

    crash> ascii 62696c2f7273752f
    62696c2f7273752f: /usr/lib

  Display an ASCII chart:

    crash> ascii
 
          0    1   2   3   4   5   6   7
        +-------------------------------
      0 | NUL DLE  SP  0   @   P   '   p
      1 | SOH DC1  !   1   A   Q   a   q
      2 | STX DC2  "   2   B   R   b   r
      3 | ETX DC3  #   3   C   S   c   s
      4 | EOT DC4  $   4   D   T   d   t
      5 | ENQ NAK  %   5   E   U   e   u
      6 | ACK SYN  &   6   F   V   f   v
      7 | BEL ETB  `   7   G   W   g   w
      8 |  BS CAN  (   8   H   X   h   x
      9 |  HT  EM  )   9   I   Y   i   y
      A |  LF SUB  *   :   J   Z   j   z
      B |  VT ESC  +   ;   K   [   k   {
      C |  FF  FS  ,   <   L   \   l   |
      D |  CR  GS  _   =   M   ]   m   }
      E |  SO  RS  .   >   N   ^   n   ~
      F |  SI  US  /   ?   O   -   o  DEL
Contents
crash-whitepaper-1.0/doc/help_pages/bt.html0000644000175000017500000003105710357525152021226 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  bt


NAME
  bt - backtrace

SYNOPSIS
  bt [-a|-r|-t|-l|-e|-f|-g] [-R ref] [ -I ip ] [-S sp] [pid | taskp]

DESCRIPTION
  Display a kernel stack backtrace.  If no arguments are given, the stack
  trace of the current context will be displayed.

       -a  displays the stack traces of the active task on each CPU.
           (only applicable to crash dumps)
       -r  display raw stack data, consisting of a memory dump of the two
           pages of memory containing the task_union structure.
       -t  display all text symbols found from the last known stack location
           to the top of the stack. (helpful if the back trace fails)
       -l  show file and line number of each stack trace text location.
       -e  search the stack for possible kernel and user mode exception frames.
       -f  display all stack data contained in a frame; this option can be
           used to determine the arguments passed to each function (x86 only);
           on IA64, the argument register contents are dumped.
       -g  use gdb stack trace code. (alpha only)
   -R ref  display stack trace only if there is a reference to this symbol
           or text address.
    -I ip  use ip as the starting text location.
    -S sp  use sp as the starting stack frame address.
      pid  displays the stack trace(s) of this pid.
    taskp  displays the stack trace the the task referenced by this hexadecimal
           task_struct pointer.

  Note that all examples below are for x86 only.  The output format will differ
  for other architectures.

EXAMPLES
  Display the stack trace of the active task(s) when the kernel panicked:

    crash> bt -a
    PID: 286    TASK: c0b3a000  CPU: 0   COMMAND: "in.rlogind"
    #0 [c0b3be90] crash_save_current_state at c011aed0
    #1 [c0b3bea4] panic at c011367c
    #2 [c0b3bee8] tulip_interrupt at c01bc820
    #3 [c0b3bf08] handle_IRQ_event at c010a551
    #4 [c0b3bf2c] do_8259A_IRQ at c010a319
    #5 [c0b3bf3c] do_IRQ at c010a653
    #6 [c0b3bfbc] ret_from_intr at c0109634
       EAX: 00000000  EBX: c0e68280  ECX: 00000000  EDX: 00000004  EBP: c0b3bfbc
       DS:  0018      ESI: 00000004  ES:  0018      EDI: c0e68284 
       CS:  0010      EIP: c012f803  ERR: ffffff09  EFLAGS: 00000246 
    #7 [c0b3bfbc] sys_select at c012f803
    #8 [c0b3bfc0] system_call at c0109598
       EAX: 0000008e  EBX: 00000004  ECX: bfffc9a0  EDX: 00000000 
       DS:  002b      ESI: bfffc8a0  ES:  002b      EDI: 00000000 
       SS:  002b      ESP: bfffc82c  EBP: bfffd224 
       CS:  0023      EIP: 400d032e  ERR: 0000008e  EFLAGS: 00000246  

  Display the stack traces of task f2814000 and PID 1592:

    crash> bt f2814000 1592
    PID: 1018   TASK: f2814000  CPU: 1   COMMAND: "java"
     #0 [f2815db4] schedule at c011af85
     #1 [f2815de4] __down at c010600f
     #2 [f2815e14] __down_failed at c01061b3
     #3 [f2815e24] stext_lock (via drain_cpu_caches) at c025fa55
     #4 [f2815ec8] kmem_cache_shrink_nr at c013a53e
     #5 [f2815ed8] do_try_to_free_pages at c013f402
     #6 [f2815f04] try_to_free_pages at c013f8d2
     #7 [f2815f1c] _wrapped_alloc_pages at c01406bd
     #8 [f2815f40] __alloc_pages at c014079d
     #9 [f2815f60] __get_free_pages at c014083e
    #10 [f2815f68] do_fork at c011cebb
    #11 [f2815fa4] sys_clone at c0105ceb
    #12 [f2815fc0] system_call at c010740c
        EAX: 00000078  EBX: 00000f21  ECX: bc1ffbd8  EDX: bc1ffbe0
        DS:  002b      ESI: 00000000  ES:  002b      EDI: bc1ffd04
        SS:  002b      ESP: 0807316c  EBP: 080731bc
        CS:  0023      EIP: 4012881e  ERR: 00000078  EFLAGS: 00000296
 
    PID: 1592   TASK: c0cec000  CPU: 3   COMMAND: "httpd"
     #0 [c0ceded4] schedule at c011af85
     #1 [c0cedf04] pipe_wait at c0153083
     #2 [c0cedf58] pipe_read at c015317f
     #3 [c0cedf7c] sys_read at c0148be6
     #4 [c0cedfc0] system_call at c010740c
        EAX: 00000003  EBX: 00000004  ECX: bffed4a3  EDX: 00000001 
        DS:  002b      ESI: 00000001  ES:  002b      EDI: bffed4a3 
        SS:  002b      ESP: bffed458  EBP: bffed488 
        CS:  0023      EIP: 4024f1d4  ERR: 00000003  EFLAGS: 00000286 
 
  In order to examine each stack frame's contents use the bt -f option.
  From the extra frame data that is displayed, the arguments passed to each
  function can be determined.  Re-examining the PID 1592 trace above:
 
    crash> bt -f 1592
    PID: 1592   TASK: c0cec000  CPU: 3   COMMAND: "httpd"
     #0 [c0ceded4] schedule at c011af85
        [RA: c0153088  SP: c0ceded4  FP: c0cedf04  SIZE: 52]
        c0ceded4: c0cedf00  c0cec000  ce1a6000  00000003  
        c0cedee4: c0cec000  f26152c0  cfafc8c0  c0cec000  
        c0cedef4: ef70a0a0  c0cec000  c0cedf28  c0cedf54  
        c0cedf04: c0153088  
     #1 [c0cedf04] pipe_wait at c0153083
        [RA: c0153184  SP: c0cedf08  FP: c0cedf58  SIZE: 84]
        c0cedf08: 00000000  c0cec000  00000000  00000000  
        c0cedf18: 00000000  c0a41fa0  c011d38b  c0394120  
        c0cedf28: 00000000  c0cec000  ceeebf30  ce4adf30  
        c0cedf38: 00000000  d4b60ce0  00000000  c0cedf58  
        c0cedf48: e204f820  ef70a040  00000001  c0cedf78  
        c0cedf58: c0153184  
     #2 [c0cedf58] pipe_read at c015317f
        [RA: c0148be8  SP: c0cedf5c  FP: c0cedf7c  SIZE: 36]
        c0cedf5c: ef70a040  c0cec000  00000000  00000000  
        c0cedf6c: 00000001  f27ae680  ffffffea  c0cedfbc  
        c0cedf7c: c0148be8  
     #3 [c0cedf7c] sys_read at c0148be6
        [RA: c0107413  SP: c0cedf80  FP: c0cedfc0  SIZE: 68]
        c0cedf80: f27ae680  bffed4a3  00000001  f27ae6a0  
        c0cedf90: 40160370  24000000  4019ba28  00000000  
        c0cedfa0: 00000000  fffffffe  bffba207  fffffffe  
        c0cedfb0: c0cec000  00000001  bffed4a3  bffed488  
        c0cedfc0: c0107413  
     #4 [c0cedfc0] system_call at c010740c
        EAX: 00000003  EBX: 00000004  ECX: bffed4a3  EDX: 00000001 
        DS:  002b      ESI: 00000001  ES:  002b      EDI: bffed4a3 
        SS:  002b      ESP: bffed458  EBP: bffed488 
        CS:  0023      EIP: 4024f1d4  ERR: 00000003  EFLAGS: 00000286 
        [RA: 4024f1d4  SP: c0cedfc4  FP: c0cedffc  SIZE: 60]
        c0cedfc4: 00000004  bffed4a3  00000001  00000001  
        c0cedfd4: bffed4a3  bffed488  00000003  0000002b  
        c0cedfe4: 0000002b  00000003  4024f1d4  00000023  
        c0cedff4: 00000286  bffed458  0000002b  
 
    Typically the arguments passed to a function will be the last values
    that were pushed onto the stack by the next higher-numbered function, i.e.,
    the lowest stack addresses in the frame above the called function's
    stack frame.  That can be verified by disassembling the calling function.
    For example, the arguments passed from sys_read() to pipe_read() above
    are the file pointer, the user buffer address, the count, and a pointer
    to the file structure's f_pos field.  Looking at the frame #3 data for
    sys_read(), the last four items pushed onto the stack (lowest addresses)
    are f27ae680, bffed4a3, 00000001, and f27ae6a0 -- which are the 4 arguments
    above, in that order.  Note that the first (highest address) stack content
    in frame #2 data for pipe_read() is c0148be8, which is the return address
    back to sys_read(). 
 
  Dump the text symbols found in the current context's stack:

    crash> bt -t
    PID: 1357   TASK: c1aa0000  CPU: 0   COMMAND: "lockd"
          START: schedule at c01190e0
      [c1aa1f28] dput at c0157dbc
      [c1aa1f4c] schedule_timeout at c0124cd4
      [c1aa1f78] svc_recv at cb22c4d8
      [c1aa1f98] put_files_struct at c011eb21
      [c1aa1fcc] nlmclnt_proc at cb237bef
      [c1aa1ff0] kernel_thread at c0105826
      [c1aa1ff8] nlmclnt_proc at cb237a60
 
  Search the current stack for possible exception frames:

    crash> bt -e
    PID: 286    TASK: c0b3a000  CPU: 0   COMMAND: "in.rlogind"
    
     KERNEL-MODE EXCEPTION FRAME AT c0b3bf44:
       EAX: 00000000  EBX: c0e68280  ECX: 00000000  EDX: 00000004  EBP: c0b3bfbc
       DS:  0018      ESI: 00000004  ES:  0018      EDI: c0e68284 
       CS:  0010      EIP: c012f803  ERR: ffffff09  EFLAGS: 00000246 
    
     USER-MODE EXCEPTION FRAME AT c0b3bfc4:
       EAX: 0000008e  EBX: 00000004  ECX: bfffc9a0  EDX: 00000000 
       DS:  002b      ESI: bfffc8a0  ES:  002b      EDI: 00000000 
       SS:  002b      ESP: bfffc82c  EBP: bfffd224 
       CS:  0023      EIP: 400d032e  ERR: 0000008e  EFLAGS: 00000246 
 
  Display the back trace from a dumpfile that resulted from the execution
  of the crash utility's "sys -panic" command:

   crash> bt
   PID: 12523  TASK: c610c000  CPU: 0   COMMAND: "crash"
    #0 [c610de64] die at c01076ec
    #1 [c610de74] do_invalid_op at c01079bc
    #2 [c610df2c] error_code (via invalid_op) at c0107256
       EAX: 0000001d  EBX: c024a4c0  ECX: c02f13c4  EDX: 000026f6  EBP: c610c000
       DS:  0018      ESI: 401de2e0  ES:  0018      EDI: c610c000
       CS:  0010      EIP: c011bbb4  ERR: ffffffff  EFLAGS: 00010296
    #3 [c610df68] panic at c011bbb4
    #4 [c610df78] do_exit at c011f1fe
    #5 [c610dfc0] system_call at c0107154
       EAX: 00000001  EBX: 00000000  ECX: 00001000  EDX: 401df154
       DS:  002b      ESI: 401de2e0  ES:  002b      EDI: 00000000
       SS:  002b      ESP: bffebf0c  EBP: bffebf38
       CS:  0023      EIP: 40163afd  ERR: 00000001  EFLAGS: 00000246
 
  Display the back trace from a dumpfile that resulted from an attempt to
  insmod the sample "crash.c" kernel module that comes as part of the
  Red Hat netdump package:

   crash> bt
   PID: 1696   TASK: c74de000  CPU: 0   COMMAND: "insmod"
    #0 [c74dfdcc] die at c01076ec
    #1 [c74dfddc] do_page_fault at c0117bbc
    #2 [c74dfee0] error_code (via page_fault) at c0107256
       EAX: 00000013  EBX: cb297000  ECX: 00000000  EDX: c5962000  EBP: c74dff28
       DS:  0018      ESI: 00000000  ES:  0018      EDI: 00000000
       CS:  0010      EIP: cb297076  ERR: ffffffff  EFLAGS: 00010282
    #3 [c74dff1c] crash_init at cb297076
    #4 [c74dff2c] sys_init_module at c011d233
    #5 [c74dffc0] system_call at c0107154
       EAX: 00000080  EBX: 08060528  ECX: 08076450  EDX: 0000000a
       DS:  002b      ESI: 0804b305  ES:  002b      EDI: 08074ed0
       SS:  002b      ESP: bffe9a90  EBP: bffe9ac8
       CS:  0023      EIP: 4012066e  ERR: 00000080  EFLAGS: 00000246
Contents
crash-whitepaper-1.0/doc/help_pages/btop.html0000644000175000017500000000525610357525152021567 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  btop


NAME
  btop - bytes to page

SYNOPSIS
  btop address ...

DESCRIPTION
  This command translates a hexadecimal address to its page number.

EXAMPLES
    crash> btop 512a000
    512a000: 512a
Contents
crash-whitepaper-1.0/doc/help_pages/dev.html0000644000175000017500000001557710357525152021410 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  dev


NAME
  dev - device data

SYNOPSIS
  dev [-i | -p]

DESCRIPTION
  If no argument is entered, this command dumps the contents of the chrdevs
  and blkdevs arrays.

    -i  display I/O port usage; on 2.4 kernels, also display I/O memory usage.
    -p  display PCI device data.

EXAMPLES
  Display the chrdevs and blkdevs array contents:

    crash> dev
    CHRDEV  NAME        OPERATIONS
       1    mem         c02c9580  <memory_fops>
       2    pty         c02c9280  <tty_fops>
       3    ttyp        c02c9280  <tty_fops>
       4    ttyS        c02c9280  <tty_fops>
       5    cua         c02c9280  <tty_fops>
       7    vcs         c02ca940  <vcs_fops>
      10    misc        c02c5480  <misc_fops>
     128    ptm         c02c9280  <tty_fops>
     136    pts         c02c9280  <tty_fops>
     162    raw         c02ca880  <raw_fops>
    
    BLKDEV  NAME        OPERATIONS
       2    fd          c02c4fc0  <floppy_fops>
       8    sd          c02cec80  <sd_fops>
      22    ide1        c02c4274  <ide_fops>
      65    sd          c02cec80  <sd_fops>
      66    sd          c02cec80  <sd_fops>

  Display PCI data:

    crash> dev -p
    PCI_DEV  BU:SL.FN CLASS: VENDOR-DEVICE
    c00051c0 00:00.0  Host bridge: Intel 440BX - 82443BX Host
    c0005250 00:01.0  PCI bridge: Intel 440BX - 82443BX AGP
    c00052e0 00:07.0  ISA bridge: Intel 82371AB PIIX4 ISA
    c0005370 00:07.1  IDE interface: Intel 82371AB PIIX4 IDE
    c0005400 00:07.2  USB Controller: Intel 82371AB PIIX4 USB
    c0005490 00:07.3  Bridge: Intel 82371AB PIIX4 ACPI
    c0005520 00:11.0  Ethernet controller: 3Com 3C905B 100bTX
    c00055b0 00:13.0  PCI bridge: DEC DC21152
    c0005640 01:00.0  VGA compatible controller: NVidia [PCI_DEVICE 28]
    c00056d0 02:0a.0  SCSI storage controller: Adaptec AIC-7890/1
    c0005760 02:0e.0  SCSI storage controller: Adaptec AIC-7880U

  Display I/O port and I/O memory usage:

    crash> dev -i
    RESOURCE    RANGE    NAME
    c03036d4  0000-ffff  PCI IO
    c0302594  0000-001f  dma1
    c03025b0  0020-003f  pic1
    c03025cc  0040-005f  timer
    c03025e8  0060-006f  keyboard
    c0302604  0080-008f  dma page reg
    c0302620  00a0-00bf  pic2
    c030263c  00c0-00df  dma2
    c0302658  00f0-00ff  fpu
    c122ff20  0170-0177  ide1
    c122f240  0213-0213  isapnp read
    c122ff40  02f8-02ff  serial(auto)
    c122ff00  0376-0376  ide1
    c03186e8  03c0-03df  vga+
    c122ff60  03f8-03ff  serial(auto)
    c123851c  0800-083f  Intel Corporation 82371AB PIIX4 ACPI
    c1238538  0840-085f  Intel Corporation 82371AB PIIX4 ACPI
    c122f220  0a79-0a79  isapnp write
    c122f200  0cf8-0cff  PCI conf1
    c1238858  dc00-dc7f  3Com Corporation 3c905B 100BaseTX [Cyclone]
    c122fc00  dc00-dc7f  00:11.0
    c12380c8  dce0-dcff  Intel Corporation 82371AB PIIX4 USB
    c1238d1c  e000-efff  PCI Bus #02
    c1237858  e800-e8ff  Adaptec AIC-7880U
    c1237458  ec00-ecff  Adaptec AHA-2940U2/W / 7890
    c1239cc8  ffa0-ffaf  Intel Corporation 82371AB PIIX4 IDE
    
    RESOURCE        RANGE        NAME
    c03036f0  00000000-ffffffff  PCI mem
    c0004000  00000000-0009ffff  System RAM
    c03026ac  000a0000-000bffff  Video RAM area
    c03026fc  000c0000-000c7fff  Video ROM
    c0302718  000c9800-000cdfff  Extension ROM
    c0302734  000ce000-000ce7ff  Extension ROM
    c0302750  000ce800-000cffff  Extension ROM
    c03026e0  000f0000-000fffff  System ROM
    c0004040  00100000-07ffdfff  System RAM
    c0302674  00100000-0028682b  Kernel code
    c0302690  0028682c-0031c63f  Kernel data
    c0004060  07ffe000-07ffffff  reserved
    c1239058  ec000000-efffffff  Intel Corporation 440BX/ZX - 82443BX/ZX Host
                                 bridge
    c1238d54  f1000000-f1ffffff  PCI Bus #02
    c1239554  f2000000-f5ffffff  PCI Bus #01
    c1237074  f4000000-f5ffffff  nVidia Corporation Riva TnT2 [NV5]
    c1238d38  fa000000-fbffffff  PCI Bus #02
    c1237874  faffe000-faffefff  Adaptec AIC-7880U
    c127ec40  faffe000-faffefff  aic7xxx
    c1237474  fafff000-faffffff  Adaptec AHA-2940U2/W / 7890
    c127eec0  fafff000-faffffff  aic7xxx
    c1239538  fc000000-fdffffff  PCI Bus #01
    c1237058  fc000000-fcffffff  nVidia Corporation Riva TnT2 [NV5]
    c1238874  fe000000-fe00007f  3Com Corporation 3c905B 100BaseTX [Cyclone]
    c0004080  fec00000-fec0ffff  reserved
    c00040a0  fee00000-fee0ffff  reserved
    c00040c0  ffe00000-ffffffff  reserved
Contents
crash-whitepaper-1.0/doc/help_pages/dis.html0000644000175000017500000002106510357525152021376 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  dis


NAME
  dis - disassemble

SYNOPSIS
  dis [-r][-l][-u] [address | symbol | (expression)] [count]

DESCRIPTION
  This command disassembles source code instructions starting (or ending) at
  a text address that may be expressed by value, symbol or expression:

            -r  (reverse) displays all instructions from the start of the 
                routine up to and including the designated address.
            -l  displays source code line number data in addition to the 
                disassembly output.
            -u  address is a user virtual address; otherwise the address is 
                assumed to be a kernel virtual address.  If this option is
                used, then -r and -l are ignored.
       address  starting hexadecimal text address.
        symbol  symbol of starting text address.
  (expression)  expression evaluating to a starting text address.
         count  the number of instructions to be disassembled (default is 1).
                If no count argument is entered, and the starting address
                is entered as a text symbol, then the whole routine will be
                disassembled.  The count argument is ignored when used with
                the -r option.

EXAMPLES
  Disassemble the sys_signal() routine without, and then with, line numbers:

    crash> dis sys_signal
    0xc0112c88 <sys_signal>:        push   %ebp
    0xc0112c89 <sys_signal+1>:      mov    %esp,%ebp
    0xc0112c8b <sys_signal+3>:      sub    $0x28,%esp
    0xc0112c8e <sys_signal+6>:      mov    0xc(%ebp),%eax
    0xc0112c91 <sys_signal+9>:      mov    %eax,0xffffffec(%ebp)
    0xc0112c94 <sys_signal+12>:     movl   $0xc0000000,0xfffffff0(%ebp)
    0xc0112c9b <sys_signal+19>:     lea    0xffffffd8(%ebp),%eax
    0xc0112c9e <sys_signal+22>:     push   %eax
    0xc0112c9f <sys_signal+23>:     lea    0xffffffec(%ebp),%eax
    0xc0112ca2 <sys_signal+26>:     push   %eax
    0xc0112ca3 <sys_signal+27>:     pushl  0x8(%ebp)
    0xc0112ca6 <sys_signal+30>:     call   0xc01124b8 <do_sigaction>
    0xc0112cab <sys_signal+35>:     test   %eax,%eax
    0xc0112cad <sys_signal+37>:     jne    0xc0112cb2 <sys_signal+42>
    0xc0112caf <sys_signal+39>:     mov    0xffffffd8(%ebp),%eax
    0xc0112cb2 <sys_signal+42>:     leave
    0xc0112cb3 <sys_signal+43>:     ret
 
    crash> dis -l sys_signal
    /usr/src/linux-2.2.5/kernel/signal.c: 1074
    0xc0112c88 <sys_signal>:        push   %ebp
    0xc0112c89 <sys_signal+1>:      mov    %esp,%ebp
    0xc0112c8b <sys_signal+3>:      sub    $0x28,%esp
    0xc0112c8e <sys_signal+6>:      mov    0xc(%ebp),%eax
    /usr/src/linux-2.2.5/kernel/signal.c: 1078
    0xc0112c91 <sys_signal+9>:      mov    %eax,0xffffffec(%ebp)
    /usr/src/linux-2.2.5/kernel/signal.c: 1079
    0xc0112c94 <sys_signal+12>:     movl   $0xc0000000,0xfffffff0(%ebp)
    /usr/src/linux-2.2.5/kernel/signal.c: 1081
    0xc0112c9b <sys_signal+19>:     lea    0xffffffd8(%ebp),%eax
    0xc0112c9e <sys_signal+22>:     push   %eax
    0xc0112c9f <sys_signal+23>:     lea    0xffffffec(%ebp),%eax
    0xc0112ca2 <sys_signal+26>:     push   %eax
    0xc0112ca3 <sys_signal+27>:     pushl  0x8(%ebp)
    0xc0112ca6 <sys_signal+30>:     call   0xc01124b8 <do_sigaction>
    /usr/src/linux-2.2.5/kernel/signal.c: 1083
    0xc0112cab <sys_signal+35>:     test   %eax,%eax
    0xc0112cad <sys_signal+37>:     jne    0xc0112cb2 <sys_signal+42>
    0xc0112caf <sys_signal+39>:     mov    0xffffffd8(%ebp),%eax
    /usr/src/linux-2.2.5/kernel/signal.c: 1084
    0xc0112cb2 <sys_signal+42>:     leave
    0xc0112cb3 <sys_signal+43>:     ret
 
  Given a return address expression of "do_no_page+65", find out the 
  function that do_no_page() calls by using the reverse flag:

    crash> dis -r (do_no_page+65)
    0xc011ea68 <do_no_page>:        push   %ebp
    0xc011ea69 <do_no_page+1>:      mov    %esp,%ebp
    0xc011ea6b <do_no_page+3>:      push   %edi
    0xc011ea6c <do_no_page+4>:      push   %esi
    0xc011ea6d <do_no_page+5>:      push   %ebx
    0xc011ea6e <do_no_page+6>:      mov    0xc(%ebp),%ebx
    0xc011ea71 <do_no_page+9>:      mov    0x10(%ebp),%edx
    0xc011ea74 <do_no_page+12>:     mov    0x14(%ebp),%edi
    0xc011ea77 <do_no_page+15>:     mov    0x28(%ebx),%eax
    0xc011ea7a <do_no_page+18>:     test   %eax,%eax
    0xc011ea7c <do_no_page+20>:     je     0xc011ea85 <do_no_page+29>
    0xc011ea7e <do_no_page+22>:     mov    0x18(%eax),%ecx
    0xc011ea81 <do_no_page+25>:     test   %ecx,%ecx
    0xc011ea83 <do_no_page+27>:     jne    0xc011eab0 <do_no_page+72>
    0xc011ea85 <do_no_page+29>:     mov    $0xffffe000,%eax
    0xc011ea8a <do_no_page+34>:     and    %esp,%eax
    0xc011ea8c <do_no_page+36>:     decl   0x30(%eax)
    0xc011ea8f <do_no_page+39>:     jns    0xc011ea9a <do_no_page+50>
    0xc011ea91 <do_no_page+41>:     lock btrl $0x0,0xc022fb60
    0xc011ea9a <do_no_page+50>:     push   %edi
    0xc011ea9b <do_no_page+51>:     mov    0x18(%ebp),%esi
    0xc011ea9e <do_no_page+54>:     push   %esi
    0xc011ea9f <do_no_page+55>:     push   %ebx
    0xc011eaa0 <do_no_page+56>:     mov    0x8(%ebp),%esi
    0xc011eaa3 <do_no_page+59>:     push   %esi
    0xc011eaa4 <do_no_page+60>:     call   0xc011e9e4 <do_anonymous_page>
    0xc011eaa9 <do_no_page+65>:     jmp    0xc011eb47 <do_no_page+223>
    
  Disassemble 10 instructions starting at user virtual address 0x81ec624:

    crash> dis -u 81ec624 10
    0x81ec624:      push   %ebp
    0x81ec625:      mov    %esp,%ebp
    0x81ec627:      sub    $0x18,%esp
    0x81ec62a:      movl   $0x1,0x8(%ebp)
    0x81ec631:      mov    0x82f9040,%eax
    0x81ec636:      mov    0x10(%eax),%edx
    0x81ec639:      and    $0x100,%edx
    0x81ec63f:      mov    0x14(%eax),%ecx
    0x81ec642:      and    $0x0,%ecx
    0x81ec645:      mov    %ecx,%eax
Contents
crash-whitepaper-1.0/doc/help_pages/eval.html0000644000175000017500000001243210357525152021544 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  eval


NAME
  eval - evaluate

SYNOPSIS
  eval [-b][-l] (expression) | value

DESCRIPTION
  This command evaluates an expression or numeric value, and displays its
  result in hexadecimal, decimal, octal and binary. If the resultant value
  is an integral number of gigabytes, megabytes, or kilobytes, a short-hand
  translation of the number will also be shown next to the hexadecimal
  value.  If the most significant bit is set, the decimal display will show
  both unsigned and signed (negative) values.  Expressions must of the format
  (x operator y), where "x" and "y" may be either numeric values or
  symbols.  The list of operators are:

                     +  -  &  |  ^  *  %  /  <<  >>

  Enclosing the expression within parentheses is optional except when the
  "|", "<<" or ">>" operators are used.  The single "value" argument may
  be a number or symbol.  Number arguments must be hexadecimal or decimal.
  A leading "0x" identifies a number as hexadecimal, but is not required
  when obvious.  Numbers may be followed by the letters "k" or "K", "m"
  or "M", and "g" or "G", which multiplies the value by a factor of 1024,
  1 megabyte or 1 gigabyte, respectively.  Numeric arguments may be preceded
  by the one's complement operator ~.
 
    -b  Indicate which bit positions in the resultant value are set.
    -l  Numeric arguments are presumed to be 64-bit values, and the result
        will be expressed as a 64-bit value. (ignored on 64-bit processors)
        However, if either operand or the resultant value are 64-bit values,
        then the result will be also be expressed as a 64-bit value.
 
 The -b and -l options must precede the expression or value arguments.

EXAMPLES
   crash> eval 128m
   hexadecimal: 8000000  (128MB)
       decimal: 134217728  
         octal: 1000000000
        binary: 00001000000000000000000000000000
    
   crash> eval 128 * 1m
   hexadecimal: 8000000  (128MB)
       decimal: 134217728  
         octal: 1000000000
        binary: 00001000000000000000000000000000
    
   crash> eval (1 << 27)
   hexadecimal: 8000000  (128MB)
       decimal: 134217728  
         octal: 1000000000
        binary: 00001000000000000000000000000000
   
   crash> eval (1 << 32)
   hexadecimal: 100000000  (4GB)
       decimal: 4294967296
         octal: 40000000000
        binary: 0000000000000000000000000000000100000000000000000000000000000000
 
   crash> eval -b 41dc065
   hexadecimal: 41dc065
       decimal: 69058661  
         octal: 407340145
        binary: 00000100000111011100000001100101
      bits set: 26 20 19 18 16 15 14 6 5 2 0 
 
   crash> eval -lb 64g
   hexadecimal: 1000000000  (64GB)
       decimal: 68719476736
         octal: 1000000000000
        binary: 0000000000000000000000000001000000000000000000000000000000000000
      bits set: 36
Contents
crash-whitepaper-1.0/doc/help_pages/exit.html0000644000175000017500000000521510357525152021567 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  exit


NAME
  exit - exit this session

SYNOPSIS
  exit  

DESCRIPTION
  Bail out of the current crash session.

NOTE
  This command is equivalent to the "q" command.
Contents
crash-whitepaper-1.0/doc/help_pages/extend.html0000644000175000017500000001763710357525152022120 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  extend


NAME
  extend - extend the crash command set

SYNOPSIS
  extend [shared-object ...] | [-u [shared-object ...]]

DESCRIPTION
  This command dynamically loads or unloads crash extension shared object
  libraries:

    shared-object     load the specified shared object file; more than one
                      one object file may be entered.
    -u shared-object  unload the specified shared object file; if no file
                      arguments are specified, unload all objects.

  If no arguments are entered, the current set of shared object files and 
  a list of their commands will be displayed.  The registered commands
  contained in each shared object file will appear automatically in the 
  "help" command screen.

  An example of a shared object prototype file, and how to compile it
  into a shared object, is appended below.

EXAMPLES
  Load two shared object files:

    crash> extend extlib1.so extlib2.so
    ./extlib1.so: shared object loaded
    ./extlib2.so: shared object loaded

  Display the current set of shared object files and their commands:

    crash> extend
    SHARED OBJECT  COMMANDS
    ./extlib1.so   echo util bin
    ./extlib2.so   smp show

  Unload one of the shared object files:

    crash> extend -u extlib1.so
    ./extlib1.so: shared object unloaded

  Unload all currently-loaded object files:

    crash> extend -u
    ./extlib2.so: shared object unloaded

CREATING A SHARED OBJECT
  The extend command loads shared object files using dlopen(3), which in
  turn calls the shared object's _init() function.  The shared object's _init()
  function should register its command set by calling register_extension(),
  passing it a pointer to an array of one or more structures of the
  following type:
 
    struct command_table_entry {
            char *name;
            cmd_func_t func;
            char **help_data,
            ulong flags;
    };
 
  Each command_table_entry structure contains the ASCII name of a command,
  the command's function address, a pointer to an array of help data strings,
  and a flags field.  The help_data field is optional; if it is non-NULL, it
  should point to an array of character strings used by the "help"
  command, and during command failures.  The flags field currently has one
  available bit setting, REFRESH_TASK_TABLE, which should be set if it is 
  preferable to reload the current set of running processes just prior to 
  executing the command (on a live system).  Terminate the array of 
  command_table_entry structures with an entry with a NULL command name.  
 
  Below is an example shared object file consisting of just one command, 
  called "echo", which simply echoes back all arguments passed to it.
  Note the comments contained within it for further details.  To build it,
  cut and paste the following output into a file, and call it, for example,
  "extlib.c".  Then compile like so:
 
    gcc -nostartfiles -shared -rdynamic -o extlib.so extlib.c
 
  The extlib.so file may be dynamically linked into crash during runtime, or
  during initialization by putting "extend extlib.so" into a .crashrc file
  located in the current directory, or in the user's $HOME directory.
  
---------------------------------- cut here ----------------------------------
 
#include "defs.h"    /* From the crash source top-level directory */

void cmd_echo();     /* Declare the commands and their help data. */
char *help_echo[];

static struct command_table_entry command_table[] = {
        "echo", cmd_echo, help_echo, 0,           /* One or more commands, */
        NULL,                                     /* terminated by NULL, */
};


_init() /* Register the command set. */
{ 
        register_extension(command_table);
}
 
/* 
 *  The _fini() function is called if the shared object is unloaded. 
 *  If desired, perform any cleanups here. 
 */
_fini() { }


/* 
 *  Arguments are passed to the command functions in the global args[argcnt]
 *  array.  See getopt(3) for info on dash arguments.  Check out defs.h and
 *  other crash commands for usage of the myriad of utility routines available
 *  to accomplish what your task.
 */
void
cmd_echo()
{
        int c;

        while ((c = getopt(argcnt, args, "")) != EOF) {
                switch(c)
                {
                default:
                        argerrs++;
                        break;
                }
        }

        if (argerrs)
                cmd_usage(pc->curcmd, SYNOPSIS);

        while (args[optind]) 
                fprintf(fp, "%s ", args[optind++]);

        fprintf(fp, "\n");
}

/* 
 *  The optional help data is simply an array of strings in a defined format.
 *  For example, the "help echo" command will use the help_echo[] string
 *  array below to create a help page that looks like this:
 * 
 *    NAME
 *      echo - echoes back its arguments
 *
 *    SYNOPSIS
 *      echo arg ...
 *
 *    DESCRIPTION
 *      This command simply echoes back its arguments.
 *
 *    EXAMPLE
 *      Echo back all command arguments:
 *
 *        crash> echo hello, world
 *        hello, world
 *
 */
 
char *help_echo[] = {
        "echo",                        /* command name */
        "echoes back its arguments",   /* short description */
        "arg ...",                     /* argument synopsis, or " " if none */
 
        "  This command simply echoes back its arguments.",
        "\nEXAMPLE",
        "  Echo back all command arguments:\n",
        "    crash> echo hello, world",
        "    hello, world",
        NULL
};
Contents
crash-whitepaper-1.0/doc/help_pages/files.html0000644000175000017500000001617310357525152021725 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  files


NAME
  files - open files

SYNOPSIS
  files [-l | -d dentry] | [-R reference] [pid | taskp] ... 

DESCRIPTION
  This command displays information about open files of a context.
  It prints the context's current root directory and current working
  directory, and then for each open file descriptor it prints a pointer
  to its file struct, a pointer to its dentry struct, a pointer to the
  inode, the file type, and the pathname.  If no arguments are entered,
  the current context is used.  The -R option, typically invoked from
  "foreach files", searches for references to a supplied number, address,
  or filename argument, and prints only the essential information leading
  up to and including the reference.  The -l and -d options are not context
  specific, and only show the data requested.

            -l  display files open by lockd server for client locks.
     -d dentry  given a hexadecimal dentry address, display its inode,
                super block, file type, and full pathname.
  -R reference  search for references to this file descriptor number,
                filename, or dentry, inode, or file structure address.
           pid  a process PID.
         taskp  a hexadecimal task_struct pointer.

EXAMPLES
  Display the open files of the current context:

    crash> files
    PID: 720    TASK: c67f2000  CPU: 1   COMMAND: "innd"
    ROOT: /    CWD: /var/spool/news/articles
     FD    FILE     DENTRY    INODE    TYPE  PATH
      0  c6b9c740  c7cc45a0  c7c939e0  CHR   /dev/null
      1  c6b9c800  c537bb20  c54d0000  REG   /var/log/news/news
      2  c6df9600  c537b420  c5c36360  REG   /var/log/news/errlog
      3  c74182c0  c6ede260  c6da3d40  PIPE
      4  c6df9720  c696c620  c69398c0  SOCK
      5  c6b9cc20  c68e7000  c6938d80  SOCK
      6  c6b9c920  c7cc45a0  c7c939e0  CHR   /dev/null
      7  c6b9c680  c58fa5c0  c58a1200  REG   /var/lib/news/history
      8  c6df9f00  c6ede760  c6da3200  PIPE
      9  c6b9c6e0  c58fa140  c5929560  REG   /var/lib/news/history.dir
     10  c7fa9320  c7fab160  c7fafd40  CHR   /dev/console
     11  c6b9c7a0  c58fa5c0  c58a1200  REG   /var/lib/news/history
     12  c377ec60  c58fa5c0  c58a1200  REG   /var/lib/news/history
     13  c4528aa0  c58fa6c0  c52fbb00  REG   /var/lib/news/history.pag
     14  c6df9420  c68e7700  c6938360  SOCK
     15  c6df9360  c68e7780  c6938120  SOCK
     16  c6b9c0e0  c68e7800  c6772000  SOCK
     17  c6b9c200  c6b5f9c0  c6b5cea0  REG   /var/lib/news/active
     21  c6b9c080  c6ede760  c6da3200  PIPE
 
  Display the files opened by the "crond" daemon, which is PID 462:

  crash> files 462
    PID: 462    TASK: f7220000  CPU: 2   COMMAND: "crond"
    ROOT: /    CWD: /var/spool
     FD    FILE     DENTRY    INODE    TYPE  PATH
      0  f7534ae0  f7538de0  f7518dc0  CHR   /dev/console
      1  f7368f80  f72c7a40  f72f27e0  FIFO  pipe:/[1456]
      2  f74f3c80  f72c79c0  f72f2600  FIFO  pipe:/[1457]
      3  f7368b60  f72a5be0  f74300c0  REG   /var/run/crond.pid
      4  f7534360  f73408c0  f72c2840  REG   /var/log/cron
      7  f7368ce0  f72c7940  f72f2420  FIFO  pipe:/[1458]
      8  f7295de0  f72c7940  f72f2420  FIFO  pipe:/[1458]
     21  f74f36e0  f747cdc0  f747e840  CHR   /dev/null
 
  The -R option is typically invoked from "foreach files".  This example
  shows all tasks that have "/dev/pts/4" open:

    crash> foreach files -R pts/4
    PID: 18633  TASK: c310a000  CPU: 0   COMMAND: "crash"
    ROOT: /    CWD: /home/CVS_pool/crash 
     FD    FILE     DENTRY    INODE    TYPE  PATH
      0  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
      1  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
      2  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
    
    PID: 18664  TASK: c2392000  CPU: 1   COMMAND: "less"
    ROOT: /    CWD: /home/CVS_pool/crash 
     FD    FILE     DENTRY    INODE    TYPE  PATH
      1  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
      2  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
    
    PID: 23162  TASK: c5088000  CPU: 1   COMMAND: "bash"
    ROOT: /    CWD: /home/CVS_pool/crash 
     FD    FILE     DENTRY    INODE    TYPE  PATH
      0  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
      1  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
      2  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
    255  c1412850  c2cb96d0  c2cad430  CHR   /dev/pts/4
    
    PID: 23159  TASK: c10fc000  CPU: 1   COMMAND: "xterm"
    ROOT: /    CWD: /homes/anderson/ 
     FD    FILE     DENTRY    INODE    TYPE  PATH
      5  c1560da0  c2cb96d0  c2cad430  CHR   /dev/pts/4
 
  Display information about the dentry at address f745fd60:

    crash> files -d f745fd60
     DENTRY    INODE    SUPERBLK  TYPE  PATH
     f745fd60  f7284640  f73a3e00  REG   /var/spool/lpd/lpd.lock
Contents
crash-whitepaper-1.0/doc/help_pages/foreach.html0000644000175000017500000001344210357525152022226 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  foreach


NAME
  foreach - display command data for multiple tasks in the system

SYNOPSIS
  foreach [[pid | taskp | name | [kernel | user]] ...] command [flag] [argument]

DESCRIPTION
  This command allows for a an examination of various kernel data associated
  with any, or all, tasks in the system, without having to set the context
  to each targeted task.

      pid  perform the command(s) on this PID.
    taskp  perform the command(s) on task referenced by this hexadecimal
           task_struct pointer.
     name  perform the command(s) on all commands with this name.  If the
           command name can be confused with a foreach command name, then
           precede the name string with a "\".
     user  perform the command(s) on all user (non-kernel) threads.
   kernel  perform the command(s) on all kernel threads.
   active  perform the command(s) on the active thread on each CPU.

  If none of the task-identifying arguments above are entered, the command
  will be performed on all tasks.

  command  select one or more of the following commands on the tasks
           selected, or on all tasks:

             bt  same as the "bt" command  (optional flags: -r -t -l -e -R -f)
             vm  same as the "vm" command  (optional flags: -p -v -m -R)
           task  same as the "task" command  (optional flag: -R)
          files  same as the "files" command  (optional flag: -R)
            net  same as the "net" command  (optional flags: -s -S -R)
            set  same as the "set" command
            sig  same as the "sig" command
           vtop  same as the "vtop" command  (optional flags: -c -u -k)

     flag  Pass this optional flag to the command selected.
 argument  Pass this argument to the command selected.
 
  A header containing the PID, task address, cpu and command name will be
  pre-pended before the command output for each selected task.  Consult the
  help page of each of the command types above for details.

EXAMPLES
  Display the stack traces for all tasks:

    crash> foreach bt
    PID: 4752   TASK: c7680000  CPU: 1   COMMAND: "xterm"
     #0 [c7681edc] schedule at c01135f6
        (void)
     #1 [c7681f34] schedule_timeout at c01131ff
        (24)
     #2 [c7681f64] do_select at c0132838
        (5, c7681fa4, c7681fa0)
     #3 [c7681fbc] sys_select at c0132dad
        (5, 8070300, 8070380, 0, 0)
     #4 [bffffb0c] system_call at c0109944
        EAX: 0000008e  EBX: 00000005  ECX: 08070300  EDX: 08070380 
        DS:  002b      ESI: 00000000  ES:  002b      EDI: 00000000 
        SS:  002b      ESP: bffffadc  EBP: bffffb0c 
        CS:  0023      EIP: 402259ee  ERR: 0000008e  EFLAGS: 00000246 
    
    PID: 557    TASK: c5600000  CPU: 0   COMMAND: "nfsd"
     #0 [c5601f38] schedule at c01135f6
        (void)
     #1 [c5601f90] schedule_timeout at c01131ff
        (c5600000)
     #2 [c5601fb8] svc_recv at c805363a
        (c0096f40, c5602800, 7fffffff, 100, c65c9f1c)
     #3 [c5601fec] (nfsd module) at c806e303
        (c5602800, c5602800, c0096f40, 6c6e0002, 50)
     #4 [c65c9f24] kernel_thread at c010834f
        (0, 0, ext2_file_inode_operations)
    
    PID: 824    TASK: c7c84000  CPU: 0   COMMAND: "mingetty"
    ...

  Display the task_struct structure for each "bash" command:

    crash> foreach bash task
    ...

  Display the open files for all tasks:

    crash> foreach files
    ...
Contents
crash-whitepaper-1.0/doc/help_pages/fuser.html0000644000175000017500000000650710357525152021747 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  fuser


NAME
  fuser - file users

SYNOPSIS
  fuser [pathname | inode]

DESCRIPTION
  This command displays the tasks using specified files or sockets.
  Tasks will be listed that reference the file as the current working
  directory, root directory, an open file descriptor, or that mmap the
  file.  If the file is held open in the kernel by the lockd server on
  behalf of a client discretionary file lock, the client hostname is
  listed.

    pathname  the full pathname of the file.
    inode     the hexadecimal inode address for the file.

EXAMPLES
  Display the tasks using file /usr/lib/libkfm.so.2.0.0

    crash> fuser /usr/lib/libkfm.so.2.0.0
     PID    TASK    COMM            USAGE
     779  c5e82000  "kwm"           mmap
     808  c5a8e000  "krootwm"       mmap
     806  c5b42000  "kfm"           mmap
     809  c5dde000  "kpanel"        mmap
Contents
crash-whitepaper-1.0/doc/help_pages/gdb.html0000644000175000017500000000701510357525152021352 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  gdb


NAME
  gdb - gdb command

SYNOPSIS
  gdb command ...

DESCRIPTION
  This command passes its arguments directly to gdb for processing.
  This is typically not necessary, but where ambiguities between crash and
  gdb command names exist, this will force the command to be executed by gdb.

EXAMPLES
    crash> gdb help
    List of classes of commands:
    
    aliases -- Aliases of other commands
    breakpoints -- Making program stop at certain points
    data -- Examining data
    files -- Specifying and examining files
    internals -- Maintenance commands
    obscure -- Obscure features
    running -- Running the program
    stack -- Examining the stack
    status -- Status inquiries
    support -- Support facilities
    tracepoints -- Tracing of program execution without stopping the program
    user-defined -- User-defined commands
    
    Type "help" followed by a class name for a list of commands in that class.
    Type "help" followed by command name for full documentation.
    Command name abbreviations are allowed if unambiguous.
Contents
crash-whitepaper-1.0/doc/help_pages/help.html0000644000175000017500000000563010357525152021547 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  help


NAME
  help - get help

SYNOPSIS
  help [command | all]

DESCRIPTION
  When entered with no argument, a list of all currently available crash
  commands is listed.  If a name of a crash command is entered, a man-like
  page for the command is displayed.  If "all" is entered, help pages
  for all commands will be displayed.  If neither of the above is entered,
  the argument string will be passed on to the gdb help command.
Contents
crash-whitepaper-1.0/doc/help_pages/input.html0000644000175000017500000001170110357525152021752 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  input

Interactive crash commands are gathered using the GNU readline library,
which implements a command line history mechanism, and a command line editing
mode.

The command line history consists of a numbered list of previously run 
commands, which can be viewed by entering "h" at any time.  A previously
run command can be re-executed in a number of manners:
 
  1. To re-execute the last command, enter "r" alone, or "!!".
  2. Enter "r" followed by the number identifying the desired command.
  3. Enter "r" followed by a uniquely-identifying set of characters from
     the beginning of the desired command string.
  4. Recycle back through the command history list by hitting the up-arrow
     key until the desired command is re-displayed, and then hit <ENTER>.
     If you go too far back, hit the down-arrow key.
 
The command line editing mode can be set to emulate either vi or emacs.
The mode can be set in the following manners, listed in increasing order of
precedence:
 
  1. The setting of the EDITOR environment variable.
  2. An entry in either in a .crashrc file, which can be located either
     in the current directory or in your home directory.  The entry must
     be of the form "set vi" or "set emacs".
  3. By use of the crash "-e" command line option, as in "-e vi" or "-e emacs".
 
To edit a previously entered command:
 
  1. Recycle back through the command history list until the desired command
     is re-displayed.
  2. Edit the line using vi or emacs editing commands, as appropriate. 
  3. Hit <ENTER>.
 
It should be noted that command line re-cycling may be accomplished by using
the CTRL-p and CTRL-n keys instead of the up- and down-arrow keys; in vi mode
you can enter <ESC>, then "k" to cycle back, or "j" to cycle forward.
 
Lastly, a set of crash commands may be entered into a regular file that can
used as input, again using standard command line syntax:

  crash> < inputfile

An input file may be also be run from the crash command line using the -i 
option:

  $ crash -i inputfile

Lastly, if a command is entered that is not recognized, it is checked against
the kernel's list of variables, structure, union or typedef names, and if 
found, the command is passed to p, struct, union or whatis.  That being the 
case, as long as a kernel variable/structure/union name is different than any
of the current commands, the appropriate command above will be executed.  If
not, the command will be passed on to the built-in gdb command for execution.
Contents
crash-whitepaper-1.0/doc/help_pages/irq.html0000644000175000017500000001240710357525152021412 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  irq


NAME
  irq - IRQ data

SYNOPSIS
  irq [-d | -b | [index ...]]

DESCRIPTION
  This command collaborates the data in an irq_desc_t, along with its
  associated hw_interrupt_type and irqaction structure data, into a
  consolidated per-IRQ display.  Alternatively, the intel interrupt
  descriptor table may be dumped, or bottom half data may be displayed.
  If no index value argument(s) nor any options are entered, the IRQ
  data for all IRQs will be displayed.

    index   a valid IRQ index.
       -d   dump the intel interrupt descriptor table.
       -b   dump bottom half data.

EXAMPLES
  Display the relevant data for IRQ 18:

    crash> irq 18
        IRQ: 18
     STATUS: 0 
    HANDLER: c02301e0  <ioapic_level_irq_type>
             typename: c01f9e0c  "IO-APIC-level"
              startup: c0110234  <unmask_IO_APIC_irq>
             shutdown: c01101cc  <mask_IO_APIC_irq>
               handle: c0110518  <do_level_ioapic_IRQ>
               enable: c0110234  <unmask_IO_APIC_irq>
              disable: c01101cc  <mask_IO_APIC_irq>
     ACTION: c009c6b0
              handler: c01ce818  <do_aic7xxx_isr>
                flags: 4000000  (SA_SHIRQ)
                 mask: 0
                 name: c0217780  "aic7xxx"
               dev_id: c0090078
                 next: c009c770
     ACTION: c009c770
              handler: c01ce818  <do_aic7xxx_isr>
                flags: 4000000  (SA_SHIRQ)
                 mask: 0
                 name: c0217780  "aic7xxx"
               dev_id: c0091078
                 next: 0
      DEPTH: 0

  Display the intel interrupt descriptor table entries:

    crash> irq -d
      [0] divide_error
      [1] debug
      [2] nmi
      [3] int3
      [4] overflow
      [5] bounds
      [6] invalid_op
      [7] device_not_available
      [8] double_fault
      [9] coprocessor_segment_overrun
     [10] invalid_TSS
     [11] segment_not_present
     [12] stack_segment
     [13] general_protection
     [14] page_fault
     [15] spurious_interrupt_bug
     [16] coprocessor_error
     [17] alignment_check
     [18] ignore_int
     [19] ignore_int
     [20] ignore_int
     [21] ignore_int
    ...

    [250] IRQ0xda_interrupt
    [251] IRQ0xdb_interrupt
    [252] IRQ0xdc_interrupt
    [253] IRQ0xdd_interrupt
    [254] IRQ0xde_interrupt
    [255] spurious_interrupt

  Display the bottom half data:

    crash> irq -b
    BH_MASK   BH_ACTIVE
    00000f17  00000000
    
    BH_BASE  FUNCTION
       0     c0114148  <timer_bh>
       1     c019b4f4  <console_bh>
       2     c0113c1c  <tqueue_bh>
       4     c01abbbc  <do_serial_bh>
       8     c0153098  <net_bh>
       9     c01b3230  <scsi_bottom_half_handler>
      10     c0113c88  <immediate_bh>
      11     c01a8c80  <kbd_bh>
Contents
crash-whitepaper-1.0/doc/help_pages/wrapper.begin0000644000175000017500000000302310357525152022411 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  CRASH_COMMAND_NAME_HERE

crash-whitepaper-1.0/doc/help_pages/kmem.html0000644000175000017500000005163510357525152021556 0ustar  troyhebetroyhebe






                        

                                

                                        
White Paper: Red Hat Crash Utility

                                

                                

                                        

                                

                                

                                        
                                

                                

                                        Contents
                                

                                

                                        
                                

                        

                        

                                

                                        


help page:  kmem








NAME
  kmem - kernel memory

SYNOPSIS
  kmem [-f|-F|-p|-c|-C|-i|-s|-S|-v|-n] [-[l|L][a|i]] [slab-name] [[-P] address]

DESCRIPTION
  This command displays information about the use of kernel memory.

        -f  displays the contents of the system free memory headers.
            also verifies that the page count equals nr_free_pages.
        -F  same as -f, but also dumps all pages linked to that header.
        -p  displays basic information about each page in the system 
            mem_map[] array.
        -c  walks through the page_hash_table and verifies page_cache_size.
        -C  same as -c, but also dumps all pages in the page_hash_table.
        -i  displays general memory usage information
        -s  displays basic kmalloc() slab data.
        -S  displays all kmalloc() slab data, including all slab objects,
            and whether each object is in use or is free.
        -v  displays the vmlist entries.
        -n  display NUMA data (if supported).
       -la  walks through the active_list and verifies nr_active_pages.
       -li  walks through the inactive_list and verifies nr_inactive_pages.
       -La  same as -la, but also dumps each page in the active_list.
       -Li  same as -li, but also dumps each page in the inactive_list.
 slab-name  when used with -s or -S, limits the command to only the slab
            of name "slab-cache".
        -P  declares that the following address argument is a physical address.
   address  when used without any flag, the address can be a kernel virtual,
            or physical address; a search is made through the symbol table,
            the kmalloc() slab subsystem, the free list, the page_hash_table,
            the vmalloc() vmlist subsystem, and the mem_map array. If found
            in any of those areas, the information will be dumped in the
            same manner as if the flags were used.
   address  when used with -s or -S, searches the kmalloc() slab subsystem
            for the slab containing of this virtual address, showing whether
            it is in use or free.
   address  when used with -f, the address can be either a page pointer,
            a physical address, or a kernel virtual address; the free_area
            header containing the page (if any) is displayed.
   address  when used with -p, the address can be either a page pointer, a
            physical address, or a kernel virtual address; its basic mem_map
            page information is displayed.
   address  when used with -c, the address must be a page pointer address;
            the page_hash_table entry containing the page is displayed.
   address  when used with -l, the address must be a page pointer address;
            the page address is displayed if it is contained with the list.
   address  when used with -v, the address can be a mapped kernel virtual
            address or physical address; the vmlist containing the address
            is displayed.

  All address arguments above must be expressed in hexadecimal format.

EXAMPLES
  Display memory usage information:

    crash> kmem -i
                  PAGES        TOTAL      PERCENTAGE
     TOTAL MEM    63602     248.4 MB         ----
          FREE      993       3.9 MB    1% of TOTAL MEM
          USED    62609     244.6 MB   98% of TOTAL MEM
        SHARED    34035     132.9 MB   53% of TOTAL MEM
       BUFFERS    10928      42.7 MB   17% of TOTAL MEM
        CACHED    35249     137.7 MB   55% of TOTAL MEM
          SLAB     2823        11 MB    4% of TOTAL MEM
    
    TOTAL HIGH        0            0    0% of TOTAL MEM
     FREE HIGH        0            0    0% of TOTAL HIGH
     TOTAL LOW    63602     248.4 MB  100% of TOTAL MEM
      FREE LOW      993       3.9 MB    1% of TOTAL LOW
    
    TOTAL SWAP   129792       507 MB         ----
     SWAP USED    14727      57.5 MB   11% of TOTAL SWAP
     SWAP FREE   115065     449.5 MB   88% of TOTAL SWAP
    
    ZONE  NAME      FREE   ACTIVE  INACTIVE_DIRTY  INACTIVE_CLEAN  MIN/LOW/HIGH
      0   DMA        240     1166               7             161  128/256/384 
      1   Normal     753    17009           27834               0  255/510/765 
      2   HighMem      0        0               0               0     0/0/0    
 
  Display and verify free memory data:

    crash> kmem -f
    NODE
      0
    ZONE  NAME        SIZE    FREE  MEM_MAP   START_PADDR  START_MAPNR
      0   DMA         4096    3372  c4000040       0            0     
    AREA  SIZE  FREE_AREA_STRUCT  BLOCKS  PAGES
      0     4k      c02eb004           2      2
      1     8k      c02eb010           3      6
      2    16k      c02eb01c           5     20
      3    32k      c02eb028           4     32
      4    64k      c02eb034           5     80
      5   128k      c02eb040           3     96
      6   256k      c02eb04c           3    192
      7   512k      c02eb058           1    128
      8  1024k      c02eb064           1    256
      9  2048k      c02eb070           5   2560
 
    ZONE  NAME        SIZE    FREE  MEM_MAP   START_PADDR  START_MAPNR
      1   Normal    225280  202269  c4044040    1000000        4096   
    AREA  SIZE  FREE_AREA_STRUCT  BLOCKS  PAGES
      0     4k      c02eb0b8           1      1
      1     8k      c02eb0c4           2      4
      2    16k      c02eb0d0           0      0
      3    32k      c02eb0dc           1      8
      4    64k      c02eb0e8           1     16
      5   128k      c02eb0f4           0      0
      6   256k      c02eb100           0      0
      7   512k      c02eb10c           0      0
      8  1024k      c02eb118           0      0
      9  2048k      c02eb124         395 202240
 
    ZONE  NAME        SIZE    FREE  MEM_MAP   START_PADDR  START_MAPNR
      2   HighMem   819200  748686  c4ee0040    38000000      229376  
    AREA  SIZE  FREE_AREA_STRUCT  BLOCKS  PAGES
      0     4k      c02eb16c          10     10
      1     8k      c02eb178           2      4
      2    16k      c02eb184           0      0
      3    32k      c02eb190           2     16
      4    64k      c02eb19c           1     16
      5   128k      c02eb1a8           1     32
      6   256k      c02eb1b4           1     64
      7   512k      c02eb1c0           0      0
      8  1024k      c02eb1cc           0      0
      9  2048k      c02eb1d8        1462 748544
    
    nr_free_pages: 954327  (verified)
 
  Dump all the base addresses of each free memory area from above:

    crash> kmem -F
    NODE
      0
    ZONE  NAME        SIZE    FREE  MEM_MAP   START_PADDR  START_MAPNR
      0   DMA         4096    3372  c4000040       0            0     
    AREA  SIZE  FREE_AREA_STRUCT
      0     4k      c02eb004      
    c400ded8
    c4042528
    AREA  SIZE  FREE_AREA_STRUCT
      1     8k      c02eb010      
    c400de50
    c400cee8
    c40424a0
    AREA  SIZE  FREE_AREA_STRUCT
      2    16k      c02eb01c      
    c400dd40
    c400cf70
    c40425b0
    c400f7d0
    c40028a0
    AREA  SIZE  FREE_AREA_STRUCT
      3    32k      c02eb028      
    c4042280
    c400f8e0
    c4002680
    c4000260
    AREA  SIZE  FREE_AREA_STRUCT
      4    64k      c02eb034      
    c400d080
    c4041e40
    ...
 
  Dump the mem_map[] array:

    crash> kmem -p
      PAGE    PHYSICAL   INODE     OFFSET  CNT FLAGS
    c0294000         0         0         0  0  DMA,reserved
    c0294028      1000         0         0  0  DMA,reserved
    c0294050      2000         0         0  0  DMA,reserved
    c0294078      3000         0    2cf000  1  DMA,uptodate
    c02940a0      4000         0         0  1  DMA,slab
    c02940c8      5000         0         0  1  DMA,slab
    c02940f0      6000         0         0  1  DMA,slab
    c0294118      7000         0         0  1  DMA,slab
    c0294140      8000         0         0  1  DMA,slab
    c0294168      9000         0         0  0  DMA,slab
    c0294190      a000         0         0  0  DMA,slab
    c02941b8      b000         0         0  0  DMA,slab
    c02941e0      c000         0         0  1  DMA,slab
    c0294208      d000         0         0  0  DMA,slab
    c0294230      e000         0         0  1  DMA,slab
    c0294258      f000         0         0  0  DMA,slab
    c0294280     10000         0     11000  1  DMA,uptodate
    c02942a8     11000         0     10000  1  DMA,uptodate
    c02942d0     12000         0     1e000  1  DMA,uptodate
    c02942f8     13000         0    153000  1  DMA,uptodate
    c0294320     14000  c3243650     4b000  1  DMA,referenced,uptodate
    c0294348     15000  c3243650     4a000  1  DMA,referenced,uptodate
    ...
    
  Use the commands above with a page pointer or a physical address argument:

    crash> kmem -f c40425b0
    NODE 
      0 
    ZONE  NAME        SIZE    FREE  MEM_MAP   START_PADDR  START_MAPNR
      0   DMA         4096    3372  c4000040       0            0     
    AREA  SIZE  FREE_AREA_STRUCT 
      2    16k      c02eb01c      
    c40425b0  (c40425b0 is 1st of 4 pages) 
 
    crash> kmem -p c035de00
      PAGE    PHYSICAL   INODE     OFFSET  CNT FLAGS
    c035de00   50c0000         0    129000  0  uptodate

    crash> kmem -p 50c0000
      PAGE    PHYSICAL   INODE     OFFSET  CNT FLAGS
    c035de00   50c0000         0    129000  0  uptodate

  Display the vmlist entry data:

    crash> kmem -v
    VM_STRUCT      ADDRESS RANGE       SIZE
     c009c560   c8000000 - c8002000    8192
     c009c620   c8002000 - c8004000    8192
     c009cda0   c8004000 - c8016000   73728
     c009cd70   c8016000 - c8019000   12288
     c009cf80   c8019000 - c801b000    8192
     c009cfb0   c801b000 - c801d000    8192
     c009cef0   c801d000 - c802d000   65536
     c3afd060   c802d000 - c8032000   20480
     c3afd090   c8032000 - c8035000   12288
     c3afd0c0   c8035000 - c8037000    8192
     c3afd150   c8037000 - c8039000    8192
     c3afd180   c8039000 - c803b000    8192
     c3afd210   c803b000 - c803d000    8192
     c3afd2a0   c803d000 - c8040000   12288
     c3afd2d0   c8040000 - c8043000   12288
     c3afd300   c8043000 - c8047000   16384
     c3afddb0   c8047000 - c804d000   24576
     c2f8a320   c804d000 - c805c000   61440
     c2f8a380   c805c000 - c8065000   36864
     c2f8a3b0   c8065000 - c806e000   36864
     c2f8aa70   c806e000 - c8095000  159744
     c2f8ab60   c8095000 - c8097000    8192
     c2f519e0   c8097000 - c8099000    8192
 
  Determine (and verify) the page cache size:

    crash> kmem -c
    page_cache_size: 18431 (verified)
 
  Dump all pages in the page_hash_table:

    crash> kmem -C
    page_hash_table[0]
    c0325b40
    c03a0598
    c03b4070
    c0364c28
    c0357690
    c02ef338
    c02d7c60
    c02c11e0
    c02a3d70
    page_hash_table[1]
    c0394ce8
    c03c4218
    c03b4048
    c0364c00
    c0357668
    c02d6e50
    c02d7dc8
    c02c0cb8
    c02db630
    c02ebad0
    page_hash_table[2]
    c037e808
    c034e248
    c03b4020
    c02ec868
    c03baa60
    ...
    page_hash_table[2047]
    c033a798
    c0390b48
    c03b4098
    c0364890
    c03576b8
    c02d2c38
    c02d7c88
    c02de5d8
    
    page_cache_size: 18437 (verified)
    
  Find the page_hash_table entry containing page c03576b8:

    crash> kmem -c c03576b8
    page_hash_table[2047]
    c03576b8
 
  Display kmalloc() slab data:

    crash> kmem -s
    CACHE     NAME                OBJSIZE  ALLOCATED     TOTAL  SLABS  SSIZE
    c02eadc0 kmem_cache               232         58        68      4     4k
    f79c2888 ip_vs_conn               128          0         0      0     4k
    f79c2970 tcp_tw_bucket             96          0         0      0     4k
    f79c2a58 tcp_bind_bucket           32         12       565      5     4k
    f79c2b40 tcp_open_request          64          0        59      1     4k
    f79c2c28 inet_peer_cache           64          1        59      1     4k
    f79c2d10 ip_fib_hash               32         11       339      3     4k
    f79c2df8 ip_dst_cache             160          8       120      5     4k
    f79c2ee0 arp_cache                128          1        30      1     4k
    c8402970 blkdev_requests           96      30208     37800    945     4k
    c8402a58 nfs_read_data            384          0         0      0     4k
    c8402b40 nfs_write_data           384          0         0      0     4k
    c8402c28 nfs_page                  96          0         0      0     4k
    c8402d10 dnotify cache             20          0         0      0     4k
    c8402df8 file lock cache           92          3       336      8     4k
    c8402ee0 fasync cache              16          0         0      0     4k
    c84027a0 uid_cache                 32          3       339      3     4k
    c84026b8 skbuff_head_cache        160        320       624     26     4k
    c84025d0 sock                     832         32       180     20     8k
    c84024e8 sigqueue                 132          0       203      7     4k
    c8402400 cdev_cache                64         19       472      8     4k
    c8402318 bdev_cache                64          8       236      4     4k
    c8402230 mnt_cache                 96         11       120      3     4k
    c8402148 inode_cache              480        817       848    106     4k
    c8402060 dentry_cache             128       1352      1470     49     4k
    c8403ee0 filp                      96        244       440     11     4k
    c8403df8 names_cache             4096          0        12     12     4k
    c8403d10 buffer_head               96      14936     16000    400     4k
    c8403c28 mm_struct                128         25       240      8     4k
    c8403b40 vm_area_struct            64        393      1298     22     4k
    c8403a58 fs_cache                  64         30       472      8     4k
    c8403970 files_cache              416         30       135     15     4k
    c8403888 signal_act              1312         32        99     33     4k
    c84037a0 size-131072(DMA)      131072          0         0      0   128k
    c84036b8 size-131072           131072          1         1      1   128k
    c84035d0 size-65536(DMA)        65536          0         0      0    64k
    c84034e8 size-65536             65536          0         0      0    64k
    c8403400 size-32768(DMA)        32768          0         0      0    32k
    c8403318 size-32768             32768          0         1      1    32k
    c8403230 size-16384(DMA)        16384          0         0      0    16k
    c8403148 size-16384             16384          0         0      0    16k
    c8403060 size-8192(DMA)          8192          0         0      0     8k
    c8401ee0 size-8192               8192          1         2      2     8k
    c8401df8 size-4096(DMA)          4096          0         0      0     4k
    c8401d10 size-4096               4096         30        30     30     4k
    c8401c28 size-2048(DMA)          2048          0         0      0     4k
    c8401b40 size-2048               2048         37       132     66     4k
    c8401a58 size-1024(DMA)          1024          0         0      0     4k
    c8401970 size-1024               1024        301       328     82     4k
    c8401888 size-512(DMA)            512          0         0      0     4k
    c84017a0 size-512                 512        141       168     21     4k
    c84016b8 size-256(DMA)            256          0         0      0     4k
    c84015d0 size-256                 256         80       435     29     4k
    c84014e8 size-128(DMA)            128          0         0      0     4k
    c8401400 size-128                 128        508       840     28     4k
    c8401318 size-64(DMA)              64          0         0      0     4k
    c8401230 size-64                   64        978      1357     23     4k
    c8401148 size-32(DMA)              32          0         0      0     4k
    c8401060 size-32                   32       1244      1808     16     4k
 
  Display all slab data in the "arp_cache" cache:

    crash> kmem -S arp_cache
    CACHE     NAME                OBJSIZE  ALLOCATED     TOTAL  SLABS  SSIZE
    f79c2ee0 arp_cache                128          1        30      1     4k
    SLAB      MEMORY    TOTAL  ALLOCATED  FREE
    f729d000  f729d0a0     30          1    29
    FREE / [ALLOCATED]
       f729d0a0  (cpu 7 cache)
       f729d120  (cpu 7 cache)
       f729d1a0  (cpu 7 cache)
       f729d220  (cpu 7 cache)
       f729d2a0  (cpu 7 cache)
       f729d320  (cpu 7 cache)
       f729d3a0  (cpu 7 cache)
       f729d420  (cpu 7 cache)
       f729d4a0  (cpu 7 cache)
       f729d520  (cpu 7 cache)
       f729d5a0  (cpu 7 cache)
       f729d620  (cpu 7 cache)
       f729d6a0  (cpu 7 cache)
       f729d720  (cpu 7 cache)
       f729d7a0  (cpu 7 cache)
       f729d820  (cpu 7 cache)
       f729d8a0  (cpu 7 cache)
       f729d920  (cpu 7 cache)
       f729d9a0  (cpu 7 cache)
       f729da20  (cpu 7 cache)
       f729daa0  (cpu 7 cache)
       f729db20  (cpu 7 cache)
       f729dba0  (cpu 7 cache)
       f729dc20  (cpu 7 cache)
       f729dca0  (cpu 7 cache)
       f729dd20  (cpu 7 cache)
       f729dda0  (cpu 7 cache)
       f729de20  (cpu 7 cache)
       f729dea0  (cpu 3 cache)
      [f729df20]
 
  Search the kmalloc() slab subsystem for address c3fbdb60:

    crash> kmem -s c3fbdb60
    CACHE     NAME                OBJSIZE  ALLOCATED     TOTAL  SLABS  SSIZE
    c8402970 blkdev_requests           96      30208     37800    945     4k
    SLAB      MEMORY    TOTAL  ALLOCATED  FREE
    c3fbd020  c3fbd0e0     40         40     0
    FREE / [ALLOCATED]
      [c3fbdb60]
 
  Make a generic search (no flags) for the same address c3fbdb60:

    crash> kmem c3fbdb60 
    CACHE     NAME                OBJSIZE  ALLOCATED     TOTAL  SLABS  SSIZE
    c8402970 blkdev_requests           96      30208     37800    945     4k
    SLAB      MEMORY    TOTAL  ALLOCATED  FREE
    c3fbd020  c3fbd0e0     40         40     0 
    FREE / [ALLOCATED]
      [c3fbdb60]
 
      PAGE     PHYSICAL   MAPPING    INDEX CNT FLAGS
    c410ee74    3fbd000         0         0  1 slab
 
  Verify the active and inactive page lists:

    crash> kmem -la -li
    nr_active_pages: 1893 (verified)
    nr_inactive_pages: 2491 (verified)

  Display NUMA data (if supported):

    crash> kmem -n
    NODE    SIZE    PGLIST_DATA   BOOTMEM_DATA   NODE_ZONES
      0    130933     c0332ee0      c0403a44      c0332ee0
                                                  c03333e0
                                                  c03338e0
    MEM_MAP   START_PADDR  START_MAPNR
    c1000030       0            0
    
    ZONE  NAME        SIZE   MEM_MAP  START_PADDR  START_MAPNR
      0   DMA         4096  c1000030            0            0
      1   Normal    126837  c1038030      1000000         4096
      2   HighMem        0         0            0            0









                                        		
                                

                        

                        

                        

                                

                                        
                                

                                

                                        Contents
crash-whitepaper-1.0/doc/help_pages/list.html0000644000175000017500000002125510357525152021573 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  list


NAME
  list - linked list

SYNOPSIS
  list [[-o] offset] [-e end] [-s struct[.member]] [-H] start

DESCRIPTION
  This command dumps the contents of a linked list.  The entries in a linked
  are typically data structures that are tied together in one of two formats:
 
  1. A starting address points to a data structure; that structure contains
     a member that is a pointer to the next structure, and so on.  The list
     typically ends when a "next" pointer value contains one of the
     following:

       a. a NULL pointer.
       b. a pointer to the start address.
       c. a pointer to the first item pointed to by the start address.
       d. a pointer to its containing structure.
  
  2. Many Linux lists are linked via embedded list_head structures contained 
     within the data structures in the list.  The linked list is headed by an
     external LIST_HEAD, which is simply a list_head structure initialized to
     point to itself, signifying that the list is empty:
 
       struct list_head {
           struct list_head *next, *prev;
       };
 
       #define LIST_HEAD_INIT(name) { &(name), &(name) } 
       #define LIST_HEAD(name) struct list_head name = LIST_HEAD_INIT(name)
 
     In the case of list_head-type lists, the "next" pointer is the address
     of the embedded list_head structure in the next structure, and not the
     address of the structure itself.  The list typically ends when the
     list_head's next pointer points back to the LIST_HEAD address.
 
  This command can handle both types of linked list; in both cases the list
  of addresses that are dumped are the addresses of the data structures
  themselves.
 
  The arguments are as follows:

  [-o] offset  The offset within the structure to the "next" pointer
               (default is 0).  If non-zero, the offset may be entered
               in either of two manners:

               1. In "structure.member" format; the "-o" is not necessary.
               2. A number of bytes; the "-o" is only necessary on processors
                  where the offset value could be misconstrued as a kernel
                  virtual address.

       -e end  If the list ends in a manner unlike the typical manners that
               are described above, an explicit ending address value may be
               entered.
    -s struct  For each address in list, format and print as this type of
               structure; use the "struct.member" format in order to display
               a particular member of the structure.
 
  The meaning of the "start" argument, which can be expressed either
  symbolically or in hexadecimal format, depends upon whether the -H option
  is pre-pended or not:
 
      start  The address of the first structure in the list.
   -H start  The address of the LIST_HEAD structure, typically expressed
             symbolically.

EXAMPLES
  Note that each task_struct is linked to its parent's task_struct via the
  p_pptr member:
 
    crash> struct task_struct.p_pptr
    struct task_struct {
       [136] struct task_struct *p_pptr;
    }
 
  That being the case, given a task_struct pointer of c169a000, show its 
  parental hierarchy back to the "init_task" (the "swapper" task):

    crash> list task_struct.p_pptr c169a000
    c169a000
    c0440000
    c50d0000
    c0562000
    c0d28000
    c7894000
    c6a98000
    c009a000
    c0252000

  Given that the "task_struct.p_pptr" offset is 136 bytes, the same
  result could be accomplished like so:

    crash> list 136 c169a000
    c169a000
    c0440000
    c50d0000
    c0562000
    c0d28000
    c7894000
    c6a98000
    c009a000
    c0252000
 
  The list of currently-registered file system types are headed up by a
  struct file_system_type pointer named "file_systems", and linked by
  the "next" field in each file_system_type structure.  The following
  sequence displays the address and name of each registered file system type:
 
    crash> p file_systems
    file_systems = $2 = (struct file_system_type *) 0xc02ebea0
    crash> list file_system_type.next -s file_system_type.name 0xc02ebea0
    c02ebea0
      name = 0xc0280372 "proc", 
    c02fd4a0
      name = 0xc02bf348 "sockfs", 
    c02eb544
      name = 0xc027c25a "tmpfs", 
    c02eb52c
      name = 0xc027c256 "shm", 
    c02ebbe0
      name = 0xc027e054 "pipefs", 
    c02ec9c0
      name = 0xc0283c13 "ext2", 
    c02ecaa8
      name = 0xc0284567 "iso9660", 
    c02ecc08
      name = 0xc0284cf5 "nfs", 
    c02edc60
      name = 0xc028d832 "autofs", 
    c02edfa0
      name = 0xc028e1e0 "devpts"
 
  In some kernels, the system run queue is a linked list headed up by the
  "runqueue_head", which is defined like so:
 
    static LIST_HEAD(runqueue_head);
 
  The run queue linking is done with the "run_list" member of the task_struct:
 
    crash> struct task_struct.run_list
    struct task_struct {
        [60] struct list_head run_list;
    }
 
  Therefore, to view the list of task_struct addresses in the run queue,
  either of the following commands will work:

    crash> list task_struct.run_list -H runqueue_head
    f79ac000
    f7254000
    f7004000
    crash> list 60 -H runqueue_head
    f79ac000
    f7254000
    f7004000
 
  Lastly, in some kernel versions, the vfsmount structures of the mounted
  filesystems are linked by the LIST_HEAD "vfsmntlist", which uses the
  mnt_list list_head of each vfsmount structure in the list.  To dump each
  vfsmount structure in the list, append the -s option:

    crash> list -H vfsmntlist vfsmount.mnt_list -s vfsmount
    c3fc9e60
    struct vfsmount {
      mnt_hash = {
        next = 0xc3fc9e60, 
        prev = 0xc3fc9e60
      }, 
      mnt_parent = 0xc3fc9e60, 
      mnt_mountpoint = 0xc3fc5dc0, 
      mnt_root = 0xc3fc5dc0, 
      mnt_instances = {
        next = 0xc3f60a74, 
        prev = 0xc3f60a74
      }, 
      mnt_sb = 0xc3f60a00, 
      mnt_mounts = {
        next = 0xf7445e08, 
        prev = 0xf7445f88
      }, 
      mnt_child = {
        next = 0xc3fc9e88, 
        prev = 0xc3fc9e88
      }, 
      mnt_count = {
        counter = 209
      }, 
      mnt_flags = 0, 
      mnt_devname = 0xc8465b20 "/dev/root", 
      mnt_list = {
        next = 0xf7445f9c, 
        prev = 0xc02eb828
      }, 
      mnt_owner = 0
    }
    f7445f60
    struct vfsmount {
    ...
Contents
crash-whitepaper-1.0/doc/help_pages/log.html0000644000175000017500000001163310357525152021400 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  log


NAME
  log - dump system message buffer

SYNOPSIS
  log [-c]

DESCRIPTION
  This command dumps the kernel log_buf contents in chronological order.
  
    -m  Display the message log level preceding each message.

EXAMPLES
  Dump the kernel message buffer:

    crash> log
    Linux version 2.2.5-15smp (root@mclinux1) (gcc version egcs-2.91.66 19990
    314/Linux (egcs-1.1.2 release)) #1 SMP Thu Aug 26 11:04:37 EDT 1999
    Intel MultiProcessor Specification v1.4
        Virtual Wire compatibility mode.
    OEM ID: DELL     Product ID: WS 410       APIC at: 0xFEE00000
    Processor #0 Pentium(tm) Pro APIC version 17
    Processor #1 Pentium(tm) Pro APIC version 17
    I/O APIC #2 Version 17 at 0xFEC00000.
    Processors: 2
    mapped APIC to ffffe000 (fee00000)
    mapped IOAPIC to ffffd000 (fec00000)
    Detected 447696347 Hz processor.
    Console: colour VGA+ 80x25
    Calibrating delay loop... 445.64 BogoMIPS
    ...
      8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
      MII transceiver found at address 24, status 782d.
      Enabling bus-master transmits and whole-frame receives.
    Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
    nfsd_init: initialized fhcache, entries=256
    ...
 
  Do the same thing, but also show the log level preceding each message:

    crash> log -m
    <4>Linux version 2.2.5-15smp (root@mclinux1) (gcc version egcs-2.91.66 19990
    314/Linux (egcs-1.1.2 release)) #1 SMP Thu Aug 26 11:04:37 EDT 1999
    <4>Intel MultiProcessor Specification v1.4
    <4>    Virtual Wire compatibility mode.
    <4>OEM ID: DELL     Product ID: WS 410       APIC at: 0xFEE00000
    <4>Processor #0 Pentium(tm) Pro APIC version 17
    <4>Processor #1 Pentium(tm) Pro APIC version 17
    <4>I/O APIC #2 Version 17 at 0xFEC00000.
    <4>Processors: 2
    <4>mapped APIC to ffffe000 (fee00000)
    <4>mapped IOAPIC to ffffd000 (fec00000)
    <4>Detected 447696347 Hz processor.
    <4>Console: colour VGA+ 80x25
    <4>Calibrating delay loop... 445.64 BogoMIPS
    ...
    <6>  8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
    <6>  MII transceiver found at address 24, status 782d.
    <6>  Enabling bus-master transmits and whole-frame receives.
    <6>Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
    <7>nfsd_init: initialized fhcache, entries=256
    ...
Contents
crash-whitepaper-1.0/doc/help_pages/mach.html0000644000175000017500000000733110357525152021527 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  mach


NAME
  mach - machine specific data

SYNOPSIS
  mach [-cm]

DESCRIPTION
  This command displays data specific to a machine type.

    -c  Display each cpu's cpuinfo structure (x86 and ia64 only).
        Display the hwrpb_struct, and each cpu's percpu_struct (alpha only).
    -m  Display the physical memory map (ia64 and x86 only).

EXAMPLES
    crash> mach
           MACHINE TYPE: i686
            MEMORY SIZE: 512 MB
                   CPUS: 2
        PROCESSOR SPEED: 1993 Mhz
                     HZ: 100
              PAGE SIZE: 4096
          L1 CACHE SIZE: 32
    KERNEL VIRTUAL BASE: c0000000
    KERNEL VMALLOC BASE: e0800000
      KERNEL STACK SIZE: 8192
 
  Display the system physical memory map:

    crash> mach -m
          PHYSICAL ADDRESS RANGE         TYPE
    0000000000000000 - 00000000000a0000  E820_RAM
    00000000000f0000 - 0000000000100000  E820_RESERVED
    0000000000100000 - 000000001ff75000  E820_RAM
    000000001ff75000 - 000000001ff77000  E820_NVS
    000000001ff77000 - 000000001ff98000  E820_ACPI
    000000001ff98000 - 0000000020000000  E820_RESERVED
    00000000fec00000 - 00000000fec90000  E820_RESERVED
    00000000fee00000 - 00000000fee10000  E820_RESERVED
    00000000ffb00000 - 0000000100000000  E820_RESERVED
Contents
crash-whitepaper-1.0/doc/help_pages/mod.html0000644000175000017500000001733110357525152021377 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  mod


NAME
  mod - module information and loading of symbols and debugging data

SYNOPSIS
  mod [ -s module [objfile] | -d module | -S | -D | -r ] 

DESCRIPTION
  With no arguments, this command displays basic information of the currently
  installed modules, consisting of the module address, name, size, and the
  object file name, if known.

  The arguments are concerned with with the loading or deleting of symbolic
  and debugging data from a module's object file:

  -s module [objfile]  Loads symbolic and debugging data from the object file
                       for the module specified.  If no objectfile argument is
                       appended, a search will be made for an object file
                       consisting of the module name with a .o suffix,
                       starting at the /lib/modules/<release> directory on
                       the host system.  If an objfile argument is appended,
                       that file will be used.
            -d module  Deletes the symbolic and debugging data of the module
                       specified.
                   -S  Load symbolic and debugging data from the object file
                       for all loaded modules.  For each module, a search
                       will be made for an object file consisting of the
                       module name with a .o suffix, starting at the
                       /lib/modules/<release> directory of the host system.
                   -D  Deletes the symbolic and debugging data of all modules.
                   -r  Reinitialize module data. All currently-loaded symbolic
                       and debugging data will be deleted, and the installed
                       module list will be updated (live system only).
 
  After symbolic and debugging data have been loaded, backtraces and text
  disassembly will be displayed appropriately.  Depending upon the processor
  architecture, data may also printed symbolically with the "p" command;
  at a minimum, the "rd" command may be used with module data symbols.
 
  If crash can recognize that the set of modules has changed while running a
  session on a live kernel, the module data will be reinitialized the next
  time this command is run; the -r option forces the reinitialization.

EXAMPLES
  Display the currently-installed modules:

    crash> mod
     MODULE   NAME         SIZE  OBJECT FILE
    c8019000  soundcore    2788  (not loaded)
    c801b000  soundlow      336  (not loaded)
    c801d000  sound       59864  (not loaded)
    c802d000  ad1848      15728  (not loaded)
    c8032000  uart401      6000  (not loaded)
    c8035000  cs4232       2472  (not loaded)
    c8043000  opl3        11048  (not loaded)
    c8047000  3c59x       18152  (not loaded)
    c804d000  sunrpc      53796  (not loaded)
    c805c000  lockd       31528  (not loaded)
    c8065000  nfsd       151896  (not loaded)
    c8092000  nfs         29752  (not loaded)
 
  Load the symbolic and debugging data of all modules:

    crash> mod -S
     MODULE   NAME         SIZE  OBJECT FILE
    c8019000  soundcore    2788  /lib/modules/2.2.5-15/misc/soundcore.o
    c801b000  soundlow      336  /lib/modules/2.2.5-15/misc/soundlow.o
    c801d000  sound       59864  /lib/modules/2.2.5-15/misc/sound.o
    c802d000  ad1848      15728  /lib/modules/2.2.5-15/misc/ad1848.o
    c8032000  uart401      6000  /lib/modules/2.2.5-15/misc/uart401.o
    c8035000  cs4232       2472  /lib/modules/2.2.5-15/misc/cs4232.o
    c8043000  opl3        11048  /lib/modules/2.2.5-15/misc/opl3.o
    c8047000  3c59x       18152  /lib/modules/2.2.5-15/net/3c59x.o
    c804d000  sunrpc      53796  /lib/modules/2.2.5-15/misc/sunrpc.o
    c805c000  lockd       31528  /lib/modules/2.2.5-15/fs/lockd.o
    c8065000  nfsd       151896  /lib/modules/2.2.5-15/fs/nfsd.o
    c8092000  nfs         29752  /lib/modules/2.2.5-15/fs/nfs.o
    
  Load the symbolic and debugging data of the soundcore module from its
  known location:
 
    crash> mod -s soundcore
     MODULE   NAME         SIZE  OBJECT FILE
    c8019000  soundcore    2788  /lib/modules/2.2.5-15/misc/soundcore.o
    
  Delete the current symbolic and debugging data of the soundcore module, 
  and then re-load it from a specified object file:
 
    crash> mod -d soundcore
    crash> mod -s soundcore /tmp/soundcore.o
     MODULE   NAME         SIZE  OBJECT FILE
    c8019000  soundcore    2788  /tmp/soundcore.o
 
  After installing a new kernel module on a live system, reinitialize the
  installed module list:

    crash> !insmod mdacon
    crash> mod
    mod: NOTE: modules have changed on this system -- reinitializing
     MODULE   NAME         SIZE  OBJECT FILE
    c8019000  soundcore    2788  (not loaded)
    c801b000  soundlow      336  (not loaded)
    c801d000  sound       59864  (not loaded)
    c802d000  ad1848      15728  (not loaded)
    c8032000  uart401      6000  (not loaded)
    c8035000  cs4232       2472  (not loaded)
    c8043000  opl3        11048  (not loaded)
    c8047000  3c59x       18152  (not loaded)
    c804d000  sunrpc      53796  (not loaded)
    c805c000  lockd       31528  (not loaded)
    c8065000  nfs         29752  (not loaded)
    c806e000  autofs       9316  (not loaded)
    c8072000  nfsd       151896  (not loaded)
    c80a1000  mdacon       3556  (not loaded)
Contents
crash-whitepaper-1.0/doc/help_pages/mount.html0000644000175000017500000001147310357525152021763 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  mount


NAME
  mount - mounted filesystem data

SYNOPSIS
  mount [-f] [-i] [vfsmount | superblock | devname | dirname | inode]

DESCRIPTION
  This command displays basic information about the currently-mounted
  filesystems.  The per-filesystem dirty inode list or list of open
  files for the filesystem may also be displayed.

     -f  dump dentries and inodes for open files in each filesystem.
     -i  dump all dirty inodes associated with each filesystem.

  Filesystems may be selected in the following forms:

    vfsmount  hexadecimal address of filesystem vfsmount structure.
  superblock  hexadecimal address of filesystem super_block structure.
     devname  device name of filesystem.
     dirname  directory where filesystem is mounted.
       inode  hexadecimal address of an open inode of a filesystem.

EXAMPLES
  Display mounted filesystem data:

    crash> mount
    VFSMOUNT SUPERBLK TYPE   DEVNAME         DIRNAME
    c0089ea0 c0088a00 ext2   /dev/root       /    
    c0089cf0 c0088c00 proc   /proc           /proc
    c0089e10 c0088800 ext2   /dev/sda5       /boot
    c0089d80 c0088600 ext2   /dev/sda6       /usr
    c0089f30 c0088400 devpts none            /dev/pts
    c3f4b010 c0088200 ext2   /dev/sda1       /home
    c6bf3d10 c0088000 nfs    home:/home1     /home1
    c49b90a0 c43a2a00 nfs    home:/usr/local /usr/local 
 
  Display the open files associated with each mounted filesystem:

    crash> mount -f
    VFSMOUNT SUPERBLK TYPE   DEVNAME         DIRNAME
    c7fb2b80 c7fb3200 ext2   /dev/root       /
    OPEN FILES:
     DENTRY    INODE    TYPE  PATH
    c6d02200  c6d0f7a0  REG   usr/X11R6/lib/libX11.so.6.1
    c6d02100  c6d0f9e0  REG   usr/X11R6/lib/libXext.so.6.3
    c6d02000  c6d0fc20  REG   usr/X11R6/lib/libICE.so.6.3
    c6d02680  c6d0f320  REG   usr/X11R6/bin/xfs
    c7106580  c70c5440  CHR   dev/psaux
    ...
 
  Display the dirty inodes associated with each mounted filesystem:

    crash> mount -i
    VFSMOUNT SUPERBLK TYPE   DEVNAME         DIRNAME
    c0089ea0 c0088a00 ext2   /dev/root       /            
    DIRTY INODES
    c7ad4008
    c2233438
    c72c4008
    c7d6b548
    c3af1a98
    c7d6b768
    c3c4e228
    ...
 
  Display the mounted filesystem containing inode c5000aa8:

    crash> mount c5000aa8
    VFSMOUNT SUPERBLK TYPE   DEVNAME         DIRNAME
    c0089f30 c0088600 ext2   /dev/sda6       /usr
Contents
crash-whitepaper-1.0/doc/help_pages/net.html0000644000175000017500000002003510357525152021401 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  net


NAME
  net - network command

SYNOPSIS
  net [-a] [[-s | -S] [-R ref] [pid | taskp]] [-n addr]

DESCRIPTION
  Display various network related data:

      -a  display the ARP cache.
      -s  display open network socket/sock addresses, their family and type,
          and their source and destination addresses and ports.
      -S  displays open network socket/sock addresses followed by a dump
          of both structures.
  -n addr translates an IP address expressed as a decimal or hexadecimal 
          value into a standard numbers-and-dots notation.
  -R ref  socket or sock address, or file descriptor.
     pid  a process PID.
   taskp  a hexadecimal task_struct pointer.

  If no arguments are entered, the list of network devices, names and IP
  addresses are displayed.  The -R option, typically invoked from "foreach net",
  and in conjunction with the -s or -S options, searches for references to a
  socket address, sock address, or a file descriptor; if found, only the
  referenced fd/socket/sock data will be displayed.

EXAMPLES
  Display the network device list:

    crash> net
     DEVICE   NAME   IP ADDRESS(ES)
    c0249f20  lo     127.0.0.1
    c7fe6d80  eth0   10.1.8.20
 
  Dump the ARP cache:

    crash> net -a
    IP ADDRESS      HW TYPE    HW ADDRESS         DEVICE  STATE
    0.0.0.0         UNKNOWN    00 00 00 00 00 00  lo      40 (NOARP)
    192.168.1.1     ETHER      00:50:54:fe:ef:23  eth0    04 (STALE)
    192.168.1.10    ETHER      00:90:27:9c:6c:79  eth0    02 (REACHABLE)
    192.168.1.118   ETHER      00:c0:4f:60:00:e2  eth0    02 (REACHABLE)
   
  Display the sockets for PID 2517, using both -s and -S output formats:

    crash> net -s 2517
    PID: 2517   TASK: c1598000  CPU: 1   COMMAND: "rlogin"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     3  c57375dc  c1ff1850  INET:STREAM      10.1.8.20:1023      10.1.16.62:513
    
    crash> net -S 2517
    PID: 2517   TASK: c1598000  CPU: 1   COMMAND: "rlogin"
    FD   SOCKET     SOCK
     3  c57375dc  c1ff1850
    
    struct socket {
      state = SS_CONNECTED,
      flags = 131072,
      ops = 0xc023f820,
      inode = 0xc5737540,
      fasync_list = 0x0,
      file = 0xc58892b0,
      sk = 0xc1ff1850,
      wait = 0xc14d9ed4,
      type = 1,
      passcred = 0 '\000',
      tli = 0 '\000'
    }
    struct sock {
      sklist_next = 0xc1ff12f0,
      sklist_prev = 0xc216bc00,
      bind_next = 0x0,
      bind_pprev = 0xc0918448,
      daddr = 1041236234,
      rcv_saddr = 336068874,
      dport = 258,
      num = 1023,
      bound_dev_if = 0,
      next = 0x0,
      pprev = 0xc0286dd4,
      state = 1 '\001',
      zapped = 0 '\000',
      sport = 65283,
      family = 2,
      reuse = 0 '\000',
      ...
   Translate the rcv_saddr from above into dotted-decimal notation:

    crash> net -n 1041236234
    10.1.16.62
 
  From "foreach", find all tasks with references to socket c08ea3cc:

    crash> foreach net -s -R c08ea3cc
    PID: 2184   TASK: c7026000  CPU: 1   COMMAND: "klines.kss"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2200   TASK: c670a000  CPU: 1   COMMAND: "kpanel"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2201   TASK: c648a000  CPU: 1   COMMAND: "kbgndwm"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 19294  TASK: c250a000  CPU: 0   COMMAND: "prefdm"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2194   TASK: c62dc000  CPU: 1   COMMAND: "kaudioserver"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2195   TASK: c6684000  CPU: 1   COMMAND: "maudio"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2196   TASK: c6b58000  CPU: 1   COMMAND: "kwmsound"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2197   TASK: c6696000  CPU: 0   COMMAND: "kfm"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2199   TASK: c65ec000  CPU: 0   COMMAND: "krootwm"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 694    TASK: c1942000  CPU: 0   COMMAND: "prefdm"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 698    TASK: c6a2c000  CPU: 1   COMMAND: "X"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
    
    PID: 2159   TASK: c4a5a000  CPU: 1   COMMAND: "kwm"
    FD   SOCKET     SOCK    FAMILY:TYPE         SOURCE:PORT     DESTINATION:PORT
     5  c08ea3cc  c50d3c80  INET:STREAM        0.0.0.0:1026         0.0.0.0:0
Contents
crash-whitepaper-1.0/doc/help_pages/output.html0000644000175000017500000000700510357525152022155 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  output


By default, crash command output is piped to "/usr/bin/less -E -X" along
with a prompt line.  This behavior can be turned off in two ways:

  1. During runtime, enter "set scroll off" or the alias "sf".
  2. Enter "set scroll off" in a .crashrc file, which can be located either
     in the current directory or in your home directory.

To restore the scrolling behavior during runtime, enter "set scroll on"
or the alias: "sn"

Command output may be piped to an external command using standard command
line pipe syntax.  For example:

  crash> log | grep eth0

Command output may be redirected to a file using standard command line syntax.
For example:

  crash> foreach bt > bt.all

Use double brackets to append the output to a pre-existing file:

  crash> ps >> crash.data

The default output radix for gdb output and certain crash commands is
hexadecimal.  This can be changed to decimal by entering "set radix 10"
or the alias "dec".  It can be reverted back to hexadecimal by entering
"set radix 16" or the alias "hex".
Contents
crash-whitepaper-1.0/doc/help_pages/p.html0000644000175000017500000001010010357525152021042 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  p


NAME
  p - print the value of an expression

SYNOPSIS
  p expression

DESCRIPTION
  This command passes its arguments on to gdb "print" command for evaluation.

    expression   The expression to be evaluated.

  The default output format is decimal, but that can be changed at any time
  with the two built-in aliases "hex" and "dec".  Alternatively, there
  are two other built-in aliases, "px" and "pd", which force the command
  output to be displayed in hexadecimal or decimal, without changing the
  default mode. 

EXAMPLES
  Print the contents of jiffies:

    crash> p jiffies
    jiffies = $6 = 166532620
    crash> px jiffies
    jiffies = $7 = 0x9ed174b
    crash> pd jiffies
    jiffies = $8 = 166533160
 
  Print the contents of the vm_area_struct "init_mm":

    crash> p init_mm
    init_mm = $5 = {
      mmap = 0xc022d540, 
      mmap_avl = 0x0, 
      mmap_cache = 0x0, 
      pgd = 0xc0101000, 
      count = {
        counter = 0x6
      }, 
      map_count = 0x1, 
      mmap_sem = {
        count = {
          counter = 0x1
        }, 
        waking = 0x0, 
        wait = 0x0
      }, 
      context = 0x0, 
      start_code = 0xc0000000, 
      end_code = 0xc022b4c8, 
      start_data = 0x0, 
      end_data = 0xc0250388, 
      start_brk = 0x0, 
      brk = 0xc02928d8, 
      start_stack = 0x0, 
      arg_start = 0x0, 
      arg_end = 0x0, 
      env_start = 0x0, 
      env_end = 0x0, 
      rss = 0x0, 
      total_vm = 0x0, 
      locked_vm = 0x0, 
      def_flags = 0x0, 
      cpu_vm_mask = 0x0, 
      swap_cnt = 0x0, 
      swap_address = 0x0, 
      segments = 0x0
    }
Contents
crash-whitepaper-1.0/doc/help_pages/pointer.html0000644000175000017500000000622010357525152022273 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  *

 
NAME
  * - pointer-to short-cut

SYNOPSIS
  * (struct or union command arguments)

DESCRIPTION
  This command is a short-cut command that replaces the requirement to enter
  "struct" or "union" command names.  For details on the arguments to
  those commands, enter "help struct" or "help union".

EXAMPLES
  Dump the page structure at address c02943c0:
 
    crash> *page c02943c0
    struct page {
      next = 0xc0fae740, 
      prev = 0xc0018fb0, 
      inode = 0x0, 
      offset = 0x3f000, 
      next_hash = 0xc02d6310, 
      count = {
        counter = 0x1
      }, 
      flags = 0x310, 
      wait = 0xc02943d8, 
      pprev_hash = 0x0, 
      buffers = 0x0
    }
Contents
crash-whitepaper-1.0/doc/help_pages/ps.html0000644000175000017500000003011410357525152021234 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  ps


NAME
  ps - display process status information

SYNOPSIS
  ps [-k|-u][-s][-p|-c|-t] [pid | taskp | command] ...

DESCRIPTION
  This command displays process status for selected, or all, processes
  in the system.  If no arguments are entered, the process data is
  is displayed for all processes.  Selected process identifiers can be
  entered in the following forms:

       pid  a process PID.
     taskp  a hexadecimal task_struct pointer.
   command  a command name.  If a command name is made up of letters that
            are all numerical values, precede the name string with a "\".
        -k  select all kernel threads.
        -u  select all user tasks.
 
  The process identifier types may be mixed.  For each task, the following
  items are displayed:
 
    1. the process PID.
    2. the parent process PID.
    3. the CPU number that the task ran on last.
    4. the task_struct address or the kernel stack pointer of the process.
       (see -s option below)
    5. the task state (RU, IN, UN, ZO, ST, DE, SW).
    6. the percentage of physical memory being used by this task.
    7. the virtual address size of this task in kilobytes.
    8. the resident set size of this task in kilobytes.
    9. the command name.
 
  The default output shows the task_struct address of each process under a
  column titled "TASK".  This can be changed to show the kernel stack 
  pointer under a column titled "KSTACKP".
 
       -s  replace the TASK column with the KSTACKP column.
 
  On SMP machines, the active task on each CPU will be highlighted by an
  angle bracket (">") preceding its information.
 
  Alternatively, information regarding parent-child relationships, or
  per-task time usage data may be displayed:
 
       -p  display the parental hierarchy of selected, or all, tasks.
       -c  display the children of selected, or all, tasks.
       -t  display the task run time, start time, and cumulative user
           and system times.

EXAMPLES
  Show the process status of all current tasks:

    crash> ps
       PID    PPID  CPU   TASK    ST  %MEM   VSZ   RSS  COMM
    >     0      0   3  c024c000  RU   0.0     0     0  [swapper]
    >     0      0   0  c0dce000  RU   0.0     0     0  [swapper]
          0      0   1  c0fa8000  RU   0.0     0     0  [swapper]
    >     0      0   2  c009a000  RU   0.0     0     0  [swapper]
          1      0   1  c0098000  IN   0.0  1096   476  init
          2      1   1  c0090000  IN   0.0     0     0  [kflushd]
          3      1   1  c000e000  IN   0.0     0     0  [kpiod]
          4      1   3  c000c000  IN   0.0     0     0  [kswapd]
          5      1   1  c0008000  IN   0.0     0     0  [mdrecoveryd]
        253      1   2  fbc4c000  IN   0.0  1088   376  portmap
        268      1   2  fbc82000  IN   0.1  1232   504  ypbind
        274    268   2  fa984000  IN   0.1  1260   556  ypbind
        321      1   1  fabf6000  IN   0.1  1264   608  syslogd
        332      1   1  fa9be000  RU   0.1  1364   736  klogd
        346      1   2  fae88000  IN   0.0  1112   472  atd
        360      1   2  faeb2000  IN   0.1  1284   592  crond
        378      1   2  fafd6000  IN   0.1  1236   560  inetd
        392      1   0  fb710000  IN   0.1  2264  1468  named
        406      1   3  fb768000  IN   0.1  1284   560  lpd
        423      1   1  fb8ac000  IN   0.1  1128   528  rpc.statd
        434      1   2  fb75a000  IN   0.0  1072   376  rpc.rquotad
        445      1   2  fb4a4000  IN   0.0  1132   456  rpc.mountd
        460      1   1  fa938000  IN   0.0     0     0  [nfsd]
        461      1   1  faa86000  IN   0.0     0     0  [nfsd]
        462      1   0  fac48000  IN   0.0     0     0  [nfsd]
        463      1   0  fb4ca000  IN   0.0     0     0  [nfsd]
        464      1   0  fb4c8000  IN   0.0     0     0  [nfsd]
        465      1   2  fba6e000  IN   0.0     0     0  [nfsd]
        466      1   1  fba6c000  IN   0.0     0     0  [nfsd]
        467      1   2  fac04000  IN   0.0     0     0  [nfsd]
        468    461   2  fa93a000  IN   0.0     0     0  [lockd]
        469    468   2  fa93e000  IN   0.0     0     0  [rpciod]
        486      1   0  fab54000  IN   0.1  1596   880  amd
        523      1   2  fa84e000  IN   0.1  1884  1128  sendmail
        538      1   0  fa82c000  IN   0.0  1112   416  gpm
        552      1   3  fa70a000  IN   0.1  2384  1220  httpd
        556    552   3  fa776000  IN   0.1  2572  1352  httpd
        557    552   2  faba4000  IN   0.1  2572  1352  httpd
        558    552   1  fa802000  IN   0.1  2572  1352  httpd
        559    552   3  fa6ee000  IN   0.1  2572  1352  httpd
        560    552   3  fa700000  IN   0.1  2572  1352  httpd
        561    552   0  fa6f0000  IN   0.1  2572  1352  httpd
        562    552   3  fa6ea000  IN   0.1  2572  1352  httpd
        563    552   0  fa67c000  IN   0.1  2572  1352  httpd
        564    552   3  fa674000  IN   0.1  2572  1352  httpd
        565    552   3  fa66a000  IN   0.1  2572  1352  httpd
        582      1   2  fa402000  IN   0.2  2968  1916  xfs
        633      1   2  fa1ec000  IN   0.2  5512  2248  innd
        636      1   3  fa088000  IN   0.1  2536   804  actived
        676      1   0  fa840000  IN   0.0  1060   384  mingetty
        677      1   1  fa590000  IN   0.0  1060   384  mingetty
        678      1   2  fa3b8000  IN   0.0  1060   384  mingetty
        679      1   0  fa5b8000  IN   0.0  1060   384  mingetty
        680      1   1  fa3a4000  IN   0.0  1060   384  mingetty
        681      1   2  fa30a000  IN   0.0  1060   384  mingetty
        683      1   3  fa5d8000  IN   0.0  1052   280  update
        686    378   1  fa3aa000  IN   0.1  2320  1136  in.rlogind
        687    686   2  f9e52000  IN   0.1  2136  1000  login
        688    687   0  f9dec000  IN   0.1  1732   976  bash
    >   700    688   1  f9d62000  RU   0.0  1048   256  gen12
 
  Display the parental hierarchy of the "crash" process on a live system:

    crash> ps -p 4249
    PID: 0      TASK: c0252000  CPU: 0   COMMAND: "swapper"
     PID: 1      TASK: c009a000  CPU: 1   COMMAND: "init"
      PID: 632    TASK: c73b6000  CPU: 1   COMMAND: "prefdm"
       PID: 637    TASK: c5a4a000  CPU: 1   COMMAND: "prefdm"
        PID: 649    TASK: c179a000  CPU: 0   COMMAND: "kwm"
         PID: 683    TASK: c1164000  CPU: 0   COMMAND: "kfm"
          PID: 1186   TASK: c165a000  CPU: 0   COMMAND: "xterm"
           PID: 1188   TASK: c705e000  CPU: 1   COMMAND: "bash"
            PID: 4249   TASK: c6b9a000  CPU: 0   COMMAND: "crash"
 
  Display all children of the "kwm" window manager:

    crash> ps -c kwm
      PID: 649    TASK: c179a000  CPU: 0   COMMAND: "kwm"
      PID: 682    TASK: c2d58000  CPU: 1   COMMAND: "kwmsound"
      PID: 683    TASK: c1164000  CPU: 1   COMMAND: "kfm"
      PID: 685    TASK: c053c000  CPU: 0   COMMAND: "krootwm"
      PID: 686    TASK: c13fa000  CPU: 0   COMMAND: "kpanel"
      PID: 687    TASK: c13f0000  CPU: 1   COMMAND: "kbgndwm"
 
  Show the time usage data for pid 18844:

    crash> ps -t 18844
    PID: 18844  TASK: c3012000  CPU: 0   COMMAND: "bash"
        RUN TIME: 00:31:23
      START TIME: 51390045
       USER TIME: 1
     SYSTEM TIME: 3
 
  Show the process status of PID 1, task f9dec000, and all nfsd tasks:

    crash> ps 1 f9dec000 nfsd
       PID    PPID  CPU   TASK    ST  %MEM   VSZ   RSS  COMM
          1      0   1  c0098000  IN   0.0  1096   476  init
        688    687   0  f9dec000  IN   0.1  1732   976  bash
        460      1   1  fa938000  IN   0.0     0     0  [nfsd]
        461      1   1  faa86000  IN   0.0     0     0  [nfsd]
        462      1   0  fac48000  IN   0.0     0     0  [nfsd]
        463      1   0  fb4ca000  IN   0.0     0     0  [nfsd]
        464      1   0  fb4c8000  IN   0.0     0     0  [nfsd]
        465      1   2  fba6e000  IN   0.0     0     0  [nfsd]
        466      1   1  fba6c000  IN   0.0     0     0  [nfsd]
        467      1   2  fac04000  IN   0.0     0     0  [nfsd]
 
  Show all kernel threads:

    crash> ps -k
       PID    PPID  CPU   TASK    ST  %MEM   VSZ   RSS  COMM
          0      0   1  c0fac000  RU   0.0     0     0  [swapper]
          0      0   0  c0252000  RU   0.0     0     0  [swapper]
          2      1   1  c0fa0000  IN   0.0     0     0  [kflushd]
          3      1   1  c03de000  IN   0.0     0     0  [kpiod]
          4      1   1  c03dc000  IN   0.0     0     0  [kswapd]
          5      1   0  c0092000  IN   0.0     0     0  [mdrecoveryd]
        336      1   0  c4a9a000  IN   0.0     0     0  [rpciod]
        337      1   0  c4830000  IN   0.0     0     0  [lockd]
        487      1   1  c4ba6000  IN   0.0     0     0  [nfsd]
        488      1   0  c18c6000  IN   0.0     0     0  [nfsd]
        489      1   0  c0cac000  IN   0.0     0     0  [nfsd]
        490      1   0  c056a000  IN   0.0     0     0  [nfsd]
        491      1   0  c0860000  IN   0.0     0     0  [nfsd]
        492      1   1  c0254000  IN   0.0     0     0  [nfsd]
        493      1   0  c0a86000  IN   0.0     0     0  [nfsd]
        494      1   0  c0968000  IN   0.0     0     0  [nfsd]
 
  Show the kernel stack pointer of each user task:

    crash> ps -us
       PID    PPID  CPU  KSTACKP  ST  %MEM   VSZ   RSS  COMM
          1      0   0  c009bedc  IN   0.0  1096    52  init
        239      1   0  c15e7ed8  IN   0.2  1332   224  pump
        280      1   1  c7cbdedc  IN   0.2  1092   208  portmap
        295      1   0  c7481edc  IN   0.0  1232     0  ypbind
        301    295   0  c7c7bf28  IN   0.1  1260   124  ypbind
        376      1   1  c5053f28  IN   0.0  1316    40  automount
        381      1   0  c34ddf28  IN   0.2  1316   224  automount
        391      1   1  c2777f28  IN   0.2  1316   224  automount
    ...
Contents
crash-whitepaper-1.0/doc/help_pages/pte.html0000644000175000017500000000572510357525152021414 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  pte


NAME
  pte - translate a page table entry

SYNOPSIS
  pte contents ...

DESCRIPTION
  This command translates the hexadecimal contents of a PTE into its physical
  page address and page bit settings.  If the PTE references a swap location,
  the swap device and offset are displayed.

EXAMPLES

    crash> pte d8e067
     PTE    PHYSICAL  FLAGS
    d8e067   d8e000   (PRESENT|RW|USER|ACCESSED|DIRTY)
 
    crash> pte 13f600
     PTE      SWAP     OFFSET
    13f600  /dev/hda2   5104
Contents
crash-whitepaper-1.0/doc/help_pages/ptob.html0000644000175000017500000000525410357525152021565 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  ptob


NAME
  ptob - page to bytes

SYNOPSIS
  ptob page_number ...

DESCRIPTION
  This command translates a page frame number to its byte value.

EXAMPLES
    crash> ptob 512a
    512a: 512a000
Contents
crash-whitepaper-1.0/doc/help_pages/ptov.html0000644000175000017500000000545010357525152021607 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  ptov


NAME
  ptov - physical to virtual

SYNOPSIS
  ptov address ...

DESCRIPTION
  This command translates a hexadecimal physical address into a kernel 
  virtual address.

EXAMPLES
  Translate physical address 56e000 into a kernel virtual address:

    crash> ptov 56e000
    VIRTUAL   PHYSICAL
    c056e000  56e000
Contents
crash-whitepaper-1.0/doc/help_pages/q.html0000644000175000017500000000520710357525152021057 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  q


NAME
  q - exit this session

SYNOPSIS
  q  

DESCRIPTION
  Bail out of the current crash session.

NOTE
  This command is equivalent to the "exit" command.
Contents
crash-whitepaper-1.0/doc/help_pages/rd.html0000644000175000017500000001600410357525152021221 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  rd


NAME
  rd - read memory

SYNOPSIS
  rd [-dDsup][-8|-16|-32|-64][-o offs][-e addr] [address|symbol] [count]

DESCRIPTION
  This command displays the contents of memory, with the output formatted
  in several different manners.  The starting address may be entered either
  symbolically or by address.  The default output size is the size of a long
  data type, and the default output format is hexadecimal.  When hexadecimal
  output is used, the output will be accompanied by an ASCII translation.

       -p  address argument is a physical address.
       -u  address argument is a user virtual address; only required on
           processors with common user and kernel virtual address spaces.
       -d  display output in signed decimal format (default is hexadecimal).
       -D  display output in unsigned decimal format (default is hexadecimal).
       -s  displays output symbolically when appropriate.
       -8  display output in 8-bit values.
      -16  display output in 16-bit values.
      -32  display output in 32-bit values (default on 32-bit machines).
      -64  display output in 64-bit values (default on 64-bit machines).
  -o offs  offset the starting address by offs.
  -e addr  display memory until reaching specified ending hexadecimal address.
  address  starting hexadecimal address:
             1  the default presumes a kernel virtual address.
             2. -p specifies a physical address.
             3. -u specifies a user virtual address, but is only necessary on
              processors with common user and kernel virtual address spaces.
   symbol  symbol of starting address to read.
    count  number of memory locations to display (default is 1).

EXAMPLES
  Display the kernel_version string:

    crash> rd kernel_version 4 
    c0226a6c:  2e322e32 35312d35 00000000 00000001   2.2.5-15........

  Display the same block of memory, with and without symbols:

    crash> rd c1157f00 52   
    c1157f00:  c0131f7a 00000400 00000015 c013206e   z...........n ..
    c1157f10:  00000100 c3d4c140 00000100 00000246   ....@.......F...
    c1157f20:  019b2065 c2a5bb90 080ac618 c02a83d0   e ............*.
    c1157f30:  40000025 01a45067 c1156000 00000000   %..@gP...`......
    c1157f40:  c011b4f7 c1156000 c2a5bb90 080ac618   .....`..........
    c1157f50:  00000001 00000000 c1a45000 c19b2000   .........P... ..
    c1157f60:  c1157f84 0000003b c022c000 c1156000   ....;....."..`..
    c1157f70:  00000000 fffffe00 bffff6fc 0000002e   ................
    c1157f80:  c022c000 ffffffff c01178ba c1156000   .."......x...`..
    c1157f90:  00000000 080ac618 bffff6ac 00000001   ................
    c1157fa0:  c1156000 c1156000 c1157fb8 c1156000   .`...`.......`..
    c1157fb0:  c1157fb8 c1156000 c1156000 c115608c   .....`...`...`..
    c1157fc0:  c01096c8 ffffffff bffff6fc 00000002   ................

    crash> rd -s c1157f00 52
    c1157f00:  alloc_fd_array+0x1a 00000400 00000015 expand_fd_array+0x72 
    c1157f10:  00000100 c3d4c140 00000100 00000246 
    c1157f20:  019b2065 c2a5bb90 080ac618 c02a83d0 
    c1157f30:  40000025 01a45067 c1156000 00000000 
    c1157f40:  do_wp_page+0x17f c1156000 c2a5bb90 080ac618 
    c1157f50:  00000001 00000000 c1a45000 c19b2000 
    c1157f60:  c1157f84 0000003b init_task_union c1156000 
    c1157f70:  00000000 fffffe00 bffff6fc 0000002e 
    c1157f80:  init_task_union ffffffff sys_wait4+0x2be c1156000 
    c1157f90:  00000000 080ac618 bffff6ac 00000001 
    c1157fa0:  c1156000 c1156000 c1157fb8 c1156000 
    c1157fb0:  c1157fb8 c1156000 c1156000 c115608c 
    c1157fc0:  system_call+0x34 ffffffff bffff6fc 00000002

  Read jiffies in hexadecimal and decimal format:

    crash> rd jiffies
    c0213ae0:  0008cc3a                              :...

    crash> rd -d jiffies
    c0213ae0:        577376

  Access the same memory in different sizes:

    crash> rd -64 kernel_version
    c0226a6c:  35312d352e322e32                    2.2.5-15

    crash> rd -32 kernel_version 2
    c0226a6c:  2e322e32 35312d35                     2.2.5-15

    crash> rd -16 kernel_version 4
    c0226a6c:  2e32 2e32 2d35 3531                       2.2.5-15

    crash> rd -8 kernel_version 8 
    c0226a6c:  32 2e 32 2e 35 2d 31 35                           2.2.5-15

  Read the range of memory from c009bf2c to c009bf60:

    crash> rd c009bf2c -e c009bf60
    c009bf2c:  c009bf64 c01328c3 c009bf64 c0132838   d....(..d...8(..
    c009bf3c:  0000002a 00000004 c57d77e8 00000104   *........w}.....
    c009bf4c:  0000000b c009a000 7fffffff 00000000   ................
    c009bf5c:  00000000                              ....
Contents
crash-whitepaper-1.0/doc/help_pages/repeat.html0000644000175000017500000000725610357525152022105 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  repeat


NAME
  repeat - repeat a command

SYNOPSIS
  repeat [-seconds] command

DESCRIPTION
  This command repeats a command indefinitely, optionally delaying a given
  number of seconds between each command execution.

    -seconds   The number of seconds to delay between command executions.
               This option must precede the command name to be executed.
 
  Command execution may be stopped with CTRL-C, or if scrolling is in effect,
  by entering "q".  This command is meant for use on a live system; it is
  hard to conceive of a reason to use it when debugging a crash dump.

EXAMPLES
  Display the value of jiffies once per second:

    crash> repeat -1 p jiffies
    jiffies = $1 = 155551079
    jiffies = $2 = 155551180
    jiffies = $3 = 155551281
    jiffies = $4 = 155551382
    jiffies = $5 = 155551483
    jiffies = $6 = 155551584
    jiffies = $7 = 155551685
    jiffies = $8 = 155551786
    jiffies = $9 = 155551887
    jiffies = $10 = 155551988
    jiffies = $11 = 155552089
    jiffies = $12 = 155552190
    jiffies = $13 = 155552291
    jiffies = $14 = 155552392
    jiffies = $15 = 155552493
    jiffies = $16 = 155552594
    jiffies = $17 = 155552695
    jiffies = $18 = 155552796
    ...
Contents
crash-whitepaper-1.0/doc/help_pages/runq.html0000644000175000017500000000620410357525152021602 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  runq


NAME
  runq - run queue

SYNOPSIS
  runq  

DESCRIPTION
  This command displays the tasks on the run queue.

EXAMPLES
    crash> runq
    PID: 435    TASK: c322c000  CPU: 1   COMMAND: "httpd"
    PID: 253    TASK: c6580000  CPU: 0   COMMAND: "portmap"
    PID: 5655   TASK: c0a86000  CPU: 1   COMMAND: "nfsd"
    PID: 3      TASK: c0fa0000  CPU: 1   COMMAND: "kflushd"
    PID: 318    TASK: c6734000  CPU: 1   COMMAND: "syslogd"
    PID: 5657   TASK: c129e000  CPU: 1   COMMAND: "nfsd"
    PID: 5653   TASK: c0d26000  CPU: 1   COMMAND: "nfsd"
    PID: 329    TASK: c4138000  CPU: 1   COMMAND: "klogd"
    PID: 2      TASK: c03dc000  CPU: 0   COMMAND: "kswapd"
Contents
crash-whitepaper-1.0/doc/help_pages/search.html0000644000175000017500000001333010357525152022060 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  search


NAME
  search - search memory

SYNOPSIS
  search [-s start | -k | -u] [-e end | -l length] [-m mask] value ...

DESCRIPTION
  This command searches a range of user or kernel memory space for given value.
  If no end nor length value is entered, the search stops at the end of
  user or kernel address space, whichever is appropriate.
 
  An optional mask value may be entered to mask off "don't care" bits.

   -s start  Start the search at this hexadecimal user or kernel virtual 
             address, or kernel symbol.
         -k  If no start address is specified, start the search at the base
             of kernel address space.  If a start address is specified, then
             this option specifies that the start address is a kernel virtual
             address. (only required on processors with overlapping
             kernel/user virtual address spaces)
         -u  If no start address is specified, start the search at the base
             of the current context's user address space.  If a start address
             is specified, then this option specifies that the start address
             is a user virtual address. (only required on processors with 
             overlapping kernel/user virtual address spaces)
     -e end  Stop the search at this hexadecimal user or kernel virtual 
             address, or kernel symbol.
  -l length  Length in bytes of address range to search.
    -m mask  Ignore the bits that are set in the hexadecimal mask value.
      value  Search for this hexadecimal value.
 
  If the -s start option is not used, then either -u or -k must be used in
  order to determine whether to search user or kernel memory.  The starting 
  address must be long-word aligned.  Address ranges that start in user space
  and end in kernel space are not accepted.

EXAMPLES
  Search the current context's address space for all instances of 0xdeadbeef:

    crash> search -u deadbeef
    bffff81c: deadbeef

  Search all kernel memory above the kernel text space for all instances
  of 0xabcd occuring in the lower 16-bits of each 32-bit word: 

    crash> search -s _etext -m ffff0000 abcd
    c071481c: abcd
    c0c2b0fc: 804abcd
    c0cf5e74: 7489abcd
    c17c0b44: c012abcd
    c1dac730: 3dbeabcd
    c226d0e8: ffffabcd
    c23ed5dc: abcd
    c3022544: 3dbeabcd
    c3069b58: 3dbeabcd
    c3e86e84: aabcd
    c3e88ed0: aabcd
    c3e8ee5c: aabcd
    c3e9df50: aabcd
    c3e9e930: aabcd
    c440a778: 804abcd
    c486eb44: 3dbeabcd
    c578f0fc: 804abcd
    c6394f90: 8ababcd
    c65219f0: 3abcd
    c661399c: abcd
    c68514ac: 8abcd
    c7e036bc: 3dbeabcd
    c7e12568: 5abcd
    c7e1256c: 5abcd

  Search the user address space of the current context for all instances
  of 0xdeadbeef:

    crash> search -u deadbeef
    81aba5c: deadbeef
    81abaa8: deadbeef
    bfffc698: deadbeef
    bffff390: deadbeef

  Search the 4K page at c532c000 for all instances of 0xffffffff:

    crash> search -s c532c000 -l 4096 ffffffff
    c532c33c: ffffffff
    c532c3fc: ffffffff

  Search the static kernel data area for all instances of c2d400eb:

    crash> search -s _etext -e _edata c2d400eb
    c022b550: c2d400eb
    c022b590: c2d400eb
    c022b670: c2d400eb
    c022b6e0: c2d400eb
    c022b7b0: c2d400eb
    c022b7e0: c2d400eb
    c022b8b0: c2d400eb
Contents
crash-whitepaper-1.0/doc/help_pages/set.html0000644000175000017500000001411010357525152021403 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  set


NAME
  set - set a process context or internal crash variable

SYNOPSIS
  set [pid | taskp | [-c cpu] | -p] | [crash_variable [setting]] | -v

DESCRIPTION
  This command either sets a new context, or gets the current context for
  display.  The context can be set by the use of:

      pid  a process PID.
    taskp  a hexadecimal task_struct pointer.
   -c cpu  sets the context to the active task on a cpu (dumpfiles only).
       -p  sets the context to the panic task, or back to the crash task on
           a live system.
       -v  display the current state of internal crash variables.

  If no argument is entered, the current context is displayed.  The context
  consists of the PID, the task pointer, the CPU, and task state.
  
  This command may also be used to set internal crash variables.  If no value
  argument is entered, the current value of the crash variable is shown.  These
  are the crash variables, acceptable arguments, and purpose:

          scroll  on | off     controls output scrolling.
           radix  10 | 16      sets output radix to 10 or 16.
         refresh  on | off     controls internal task list refresh.
       print_max  number       set maximum number of array elements to print.
         console  device-name  sets debug console device.
           debug  number       sets crash debug level.
            core  on | off     if on, drops core when the next error message
                               is displayed.
            hash  on | off     controls internal list verification.
          silent  on | off     turns off initialization messages; turns off
                               crash prompt during input file execution. 
                               (scrolling is turned off if silent is on)
            edit  vi | emacs   set line editing mode (from .crashrc file only).
        namelist  filename     name of kernel (from .crashrc file only).
        dumpfile  filename     name of core dumpfile (from .crashrc file only).
 
  Internal variables may be set in four manners:

    1. entering the set command in $HOME/.crashrc.
    2. entering the set command in .crashrc in the current directory.
    3. executing an input file containing the set command.
    4. during runtime with this command.

  During initialization, $HOME/.crashrc is read first, followed by the
  .crashrc file in the current directory.  Set commands in the .crashrc file
  in the current directory override those in $HOME/.crashrc.  Set commands 
  entered with this command or by runtime input file override those
  defined in either .crashrc file.  Multiple set command arguments or argument
  pairs may be entered in one command line.

EXAMPLES
  Set the current context to task c2fe8000:

    crash> set c2fe8000
         PID: 15917
     COMMAND: "bash"
        TASK: c2fe8000  
         CPU: 0
       STATE: TASK_INTERRUPTIBLE

  Set the context back to the panicking task:

    crash> set -p
         PID: 698
     COMMAND: "gen12"
        TASK: f9d78000
         CPU: 2
       STATE: TASK_RUNNING (PANIC)

  Turn off output scrolling:

    crash> set scroll off
    scroll: off
 
  Show the current state of crash internal variables:

    crash> set -v
            scroll: on
             radix: 10 (decimal)
           refresh: on
         print_max: 256
           console: /dev/pts/2
             debug: 0
              core: off
              hash: on
            silent: off
              edit: vi
          namelist: vmlinux
          dumpfile: vmcore
 
  Show the current context:

    crash> set
         PID: 1525
     COMMAND: "bash"
        TASK: c1ede000
         CPU: 0
       STATE: TASK_INTERRUPTIBLE
Contents
crash-whitepaper-1.0/doc/help_pages/sig.html0000644000175000017500000001506610357525152021405 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  sig


NAME
  sig - task signal handling

SYNOPSIS
  sig [[-l] | [-s sigset]] | [pid | taskp] ...

DESCRIPTION
  This command displays signal-handling data of one or more tasks.  Multiple
  task or PID numbers may be entered; if no arguments are entered, the signal
  handling data of the current context will be displayed.  The default display
  shows:
 
    1.  Whether the task has an unblocked signal pending.
    2.  The contents of the "signal" and "blocked" sigset_t structures
        from the task_struct, both of which are represented as a 64-bit 
        hexadecimal value.
    3.  A formatted dump of the "sig" signal_struct structure referenced by
        the task_struct.  For each defined signal, it shows the sigaction
        structure address, the signal handler, the signal sigset_t mask 
        (also expressed as a 64-bit hexadecimal value), and the flags.
    4.  For each queued signal, if any, its signal number and associated
        siginfo structure address.
 
  The -l option lists the signal numbers and their name(s).  The -s option
  translates a 64-bit hexadecimal value representing the contents of a
  sigset_t structure into the signal names whose bits are set.
 
        pid  a process PID.
      taskp  a hexadecimal task_struct pointer.
         -l  displays the defined signal numbers and names.
  -s sigset  translates a 64-bit hexadecimal value representing a sigset_t
             into a list of signal names associated with the bits set.

EXAMPLES
  Dump the signal-handling data of PID 614:

    crash> sig 614
    PID: 614    TASK: c6f26000  CPU: 1   COMMAND: "httpd"
    SIGPENDING: no
        SIGNAL: 0000000000000000
       BLOCKED: 0000000000000000
    SIGNAL_STRUCT: c1913800  COUNT: 1
     SIG SIGACTION  HANDLER       MASK       FLAGS   
     [1]  c1913804  8057c98 0000000000000201 0 
     [2]  c1913818  8057c8c 0000000000000000 0 
     [3]  c191382c  SIG_DFL 0000000000000000 0 
     [4]  c1913840  8057bd8 0000000000000000 80000000 (SA_RESETHAND)
     [5]  c1913854  SIG_DFL 0000000000000000 0 
     [6]  c1913868  8057bd8 0000000000000000 80000000 (SA_RESETHAND)
     [7]  c191387c  8057bd8 0000000000000000 80000000 (SA_RESETHAND)
     [8]  c1913890  SIG_DFL 0000000000000000 0 
     [9]  c19138a4  SIG_DFL 0000000000000000 0 
    [10]  c19138b8  8057c98 0000000000000201 0 
    [11]  c19138cc  8057bd8 0000000000000000 80000000 (SA_RESETHAND)
    [12]  c19138e0  SIG_DFL 0000000000000000 0 
    [13]  c19138f4  SIG_IGN 0000000000000000 0 
    [14]  c1913908  SIG_DFL 0000000000000000 0 
    [15]  c191391c  8057c8c 0000000000000000 0 
    [16]  c1913930  SIG_DFL 0000000000000000 0 
    [17]  c1913944  SIG_DFL 0000000000000000 0 
    [18]  c1913958  SIG_DFL 0000000000000000 0 
    [19]  c191396c  SIG_DFL 0000000000000000 0 
    [20]  c1913980  SIG_DFL 0000000000000000 0 
    [21]  c1913994  SIG_DFL 0000000000000000 0 
    [22]  c19139a8  SIG_DFL 0000000000000000 0 
    [23]  c19139bc  SIG_DFL 0000000000000000 0 
    [24]  c19139d0  SIG_DFL 0000000000000000 0 
    [25]  c19139e4  SIG_DFL 0000000000000000 0 
    [26]  c19139f8  SIG_DFL 0000000000000000 0 
    [27]  c1913a0c  SIG_DFL 0000000000000000 0 
    [28]  c1913a20  SIG_DFL 0000000000000000 0 
    [29]  c1913a34  SIG_DFL 0000000000000000 0 
    [30]  c1913a48  SIG_DFL 0000000000000000 0 
    [31]  c1913a5c  SIG_DFL 0000000000000000 0 
    SIGQUEUE: (empty)
    
  Translate the sigset_t mask value, cut-and-pasted from the signal handling
  data from signals 1 and 10 above:
 
    crash> sig -s 0000000000000201
    SIGHUP SIGUSR1
 
  List the signal numbers and their names:
 
    crash> sig -l
     [1] SIGHUP
     [2] SIGINT
     [3] SIGQUIT
     [4] SIGILL
     [5] SIGTRAP
     [6] SIGABRT/SIGIOT
     [7] SIGBUS
     [8] SIGFPE
     [9] SIGKILL
    [10] SIGUSR1
    [11] SIGSEGV
    [12] SIGUSR2
    [13] SIGPIPE
    [14] SIGALRM
    [15] SIGTERM
    [16] SIGSTKFLT
    [17] SIGCHLD/SIGCLD
    [18] SIGCONT
    [19] SIGSTOP
    [20] SIGTSTP
    [21] SIGTTIN
    [22] SIGTTOU
    [23] SIGURG
    [24] SIGXCPU
    [25] SIGXFSZ
    [26] SIGVTALRM
    [27] SIGPROF
    [28] SIGWINCH
    [29] SIGIO/SIGPOLL
    [30] SIGPWR
    [31] SIGSYS
Contents
crash-whitepaper-1.0/doc/help_pages/struct.html0000644000175000017500000002327010357525152022143 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  struct


NAME
  struct - structure contents

SYNOPSIS
  struct struct_name[.member] [[-o][-l offset][-r] [address | symbol] [count]]

DESCRIPTION
  This command displays either a structure definition, or a formatted display
  of the contents of a structure at a specified address.  When no address is
  specified, the structure definition is shown along with the structure size.
  A structure member may be appended to the structure name in order to limit
  the scope of the data displayed to that particular member; when no address
  is specified, the member's offset and definition are shown.

    struct_name  name of a C-code structure used by the kernel.
        .member  name of a structure member.
             -o  show member offsets when displaying structure definitions.
      -l offset  if the address argument is a pointer to a list_head structure
                 that is embedded in the target data structure, the offset
                 to the list_head member may be entered in either of the
                 following manners:
                   1. in "structure.member" format.
                   2. a number of bytes. 
             -r  raw dump of structure data.
        address  hexadecimal address of a structure; if the address points
                 to an embedded list_head structure contained within the
                 target data structure, then the "=l" option must be used.
         symbol  symbolic reference to the address of a structure.
          count  count of structures to dump from an array of structures;
                 if used, this must be the last argument entered.

  Structure data, sizes, and member offsets are shown in the current output
  radix.
 
  Please note that in the vast majority of cases, the "struct" command
  name may be dropped; if the structure name does not conflict with any crash
  or gdb command name, then the "struct_name[.member]" argument will be
  recognized as a structure name, and this command automatically executed.
  See the NOTE below.

EXAMPLES
  Display the vm_area_struct at address c1e44f10:

    crash> struct vm_area_struct c1e44f10
    struct vm_area_struct {
      vm_mm = 0xc2857750,
      vm_start = 0x8048000, 
      vm_end = 0x80a5000, 
      vm_next = 0xc1e44a10,
      vm_page_prot = {
        pgprot = 0x25      
      },
      vm_flags = 0x1875,
      vm_avl_height = 0x2,   
      vm_avl_left = 0xc30fe200,
      vm_avl_right = 0xc30fed00,
      vm_next_share = 0x0,       
      vm_pprev_share = 0xc1e44a30,
      vm_ops = 0xc0215ca0,
      vm_offset = 0x0,       
      vm_file = 0xc0bfdc70,
      vm_pte = 0   
    }

  Display the definition and size of a vm_area_struct structure.  This first
  example below displays just the structure and size.  The second example
  uses the -o option to also display member offsets.  Both examples were
  run with the output radix set to 10 (decimal):

    crash> struct vm_area_struct
    struct vm_area_struct {
        struct mm_struct *vm_mm;
        long unsigned int vm_start;
        long unsigned int vm_end;
        struct vm_area_struct *vm_next;
        pgprot_t vm_page_prot;
        short unsigned int vm_flags;
        short int vm_avl_height;
        struct vm_area_struct *vm_avl_left;
        struct vm_area_struct *vm_avl_right;
        struct vm_area_struct *vm_next_share;
        struct vm_area_struct **vm_pprev_share;
        struct vm_operations_struct *vm_ops;
        long unsigned int vm_offset;
        struct file *vm_file;
        long unsigned int vm_pte;
    }
    SIZE: 56

    crash> struct vm_area_struct -o
    struct vm_area_struct {
       [0] struct mm_struct *vm_mm;
       [4] long unsigned int vm_start;
       [8] long unsigned int vm_end;
      [12] struct vm_area_struct *vm_next;
      [16] pgprot_t vm_page_prot;
      [20] short unsigned int vm_flags;
      [22] short int vm_avl_height;
      [24] struct vm_area_struct *vm_avl_left;
      [28] struct vm_area_struct *vm_avl_right;
      [32] struct vm_area_struct *vm_next_share;
      [36] struct vm_area_struct **vm_pprev_share;
      [40] struct vm_operations_struct *vm_ops;
      [44] long unsigned int vm_offset;
      [48] struct file *vm_file;
      [52] long unsigned int vm_pte;
    }
    SIZE: 56

  Display the pgd member of the mm_struct at address c2857bd0:

    crash> struct mm_struct.pgd c2857bd0
      pgd = 0xc168c000,

  Display the definition and offset of the pgd member of an mm_struct:

    crash> struct mm_struct.pgd
    struct mm_struct {
       [12] pgd_t *pgd;
    }

  Display the array of tcp_sl_timer structures declared by tcp_slt_array[]:

    crash> struct tcp_sl_timer tcp_slt_array 4
    struct tcp_sl_timer {
      count = {
        counter = 0x0       
      },
      period = 0x32,      
      last = 0x1419e4,  
      handler = 0xc0164854  <tcp_syn_recv_timer>
    }
    struct tcp_sl_timer {
      count = {
        counter = 0x2       
      },
      period = 0x753,     
      last = 0x14a6df,  
      handler = 0xc01645b0  <tcp_keepalive>
    }
    struct tcp_sl_timer {
      count = {
        counter = 0x0       
      },
      period = 0x2ee,     
      last = 0x143134,  
      handler = 0xc016447c  <tcp_twkill>
    }
    struct tcp_sl_timer {
      count = {
        counter = 0x0       
      },
      period = 0x64,      
      last = 0x143198,  
      handler = 0xc0164404  <tcp_bucketgc>
    }
 
  Without using the "struct" command name, display the the "d_child" 
  list_head member from a dentry structure:

    crash> dentry.d_child 0xe813cb4
      d_child = {
        next = 0x3661344,
        prev = 0xdea4bc4
      },
 
  Display the child dentry structure referenced by the "next" pointer above.
  Since the "next" address of 0x3661344 above is a pointer to an embedded
  list_head structure within the child dentry structure, the -l option
  is required:

    crash> dentry -l dentry.d_child 0x3661344
    struct dentry {
      d_count = {
        counter = 1
      }, 
      d_flags = 0, 
      d_inode = 0xf9aa604, 
      d_parent = 0x11152b1c, 
      d_hash = {
        next = 0x11fb3fc0, 
        prev = 0x11fb3fc0
      }, 
      d_lru = {
        next = 0x366133c, 
        prev = 0x366133c
      }, 
      d_child = {
        next = 0x36613cc, 
        prev = 0xe813cd4
      }, 
      d_subdirs = {
        next = 0x366134c, 
        prev = 0x366134c
      }, 
      d_alias = {
        next = 0xf9aa614, 
        prev = 0xf9aa614
      }, 
      d_mounted = 0, 
      d_name = {
        name = 0x3661384 "boot.log", 
        len = 8, 
        hash = 1935169207
      }, 
      d_time = 1515870810, 
      d_op = 0x0, 
      d_sb = 0x11fc9c00, 
      d_vfs_flags = 0, 
      d_fsdata = 0x0, 
      d_extra_attributes = 0x0, 
      d_iname = "boot.log\000"
    }

NOTE
  If the structure name does not conflict with any crash command name, the
  "struct" command may be dropped.  Accordingly, the examples above could
  also have been accomplished like so:

    crash> vm_area_struct c1e44f10
    crash> vm_area_struct
    crash> vm_area_struct -o
    crash> mm_struct.pgd c2857bd0
    crash> mm_struct.pgd
    crash> tcp_sl_timer tcp_slt_array 4

  Lastly, the short-cut "*" pointer-to command may also be used to negate
  the need to enter the "struct" command name (enter "help *" for details).
Contents
crash-whitepaper-1.0/doc/help_pages/swap.html0000644000175000017500000000543210357525152021571 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  swap


NAME
  swap - swap device information

SYNOPSIS
  swap  

DESCRIPTION
  This command displays information for each configured swap device.

EXAMPLES
    crash> swap
    FILENAME           TYPE         SIZE       USED   PCT  PRIORITY
    /dev/sda8        PARTITION     136516k    47896k  35%     -1
Contents
crash-whitepaper-1.0/doc/help_pages/sym.html0000644000175000017500000001561210357525152021430 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  sym


NAME
  sym - translate a symbol to its virtual address, or vice-versa 

SYNOPSIS
  sym [-l] | [-M] | [-m module] | [-p|-n] | [-q string] | [symbol | vaddr]

DESCRIPTION
  This command translates a symbol to its virtual address, or a static 
  kernel virtual address to its symbol -- or to a symbol-plus-offset value,
  if appropriate.  Additionally, the symbol type is shown in parentheses, 
  and if the symbol is a known text value, the file and line number are shown.

              -l  dumps all symbols and their values.
              -M  dumps the current set of module symbols.
       -m module  dumps the current set of symbols for a specified module.
              -p  display the target symbol and the previous symbol.
              -n  display the target symbol and the next symbol.
       -q string  searches for all symbols containing "string".
          symbol  a kernel text or data symbol.
           vaddr  a kernel virtual address.

EXAMPLES
  Translate data symbol jiffies to its value, and vice-versa:

    crash> sym jiffies
    c0213ae0 (D) jiffies

    crash> sym c0213ae0
    c0213ae0 (D) jiffies

  Translate a text address to its symbolic value and source file:

    crash> sym c0109944
    c0109944 (T) system_call+0x34  ../linux-2.2.5/arch/i386/kernel/signal.c: 723

  Dump the whole symbol table:

    crash> sym -l
    c0100000 (T) _stext
    c0100000 (A) _text
    c0100000 (t) startup_32
    c0100000 (T) stext
    c01000a4 (t) checkCPUtype
    c0100139 (t) is486
    c0100148 (t) is386
    c01001b1 (t) L6
    c01001b3 (t) ready
    c01001b4 (t) check_x87
    c01001da (t) setup_idt
    c01001f7 (t) rp_sidt
    c0100204 (T) stack_start
    c010020c (t) int_msg
    c0100220 (t) ignore_int
    c0100242 (t) idt_descr
    c0100244 (T) idt
    c010024a (t) gdt_descr
    c010024c (T) gdt
    c0101000 (T) swapper_pg_dir
    c0102000 (T) pg0
    c0103000 (T) empty_bad_page
    c0104000 (T) empty_bad_page_table
    c0105000 (T) empty_zero_page
    ...

  Find all symbols containing the string "pipe":

    crash> sym -q pipe
    c010ec60 (T) sys_pipe  
    c012f660 (t) pipe_read  
    c012f7b8 (t) pipe_write  
    c012f9c0 (t) pipe_lseek  
    c012f9d0 (t) bad_pipe_r  
    c012f9dc (t) bad_pipe_w  
    c012f9e8 (t) pipe_ioctl  
    c012fa18 (t) pipe_poll  
    c012fb00 (t) pipe_release  
    c012fb48 (t) pipe_read_release  
    c012fb5c (t) pipe_write_release  
    c012fb70 (t) pipe_rdwr_release  
    c012fba0 (t) pipe_read_open  
    c012fbb0 (t) pipe_write_open  
    c012fbc0 (t) pipe_rdwr_open  
    c012fbec (t) get_pipe_inode  
    c012fcc4 (T) do_pipe  
    c023a920 (D) read_pipe_fops  
    c023a960 (D) write_pipe_fops  
    c023a9a0 (D) rdwr_pipe_fops  
    c023a9e0 (D) pipe_inode_operations  

  Dump the symbols of the uart401 module, both before, and then after,
  the complete set of symbols are loaded with the "mod -s" command:

    crash> sym -m uart401
    c8032000 MODULE START: uart401
    c8032138 (?) uart401intr  
    c803235c (?) attach_uart401  
    c8032638 (?) probe_uart401  
    c80326d4 (?) unload_uart401  
    c8033770 MODULE END: uart401
    crash> mod -s uart401
     MODULE   NAME         SIZE  OBJECT FILE
    c8032000  uart401      6000  /lib/modules/2.2.14/misc/uart401.o
    crash> sym -m uart401
    c8032000 MODULE START: uart401
    c8032050 (t) my_notifier_call  
    c8032084 (t) uart401_status  
    c8032098 (t) uart401_cmd  
    c80320a8 (t) uart401_read  
    c80320bc (t) uart401_write  
    c80320cc (t) uart401_input_loop  
    c8032138 (T) uart401intr  
    c8032168 (t) uart401_open  
    c80321c8 (t) uart401_close  
    c80321f4 (t) uart401_out  
    c80322ac (t) uart401_start_read  
    c80322b4 (t) uart401_end_read  
    c80322bc (t) uart401_kick  
    c80322c4 (t) uart401_buffer_status  
    c80322cc (t) enter_uart_mode  
    c803235c (T) attach_uart401  
    c803259c (t) reset_uart401  
    c8032638 (T) probe_uart401  
    c80326d4 (T) unload_uart401  
    c8032760 (T) init_module
    c80327cc (T) cleanup_module  
    c8032b00 (d) sound_notifier  
    c8032b0c (d) detected_devc  
    c8032b20 (d) std_synth_info  
    c8032bc0 (d) std_midi_synth  
    c8033600 (d) uart401_operations  
    c80336c4 (D) io  
    c80336c8 (D) irq  
    c80336e0 (b) hw_info.508  
    c8033770 MODULE END: uart401

  Display the value of jiffies, along with the next and previous symbols:

    crash> sym -np jiffies
    c023027c (D) prof_shift
    c0230280 (D) jiffies 
    c02302a0 (D) task
Contents
crash-whitepaper-1.0/doc/help_pages/sys.html0000644000175000017500000001204410357525152021432 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  sys


NAME
  sys - system data

SYNOPSIS
  sys [-c [name|number]] 

DESCRIPTION
  This command displays system-specific data.  If no arguments are entered,
  the same system data shown during crash invocation is shown.

    -c [name|number]  If no name or number argument is entered, dump all
                      sys_call_table entries.  If a name string is entered,
                      search the table for all entries containing the string.
                      If a number is entered, the table entry associated with
                      that number is displayed.  If the current output radix
                      has been set to 16, the system call numbers will be 
                      displayed in hexadecimal.
    -panic            Panic a live system.  Requires write permission to
                      /dev/mem.  Results in the crash context causing an
                      "Attempted to kill the idle task!" panic.  (The dump
                      will indicate that the crash context has a PID of 0).

EXAMPLES
  Display essential system information:

    crash> sys
          KERNEL: vmlinux.4
        DUMPFILE: lcore.cr.4
            CPUS: 4
            DATE: Mon Oct 11 18:48:55 1999
          UPTIME: 10 days, 14:14:39
    LOAD AVERAGE: 0.74, 0.23, 0.08
           TASKS: 77
        NODENAME: test.mclinux.com
         RELEASE: 2.2.5-15smp
         VERSION: #24 SMP Mon Oct 11 17:41:40 CDT 1999
         MACHINE: i686  (500 MHz)
          MEMORY: 1 GB

  Dump the system call table:

    crash> sys -c
    NUM  SYSTEM CALL                FILE AND LINE NUMBER
      0  sys_ni_syscall             ../kernel/sys.c: 48
      1  sys_exit                   ../kernel/exit.c: 404
      2  sys_fork                   ../arch/i386/kernel/process.c: 771
      3  sys_read                   ../fs/read_write.c: 117
      4  sys_write                  ../fs/read_write.c: 146
      5  sys_open                   ../fs/open.c: 754
      6  sys_close                  ../fs/open.c: 839
      7  sys_waitpid                ../kernel/exit.c: 503
      8  sys_creat                  ../fs/open.c: 789
      9  sys_link                   ../fs/namei.c: 1213
     10  sys_unlink                 ../fs/namei.c: 1074
     11  sys_execve                 ../arch/i386/kernel/process.c: 806
    ...

  Find the system call number of the select system call:

    crash> sys -c select
    NUM  SYSTEM CALL                FILE AND LINE NUMBER
     65  sys_select                 ../fs/select.c: 259
 
    If the current output radix has been set to 16, the system call numbers
    will be displayed in hexadecimal.
Contents
crash-whitepaper-1.0/doc/help_pages/task.html0000644000175000017500000001110010357525152021546 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  task


NAME
  task - task_struct contents

SYNOPSIS
  task [-R member[,member]] [pid | taskp] ...

DESCRIPTION
  This command dumps a formatted display of the contents of a task_struct.
  Multiple task or PID numbers may be entered; if no arguments are entered,
  the task_struct of the current context is displayed.  The -R option,
  typically invoked indirectly via "foreach task", pares the output down
  to one or more structure members.
 
        pid  a process PID.
      taskp  a hexadecimal task_struct pointer.
  -R member  a comma-separated list of one or more task_struct members.

EXAMPLES
  Dump the task_struct structure of the current context:

    crash> task
    PID: 18138  TASK: c7d12000  CPU: 1   COMMAND: "crash"
    struct task_struct {
      state = 0, 
      flags = 0, 
      sigpending = 0, 
      addr_limit = {
        seg = 3221225472
      }, 
      exec_domain = 0xc02386c0, 
      need_resched = 0, 
      counter = 2, 
      priority = 20, 
      avg_slice = 0, 
      has_cpu = 1, 
      processor = 1, 
      last_processor = 0, 
      lock_depth = 0, 
      next_task = 0xc7808000, 
      prev_task = 0xc6d1c000, 
      next_run = 0xc0252000, 
      prev_run = 0xc0252000, 
      binfmt = 0xc023abd0, 
      exit_code = 0, 
      exit_signal = 17, 
         .
         .
         .
      files = 0xc09a7d60, 
      mm = 0xc753fb50, 
      sigmask_lock = {
        lock = 0
      }, 
      sig = 0xc4745800, 
      signal = {
        sig = {65536, 0}
      }, 
      blocked = {
        sig = {65536, 0}
      }, 
      sigqueue = 0x0, 
      sigqueue_tail = 0xc7d124ac, 
      sas_ss_sp = 0, 
      sas_ss_size = 0
    }
 
  Display the ngroups and groups task_struct members for PID 2958:

    crash> task -R ngroups,groups 2958
    PID: 2958   TASK: c6718000  CPU: 0   COMMAND: "bash"
      ngroups = 6, 
      groups = {504, 8, 9, 1000, 1007, 1006, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
 
  NOTE: When this command is invoked directly (i.e., not from "foreach"), it
  is not necessary to include the "-R" before the task_struct member name(s).
Contents
crash-whitepaper-1.0/doc/help_pages/timer.html0000644000175000017500000001035510357525152021737 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  timer


NAME
  timer - timer queue data

SYNOPSIS
  timer  

DESCRIPTION
  This command displays the timer queue entries, both old- and new-style,
  in chronological order.  In the case of the old-style timers, the
  timer_table array index is shown; in the case of the new-style timers, 
  the timer_list address is shown.

EXAMPLES
    crash> timer
    JIFFIES
      68102
    EXPIRES  TIMER_LIST/TABLE  FUNCTION
      68346      c0241934      c01775d4  <tcp_sltimer_handler>
      68379      c0241204      c01696d8  <dev_do_watchdog>
      68523      c7fcdfc0      c0112d6c  <process_timeout>
      68718      c7fd8edc      c018719c  <irlmp_discovery_timer_expired>
      68723   timer_table[2]   c01c707c  <rs_timer>
      68742      c20c1f7c      c0112d6c  <process_timeout>
      68742      c20c1f7c      c0112d6c  <process_timeout>
      68742      c20c1f7c      c0112d6c  <process_timeout>
      68752      c7fd1fc4      c0112d6c  <process_timeout>
      68752      c7fd1fc4      c0112d6c  <process_timeout>
      68989      c0241d40      c0168060  <neigh_periodic_timer>
      69028      c2533f7c      c0112d6c  <process_timeout>
      69134      c22dd868      c0181948  <unix_destroy_timer>
      71574      c0241430      c0169ea4  <rt_check_expire>
      72179      c7fb1c48      c01cb9a0  <vortex_timer>
      73144      c1b17f10      c0112d6c  <process_timeout>
      73259      c17a5f10      c0112d6c  <process_timeout>
     112929      c203ff10      c0112d6c  <process_timeout>
     372010      c2323f7c      c0112d6c  <process_timeout>
     372138      c2191f10      c0112d6c  <process_timeout>
    8653052      c1f13f10      c0112d6c  <process_timeout>
Contents
crash-whitepaper-1.0/doc/help_pages/union.html0000644000175000017500000001320310357525152021742 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  union


NAME
  union - union contents

SYNOPSIS
  union union_name[.member] [[-o][-l offset][-r] [address | symbol] [count]]

DESCRIPTION
  This command displays either a union definition, or a formatted display
  of the contents of a union at a specified address.  When no address is
  specified, the union definition is shown along with the union size.
  A union member may be appended to the structure name in order to limit
  the scope of the data displayed to that particular member; when no address
  is specified, the member's offset (always 0) and definition are shown.

     union_name  name of a C-code union used by the kernel.
        .member  name of a union member.
             -o  show member offsets when displaying union definitions.
                 (always 0)
      -l offset  if the address argument is a pointer to a list_head structure
                 that is embedded in the target union structure, the offset
                 to the list_head member may be entered in either of the
                 following manners:
                   1. in "structure.member" format.
                   2. a number of bytes. 
             -r  raw dump of union data.
        address  hexadecimal address of a union; if the address points
                 to an embedded list_head structure contained within the
                 target union structure, then the "=l" option must be used.
         symbol  symbolic reference to the address of a union.
          count  count of unions to dump from an array of unions; if used,
                 this must be the last argument entered.

  Union data, sizes, and member offsets are shown in the current output radix.
 
  Please note that in the vast majority of cases, the "union" command
  name may be dropped; if the union name does not conflict with any crash
  or gdb command name, then the "union_name[.member]" argument will be
  recognized as a union name, and this command automatically executed.
  See the NOTE below.

EXAMPLES

  Display the bdflush_param union definition, and then an instance of it:

    crash> union bdflush_param
    union bdflush_param {
        struct {
            int nfract;
            int ndirty;
            int nrefill;
            int nref_dirt;
            int dummy1;
            int age_buffer;
            int age_super;
            int dummy2;
            int dummy3;
        } b_un;
        unsigned int data[9];
    }
    
    SIZE: 36  (0x24)
 
    crash> union bdflush_param bdf_prm
    union bdflush_param {
      b_un = {
        nfract = 40, 
        ndirty = 500, 
        nrefill = 64, 
        nref_dirt = 256, 
        dummy1 = 15, 
        age_buffer = 3000, 
        age_super = 500, 
        dummy2 = 1884, 
        dummy3 = 2
      }, 
      data = {40, 500, 64, 256, 15, 3000, 500, 1884, 2}
    }

NOTE
  If the union name does not conflict with any crash command name, the
  "union" command may be dropped.  Accordingly, the examples above could
  also have been accomplished like so:

    crash> bdflush_param
    crash> bdflush_param bdf_prm
 
  Lastly, the short-cut "*" (pointer-to) command may also be used to negate
  the need to enter the "union" command name (enter "help *" for details).
Contents
crash-whitepaper-1.0/doc/help_pages/vm.html0000644000175000017500000002413610357525152021243 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  vm


NAME
  vm - virtual memory

SYNOPSIS
  vm [-p | -v | -m | [-R reference] | [-f vm_flags]] [pid | taskp] ... 

DESCRIPTION
  This command displays basic virtual memory information of a context,
  consisting of a pointer to its mm_struct and page dirctory, its RSS and 
  total virtual memory size; and a list of pointers to each vm_area_struct,
  its starting and ending address, vm_flags value, and file pathname.  If no
  arguments are entered, the current context is used.  Additionally, the -p 
  option translates each virtual page of each VM area to its physical address.
  The -R option, typically invoked from "foreach vm", searches for references
  to a supplied number, address, or filename argument, and prints only the
  essential information leading up to and including the reference.  
  Alternatively, the -m or -v options may be used to dump the task's mm_struct
  or all of its vm_area_structs respectively.  The -p, -v, -m, -R and -f
  options are all mutually exclusive.

            -p  translate each virtual page to its physical address, or if
                the page is not mapped, its swap device and offset, or
                filename and offset.
  -R reference  search for references to this number or filename.
            -m  dump the mm_struct assocated with the task.
            -v  dump all of the vm_area_structs associated with the task.
   -f vm_flags  translate the bits of a FLAGS (vm_flags) value.
           pid  a process PID.
         taskp  a hexadecimal task_struct pointer.

EXAMPLES
  Display the virtual memory data of the current context:

    crash> vm
    PID: 30986  TASK: c0440000  CPU: 0   COMMAND: "bash"
       MM       PGD       RSS    TOTAL_VM
    c303fe20  c4789000    88k      1728k
      VMA      START      END     FLAGS  FILE
    c0d1f540   8048000   80ad000  1875   /bin/bash
    c0d1f400   80ad000   80b3000  1873   /bin/bash
    c0d1f880   80b3000   80ec000    77
    c0d1f0c0  40000000  40012000   875   /lib/ld-2.1.1.so
    c0d1f700  40012000  40013000   873   /lib/ld-2.1.1.so
    c0d1fe00  40013000  40014000    77
    c0d1f580  40014000  40016000    73
    c0d1f280  4001a000  4004b000    75   /usr/lib/libncurses.so.4.2
    c0d1f100  4004b000  40054000    73   /usr/lib/libncurses.so.4.2
    c0d1f600  40054000  40057000    73
    c0d1f9c0  40057000  40059000    75   /lib/libdl-2.1.1.so
    c0d1f800  40059000  4005a000    73   /lib/libdl-2.1.1.so
    c0d1fd00  4005a000  40140000    75   /lib/libc-2.1.1.so
    c0d1fe40  40140000  40145000    73   /lib/libc-2.1.1.so
    c0d1f780  40145000  40148000    73
    c0d1f140  40148000  40150000    75   /lib/libnss_files-2.1.1.so
    c0d1fa80  40150000  40151000    73   /lib/libnss_files-2.1.1.so
    c0d1fb00  40151000  4015a000    75   /lib/libnss_nisplus-2.1.1.so
    c5f754e0  4015a000  4015b000    73   /lib/libnss_nisplus-2.1.1.so
    c0d1fec0  4015b000  4016d000    75   /lib/libnsl-2.1.1.so
    c5f75460  4016d000  4016e000    73   /lib/libnsl-2.1.1.so
    c5f75420  4016e000  40170000    73
    c5f753e0  40170000  40178000    75   /lib/libnss_nis-2.1.1.so
    c5f753a0  40178000  40179000    73   /lib/libnss_nis-2.1.1.so
    c0d1f240  bfffc000  c0000000   177

  Display the virtual memory data along with page translations for PID 386:

    crash> vm -p 386
    PID: 386    TASK: c11cc000  CPU: 0   COMMAND: "atd"
       MM       PGD       RSS    TOTAL_VM
    c7e30560  c10e5000    104k     1112k
      VMA      START      END     FLAGS  FILE
    c0fbe6a0   8048000   804b000  1875   /usr/sbin/atd
     VIRTUAL  PHYSICAL
     8048000  20e1000
     8049000  17c6000
     804a000  1f6f000
      VMA      START      END     FLAGS  FILE
    c61e0ba0   804b000   804d000  1873   /usr/sbin/atd
     VIRTUAL  PHYSICAL
     804b000  254d000
     804c000  6a9c000
      VMA      START      END     FLAGS  FILE
    c61e04e0   804d000   8050000    77   
     VIRTUAL  PHYSICAL
     804d000  219d000
     804e000  2617000
     804f000  SWAP: /dev/sda8  OFFSET: 24225
      VMA      START      END     FLAGS  FILE
    c61e0720  40000000  40012000   875   /lib/ld-2.1.1.so
     VIRTUAL  PHYSICAL
    40000000  FILE: /lib/ld-2.1.1.so  OFFSET: 0
    40001000  FILE: /lib/ld-2.1.1.so  OFFSET: 1000
    40002000  FILE: /lib/ld-2.1.1.so  OFFSET: 2000
    40003000  FILE: /lib/ld-2.1.1.so  OFFSET: 3000
    40004000  FILE: /lib/ld-2.1.1.so  OFFSET: 4000
    40005000  FILE: /lib/ld-2.1.1.so  OFFSET: 5000
    ...
 
  Although the -R option is typically invoked from "foreach vm", it can be
  executed directly.  This example displays all VM areas with vm_flags of 75:

    crash> vm -R 75
    PID: 694    TASK: c0c76000  CPU: 1   COMMAND: "crash"
       MM       PGD      RSS    TOTAL_VM
    c6c43110  c0fe9000  8932k    10720k 
      VMA       START      END   FLAGS  FILE
    c322c0d0  40019000  4004a000    75  /usr/lib/libncurses.so.4.2
    c67537c0  40056000  40071000    75  /lib/libm-2.1.1.so
    c6753d00  40072000  40074000    75  /lib/libdl-2.1.1.so
    c6753540  40075000  40081000    75  /usr/lib/libz.so.1.1.3
    c6753740  40085000  4016b000    75  /lib/libc-2.1.1.so
 
  One reason to use -R directly is to pare down the output associated with
  the -p option on a task with a huge address space.  This example displays
  the page data associated with virtual address 40121000:

    crash> vm -R 40121000
    PID: 694    TASK: c0c76000  CPU: 0   COMMAND: "crash"
       MM       PGD      RSS    TOTAL_VM
    c6c43110  c0fe9000  8928k    10720k 
      VMA       START      END   FLAGS  FILE
    c6753740  40085000  4016b000    75  /lib/libc-2.1.1.so
    VIRTUAL   PHYSICAL
    40121000  FILE: /lib/libc-2.1.1.so  OFFSET: 9c000
 
  Display the mm_struct for PID 4777:

    crash> vm -m 4777
    PID: 4777   TASK: c0896000  CPU: 0   COMMAND: "bash"
    struct mm_struct {
      mmap = 0xc6caa1c0, 
      mmap_avl = 0x0, 
      mmap_cache = 0xc6caabc0, 
      pgd = 0xc100a000, 
      count = {
        counter = 0x1
      }, 
      map_count = 0x14, 
      mmap_sem = {
        count = {
          counter = 0x1
        }, 
        waking = 0x0, 
        wait = 0x0
      }, 
      context = 0x0, 
      start_code = 0x8048000, 
      end_code = 0x809c6f7, 
      start_data = 0x0, 
      end_data = 0x80a2090, 
      start_brk = 0x80a5420, 
      brk = 0x80b9000, 
      start_stack = 0xbffff9d0, 
      arg_start = 0xbffffad1, 
      arg_end = 0xbffffad7, 
      env_start = 0xbffffad7, 
      env_end = 0xbffffff2, 
      rss = 0xf6, 
      total_vm = 0x1a3, 
      locked_vm = 0x0, 
      def_flags = 0x0, 
      cpu_vm_mask = 0x0, 
      swap_cnt = 0x23d,
      swap_address = 0x0, 
      segments = 0x0
    }
 
  Display all of the vm_area_structs for task c47d4000:

    crash> vm -v c47d4000
    PID: 4971   TASK: c47d4000  CPU: 1   COMMAND: "login"
    struct vm_area_struct {
      vm_mm = 0xc4b0d200, 
      vm_start = 0x8048000, 
      vm_end = 0x804d000, 
      vm_next = 0xc3e3abd0, 
      vm_page_prot = {
        pgprot = 0x25
      }, 
      vm_flags = 0x1875, 
      vm_avl_height = 0x1, 
      vm_avl_left = 0x0, 
      vm_avl_right = 0x0, 
      vm_next_share = 0x0, 
      vm_pprev_share = 0xc3e3abf0, 
      vm_ops = 0xc02392a0, 
      vm_offset = 0x0, 
      vm_file = 0xc1e23660, 
      vm_pte = 0x0
    }
    struct vm_area_struct {
      vm_mm = 0xc4b0d200, 
      vm_start = 0x804d000, 
      vm_end = 0x804e000, 
      vm_next = 0xc3e3a010, 
      vm_page_prot = {
        pgprot = 0x25
      }, 
      vm_flags = 0x1873, 
      vm_avl_height = 0x2, 
      vm_avl_left = 0xc3e3a810, 
      vm_avl_right = 0xc3e3a010, 
      vm_next_share = 0xc3e3a810, 
      vm_pprev_share = 0xc3699c14
      ...
 
  Translate a FLAGS value:

    crash> vm -f 3875
    3875: (READ|EXEC|MAYREAD|MAYWRITE|MAYEXEC|DENYWRITE|EXECUTABLE|LOCKED)
Contents
crash-whitepaper-1.0/doc/help_pages/vtop.html0000644000175000017500000001424710357525152021613 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  vtop


NAME
  vtop - virtual to physical

SYNOPSIS
  vtop [-c [pid | taskp]] [-u|-k] address ...

DESCRIPTION
  This command translates a user or kernel virtual address to its physical
  address.  Also displayed is the PTE translation, the vm_area_struct data
  for user virtual addresses, the mem_map page data associated with the
  physical page, and the swap location or file location if the page is
  not mapped.  The -u and -k options specify that the address is a user
  or kernel virtual address; -u and -k are not necessary on processors whose
  virtual addresses self-define themselves as user or kernel.  User addresses
  are translated with respect to the current context unless the -c option
  is used.  Kernel virtual addresses are translated using the swapper_pg_dir
  as the base page directory unless the -c option is used.
 
   -u                 The address is a user virtual address; only required
                      on processors with overlapping user and kernel virtual
                      address spaces.
   -k                 The address is a kernel virtual address; only required
                      on processors with overlapping user and kernel virtual
                      address spaces.
   -c [pid | taskp]   Translate the virtual address from the page directory
                      of the specified PID or hexadecimal task_struct pointer.
                      However, if this command is invoked from "foreach vtop",
                      the pid or taskp argument should NOT be entered; the
                      address will be translated using the page directory of
                      each task specified by "foreach".
   address            A hexadecimal user or kernel virtual address.

EXAMPLES
  Translate user virtual address 80b4000:

    crash> vtop 80b4000
    VIRTUAL   PHYSICAL
    80b4000   660f000
    
    PAGE DIRECTORY: c37f0000
      PGD: c37f0080 => e0d067
      PMD: c37f0080 => e0d067
      PTE: c0e0d2d0 => 660f067
     PAGE: 660f000
 
      PTE    PHYSICAL  FLAGS
    660f067   660f000  (PRESENT|RW|USER|ACCESSED|DIRTY)
    
      VMA      START      END      FLAGS  FILE
    c773daa0   80b4000   810c000    77
    
      PAGE    PHYSICAL   INODE     OFFSET  CNT FLAGS
    c0393258   660f000         0     17000  1  uptodate
    
  Translate kernel virtual address c806e000, first using swapper_pg_dir
  as the page directory base, and secondly, using the page table base
  of PID 1359:

    crash> vtop c806e000
    VIRTUAL   PHYSICAL
    c806e000  2216000
    
    PAGE DIRECTORY: c0101000
      PGD: c0101c80 => 94063
      PMD: c0101c80 => 94063
      PTE: c00941b8 => 2216063
     PAGE: 2216000
 
      PTE    PHYSICAL  FLAGS
    2216063   2216000  (PRESENT|RW|ACCESSED|DIRTY)
    
      PAGE    PHYSICAL   INODE     OFFSET  CNT FLAGS
    c02e9370   2216000         0         0  1  
    
    crash> vtop -c 1359 c806e000
    VIRTUAL   PHYSICAL
    c806e000  2216000
    
    PAGE DIRECTORY: c5caf000
      PGD: c5cafc80 => 94063
      PMD: c5cafc80 => 94063
      PTE: c00941b8 => 2216063
     PAGE: 2216000
    
      PTE    PHYSICAL  FLAGS
    2216063   2216000  (PRESENT|RW|ACCESSED|DIRTY)
 
      PAGE    PHYSICAL   INODE     OFFSET  CNT FLAGS
    c02e9370   2216000         0         0  1  
    
  Determine swap location of user virtual address 40104000:

    crash> vtop 40104000
    VIRTUAL   PHYSICAL
    40104000  (not mapped)
    
    PAGE DIRECTORY: c40d8000
      PGD: c40d8400 => 6bbe067
      PMD: c40d8400 => 6bbe067
      PTE: c6bbe410 => 58bc00  
 
     PTE      SWAP     OFFSET
    58bc00  /dev/sda8   22716
    
      VMA      START      END     FLAGS  FILE
    c7200ae0  40104000  40b08000    73   
 
    SWAP: /dev/sda8  OFFSET: 22716
Contents
crash-whitepaper-1.0/doc/help_pages/waitq.html0000644000175000017500000000721410357525152021744 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  waitq


NAME
  waitq - list tasks queued on a wait queue

SYNOPSIS
  waitq  [ symbol ] | [ struct.member struct_addr ] | [ address ]

DESCRIPTION
  This command walks the wait queue list displaying the tasks which 
  are blocked on the specified wait queue.  The command differentiates
  between the old- and new-style wait queue structures used by the kernel.
  It can be invoked with the following argument types:
 
                     symbol  a global symbol of a wait queue.
  struct.member struct_addr  a structure name and wait queue member combination
                             followed by the structure's hexadecimal address.
                    address  a hexadecimal wait queue pointer.

EXAMPLES

  Find out if any tasks are blocked on the "buffer_wait" wait queue:

    crash> waitq buffer_wait
    wait queue "buffer_wait" (c02927f0) is empty
 
  See who is blocked on the "wait_chldexit" queue of task c5496000:

    crash> waitq task_struct.wait_chldexit c5496000
    PID: 30879  TASK: c5496000  CPU: 0   COMMAND: "bash"
 
  Display the task list waiting on a known task queue:

    crash> waitq c3534098
    PID: 13691  TASK: c3534000  CPU: 1   COMMAND: "bash"
Contents
crash-whitepaper-1.0/doc/help_pages/whatis.html0000644000175000017500000001053110357525152022112 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  whatis


NAME
  whatis - search symbol table for data or type information

SYNOPSIS
  whatis [struct | union | typedef | symbol] 

DESCRIPTION
  This command displays the definition of structures, unions, typedefs or
  text/data symbols.

    struct  a structure name. The output is the same as if the "struct"
            command was used.
     union  a union name. The output is the same as if the "union" command
            was used.
   typedef  a typedef name. If the typedef translates to a structure or union
            the output is the same as if the "struct" or "union" command
            was used. If the typedef is a primitive datatype, the one-line
            declaration is displayed.
    symbol  a kernel symbol.  

EXAMPLES
   Display the definition of a linux_binfmt structure:

    crash> whatis linux_binfmt
    struct linux_binfmt {
      struct linux_binfmt  *next;
      struct module  *module;
      int (*load_binary) ();
      int (*load_shlib) ();
      int (*core_dump) ();
    };
    
  Since a kmem_bufctl_t is typedef'd to be a kmem_bufctl_s structure, the
  output of the following two commands is identical:

    crash> whatis kmem_bufctl_s
    struct kmem_bufctl_s {
      union {
        struct kmem_bufctl_s  *buf_nextp;
        kmem_slab_t *buf_slabp;
        void *buf_objp;
      } u;
    };
    
    crash> whatis kmem_bufctl_t
    struct kmem_bufctl_s {
      union {
        struct kmem_bufctl_s *buf_nextp;
        kmem_slab_t *buf_slabp;
        void *buf_objp;
      } u;
    };
    SIZE: 4  (0x4)
    
  Display the type data of sys_read() and jiffies text and data symbols:

    crash> whatis sys_read
    ssize_t sys_read(unsigned int, char *, size_t);
 
    crash> whatis jiffies
    long unsigned int jiffies;

  Display definition of a kdev_t typedef:

    crash> whatis kdev_t
    typedef short unsigned int kdev_t;
    SIZE: 2  (0x2)
Contents
crash-whitepaper-1.0/doc/help_pages/wr.html0000644000175000017500000000760210357525152021250 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
Contents

help page:  wr


NAME
  wr - write memory

SYNOPSIS
  wr [-u|-k|-p] [-8|-16|-32|-64] [address|symbol] value

DESCRIPTION
  This command modifies the contents of memory.  The starting address may be
  entered either symbolically or by address.  The default modification size 
  is the size of a long data type.  Write permission must exist on the
  /dev/mem.  When writing to memory on a live system, this command should 
  obviously be used with great care.

       -u  address argument is a user virtual address.
       -k  address argument is a kernel virtual address.
       -p  address argument is a physical address.
       -8  write data in an 8-bit value.
      -16  write data in a 16-bit value.
      -32  write data in a 32-bit values (default on 32-bit machines).
      -64  write data in a 64-bit values (default on 64-bit machines).
  address  address to write.  The address is considered virtual unless the
           -p option is used.  If a virtual address is specified, the
           -u or -k options are necessary only if the address space cannot
           be determined from the address value itself.  If a user virtual
           address is specified, the address space of the current context
           implied.  The address must be expressed in hexadecimal format.
   symbol  symbol of starting address to write.
    value  the value of the data to write.

EXAMPLES
  Turn on a debug flag:

    crash> wr my_debug_flag 1
Contents
crash-whitepaper-1.0/doc/help_pages/wrapper.end0000644000175000017500000000175310357525152022103 0ustar troyhebetroyhebe
Contents
crash-whitepaper-1.0/doc/build_procedure.html0000644000175000017500000002763010357525152021663 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Build Procedure

Starting with the Red Hat Enterprise Linux 3 release, the crash utility is automatically installed during system installation if the Development Tools package set is selected. However, for all other kernel versions, or if it was not selected during system installation, the binary RPM can be installed, or if desired, the sources re-built and installed.

If the crash utility is not pre-installed, and if all dependencies are met on the target system, install the binary RPM like so: 

# rpm -ivh crash-3.7-1.i386.rpm
Preparing...                ########################################### [100%]
   1:crash                  ########################################### [100%]
#

The crash executable will be installed in the /usr/bin directory.

Alternatively, the crash source code can be rebuilt. The crash utility's source files come packaged in two typical formats, a source RPM file, or a compressed tar image. So, for example, crash version 3.7-1 can be built from either crash-3.7-1.src.rpm or crash-3.7-1.tar.gz.

In either case, the source file layout consists of a top-level directory containing a set of crash-specific files, a compressed tar image containing the full, unmodified, gdb source tree, and a small number of modified gdb files required to merge the two entities. The build procedure does the following:

  1. the unmodified gdb sources are extracted into a subdirectory of the top-level crash source directory, and overlayed by the small set of modified gdb files.
  2. the files in the gdb source tree are built first, creating the libbfd.a, libreadline.a, libopcodes.a, libiberty.a and libgdb.a libraries.
  3. the crash sources files in the top-level directory are then compiled into a crashlib.a library.
  4. the objects are all linked into the crash executable, located in the top-level directory.
Depending upon the speed of the host system, the complete build may take several minutes, primarily due to the time consumed by the build of the gdb portion.

Building from the source RPM

To build from the source RPM, install the crash-3.7-1.src.rpm, cd to the appropriate SPECS directory, and build the package: 

# rpm -Uvh crash-3.7-1.src.rpm
   1:crash                  ########################################### [100%]
# cd /usr/src/redhat/SPECS
# rpmbuild -ba crash.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.19005
+ umask 022
+ cd /usr/src/redhat/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ cd /usr/src/redhat/BUILD
+ rm -rf crash-3.7-1
+ /bin/gzip -dc /usr/src/redhat/SOURCES/crash-3.7-1.tar.gz
+ tar -xvvf -
drwxr-xr-x root/root         0 2002-09-12 16:51:40 crash-3.7-1/
-rwxrwxr-x root/root     31916 2002-09-12 16:51:40 crash-3.7-1/main.c
-rwxrwxr-x root/root    103454 2002-09-12 16:51:40 crash-3.7-1/tools.c
-rwxrwxr-x root/root      5802 2002-09-12 16:51:40 crash-3.7-1/global_data.c
-rwxrwxr-x root/root    225343 2002-09-12 16:51:40 crash-3.7-1/memory.c
-rwxrwxr-x root/root     75492 2002-09-12 16:51:40 crash-3.7-1/filesys.c
-rwxrwxr-x root/root    211519 2002-09-12 16:51:40 crash-3.7-1/help.c
-rwxrwxr-x root/root    110604 2002-09-12 16:51:40 crash-3.7-1/task.c
-rwxrwxr-x root/root    101805 2002-09-12 16:51:40 crash-3.7-1/kernel.c
-rwxrwxr-x root/root      2198 2002-09-12 16:51:40 crash-3.7-1/test.c
-rwxrwxr-x root/root     18949 2002-09-12 16:51:40 crash-3.7-1/gdb_interface.c
-rwxrwxr-x root/root     20239 2002-09-12 16:51:40 crash-3.7-1/configure.c
-rwxrwxr-x root/root     29931 2002-09-12 16:51:40 crash-3.7-1/net.c
-rwxrwxr-x root/root     99654 2002-09-12 16:51:40 crash-3.7-1/dev.c
-rwxrwxr-x root/root     76146 2002-09-12 16:51:40 crash-3.7-1/alpha.c
-rwxrwxr-x root/root     74638 2002-09-12 16:51:40 crash-3.7-1/x86.c
-rwxrwxr-x root/root     42109 2002-09-12 16:51:40 crash-3.7-1/ppc.c
-rwxrwxr-x root/root     76357 2002-09-12 16:51:40 crash-3.7-1/ia64.c

  (complete output not shown)

Requires: libc.so.6 libc.so.6(GLIBC_2.0) libc.so.6(GLIBC_2.1) libc.so.6(GLIBC_2.
2) libc.so.6(GLIBC_2.2.3) libc.so.6(GLIBC_2.3) libdl.so.2 libdl.so.2(GLIBC_2.0)
libdl.so.2(GLIBC_2.1) libm.so.6 libm.so.6(GLIBC_2.0) libncurses.so.5 libz.so.1
Processing files: crash-debuginfo-3.7-1
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHave
Prefix) <= 4.0-1
Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/crash-root
Wrote: /usr/src/redhat/SRPMS/crash-3.7-1.src.rpm
Wrote: /usr/src/redhat/RPMS/i386/crash-3.7-1.i386.rpm
Wrote: /usr/src/redhat/RPMS/i386/crash-debuginfo-3.7-1.i386.rpm
#

Lastly, install the binary RPM, which copies the crash executable to the /usr/bin directory: 

# rpm -ivh /usr/src/redhat/RPMS/i386/crash-3.7-1.i386.rpm 
Preparing...                ########################################### [100%]
   1:crash
#

Building from the tar image

To build from the compressed tar image, simply uncompress/extract the source files, cd into the resultant source directory, and enter make: 

# tar xvzmf crash-3.7-1.tar.gz
crash-3.7-1/
crash-3.7-1/main.c
crash-3.7-1/tools.c
crash-3.7-1/global_data.c
crash-3.7-1/memory.c
crash-3.7-1/filesys.c
crash-3.7-1/help.c
crash-3.7-1/task.c
crash-3.7-1/kernel.c
crash-3.7-1/test.c
crash-3.7-1/gdb_interface.c
crash-3.7-1/configure.c
crash-3.7-1/net.c
crash-3.7-1/dev.c
crash-3.7-1/alpha.c
crash-3.7-1/x86.c
crash-3.7-1/ppc.c
crash-3.7-1/ia64.c
crash-3.7-1/s390.c
crash-3.7-1/s390x.c
crash-3.7-1/extensions.c
crash-3.7-1/remote.c
crash-3.7-1/va_server.c
crash-3.7-1/va_server_v1.c
crash-3.7-1/symbols.c
crash-3.7-1/cmdline.c
crash-3.7-1/lkcd_common.c
crash-3.7-1/lkcd_v1.c
crash-3.7-1/lkcd_v2_v3.c
crash-3.7-1/lkcd_v5.c
crash-3.7-1/lkcd_v7.c
crash-3.7-1/s390_dump.c
crash-3.7-1/s390x_dump.c
crash-3.7-1/lkcd_x86_trace.c
crash-3.7-1/netdump.c
crash-3.7-1/builtin_debug_data.c
crash-3.7-1/unwind.c
crash-3.7-1/unwind_decoder.c
crash-3.7-1/defs.h
crash-3.7-1/va_server.h
crash-3.7-1/vas_crash.h
crash-3.7-1/netdump.h
crash-3.7-1/unwind.h
crash-3.7-1/unwind_i.h
crash-3.7-1/rse.h
crash-3.7-1/lkcd_vmdump_v1.h
crash-3.7-1/lkcd_vmdump_v2_v3.h
crash-3.7-1/lkcd_dump_v5.h
crash-3.7-1/lkcd_x86_trace.h
crash-3.7-1/ibm_common.h
crash-3.7-1/Makefile
crash-3.7-1/gdb-5.1/
crash-3.7-1/gdb-5.1/gdb/
crash-3.7-1/gdb-5.1/gdb/Makefile.in
crash-3.7-1/gdb-5.1/gdb/main.c
crash-3.7-1/gdb-5.1/gdb/symtab.c
crash-3.7-1/gdb-5.1/gdb/target.c
crash-3.7-1/gdb-5.1/gdb/blockframe.c
crash-3.7-1/gdb-5.1/gdb/alpha-tdep.c
crash-3.7-1/gdb-5.1/gdb/symfile.c
crash-3.7-1/gdb-5.1/gdb/elfread.c
crash-3.7-1/gdb-5.1/gdb/ui-file.c
crash-3.7-1/gdb-5.1/gdb/utils.c
crash-3.7-1/gdb-5.1/gdb/gnu-regex.c
crash-3.7-1/COPYING
crash-3.7-1/gdb-5.1.tar.gz
crash-3.7-1/README
# cd crash-3.7-1
# make
TARGET: X86
 CRASH: 3.7-1
   GDB: gdb-5.1
gdb-5.1/gdb/CONTRIBUTE
gdb-5.1/gdb/COPYING
gdb-5.1/gdb/ChangeLog
gdb-5.1/gdb/ChangeLog-1990
gdb-5.1/gdb/ChangeLog-1991
gdb-5.1/gdb/ChangeLog-1992
gdb-5.1/gdb/ChangeLog-1993
gdb-5.1/gdb/ChangeLog-1994
gdb-5.1/gdb/ChangeLog-1995
gdb-5.1/gdb/ChangeLog-1996
gdb-5.1/gdb/ChangeLog-1997
gdb-5.1/gdb/ChangeLog-1998
gdb-5.1/gdb/ChangeLog-1999
gdb-5.1/gdb/ChangeLog-2000
gdb-5.1/gdb/ChangeLog-3.x
gdb-5.1/gdb/MAINTAINERS
gdb-5.1/gdb/NEWS
gdb-5.1/gdb/README
gdb-5.1/gdb/TODO
gdb-5.1/gdb/a29k-tdep.c
gdb-5.1/gdb/a68v-nat.c
gdb-5.1/gdb/abug-rom.c

  (complete output not shown)

ar -rs crashlib.a main.o tools.o global_data.o memory.o filesys.o help.o task.o
build_data.o kernel.o test.o gdb_interface.o net.o dev.o alpha.o x86.o ppc.o ia6
4.o s390.o s390x.o extensions.o remote.o va_server.o va_server_v1.o symbols.o cm
dline.o lkcd_common.o lkcd_v1.o lkcd_v2_v3.o lkcd_v5.o lkcd_v7.o s390_dump.o s39
0x_dump.o netdump.o builtin_debug_data.o lkcd_x86_trace.o unwind.o
gcc -g -O2         -o `cat mergeobj` \
main.o libgdb.a cli-decode.o cli-script.o cli-cmds.o cli-setshow.o cli-utils.o m
i-out.o mi-console.o mi-cmds.o mi-cmd-var.o mi-cmd-break.o mi-cmd-stack.o mi-cmd
-disas.o mi-main.o mi-parse.o mi-getopt.o    ../bfd/libbfd.a ../readline/libread
line.a ../opcodes/libopcodes.a  ../libiberty/libiberty.a -lncurses     -lm  ../l
ibiberty/libiberty.a \
-ldl -rdynamic `cat mergelibs`
#

The resultant crash executable will be located in the current top-level source directory. Install it in /usr/bin by entering: 

# make install
/usr/bin/install crash /usr/bin
#


< Prev Contents Next >
crash-whitepaper-1.0/doc/GFDL.html0000644000175000017500000005261510357525152017231 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents   


                GNU Free Documentation License
                  Version 1.2, November 2002


 Copyright (C) 2000,2001,2002  Free Software Foundation, Inc.
     51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.


0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other
functional and useful document "free" in the sense of freedom: to
assure everyone the effective freedom to copy and redistribute it,
with or without modifying it, either commercially or noncommercially.
Secondarily, this License preserves for the author and publisher a way
to get credit for their work, while not being considered responsible
for modifications made by others.

This License is a kind of "copyleft", which means that derivative
works of the document must themselves be free in the same sense.  It
complements the GNU General Public License, which is a copyleft
license designed for free software.

We have designed this License in order to use it for manuals for free
software, because free software needs free documentation: a free
program should come with manuals providing the same freedoms that the
software does.  But this License is not limited to software manuals;
it can be used for any textual work, regardless of subject matter or
whether it is published as a printed book.  We recommend this License
principally for works whose purpose is instruction or reference.


1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work, in any medium, that
contains a notice placed by the copyright holder saying it can be
distributed under the terms of this License.  Such a notice grants a
world-wide, royalty-free license, unlimited in duration, to use that
work under the conditions stated herein.  The "Document", below,
refers to any such manual or work.  Any member of the public is a
licensee, and is addressed as "you".  You accept the license if you
copy, modify or distribute the work in a way requiring permission
under copyright law.

A "Modified Version" of the Document means any work containing the
Document or a portion of it, either copied verbatim, or with
modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of
the Document that deals exclusively with the relationship of the
publishers or authors of the Document to the Document's overall subject
(or to related matters) and contains nothing that could fall directly
within that overall subject.  (Thus, if the Document is in part a
textbook of mathematics, a Secondary Section may not explain any
mathematics.)  The relationship could be a matter of historical
connection with the subject or with related matters, or of legal,
commercial, philosophical, ethical or political position regarding
them.

The "Invariant Sections" are certain Secondary Sections whose titles
are designated, as being those of Invariant Sections, in the notice
that says that the Document is released under this License.  If a
section does not fit the above definition of Secondary then it is not
allowed to be designated as Invariant.  The Document may contain zero
Invariant Sections.  If the Document does not identify any Invariant
Sections then there are none.

The "Cover Texts" are certain short passages of text that are listed,
as Front-Cover Texts or Back-Cover Texts, in the notice that says that
the Document is released under this License.  A Front-Cover Text may
be at most 5 words, and a Back-Cover Text may be at most 25 words.

A "Transparent" copy of the Document means a machine-readable copy,
represented in a format whose specification is available to the
general public, that is suitable for revising the document
straightforwardly with generic text editors or (for images composed of
pixels) generic paint programs or (for drawings) some widely available
drawing editor, and that is suitable for input to text formatters or
for automatic translation to a variety of formats suitable for input
to text formatters.  A copy made in an otherwise Transparent file
format whose markup, or absence of markup, has been arranged to thwart
or discourage subsequent modification by readers is not Transparent.
An image format is not Transparent if used for any substantial amount
of text.  A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain
ASCII without markup, Texinfo input format, LaTeX input format, SGML
or XML using a publicly available DTD, and standard-conforming simple
HTML, PostScript or PDF designed for human modification.  Examples of
transparent image formats include PNG, XCF and JPG.  Opaque formats
include proprietary formats that can be read and edited only by
proprietary word processors, SGML or XML for which the DTD and/or
processing tools are not generally available, and the
machine-generated HTML, PostScript or PDF produced by some word
processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself,
plus such following pages as are needed to hold, legibly, the material
this License requires to appear in the title page.  For works in
formats which do not have any title page as such, "Title Page" means
the text near the most prominent appearance of the work's title,
preceding the beginning of the body of the text.

A section "Entitled XYZ" means a named subunit of the Document whose
title either is precisely XYZ or contains XYZ in parentheses following
text that translates XYZ in another language.  (Here XYZ stands for a
specific section name mentioned below, such as "Acknowledgements",
"Dedications", "Endorsements", or "History".)  To "Preserve the Title"
of such a section when you modify the Document means that it remains a
section "Entitled XYZ" according to this definition.

The Document may include Warranty Disclaimers next to the notice which
states that this License applies to the Document.  These Warranty
Disclaimers are considered to be included by reference in this
License, but only as regards disclaiming warranties: any other
implication that these Warranty Disclaimers may have is void and has
no effect on the meaning of this License.


2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either
commercially or noncommercially, provided that this License, the
copyright notices, and the license notice saying this License applies
to the Document are reproduced in all copies, and that you add no other
conditions whatsoever to those of this License.  You may not use
technical measures to obstruct or control the reading or further
copying of the copies you make or distribute.  However, you may accept
compensation in exchange for copies.  If you distribute a large enough
number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and
you may publicly display copies.


3. COPYING IN QUANTITY

If you publish printed copies (or copies in media that commonly have
printed covers) of the Document, numbering more than 100, and the
Document's license notice requires Cover Texts, you must enclose the
copies in covers that carry, clearly and legibly, all these Cover
Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on
the back cover.  Both covers must also clearly and legibly identify
you as the publisher of these copies.  The front cover must present
the full title with all words of the title equally prominent and
visible.  You may add other material on the covers in addition.
Copying with changes limited to the covers, as long as they preserve
the title of the Document and satisfy these conditions, can be treated
as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit
legibly, you should put the first ones listed (as many as fit
reasonably) on the actual cover, and continue the rest onto adjacent
pages.

If you publish or distribute Opaque copies of the Document numbering
more than 100, you must either include a machine-readable Transparent
copy along with each Opaque copy, or state in or with each Opaque copy
a computer-network location from which the general network-using
public has access to download using public-standard network protocols
a complete Transparent copy of the Document, free of added material.
If you use the latter option, you must take reasonably prudent steps,
when you begin distribution of Opaque copies in quantity, to ensure
that this Transparent copy will remain thus accessible at the stated
location until at least one year after the last time you distribute an
Opaque copy (directly or through your agents or retailers) of that
edition to the public.

It is requested, but not required, that you contact the authors of the
Document well before redistributing any large number of copies, to give
them a chance to provide you with an updated version of the Document.


4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under
the conditions of sections 2 and 3 above, provided that you release
the Modified Version under precisely this License, with the Modified
Version filling the role of the Document, thus licensing distribution
and modification of the Modified Version to whoever possesses a copy
of it.  In addition, you must do these things in the Modified Version:

A. Use in the Title Page (and on the covers, if any) a title distinct
   from that of the Document, and from those of previous versions
   (which should, if there were any, be listed in the History section
   of the Document).  You may use the same title as a previous version
   if the original publisher of that version gives permission.
B. List on the Title Page, as authors, one or more persons or entities
   responsible for authorship of the modifications in the Modified
   Version, together with at least five of the principal authors of the
   Document (all of its principal authors, if it has fewer than five),
   unless they release you from this requirement.
C. State on the Title page the name of the publisher of the
   Modified Version, as the publisher.
D. Preserve all the copyright notices of the Document.
E. Add an appropriate copyright notice for your modifications
   adjacent to the other copyright notices.
F. Include, immediately after the copyright notices, a license notice
   giving the public permission to use the Modified Version under the
   terms of this License, in the form shown in the Addendum below.
G. Preserve in that license notice the full lists of Invariant Sections
   and required Cover Texts given in the Document's license notice.
H. Include an unaltered copy of this License.
I. Preserve the section Entitled "History", Preserve its Title, and add
   to it an item stating at least the title, year, new authors, and
   publisher of the Modified Version as given on the Title Page.  If
   there is no section Entitled "History" in the Document, create one
   stating the title, year, authors, and publisher of the Document as
   given on its Title Page, then add an item describing the Modified
   Version as stated in the previous sentence.
J. Preserve the network location, if any, given in the Document for
   public access to a Transparent copy of the Document, and likewise
   the network locations given in the Document for previous versions
   it was based on.  These may be placed in the "History" section.
   You may omit a network location for a work that was published at
   least four years before the Document itself, or if the original
   publisher of the version it refers to gives permission.
K. For any section Entitled "Acknowledgements" or "Dedications",
   Preserve the Title of the section, and preserve in the section all
   the substance and tone of each of the contributor acknowledgements
   and/or dedications given therein.
L. Preserve all the Invariant Sections of the Document,
   unaltered in their text and in their titles.  Section numbers
   or the equivalent are not considered part of the section titles.
M. Delete any section Entitled "Endorsements".  Such a section
   may not be included in the Modified Version.
N. Do not retitle any existing section to be Entitled "Endorsements"
   or to conflict in title with any Invariant Section.
O. Preserve any Warranty Disclaimers.

If the Modified Version includes new front-matter sections or
appendices that qualify as Secondary Sections and contain no material
copied from the Document, you may at your option designate some or all
of these sections as invariant.  To do this, add their titles to the
list of Invariant Sections in the Modified Version's license notice.
These titles must be distinct from any other section titles.

You may add a section Entitled "Endorsements", provided it contains
nothing but endorsements of your Modified Version by various
parties--for example, statements of peer review or that the text has
been approved by an organization as the authoritative definition of a
standard.

You may add a passage of up to five words as a Front-Cover Text, and a
passage of up to 25 words as a Back-Cover Text, to the end of the list
of Cover Texts in the Modified Version.  Only one passage of
Front-Cover Text and one of Back-Cover Text may be added by (or
through arrangements made by) any one entity.  If the Document already
includes a cover text for the same cover, previously added by you or
by arrangement made by the same entity you are acting on behalf of,
you may not add another; but you may replace the old one, on explicit
permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License
give permission to use their names for publicity for or to assert or
imply endorsement of any Modified Version.


5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this
License, under the terms defined in section 4 above for modified
versions, provided that you include in the combination all of the
Invariant Sections of all of the original documents, unmodified, and
list them all as Invariant Sections of your combined work in its
license notice, and that you preserve all their Warranty Disclaimers.

The combined work need only contain one copy of this License, and
multiple identical Invariant Sections may be replaced with a single
copy.  If there are multiple Invariant Sections with the same name but
different contents, make the title of each such section unique by
adding at the end of it, in parentheses, the name of the original
author or publisher of that section if known, or else a unique number.
Make the same adjustment to the section titles in the list of
Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled "History"
in the various original documents, forming one section Entitled
"History"; likewise combine any sections Entitled "Acknowledgements",
and any sections Entitled "Dedications".  You must delete all sections
Entitled "Endorsements".


6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents
released under this License, and replace the individual copies of this
License in the various documents with a single copy that is included in
the collection, provided that you follow the rules of this License for
verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute
it individually under this License, provided you insert a copy of this
License into the extracted document, and follow this License in all
other respects regarding verbatim copying of that document.


7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate
and independent documents or works, in or on a volume of a storage or
distribution medium, is called an "aggregate" if the copyright
resulting from the compilation is not used to limit the legal rights
of the compilation's users beyond what the individual works permit.
When the Document is included in an aggregate, this License does not
apply to the other works in the aggregate which are not themselves
derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these
copies of the Document, then if the Document is less than one half of
the entire aggregate, the Document's Cover Texts may be placed on
covers that bracket the Document within the aggregate, or the
electronic equivalent of covers if the Document is in electronic form.
Otherwise they must appear on printed covers that bracket the whole
aggregate.


8. TRANSLATION

Translation is considered a kind of modification, so you may
distribute translations of the Document under the terms of section 4.
Replacing Invariant Sections with translations requires special
permission from their copyright holders, but you may include
translations of some or all Invariant Sections in addition to the
original versions of these Invariant Sections.  You may include a
translation of this License, and all the license notices in the
Document, and any Warranty Disclaimers, provided that you also include
the original English version of this License and the original versions
of those notices and disclaimers.  In case of a disagreement between
the translation and the original version of this License or a notice
or disclaimer, the original version will prevail.

If a section in the Document is Entitled "Acknowledgements",
"Dedications", or "History", the requirement (section 4) to Preserve
its Title (section 1) will typically require changing the actual
title.


9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except
as expressly provided for under this License.  Any other attempt to
copy, modify, sublicense or distribute the Document is void, and will
automatically terminate your rights under this License.  However,
parties who have received copies, or rights, from you under this
License will not have their licenses terminated so long as such
parties remain in full compliance.


10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions
of the GNU Free Documentation License from time to time.  Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.  See
http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number.
If the Document specifies that a particular numbered version of this
License "or any later version" applies to it, you have the option of
following the terms and conditions either of that specified version or
of any later version that has been published (not as a draft) by the
Free Software Foundation.  If the Document does not specify a version
number of this License, you may choose any version ever published (not
as a draft) by the Free Software Foundation.
< Prev Contents  
crash-whitepaper-1.0/doc/command_set.html0000644000175000017500000006362410357525152021010 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

The Command Set

Each crash command generally falls into one of the following categories:

The remainder of this section breaks the command set into categories, and gives a short description of each command in that category. However, for complete details and examples, recall that the crash utility has a self-contained help page for each command; to view the full help page, click on the command name next to its description below.

Symbolic Display of Kernel Text or Data

The following commands typically take full advantage of the power of gdb to display kernel data structures symbolically.
Command
Description
struct
Displays a formatted kernel data structure type located at a given address, or at an address referred to by a symbol; if no address is specified, the structure definition is displayed. The output can be narrowed down to a singular member of the structure, or to display the offset of every member from the beginning of the structure. A count may be appended to display an array of structures. Its usage is so common that two short-cuts exist such that the user need not enter the "struct" command name:
  1. The "pointer-to" * command below can be substituted.
  2. If a structure name is entered as the first token on a command line, the "struct" command is actually not necessary.
union
Same as struct command, but used for kernel data types defined as unions instead of structures..
*
"Pointer-to" command wich can be used in lieu of entering struct or union; the gdb module first determines whether the argument is a structure or a union, and then calls the appropriate function.
p
Displays the contents of a kernel variable; the arguments are passed on to gdb's print command for proper formatting. Two builtin aliases, px and pd, set the numerical output radix to hexadecimal or decimal for the print operation, temporarily overriding the current default.
whatis
Displays all available symbol table information concerning a data type or a data symbol.
sym
Translates a kernel symbol name to its kernel virtual address and section, or a kernel virtual address to its symbol name and section. It can also be used to dump the complete list of kernel symbols, or to query the symbol list for all symbols containing a given sub-string.
dis
Disassembles the text of complete kernel function, or from a specified address for a given number of instructions, or from the beginning of a function up to a specified address.

System State

The majority of crash commands come from the following set of "kernel-aware" commands, which delve into various kernel subsystems on a system-wide or per-task basis. The task-specific commands are context-sensitive, meaning that they act upon the current context unless a PID or task address is specified as an argument.
Command
Description
bt
Arguably the most useful crash command, bt displays a task's kernel stack back-trace, including full exception frame dumps. It is context-sensitive, although the -a option will display the stack traces of the active task on each CPU. This command is often used within the foreach wrapper command in order to display the back traces of all tasks with one command.
dev
Displays data concerning the character and block device assignments, I/O port usage, I/O memory usage, and PCI device data.
files
This context-sensitive command displays the task's current root directory and working directories, and then for each open file descriptor, shows:
  • its file struct address
  • its dentry struct address
  • its inode struct address
  • the file type
  • the file's full pathname
Another option acts upon a specified dentry address, showing:
  • its inode struct address
  • its superblock struct address
  • the file type
  • the file's full pathname
It can be called from the foreach wrapper command.
fuser
Displays a list of tasks that reference a specified filename or inode address as the current root or working directory, an open file descriptor, or which mmap the file.
irq
Display data concerning interrupt request numbers and bottom-half handling.
kmem
This command has numerous options that delve into the state of several kernel memory subsystems:
  • general memory usage, similar in scope to /proc/meminfo
  • kmalloc slab memory allocator, including an option that lists each slab object and its state, verifying the slab chain
  • display and verification of free page lists
  • vmalloc memory allocator vmlist contents
  • display and verification of the page cache
  • the mem_map page list
  • display NUMA data, if applicable
Also, given an address, this command searches the symbol table, the slab subsystem, the free list, the page_hash_table, the vmlist, and the mem_map array, displaying where it was found.
log
Dumps the kernel message buffer chronologically, accounting for any wrap-around.
mach
Displays machine and/or processor specific data.
mod
Displays the list of currently-loaded kernel modules. More importantly, it loads the debug data from the module object files if they are available, allowing symbolic debugging capability of kernel modules.
mount
For each mounted filesystem, or for just a specified filesystem, displays:
  • its vfsmount struct address
  • its super_block struct address
  • its type
  • its device name
  • its mount point
Options exist to dump a list of a specified filesystem's open files or dirty inodes. Filesystems may be specified by vfsmount, super_block, or inode addresses, or by device name or mount point names.
net
Displays various network-related data:
  • displays each configured network device's net_device address, its name, and IP address
  • displays the ARP cache
  • context-sensitive display of information concerning the open sockets of a task
  • translates an IP address expressed as a decimal or hexadecimal value into a standard numbers-and-dots notation
It can be called from the foreach wrapper command.
ps
Useful process status command, in typical Linux ps command type output, containing:
  • PID number
  • PPID number
  • CPU number
  • task address
  • process state
  • percent of physical memory consumed
  • virtual address size
  • resident set size
  • command name
Also has an option to show a task's parental hierarchy back to the init process, and another to show all children of a task.
pte
This command translates the contents of a PTE into its physical page address and page bit settings, or if it references a swap location, the swap device and offset.
runq
Displays list of tasks on the run queue.
sig
A context-sensitive command which displays a task's signal information, including:
  • whether an unblocked signal is pending
  • the pending and blocked signals
  • the handler data for each signal
  • queued signals, if any
Other options list the signal number/names combination for a processor type, and translate the contents of a sigset_t into the signal names whose bits are set. It can be called from the foreach wrapper command.
swap
For each configured swap device, this command displays the same data that is shown by the Linux command swapon -s.
sys
Re-displays the same system-related data that is seen during crash initialization:
  • the kernel object filename
  • the dumpfile name
  • the number of CPUS
  • the date
  • system uptime
  • system load average
  • the number of tasks
  • the nodename
  • the kernel release and version data
  • the processor type and speed
  • the amount of memory
  • the panic string
Other options display information concerning the system call table, and one allows the root user to panic a live system.
task
This context-sensitive command displays a task's complete task_struct contents, or one or more members of the structure. This command is often used within the foreach wrapper command in order to display task_struct data for all tasks with one command.
timer
Displays the timer queue entries in chronological order, listing the target function names, the current value of jiffies, and the expiration time of each entry.
vm
This powerful, context-sensitive command displays a wealth of information concerning a task's virtual memory data, including:
  • its mm_struct address
  • its page directory address
  • its resident set size
  • its total virtual memory size
  • each vm_area_struct address, along with its start and ending virtual address, flags, and source file if applicable.
  • optionally, every virtual page referenced by a vm_area_struct can be translated into its physical address, or if not resident, its file and offset.
Other options tranlate the flags of a vm_area_struct, or display the full contents of a task's mm_struct or of each vm_area_struct. It can be called from the foreach wrapper command.
vtop
This context-sensitive command translates a user or kernel virtual address to its physical address. Also displayed are:
  • the full PTE translation from page directory through to the page table
  • the vm_area_struct data for user virtual addresses
  • the mem_map page data associated with the physical page
  • the swap location or file location if a user virtual page is not currently mapped
It can be called from the foreach wrapper command.
waitq
Lists the tasks linked on a specified kernel wait queue.

Utility Functions

The following commands are a set of useful helper commands serving various purposes, some simple, others quite powerful.
Command
Description
ascii
Translates a numerical value into its ASCII components; with no arguments, displays an ASCII chart.
btop
Translates a byte value (physical address) to its page number.
eval
A simple calculator, evaluates an expression and displays the result in hexadecimal, decimal, octal and binary, and optionally showing the bit numbers set in the result.
list
Dumps the entries of a linked list of structures. It can handle lists of structures that are singly-linked with simple "next" pointers, or those with embedded list_head structures. The output may be constrained to simply display the address of each structure in the list, or if directed, also dump each complete structure, or just one member of each structure. The gathered list entries are hashed, so a corrupted list that loops back upon itself will be recognized.
ptob
translates a page frame number to its byte value (physical address).
ptov
Translates a physical address into a kernel virtual address by adding the appropriate PAGE_OFFSET value.
search
Searches a range of user or kernel memory space for given value, with an optional "don't care" bit-mask argument.
rd
Displays a specified amount of user virtual, kernel virtual, or physical memory in several formats, such as 8, 16, 32 or 64 byte values, hexadecimal or decimal, symbolically, and with ASCII translations. When reading user virtual addresses, the command is context-sensitive.
wr
Modifies the contents of memory on a live system. Write permission on /dev/mem is required; this command should obviously be used with great care. The write operation is constrained to one 8, 16, 32 or 64 byte location.

Session Control Commands

The following commands typcally aid in the efficient running of a crash session.
Command
Description
alias
Creates a single-word alias for a command string. Several aliases are built into crash; user-defined aliases may also be defined in a .crashrc file, or during a crash session by entering it on the command line or reading it from an input file.
exit
Shuts down the crash session (same as q).
extend
Extend the crash command set by dynamically loading a shared object library containing one or more user-written commands.
foreach
Quite often it is helpful, or even necessary, to run the same crash context-sensitive command on a number of tasks by just entering one command. This wrapper command sets off the execution of a given crash command on each of a defined set of tasks, temporarily changing the current context to that of the targeted task before running the command. The set of tasks that are issued the given command can be defined by:
  • one or more PID numbers
  • one or more task numbers
  • one or more command name
  • all user tasks
  • all kernel tasks
  • the active task on each CPU
The identifiers above may be mixed if it makes sense, such as using a combination of PIDs, task addresses, and command names. The context-sensitive commands that can be issued to the selected tasks are: A header containing the PID, task address, CPU and command name will be pre-pended before the command output for each selected task.
gdb
This command passes its arguments directly to gdb for processing. This is typically not necessary, but where ambiguities between crash and gdb command names exist, this will force the command to be executed by gdb.
repeat
This wrapper command repeats a crash command indefinitely, optionally delaying a given number of seconds between each command execution. Obviously this command is only useful when running on a live system.
set
This primary purpose for this command is to set the crash context to a new task, or to display the current context. It can also be used to view or change one of a set of internal crash variables that modify program behavior, such as the default output radix or scrolling behavior. It can be called from the foreach wrapper command for viewing the context data of each task.
q
Shuts down the crash session (same as exit).

< Prev Contents Next >
crash-whitepaper-1.0/doc/context.html0000644000175000017500000002332510357525152020175 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Crash Context

Upon a successful invocation of a crash session, one of the existing Linux tasks is selected as the current context. It is important to be aware of the current context because several crash commands are "context-sensitive", meaning that the command is executed from the view-point of the current context. Therefore, the output of context-sensitive commands can vary depending upon which context is current.

Upon invocation of a crash session, the selection of the current context is based upon the following criteria:

On dumpfiles:

  • The task that was running when die() was called.
  • The task that was running when panic() was called.
  • The task that was running when an ALT-SYSRQ-c keyboard interrupt was received.
  • The task that was running when the character "c" was copied to /proc/sysrq-trigger.

On a live system:

  • the crash task itself.

The current context selection is shown in the session invocation data. For example, here is a session begun on a dumpfile that was created when an insmod task's attempt to install a module resulted in an "oops" violation:

# crash tmp/vm*

crash 3.7-1
Copyright (C) 2002, 2003  Red Hat, Inc.
Copyright (C) 1998-2003 Hewlett-Packard Co
Copyright (C) 1999, 2002 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb Red Hat Linux (5.3post-0.20021129.36rh)
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...

      KERNEL: tmp/vmlinux
DEBUG KERNEL: tmp/vmlinux.dbg
    DUMPFILE: tmp/vmcore
        CPUS: 1
        DATE: Wed Mar 27 11:02:31 2002
      UPTIME: 00:07:24
LOAD AVERAGE: 0.43, 0.42, 0.19
       TASKS: 68
    NODENAME: anderson.boston.redhat.com
     RELEASE: 2.4.9-26beta.48enterprise
     VERSION: #1 SMP Thu Mar 21 12:33:05 EST 2002
     MACHINE: i686  (501 Mhz)
      MEMORY: 128 MB
       PANIC: "Oops: 0002" (check log for details)
         PID: 1696
     COMMAND: "insmod"
        TASK: c74de000
         CPU: 0
       STATE: TASK_RUNNING (PANIC)

crash>

During runtime, the current context can always be displayed by entering the set command with no arguments:

crash> set
    PID: 1696
COMMAND: "insmod"
   TASK: c74de000
    CPU: 0
  STATE: TASK_RUNNING (PANIC)
crash>

Changing the Crash Context

The current context can be changed to a new task via the set command. Either of two "handles" may be used to identify a task, the PID number, or the kernel address of the task's task_struct. For example: 

crash> set 1
    PID: 1
COMMAND: "init"
   TASK: c7f98000
    CPU: 0
  STATE: TASK_RUNNING
crash> set c0a52000
    PID: 1503
COMMAND: "cat"
   TASK: c0a52000
    CPU: 0
  STATE: TASK_INTERRUPTIBLE
crash>

Alternatively, the current context can be set to the task running on a given CPU number, or back to the panicking task. Using the same dumpfile session shown above, in which there is only one CPU, the original context may be restored using the -c CPU-number or the -p ("panic task") options: 

crash> set -c 0
    PID: 1696
COMMAND: "insmod"
   TASK: c74de000
    CPU: 0
  STATE: TASK_RUNNING (PANIC)
crash> set -p
    PID: 1696
COMMAND: "insmod"
   TASK: c74de000
    CPU: 0
  STATE: TASK_RUNNING (PANIC)
crash>

Context-Sensitive Commands

It is important to be aware that several crash commands are context-sensitive. For example, the files command displays data about the open files of a task. If it is issued with no arguments, it displays the open files data of the current context. In this example, the current context happens to be PID 642, the klogd daemon:

crash> files
PID: 642    TASK: cd92c000  CPU: 0   COMMAND: "klogd"
ROOT: /    CWD: /
 FD    FILE     DENTRY    INODE    TYPE  PATH
  0  ce06c800  ce29ec60  cd8df900  REG   /proc/kmsg
  1  ce06cf20  ce29ebe0  cd8df740  SOCK  socket:/[858]
  2  ce06c5c0  ce2423a0  ce462c80  REG   /boot/System.map-2.4.9-e.3enterprise

However, if the files command is issued with either of the two task handles as an argument, then it will display the open files data of the specified task. In this example, PID 12731 is specified: 

crash> files 12731
PID: 12731  TASK: c8150000  CPU: 0   COMMAND: "vi"
ROOT: /    CWD: /tmp
 FD    FILE     DENTRY    INODE    TYPE  PATH
  0  c988cd80  ced919a0  c87fc3c0  CHR   /dev/pts/11
  1  c988cd80  ced919a0  c87fc3c0  CHR   /dev/pts/11
  2  c988cd80  ced919a0  c87fc3c0  CHR   /dev/pts/11
  4  c2927ae0  c6cad8a0  cd6d5040  REG   /tmp/.crontab.12730.swp
  5  c2927a80  c6cad9a0  c5764ac0  REG   /tmp/crontab.12730

This type of context-sensitive behaviour is also exhibited by the vm, bt, sig, set, net and task commands. Unless a PID or task address is specified as an argument, the output will reflect data concerning the current context.

Other commands may simply default to the current context. For example, the rd command can read memory from an address that is specified as a user-space address. Since the rd command does not accept a PID or task address as an argument, it would be necessary to be aware that the user-space access will come from the address space of the current context.


< Prev Contents Next >
crash-whitepaper-1.0/doc/extensions.html0000644000175000017500000004042210357525152020705 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Command Extensions

Frequently users wish to add an additional option to an existing crash command, or add a brand new command, in order to address a kernel issue they are debugging or developing. For those reasons, the crash utility was designed with extensibility in mind. There are two ways to add new functionality:

  1. adding new code and compiling it into the crash executable,
  2. creating a shared object library that can be dynamically loaded by using the extend command.

This section consists of a quick guide that describes how to get started using both methods.

Adding new code and compiling it into the crash executable

The current set of crash commands can be seen by entering the help command with no arguments: 

crash> help

*              files          mod            runq           union
alias          foreach        mount          search         vm
ascii          fuser          net            set            vtop
bt             gdb            p              sig            waitq
btop           help           ps             struct         whatis
dev            irq            pte            swap           wr
dis            kmem           ptob           sym            q
eval           list           ptov           sys
exit           log            rd             task
extend         mach           repeat         timer

crash version: 3.7-1   gdb version: Red Hat Linux (5.3post-0.20021129.36rh)
For help on any command above, enter "help ".
For help on input options, enter "help input".
For help on output options, enter "help output".

crash>

For each command in the menu above, there is an entry in a data structure in the file global_data.c, which is located in the top-level directory of the crash source code tree: 

/*
 *  To add a new command, declare it in defs.h and enter it in this table.
 */

struct command_table_entry base_command_table[] = {
	{"*", 	    cmd_pointer, help_pointer, 0},
	{"alias",   cmd_alias,   help_alias,   0},
        {"ascii",   cmd_ascii,   help_ascii,   0},
        {"bt",      cmd_bt,      help_bt,      REFRESH_TASK_TABLE},
	{"btop",    cmd_btop,    help_btop,    0},
	{"dev",     cmd_dev,     help_dev,     0},
	{"dis",     cmd_dis,     help_dis,     0},
	{"eval",    cmd_eval,    help_eval,    0},
	{"exit",    cmd_quit,    help_exit,    0},
	{"extend",  cmd_extend,  help_extend,  0},
	{"files",   cmd_files,   help_files,   REFRESH_TASK_TABLE},
	{"foreach", cmd_foreach, help_foreach, REFRESH_TASK_TABLE},
	{"fuser",   cmd_fuser,   help_fuser,   REFRESH_TASK_TABLE},
	{"gdb",     cmd_gdb,     help_gdb,     REFRESH_TASK_TABLE},
        {"help",    cmd_help,    help_help,    0},
	{"irq",     cmd_irq,     help_irq,     0},
	{"kmem",    cmd_kmem,    help_kmem,    0},
	{"list",    cmd_list,    help__list,   REFRESH_TASK_TABLE},
	{"log",     cmd_log,     help_log,     0},
	{"mach",    cmd_mach,    help_mach,    0},
	{"mod",     cmd_mod,     help_mod,     0},
	{"mount",   cmd_mount,   help_mount,   0},
	{"net",	    cmd_net,	 help_net,     REFRESH_TASK_TABLE},
	{"p",       cmd_p,       help_p,       0},
	{"ps",      cmd_ps,      help_ps,      REFRESH_TASK_TABLE},
	{"pte",     cmd_pte,     help_pte,     0},
	{"ptob",    cmd_ptob,    help_ptob,    0},
	{"ptov",    cmd_ptov,    help_ptov,    0},
        {"q",       cmd_quit,    help_quit,    0},
        {"rd",      cmd_rd,      help_rd,      0},
	{"repeat",  cmd_repeat,  help_repeat,  0},
	{"runq",    cmd_runq,    help_runq,    REFRESH_TASK_TABLE},
        {"search",  cmd_search,  help_search,  0},
        {"set",     cmd_set,     help_set,     REFRESH_TASK_TABLE},
        {"sig",     cmd_sig,     help_sig,     REFRESH_TASK_TABLE},
        {"struct",  cmd_struct,  help_struct,  0},
	{"swap",    cmd_swap,    help_swap,    0},
        {"sym",     cmd_sym,     help_sym,     0},
        {"sys",     cmd_sys,     help_sys,     REFRESH_TASK_TABLE},
        {"task",    cmd_task,    help_task,    REFRESH_TASK_TABLE},
	{"test",    cmd_test,    NULL,         HIDDEN_COMMAND},
        {"timer",   cmd_timer,   help_timer,   0},
	{"union",   cmd_union,   help_union,   0},
	{"vm",      cmd_vm,      help_vm,      REFRESH_TASK_TABLE},
	{"vtop",    cmd_vtop,    help_vtop,    REFRESH_TASK_TABLE},
	{"waitq",   cmd_waitq,   help_waitq,   REFRESH_TASK_TABLE},
	{"whatis",  cmd_whatis,  help_whatis,  0},
	{"wr",      cmd_wr,      help_wr,      0},
	{(char *)NULL}
};

Each entry consists of the following simple data structure: 

struct command_table_entry {               /* one for each command in menu */
        char *name;
        cmd_func_t func;
        char **help_data;
        ulong flags;
};

The structure members consist of:

name The character string that appears in the help menu output. It must be unique among other crash commands, and preferably should not clash with any gdb command.
func A pointer to the function that performs the command. To add an option to an existing command, find the file that contains this function, and modify it as desired.
help_data A pointer to an array of character strings that make up the help data for that command. Although optional, it's certainly useful to create help data for any new commands or options.
flags If REFRESH_TASK_TABLE, the set of running tasks on a live system will be updated just prior to executing the command.
If HIDDEN_COMMAND, the command will not be shown in the help menu. The only command with this flag is the test command, whose function cmd_test() exists solely as an builtin aid for quickly developing new or temporary commands.

For a newly-written command to appear in the help menu, it simply requires a reference to it in the structure above. To test a new command without adding it to the menu, use the hidden "test" command, found in test.c: 

void
cmd_test(void)
{
        int c;

        while ((c = getopt(argcnt, args, "")) != EOF) {
                switch(c)
                {
                default:
                        argerrs++;
                        break;
                }
        }

        if (argerrs)
                cmd_usage(pc->curcmd, SYNOPSIS);

        while (args[optind]) {
                ;
                optind++;
        }
}

The test command contains the basic template used by crash commands to accept dash-arguments, which are fielded by the getopt() routine, while all other command line arguments are fielded in the while loop. To add an option to the test command (or any other existing command), simply fit it into the appropriate argument-gathering mechanism. Or, for that matter, if no arguments are required, put the functionality at the end of the command's function. Here is a trivial example of a change to the cmd_test() function, with the modifications highlighted: 

void
cmd_test(void)
{
        int c;

        while ((c = getopt(argcnt, args, "xa:")) != EOF) {
                switch(c)
                {
                case 'x':
                        fprintf(fp, "arg: -x\n");
                        break;
                case 'a':
                        fprintf(fp, "arg: -a %s\n", optarg);
                        break;
                default:
                        argerrs++;
                        break;
                }
        }

        if (argerrs)
                cmd_usage(pc->curcmd, SYNOPSIS);

        while (args[optind]) {
                fprintf(fp, "arg: %s\n", args[optind]);
                ;
                optind++;
        }

        fprintf(fp, "do test work here...\n");
}

Then re-compile crash by entering make: 

# make
TARGET: X86
 CRASH: 3.7-1
   GDB: 5.3post-0.20021129.36rh

cc -c -g -DX86 -D_FILE_OFFSET_BITS=64 build_data.c
cc -c -g -DX86 -D_FILE_OFFSET_BITS=64 test.c
ar -rs crashlib.a main.o tools.o global_data.o memory.o filesys.o help.o task.o
build_data.o kernel.o test.o gdb_interface.o net.o dev.o alpha.o x86.o ppc.o ia6
4.o s390.o s390x.o extensions.o remote.o va_server.o va_server_v1.o symbols.o cm
dline.o lkcd_common.o lkcd_v1.o lkcd_v2_v3.o lkcd_v5.o lkcd_v7.o s390_dump.o s39
0x_dump.o netdump.o lkcd_x86_trace.o unwind.o
rm -f gdb
gcc -g -O2         \
-o `cat mergeobj` main.o cli-dump.o cli-decode.o cli-script.o cli-cmds.o cli-set
show.o cli-utils.o mi-out.o mi-console.o mi-cmds.o mi-cmd-var.o mi-cmd-break.o m
i-cmd-stack.o mi-cmd-disas.o mi-main.o mi-parse.o mi-getopt.o libgdb.a \
           ../bfd/libbfd.a ../readline/libreadline.a ../opcodes/libopcodes.a  ..
/libiberty/libiberty.a -lncurses     -lm   ../libiberty/libiberty.a  -ldl -rdyna
mic `cat mergelibs`
#

Then run the test command with the tree possible argument types: 

crash> test -x -a dasharg this that the other
arg: -x
arg: -a dasharg
arg: this
arg: that
arg: the
arg: other
do test work here...
crash>

The easiest way to implement a new command's functionality is to use an existing command as a template. There is a broad range of utility routines that handle argument strings, read memory, access data structures and their members, display data, and so on, that obviate the need to reinvent the wheel to accomplish a task. Look at what other similar commands do, and copy their mechanisms.

Creating a shared object library and loading it with extend

While adding a new command and/or command option in the manner above is useful, it would require the same integration mechanism with each subsequent release of the crash utility. Since that could become tedious, another extension mechanism exists in which share objects containing one or more crash commands can be written, and then dynamically attached to a running crash session, using the extend command. Once loaded, the command(s) in the shared object library automatically appear in the help menu, as if they were compiled into the crash executable. As an quick aid in creating a shared object, the help page for the extend contains an example C program tagged onto the end, which adds a new echo command (which simply echoes back all arguments). The C program piece can be cut and pasted into a file, say extlib.c for example, and then compiled like so: 

# gcc -nostartfiles -shared -rdynamic -o extlib.so extlib.c
#

The resultant extlib.so file may be dynamically linked into crash during runtime using the extend command: 

crash> extend extlib.so
./extlib.so: shared object loaded
crash>

Or, to automatically load the shared library during crash session initialization, put the following string into a .crashrc file located in the current directory, or in the user's $HOME directory: 

extend extlib.so

Here is the help menu once the library is loaded; note the integration of the new echo command: 

crash> help

*              extend         mach           repeat         timer
alias          files          mod            runq           union
ascii          foreach        mount          search         vm
bt             fuser          net            set            vtop
btop           gdb            p              sig            waitq
dev            help           ps             struct         whatis
dis            irq            pte            swap           wr
echo           kmem           ptob           sym            q
eval           list           ptov           sys
exit           log            rd             task

crash version: 3.7-1    gdb version: Red Hat Linux (5.3post-0.20021129.36rh)
For help on any command above, enter "help ".
For help on input options, enter "help input".
For help on output options, enter "help output".

With this extension mechanism, the most that would be required to use the shared library with subsequent versions of crash would be a simple re-compile of the extlib.c file.

< Prev Contents Next >
crash-whitepaper-1.0/doc/help.html0000644000175000017500000003217610357525152017445 0ustar troyhebetroyhebe

White Paper: Enhanced Console Access
< Prev Contents Next >

Builtin Help

Readily available help information is built into the crash utility. During a session, entering the help command with no argument shows the following menu:

crash> help
* files mod runq union
alias foreach mount search vm
ascii fuser net set vtop
bt gdb p sig waitq
btop help ps struct whatis
dev irq pte swap wr
dis kmem ptob sym q
eval list ptov sys
exit log rd task
extend mach repeat timer 

crash version: 3.7-1   gdb version: Red Hat Linux (5.3post-0.20021129.36rh)
For help on any command above, enter "help <command>".
For help on input options, enter "help input".
For help on output options, enter "help output".

crash>

Each command has its own man-like help page, which can be viewed by clicking on the command name above. Each help page details the syntax of the command and its available options, a description of the command in general, a description of each option, and a set of examples. During a crash session, a command's help page can be displayed by entering help followed by the command name. So, for example, to get help on how to use the set command:

crash> help set

NAME
  set - set a process context or internal crash variable

SYNOPSIS
  set [pid | taskp | [-c cpu] | -p] | [crash_variable [setting]] | -v

DESCRIPTION
  This command either sets a new context, or gets the current context for
  display.  The context can be set by the use of:

      pid  a process PID.
    taskp  a hexadecimal task_struct pointer.
   -c cpu  sets the context to the active task on a cpu (dumpfiles only).
       -p  sets the context to the panic task, or back to the crash task on
           a live system.
       -v  display the current state of internal crash variables.

  If no argument is entered, the current context is displayed.  The context
  consists of the PID, the task pointer, the CPU, and task state.
  
  This command may also be used to set internal crash variables.  If no value
  argument is entered, the current value of the crash variable is shown.  These
  are the crash variables, acceptable arguments, and purpose:

       scroll  on | off     controls output scrolling.
        radix  10 | 16      sets output radix to 10 or 16.
      refresh  on | off     controls internal task list refresh.
    print_max  number       set maximum number of array elements to print.
      console  device-name  sets debug console device.
        debug  number       sets crash debug level.
         core  on | off     if on, drops core when the next error message
                            is displayed.
         hash  on | off     controls internal list verification.
       silent  on | off     turns off initialization messages; turns off crash
                            prompt during input file execution. (scrolling is
                            turned off if silent is on)
         edit  vi | emacs   set line editing mode (from .crashrc file only).
     namelist  filename     name of kernel (from .crashrc file only).
     dumpfile  filename     name of core dumpfile (from .crashrc file only).
 
  Internal variables may be set in four manners:

    1. entering the set command in $HOME/.crashrc.
    2. entering the set command in .crashrc in the current directory.
    3. executing an input file containing the set command.
    4. during runtime with this command.

  During initialization, $HOME/.crashrc is read first, followed by the
  .crashrc file in the current directory.  Set commands in the .crashrc file
  in the current directory override those in $HOME/.crashrc.  Set commands 
  entered with this command or by runtime input file override those
  defined in either .crashrc file.  Multiple set command arguments or argument
  pairs may be entered in one command line.

EXAMPLES
  Set the current context to task c2fe8000:

    crash> set c2fe8000
         PID: 15917
     COMMAND: "bash"
        TASK: c2fe8000  
         CPU: 0
       STATE: TASK_INTERRUPTIBLE

  Set the context back to the panicking task:

    crash> set -p
         PID: 698
     COMMAND: "gen12"
        TASK: f9d78000
         CPU: 2
       STATE: TASK_RUNNING (PANIC)

  Turn off output scrolling:

    crash> set scroll off
    scroll: off
 
  Show the current state of crash internal variables:

    crash> set -v
       scroll: on
        radix: 10 (decimal)
      refresh: on
    print_max: 256
      console: /dev/pts/2
        debug: 0
         core: off
         hash: on
       silent: off
         edit: vi
     namelist: vmlinux
     dumpfile: vmcore
 
  Show the current context:

    crash> set
         PID: 1525
     COMMAND: "bash"
        TASK: c1ede000
         CPU: 0
       STATE: TASK_INTERRUPTIBLE

If for some reason a crash session cannot be invoked, but help information for a particular crash command is desired, the same help page can be displayed from a shell command line using the -h option to crash:

# crash -h ascii

NAME
  ascii - translate a hexadecimal string to ASCII

SYNOPSIS
  ascii value ...

DESCRIPTION
  Translates 32-bit or 64-bit hexadecimal values to ASCII.  If no argument
  is entered, an ASCII chart is displayed.

EXAMPLES
  Translate the hexadecimal value of 0x62696c2f7273752f to ASCII:

    crash> ascii 62696c2f7273752f
    62696c2f7273752f: /usr/lib

  Display an ASCII chart:

    crash> ascii
 
          0    1   2   3   4   5   6   7
        +-------------------------------
      0 | NUL DLE  SP  0   @   P   '   p
      1 | SOH DC1  !   1   A   Q   a   q
      2 | STX DC2  "   2   B   R   b   r
      3 | ETX DC3  #   3   C   S   c   s
      4 | EOT DC4  $   4   D   T   d   t
      5 | ENQ NAK  %   5   E   U   e   u
      6 | ACK SYN  &   6   F   V   f   v
      7 | BEL ETB  `   7   G   W   g   w
      8 |  BS CAN  (   8   H   X   h   x
      9 |  HT  EM  )   9   I   Y   i   y
      A |  LF SUB  *   :   J   Z   j   z
      B |  VT ESC  +   ;   K   [   k   {
      C |  FF  FS  ,   <   L   \   l   |
      D |  CR  GS  _   =   M   ]   m   }
      E |  SO  RS  .   >   N   ^   n   ~
      F |  SI  US  /   ?   O   -   o  DEL

#

Lastly, help concerning command input and output can be displayed by entering help input or help output during runtime, or crash -h input or crash -h output from a shell command line.
< Prev Contents Next >
crash-whitepaper-1.0/doc/index.html0000644000175000017500000001353110357525152017616 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility

by David Anderson
<anderson@redhat.com>

Copyright © 2003 by Red Hat Software, Inc.

Permission is granted to copy, distribute and/or modify this document under the terms of
the GNU Free Documentation License, Version 1.2 or any later version published by the
Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU Free Documentation License".

Content

  1. Why Crash?
  2. Prerequisites
  3. Build Procedure
  4. Invocation
  5. Command Input
  6. Command Output
  7. Crash Context
  8. Builtin Help
  9. The Command Set
  10. Crash Usage: A Case Study
  11. Command Extensions
  12. GNU Free Documentation License

Abstract

The Red Hat crash analysis utility is loosely based on the SVR4 UNIX crash command, but has been significantly enhanced by completely merging it with the GNU gdb debugger. The marriage of the two effectively combines the kernel-specific nature of the traditional UNIX crash utility with the source code level debugging capabilities of gdb. The utility can be used to investigate:

The current set of commands consist of common kernel core analysis tools such as kernel stack back traces of all processes, source code disassembly, formatted kernel structure and variable displays, virtual memory data, dumps of linked-lists, etc., along with several commands that delve deeper into specific kernel subsystems. Relevant gdb commands may also be entered, which in turn are passed on to the gdb module for execution.

The crash utility is designed to be independent of Linux version dependencies. When new kernel source code impacts the correct functionality of crash and its command set, the utility will be updated to recognize new kernel code changes while maintaining backwards compatibility with earlier releases.

Next >
crash-whitepaper-1.0/doc/input.html0000644000175000017500000002022710357525152017646 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Command Input

Upon a successful session invocation on a dump file or a live kernel, the crash> prompt will appear. Interactive crash commands are gathered using the GNU readline library, taking advantage of its command line history mechanism, and its vi or emacs command line editing modes. Commands may also be issued to crash from a file.

Command Line History

The command line history consists of a numbered list of previously-run commands. The full list of commands may be viewed by entering h at any time. For example: 

crash> h
[1] bt -a
[2] ps
[3] foreach bt
[4] set 
[5] dis -rl c0221141
crash>

Commands in the history list may be re-run in the following manners

  1. To re-run the last command executed, simply enter r or !! and then ENTER.
  2. Enter r followed by the appropriate history list number, and then ENTER.
  3. Enter r followed by a uniquely-identifying set of characters from the beginning of the previously-entered command string, and then ENTER.
  4. Recycle back through the command history list using the up-arrow and down-arrow keys until the desired command is re-displayed, and then ENTER.
  5. Recycle back through the command history list using the key-strokes appropriate for the command line editing mode being used (vi or emacs) until the desired command is re-displayed, and then ENTER.

Command Line Editing

The command line editing mode may be set to either vi (the default) or emacs. The mode may set in the following manners, listed in increasing order of precedence:

  1. Do none of the following, in which case the default vi editing mode will be used.
  2. Set the EDITOR environment variable to either vi or emacs.
  3. Create an entry in a .crashrc file in the user's $HOME directory. The entry must be a line of one of the following forms: 

    set vi
set emacs

  4. Create an entry in a .crashrc file be located in the current directory, in the form shown in (3) above.
  5. Use the -e command line option, as in: 

    # crash -e [vi | emacs] ...

Given either editing mode, any previously entered command line can be brought back by entering the mode-specific key-stroke(s), the command line edited using the proper mode, and then run by hitting ENTER.

Command Line Input from a File

An input file consisting of a list of commands may be fed to crash in the following manners:

  1. Upon invocation, as in: 

    # crash -i inputfile

  2. Upon invocation, by entering the commands in a .crashrc file, which can be either in the user's $HOME directory or in the current directory.
  3. On the command line during a crash session, as in: 

    4. crash> < inputfile

In all of the three cases above, after the list of commands in the file have completed, the crash> prompt will appear and commands may then be entered interactively (unless one of the file commands happens to be the exit command).

Numerical Arguments

Numerical arguments are typically presumed to be decimal unless the argument contains an a, b, c, d, e or f. In those cases, the preceding 0x is not required. For hexadecimal numbers that do not contain one of those 6 characters, the preceding 0x is required. So, for example, a value of 1 gigabyte would have to be expressed as 0x40000000, whereas 3 gigabytes could be expressed as c0000000.

It should be noted that several commands will only accept hexadecimal numerical arguments. For example, the rd ("read") command only accepts hexadecimal addresses. Therefore a read from user address of 0x40017000 could be entered as:

crash> rd 40017000 40
40017000:  20000824 00000010 00000048 00000063   $.. ....H...c...
40017010:  00000082 000000ba 000000bb 000000cd   ................
40017020:  000000ce 000000cf 000000d7 000000db   ................
40017030:  000000dc 000000dd 000000de 000000e2   ................
40017040:  000000ed 00000167 6c676e45 20687369   ....g...English
40017050:  61636f6c 6620656c 7420726f 55206568   locale for the U
40017060:  46004153 20656572 74666f53 65726177   SA.Free Software
40017070:  756f4620 7461646e 2c6e6f69 636e4920    Foundation, Inc
40017080:  3935002e 6d655420 20656c70 63616c50   ..59 Temple Plac
40017090:  202d2065 74697553 33332065 42202c30   e - Suite 330, B


< Prev Contents Next >
crash-whitepaper-1.0/doc/invocation.html0000644000175000017500000005720510357525152020666 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Invocation

When crash is run on a dumpfile, at least two arguments are always required:

  1. The kernel object filename, often referred to as the kernel namelist. When initially built from the kernel sources, its name is vmlinux; typically it is then copied to the /boot directory, where the operating system release number is appended, as in vmlinux-2.4.21-4.ELsmp.
  2. The dumpfile name. For example, Netdump-generated dumpfiles are named vmcore.
For example: 

# crash vmlinux vmcore

When crash is run on a live system, /dev/mem is used as the memory image. Therefore, only the kernel object filename is required: 

# crash vmlinux

Furthermore, when crash is run on a live system, the vmlinux argument is not required when the kernel object file is located in any of the following locations:

  • / (root directory)
  • /boot
  • /boot/efi/EFI/redhat
  • any subdirectory of /usr/src
  • /usr/src/redhat/BUILD/kernel-x.x.x/linux
  • /usr/lib/debug/lib/modules/<release><flavor>

When the vmlinux file is not entered on the command line, a search will be made in all of the directories above until a kernel object file is found that contains a version string that matches the running kernel, as indicated by /proc/version. If a matching kernel is found, then crash may be invoked on a live system simply by entering: 

# crash

In the examples above, it is presumed that the vmlinux kernel has been built with the -g C flag, which traditionally has not been done by default. To address this requirement, Red Hat Enterprise Linux 3 (RHEL 3) and Red Hat Enterprise Linux 4 (RHEL 4) kernels are now built with -g C flag. The manner of accessing the debug data for RHEL 3 and RHEL 4 kernels is described in the following two sections. Unfortunately, since Red Hat Advanced Server 2.1 kernels are not built with -g, the kernel must be rebuilt; directions for rebuilding Red Hat Advanced Server 2.1 kernels can be found here.

RHEL-3 Kernels

In RHEL 3, the vmlinux kernel debug information is stripped and stored in a separate debuginfo file. The stripped vmlinux file in /boot has an embedded link to its associated debuginfo file in /usr/lib/debug/boot, so that the crash utility (and the built-in gdb module) knows where to find it:

RHEL 3 kernel namelist in /boot RHEL 3 kernel debuginfo file in /usr/lib/debug/boot
vmlinux-<release>.EL
vmlinux-<release>.ELsmp
vmlinux-<release>.ELhugemem 		vmlinux-<release>.EL.debug
vmlinux-<release>.ELsmp.debug
vmlinux-<release>.ELhugemem.debug

The debuginfo files for a specific kernel <release> come from a separate RPM that must be installed for the crash utility to work. For example, the i686 RPM for the examples above would be named kernel-debuginfo-<release>.i686.rpm, and would install the debuginfo file for all three of the kernel flavors.

For example, to run crash on a live system, the associated debuginfo package must be installed: 

# uname -r
2.4.21-4.ELsmp
# rpm -ivh kernel-debuginfo-2.4.21-4.EL.i686.rpm
Preparing...                ########################################### [100%]
   1:kernel-debuginfo       ########################################### [100%]
# ls /usr/lib/debug/boot
vmlinux-2.4.21-4.EL.debug
vmlinux-2.4.21-4.ELhugemem.debug
vmlinux-2.4.21-4.ELsmp.debug
#

Accordingly, if the running kernel's vmlinux file is in one the search locations above, and its associated debuginfo file is located in the /usr/lib/debug/boot directory or in the current directory from which crash is invoked, no arguments are required to run on a live system: 

# crash

However, if the linked debuginfo file is not in either of those locations, it can be added to the crash command line along with the vmlinux filename. So, for example, if the debuginfo file was located in /tmp: 

# crash /boot/vmlinux-2.4.21-4.ELsmp /tmp/vmlinux-2.4.21-4.ELsmp.debug

For analyzing dumpfiles however, the vmlinux file name must be on the command line along with the dumpfile name, as in the following examples: 

# crash /boot/vmlinux-2.4.21-4.ELsmp vmcore

or if the debuginfo file is not in the standard location: 

# crash /boot/vmlinux-2.4.21-4.ELsmp /tmp/vmlinux-2.4.21-4.ELsmp.debug vmcore

RHEL-4 Kernels

The procedure has been made much simpler for RHEL-4 kernels. The kernel is built with the -g flag, and the resultant vmlinux file is stored in the associated debuginfo package. After installing the debuginfo package, the vmlinux file for each kernel flavor of a given RHEL 4 release will be installed in the directory named:

/usr/lib/debug/lib/modules/<release><flavor>/vmlinux

where for i686 kernels, <flavor> can be either hugemem, smp, or nothing (for uniprocessor kernels). For example: 

# uname -r
2.6.9-6.39.ELsmp
# rpm -ivh kernel-debuginfo-2.6.9-6.39.EL.i686.rpm
Preparing...                ########################################### [100%]
   1:kernel-debuginfo       ########################################### [100%]
#
# find /usr/lib/debug/lib/modules/2.6.9-6.39.EL* -name vmlinux
/usr/lib/debug/lib/modules/2.6.9-6.39.ELhugemem/vmlinux
/usr/lib/debug/lib/modules/2.6.9-6.39.ELsmp/vmlinux
/usr/lib/debug/lib/modules/2.6.9-6.39.EL/vmlinux
#

Once the debuginfo package is installed, crash can be invoked on the live system with no arguments, because the vmlinux file will be found automatically: 

# crash

To run crash on a dumpfile, however, the appropriate vmlinux file and the dumpfile name must both be on the command line, as in: 

# crash /usr/lib/debug/lib/modules/2.6.9-6.39.ELsmp/vmlinux vmcore

Kernels built without -g flag

If the running kernel was not built with the -g C flag, then it is necessary to rebuild a kernel of the same configuration with the -g C flag. The essential change done by this kernel rebuild task is a modification of top-level Makefile of the kernel source tree, such that the CFLAGS definition contains the -g flag. For example, this is the line that must be changed: 

CFLAGS := $(CPPFLAGS) -Wall -Wstrict-prototypes -Wno-trigraphs -O2 \
          -fno-strict-aliasing -fno-common

by adding the -g flag: 

CFLAGS := $(CPPFLAGS) -Wall -Wstrict-prototypes -Wno-trigraphs -O2 \
          -fno-strict-aliasing -fno-common -g

For example, since Red Hat Advanced Server 2.1 kernels are not built with -g, a kernel rebuild is required. For a detailed example of how to rebuild a Red Hat Advanced Server 2.1 kernel with the -g flag, please refer to these directions.

Upon rebuilding the kernel, a new vmlinux file will be created that contains the debug data required by crash. However, the symbol values will not match those of the running or dumped kernel. To deal with this inequity, the actual symbol values can be gathered from either the original non-debug vmlinux file or from its associated System.map file. That being the case, two arguments must be supplied to crash to fully describe the running/dumped kernel, the newly-created vmlinux file compiled with -g, as well as a source of the real symbol values. So, for example, if the vmlinux file built with -g were renamed to vmlinux.dbg, the invocation line would look like this on a live system: 

# crash vmlinux vmlinux.dbg
(or)
# crash /boot/System.map vmlinux.dbg
(or)
# crash -S vmlinux.debug

The -S argument above is simply an alternative to entering the default /boot/System.map string.

Similarly, when looking at a dumpfile, two arguments are required to describe the dumped kernel, along with the vmcore image: 

# crash vmlinux vmlinux.dbg vmcore
(or)
# crash /boot/System.map vmlinux.dbg vmcore 
(or)
# crash -S vmlinux.dbg vmcore

Again, for a detailed example of how to rebuild a Red Hat Advanced Server 2.1 kernel with the -g flag, refer to these directions.

Invocation output

The arguments may be entered in any order. If the file arguments are not in the current directory, absolute pathnames must be used. When in doubt, simply enter crash -h to get an explanation of the command line arguments: 

# crash -h

Usage:
  crash [-h [opt]][-v][-s][-i file][-d num] [-S] [mapfile] [namelist] [dumpfile]

  [namelist]
    The [namelist] argument is a pathname to an uncompressed kernel image
    (a vmlinux file) that has been compiled with the "-g" switch, or
    that has an accessible, associated, debuginfo file.  If the [dumpfile]
    argument is entered, then the [namelist] argument must be entered
    If the [namelist] argument is not entered when running on a live
    system, a search will be made in several typical directories for
    for a kernel namelist file that matches the live system.
 
  [dumpfile]
    The [dumpfile] argument is a pathname to a kernel memory core dump
    file.  If the [dumpfile] argument is not entered, the session will be
    invoked on the live system using /dev/mem, which usually requires root
    privileges.
 
  [mapfile]
    If the live system kernel, or the kernel from which the [dumpfile] 
    was derived, was not compiled with the -g switch, then the additional
    [mapfile] argument is required.  The [mapfile] argument may consist
    of either the associated System.map file, or the non-debug kernel
    namelist.  However, if the [mapfile] argument is used, then the
    [namelist] argument must be a kernel namelist of a similar kernel
    version that was built with the -g switch.
 
  [-S]
    Use "/boot/System.map" as the [mapfile].
 
    Examples when running on a live system:
 
      $ crash
      $ crash /usr/tmp/vmlinux
      $ crash /boot/System.map vmlinux.dbg
      $ crash -S vmlinux.dbg
      $ crash vmlinux vmlinux.dbg
 
    Examples when running on a dumpfile:
 
      $ crash vmlinux vmcore
      $ crash /boot/System.map vmlinux.dbg vmcore
      $ crash -S vmlinux.dbg vmcore
      $ crash vmlinux vmlinux.dbg vmcore
 
  [-h [opt]]
    The -h option alone displays this message.  If the [opt] argument is
    a crash command name, the help page for that command is displayed.  If
    the string "input" is entered, a page describing the various crash
    command line input options is displayed.  If the string "output" is
    entered, a page describing command line output options is displayed.
 
  [-v]
    Display the versions of crash and gdb making up this executable.
 
  [-s]
    Do not display any version, GPL, or crash initialization data; proceed
    directly to the "crash>" prompt.
 
  [-i file]
    Execute the crash command(s) in [file] prior to accepting any user
    input from the "crash>" prompt.
 
  [-d num]
    Set crash debug level [num].  The higher the number, the more debug data
    will be printed during crash runtime.

Given that all invocation arguments are in order, here is an example of a successful invocation on a dumpfile, running a kernel that was built with -g, along with a vmcore dump file was created by the Red Hat Netdump facility: 

# crash vmlinux-2.4.20-2.1.15.entsmp vmcore

crash 3.7-1
Copyright (C) 2002, 2003  Red Hat, Inc.
Copyright (C) 1998-2003  Hewlett-Packard Co
Copyright (C) 1999, 2002  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb Red Hat Linux (5.3post-0.20021129.36rh)
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...

      KERNEL: vmlinux-2.4.20-2.1.15.entsmp
    DUMPFILE: vmcore
        CPUS: 1
        DATE: Wed Mar 12 10:12:56 2003
      UPTIME: 00:38:25
LOAD AVERAGE: 1.16, 0.74, 0.30
       TASKS: 60
    NODENAME: dhcp64-220.boston.redhat.com
     RELEASE: 2.4.20-2.1.15.entsmp
     VERSION: #1 SMP Tue Mar 11 16:12:22 EST 2003
     MACHINE: i686  (501 Mhz)
      MEMORY: 128 MB
       PANIC: "Oops: 0002" (check log for details)
         PID: 0
     COMMAND: "swapper"
        TASK: c038e000
         CPU: 0
       STATE: TASK_RUNNING (PANIC)

crash>

This next example shows the output when the panicking kernel was not built with -g. In this case, a similar kernel type was built with -g, and the resultant kernel object file was renamed as vmlinux.dbg. Note that there will be a message concerning the patching of gdb data; this indicates that the non-matching symbol values from the vmlinux.dbg are being over-written by the correct symbol values found in the original vmlinux file: 

# crash vmlinux vmlinux.dbg vmcore

crash 3.7-1
Copyright (C) 2002, 2003  Red Hat, Inc.
Copyright (C) 1998-2003  Hewlett-Packard Co
Copyright (C) 1999, 2002 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
GNU gdb Red Hat Linux (5.3post-0.20021129.36rh)
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...

please wait... (patching 16053 gdb minimal_symbol values)

DEBUG KERNEL: vmlinux.dbg
    DUMPFILE: vmcore
        CPUS: 1
        DATE: Wed Mar 27 11:02:31 2002
      UPTIME: 00:07:24
LOAD AVERAGE: 0.43, 0.42, 0.19
       TASKS: 68
    NODENAME: anderson.boston.redhat.com
     RELEASE: 2.4.9-26beta.48enterprise
     VERSION: #1 SMP Thu Mar 21 12:33:05 EST 2002
     MACHINE: i686  (501 Mhz)
      MEMORY: 128 MB
       PANIC: "Oops: 0002" (check log for details)
         PID: 1696
     COMMAND: "insmod"
        TASK: c74de000  
         CPU: 0
       STATE: TASK_RUNNING (PANIC)

crash>

Invocation on a live system looks essentially the same, except that the DUMPFILE will be indicated as /dev/mem. In the following example, no arguments were entered, because the running RHEL 3 kernel was found in the /boot directory, and its associated debuginfo file in the /usr/lib/debug/boot directory. The debuginfo file is listed next to the DEBUGINFO tag: 

# crash

crash 3.7-1
Copyright (C) 2002, 2003  Red Hat, Inc.
Copyright (C) 1998-2003  Hewlett-Packard Co
Copyright (C) 1999, 2002  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.

GNU gdb Red Hat Linux (5.3post-0.20021129.36rh)
Copyright 2003 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...

      KERNEL: /boot/vmlinux-2.4.21-4.ELhugemem
   DEBUGINFO: /usr/lib/debug/boot/vmlinux-2.4.21-4.ELhugemem.debug
    DUMPFILE: /dev/mem
        CPUS: 2
        DATE: Thu Aug 21 11:24:38 2003
      UPTIME: 1 days, 23:14:11
LOAD AVERAGE: 0.14, 0.10, 0.08
       TASKS: 106
    NODENAME: crash.boston.redhat.com
     RELEASE: 2.4.21-4.ELhugemem
     VERSION: #1 SMP Wed Aug 13 21:33:31 EDT 2003
     MACHINE: i686  (1993 Mhz)
      MEMORY: 511.5 MB
         PID: 4757
     COMMAND: "crash"
        TASK: 19b82000
         CPU: 1
       STATE: TASK_RUNNING (ACTIVE)

crash>

Invocation Errors

Invocation errors will cause the crash session to abort upon initialization. Typically they occur as the result of one of the following reasons:

  1. The vmlinux file contains no debug data (i.e., was built without the -g flag), and no additional debug kernel object file name was entered on the command line. The error message will be of the form: 

    crash: /boot/vmlinux-2.4.18-14: no debugging data available

  2. The vmlinux file does not match the dumpfile. The error message will be of the form: 

    crash: vmlinux and tmp/vmcore do not match!

  3. The vmlinux file could not be found on a live system. The error message will be of the form: 

    crash: cannot find booted kernel -- please enter namelist argument

  4. The associated debuginfo file cannot be found. The error message will be of the form: 

    crash: /boot/vmlinux-2.4.21-4.ELsmp: no debugging data available
crash: vmlinux-2.4.21-4.ELsmp.debug: debuginfo file not found


< Prev Contents Next >
crash-whitepaper-1.0/doc/prerequisites.html0000644000175000017500000001264310357525152021416 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Prerequisites

The crash utility has the following prerequisites:

kernel object file:

A vmlinux kernel object file, often referred to as the namelist in this document, which must have been built with the -g C flag so that it will contain the debug data required for symbolic debugging. The vmlinux file associated with the running kernel is typically found in the /boot directory; it will have the operating system release string appended to it, for example, vmlinux-2.4.21-4.ELsmp.

Ideally the kernel object file is the same kernel object file that is associated with the memory image. However, in circumstances where the vmlinux file associated with the crash dump or live system was not built with the -g flag, there are work-arounds discussed later in the Invocation section.


memory image:

This may consist of a kernel crash dump file generated from any of the three crash dump facilities (Red Hat Netdump, Mission Critical Linux Mcore, or LKCD), or live system memory accessed via /dev/mem. If no dump file argument is issued on the crash command line, live system memory will be used by default. When examining a live system, root privileges are required.

Support for the Red Hat Netdump dumpfile format is actively maintained and supported. Support for the Mcore dumpfiles exists only because the kernel patch that creates them is no longer being updated or maintained. Support for 2.4-based LKCD dumpfiles exists on an as-is basis, and is not actively maintained. However, if LKCD eventually becomes part of the mainstream kernel in the future, maintenance will be resumed.


platform processor types:

The crash utility is actively developed and tested on Intel x86 and IA64 processors. Legacy support for the Alpha and 32-bit PowerPC platforms exists, but no longer actively maintained. Support for the AMD64, PPC64, s390 and s390x processors are targetted for future support.

Linux kernel versions:

The crash utility is backwards-compatible to at least Red Hat 6.0 (Linux version 2.2.5-15), up to Red Hat Enterprise Linux 3 (Linux version 2.4.21+). The intent has always been to make the utility independent of Linux version dependencies, building in recognition of major kernel code changes so as to adapt to new kernel versions, while maintaining backwards compatibility.


< Prev Contents Next >
crash-whitepaper-1.0/doc/new_page.html0000644000175000017500000000736610357525152020305 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

New Page

yada, yada, yada...

crash> help

*              files          mod            runq           union
alias          foreach        mount          search         vm
ascii          fuser          net            set            vtop
bt             gdb            p              sig            waitq
btop           help           ps             struct         whatis
dev            irq            pte            swap           wr
dis            kmem           ptob           sym            q
eval           list           ptov           sys
exit           log            rd             task
extend         mach           repeat         timer

crash version: 3.7-1   gdb version: Red Hat Linux (5.3post-0.20021129.36rh)
For help on any command above, enter "help ".
For help on input options, enter "help input".
For help on output options, enter "help output".

crash>

yada, yada, yada...
< Prev Contents Next >
crash-whitepaper-1.0/doc/output.html0000644000175000017500000002027210357525152020047 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Command Output

crash commands can often be verbose, and it's helpful to control the output, as well as to be able to scroll backwards to view previous command output. So, by default, command output that would overflow the user's display screen is piped to /usr/bin/less, along with a prompt line that informs the user how to scroll forward, backward, or to quit the command. For example, here is an example of what a ps command might look like: 

crash> ps
   PID    PPID  CPU   TASK    ST  %MEM   VSZ   RSS  COMM
      0      0   0  c030a000  RU   0.0     0     0  [swapper]
      1      0   0  cff98000  IN   0.2  1412   468  init
      2      1   0  c1446000  IN   0.0     0     0  [keventd]
      3      1   0  cfffa000  IN   0.0     0     0  [kapm-idled]
      4      0   0  cfff8000  IN   0.0     0     0  [ksoftirqd_CPU0]
      5      0   0  cffee000  IN   0.0     0     0  [kswapd]
      6      0   0  cffec000  IN   0.0     0     0  [kreclaimd]
      7      0   0  c1826000  IN   0.0     0     0  [bdflush]
      8      0   0  c1824000  IN   0.0     0     0  [kupdated]
      9      1   0  cff90000  IN   0.0     0     0  [mdrecoveryd]
     13      1   0  cf07a000  IN   0.0     0     0  [kjournald]
     89      1   0  ce804000  IN   0.0     0     0  [khubd]
    184      1   0  ce4d4000  IN   0.0     0     0  [kjournald]
    572      1   0  cd938000  IN   0.0   440    48  dhcpcd
    637      1   0  ce4a4000  IN   0.2  1476   612  syslogd
    642      1   0  cd92c000  IN   0.2  2092   432  klogd
    663      1   0  ce2bc000  IN   0.2  1564   612  portmap
    691      1   0  cd84a000  IN   0.3  1652   668  rpc.statd
    803      1   0  cd756000  IN   0.2  1400   452  apmd
    828      1   0  cd6c2000  IN   0.3 18024   684  ypbind
    830    828   0  cd76e000  IN   0.3 18024   684  ypbind
    831    830   0  cd71c000  IN   0.3 18024   684  ypbind
 
-- MORE --  forward: <SPACE>, <ENTER> or j  backward: b or k  quit: q

This default output scrolling behavior can be turned off by entering the following line in a .crashrc file located in either the $HOME or current directories: 

set scroll off

During runtime, the following commands (or their respective builtin aliases) can be used to turn the scrolling behavior off, and back on, again: 

crash> set scroll off
scroll: off
crash> set scroll on
scroll: on
crash> alias
ORIGIN   ALIAS    COMMAND
builtin  man      help 
builtin  ?        help 
builtin  quit     q 
builtin  sf       set scroll off
builtin  sn       set scroll on
builtin  hex      set radix 16 
builtin  dec      set radix 10 
builtin  g        gdb 
builtin  px       p -x 
builtin  pd       p -d 
builtin  for      foreach 
builtin  size     * 
builtin  dmesg    log
crash> sf
scroll: off
crash> sn
scroll: on
crash>

Alternatively, command output may be redirected to a pipe or to a file using standard shell redirection syntax. For examples: 

crash> task | grep uid
  uid = 3369, 
  euid = 3369, 
  suid = 3369, 
  fsuid = 3369,
crash> foreach bt > bt.all
crash> ps >> process.data
crash> kmem -p | grep slab > slab.pages
crash>

When a command's output is redirected to a pipe or file, the default /usr/bin/less behavior is turned off for that particular command.

Numerical Output

The default numerical output radix for non-pointer values is decimal, which is most often noticed when using the builtin gdb capability of printing formatted data structures. During runtime, the following commands (or their respective builtin aliases) can be used to toggle the output radix from decimal to hexadecimal, and back again: 

crash> set radix 16
output radix: 16 (hex)
crash> set scroll 19
output radix: 10 (decimal)
crash> alias
ORIGIN   ALIAS    COMMAND
builtin  man      help 
builtin  ?        help 
builtin  quit     q 
builtin  sf       set scroll off
builtin  sn       set scroll on
builtin  hex      set radix 16 
builtin  dec      set radix 10
builtin  g        gdb 
builtin  px       p -x 
builtin  pd       p -d 
builtin  for      foreach 
builtin  size     * 
builtin  dmesg    log
crash> hex
output radix: 16 (hex)
crash> dec
output radix: 10 (decimal)
crash>

Alternatively, the px or pd aliases coerce the "print" command p, to override the current output radix. For example, here the changing value of jiffies on a live system is printed using the current default radix, then in hexadecimal, and lastly in decimal: 

crash> p jiffies
jiffies = $4 = 69821055
crash> px jiffies
jiffies = $5 = 0x42963aa
crash> pd jiffies
jiffies = $6 = 69821656
crash>


< Prev Contents Next >
crash-whitepaper-1.0/doc/why_crash.html0000644000175000017500000001134510357525152020477 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Why Crash?

A current limitation of the Linux operating system is the lack of a built-in traditional UNIX-like kernel crash dump facility. This has been addressed by the Red Hat Netdump facility, the 2.4-based LKCD (Linux Kernel Crash Dump) kernel patch, and the Mission Critical Linux Mcore kernel patch. But the creation of kernel crash dump files is only half of the picture; a utility is required to be able to recognize the dumpfile format in order to read it, and to offer a useful set of commands to make sense of it.

Furthermore, to examine the contents of a live system's kernel internals from user space, the only readily available option has been to use gdb on /proc/kcore. While gdb is an incredibly powerful tool, it is designed to debug user programs, and is not at all "kernel-aware". Consequently, using gdb alone has limited usefulness when looking at kernel memory, essentially constrained to the printing of kernel data structures if the vmlinux file was built with the -g C flag, the disassembly of kernel text, and raw data dumps.

As far as kernel crash dump files are concerned, the Red Hat Netdump facility creates dump files that are readable by gdb, but aside from giving it the capability of displaying the panicking task's stack trace, it has the same constraints as when reading /proc/kcore. However, gdb cannot read LKCD or Mcore dump files.

That being the state of things, the crash utility was developed as a convenient means to cover all four bases, i.e., the three dumpfile formats as well as live systems. Moreover, it is also designed to be easily enhanced to suit the specific needs of the kernel developers or analysts using it; the builtin command set can easily be extended or enhanced, and external command modules may be written and dynamically attached.


< Prev Contents Next >
crash-whitepaper-1.0/doc/usage.html0000644000175000017500000004624410357525152017622 0ustar troyhebetroyhebe

White Paper: Red Hat Crash Utility
< Prev Contents Next >

Crash Usage: A Case Study

The steps taken to debug a kernel crash dump are not etched in stone, and the crash commands used to debug a kernel issue vary according to the problem exhibited. The section contains of a case study that shows how the capabilities of the crash utility were used to to debug a specific kernel problem. However, before doing so, it should be noted that the following commands are typically the most commonly-used:

bt Display the backtrace of the current context, or as specified with arguments. This command is typically the first command entered after starting a dumpfile session. Since the initial context is the panic context, it will show the function trace leading up to the kernel panic. bt -a will show the trace of the active task on each CPU, since there may be an interrelationship between the panicking task on one CPU and the running task(s) on the other CPU(s). When bt is given as the argument to foreach. displays the backtraces of all tasks.
struct Print the contents of a data structure at a specified address. This command is so common that it is typically unnecessary to enter the struct command name on the command line; if the first command line argument is not a crash or gdb command, but it is the name of a known data structure, then all the command line arguments are passed to the struct command. So for example, the following two commands yield the same result: 

crash> struct vm_area_struct d3cb2600


crash> vm_area_struct d3cb2600
set Set a new task context by PID, task address, or cpu. Since several crash commands are context-sensitive, it's helpful to be able to change the context to avoid having to pass the PID or task address to those context-sensitive commands in order to access the data of a task that is not the current context.
p Prints the contents of a kernel variable; since it's a gateway to the print command of the mbedded gdb module, it can also be used to print complex C language expressions.
rd Read memory, which may be either kernel virtual, user virtual, or physical, and display it several different formats and sizes.
ps Lists basic task information for each process; it can also display parent and child hierarchies.
log Dump the kernel log_buf, which often contains clues leading up to a subsequent kernel crash.
foreach Execute a crash command on all tasks, or those specified, in the system; can be used with bt, vm, task, files, net, set, sig and vtop.
files Dump the open file descriptor data of a task; most usefully, the file, dentry and inode structure addresses for each open file descriptor.
vm Dump the virtual memory map of a task, including the vital information concerning each vm_area_struct making up a task's address space. It can also dump the physical address of each page in the address space, or if not mapped, its location in a file or on the swap device.

A Case Study: "kernel BUG at pipe.c:120!"

Upon bringing up a crash session, a great deal of information can be gained just by the invocation data. Here is what what displayed in this particular case: 

...
      KERNEL: vmlinux-2.4.9-e.10.13enterprise-g
    DUMPFILE: vmcore-incomplete
        CPUS: 2
        DATE: Mon Feb 17 08:20:56 2003
      UPTIME: 4 days, 20:04:41
LOAD AVERAGE: 0.95, 1.04, 1.25
       TASKS: 110
    NODENAME: testbox.redhat.com
     RELEASE: 2.4.9-e.10.13enterprise
     VERSION: #1 SMP Mon Feb 3 12:59:26 EST 2003
     MACHINE: i686  (2788 Mhz)
      MEMORY: 6 GB
       PANIC: "kernel BUG at pipe.c:120!"
         PID: 20571
     COMMAND: "imp"
        TASK: d1566000
         CPU: 1
       STATE: TASK_RUNNING (PANIC)

crash>

In this case the PANIC string "kernel BUG at pipe.c:120!" points to the exact kernel source code line at which the panic occurred.

Then, getting a backtrace of panicking task is typically the first order of the day: 

crash> bt
PID: 20571  TASK: d1566000  CPU: 1   COMMAND: "imp"
 #0 [d1567e44] die at c010785c
 #1 [d1567e54] do_invalid_op at c0107b2c
 #2 [d1567f0c] error_code (via invalid_op) at c01073de
    EAX: 0000001d  EBX: ed87b2e0  ECX: c02f6064  EDX: 00005fa1  EBP: 00001000
    DS:  0018      ESI: f640e740  ES:  0018      EDI: 00001000
    CS:  0010      EIP: c0150b6d  ERR: ffffffff  EFLAGS: 00010292
 #3 [d1567f48] pipe_read at c0150b6d
 #4 [d1567f6c] sys_read at c01468d4
 #5 [d1567fc0] system_call at c01072dc
    EAX: 00000003  EBX: 0000000a  ECX: 40b4e05c  EDX: 00002000
    DS:  002b      ESI: 00002000  ES:  002b      EDI: 40b4e05c
    SS:  002b      ESP: bffe9e88  EBP: bffe9eb8
    CS:  0023      EIP: 40aaa1d4  ERR: 00000003  EFLAGS: 00000286

The backtrace shows that the call to die() was generated by an invalid_op exception. The exception was caused by the BUG() call in the pipe_read() function: 

        if (count && PIPE_WAITING_WRITERS(*inode) &&
                !(filp->f_flags & O_NONBLOCK)) {
                /*
                 * We know that we are going to sleep: signal
                 * writers synchronously that there is more
                 * room.
                 */
                wake_up_interruptible_sync(PIPE_WAIT(*inode));
                if (!PIPE_EMPTY(*inode))
                        BUG();
                goto do_more_read;
        }

In the code segment above, the pipe_read() code has previously down'd the semaphore of the inode associated with the pipe, giving it exclusive access. It had read all data in the pipe, but still needed more to satisfy the count requested. Finding that there was a writer with more data -- and who was waiting on the semaphore -- it woke up the writer. However, after doing the wakeup, it did a sanity-check on the pipe contents, and found that it was no longer empty -- which is theoretically impossible since it was still holding the semaphore. It appeared that the writer process wrote to the pipe while the reader process still had exclusive access -- somehow overriding the semaphore.

Since the semaphore mechanism was seemingly not working, it was first necessary to look at the actual semaphore structure associated with the pipe's inode. This first required looking at the first argument to the pipe_read() function; the whatis command shows that it is a struct file pointer: 

crash> whatis pipe_read
ssize_t pipe_read(struct file *, char *, size_t, loff_t *);
crash>

Using the bt -f option, each frame in the backtrace is expanded to show all stack data in the frame. Looking at the expansion of the sys_read() frame, we can see that the last thing pushed on the stack before calling pipe_read() was the file pointer address of edf3f740: 

...
#3 [d1567f48] pipe_read at c0150b6d
    [RA: c01468d6  SP: d1567f4c  FP: d1567f6c  SIZE: 36]
    d1567f4c: c026701c  00000078  fffffff2  00001000
    d1567f5c: 00000000  edf3f740  ffffffea  00002000
    d1567f6c: c01468d6
 #4 [d1567f6c] sys_read at c01468d4
    [RA: c01072e3  SP: d1567f70  FP: d1567fc0  SIZE: 84]
    d1567f70: edf3f740  40b4f05c  00002000  edf3f760
    d1567f80: c03683d0  fffffffb  00000001  c0120d3b
    d1567f90: 00000046  00000046  0000000b  c0350960
    d1567fa0: 0000000b  f639eb00  c0108e0e  00000020
    d1567fb0: d1566000  00002000  40b4e05c  bffe9eb8
    d1567fc0: c01072e3
...

The task at hand is finding the inode containing the suspect semaphore from the file structure address. The file structure's f_dentry member points to its dentry structure, whose d_inode member in turn points to the pipe's inode. The struct command can be used to dump the complete contents of a data structure at a given address; by tagging the .member onto the structure name, we can print just the member desired. By following the structure chain, the inode address can be determined like so: 

crash> struct file.f_dentry edf3f740
  f_dentry = 0xdb0ec440,
crash> struct dentry.d_inode db0ec440
  d_inode = 0xf640e740,
crash> struct inode.i_sem f640e740
  i_sem = {
    count = {
      counter = 2
    },
    sleepers = 0,
    wait = {
      lock = {
        lock = 1
      },
      task_list = {
        next = 0xf640e7ac,
        prev = 0xf640e7ac
      }
    }
  },
crash>

The dump of the semaphore structure above showed the problem: the counter value of 2 is illegal. It should never be greater than 1; in this case a value of 2 allows two successful down operations, i.e., giving two tasks access to the pipe at the same time.

(As an aside, determining the inode address above could also be accomplished by using the context-sensitive files command, which dumps the associated file, dentry and inode structure addresses for each open file descriptor of a task. The dumped file descriptor list would contain one with a reference to the file structure at edf3f740, and would also show the associated inode address of f640e740.)

Before getting a dumpfile, this same panic had occurred several times. It was erroneously presumed that the problem was in the pipe-handling code, but it was eventually determined not to be the case. By instrumenting a kernel with debug code, the starting counter value of a pipe was found to be 3. Compounding that problem was the fact that the inode slab cache is one of a few special cases that presume that the freed inode's contents are left in a legitimate state so that they do not have to be completely reinitialized with each subsequent reallocation. So when the pipe's inode was created, it received an inode with a bogus counter value.

Confirming the existence of bogus inode structures in the slab cache was a multi-stepped procedure. Using the command kmem command to access the inode slab cache, we can get the addresses of all free and currently-allocated inodes. Since there are typically several thousand inodes, the output is extremely verbose, but here is the beginning of it: 

crash> kmem -S inode_cache
CACHE    NAME                 OBJSIZE  ALLOCATED     TOTAL  SLABS  SSIZE
c7666564 inode_cache              448      11563     12339   1371     4k
SLAB      MEMORY    TOTAL  ALLOCATED  FREE
d1d82000  d1d82040      9          9     0
FREE / [ALLOCATED]
  [d1d82040]
  [d1d82200]
  [d1d823c0]
  [d1d82580]
  [d1d82740]
  [d1d82900]
  [d1d82ac0]
  [d1d82c80]
  [d1d82e40]
SLAB      MEMORY    TOTAL  ALLOCATED  FREE
f4e52000  f4e52040      9          7     2
FREE / [ALLOCATED]
   f4e52040  (cpu 1 cache)
   f4e52200  (cpu 1 cache)
  [f4e523c0]
  [f4e52580]
  [f4e52740]
  [f4e52900]
  [f4e52ac0]
  [f4e52c80]
  [f4e52e40]
...

In the truncated output above, all of the inode address in the slab cache are dumped; the ones currently in use are surrounded by brackets, the free ones are not. So, for example, the inodes at addresses f4e52040 and f4e52200 are free; the others are not. The full output was piped to a script that pulled out just the free inode addresses (i.e., output lines starting with three spaces), and redirected them into a file. The file was modified to be a crash input file by making each extracted inode address to be the arguments of the struct command, using its short-cut method that allows the dropping of the struct command name; therefore the input file contained hundreds of crash commands of the form: 

inode.i_sem f4e52040
inode.i_sem f4e52200
inode.i_sem f5cdc040
inode.i_sem f5cdc200
inode.i_sem f5cdc3c0
inode.i_sem f5cdc580
...

Note that the struct command would be used by default above, as documented in its help page; if the first command line argument is not a crash or gdb command, but it is the name of a known data structure, it passes the arguments to the struct command.

Using the capability of feeding an input file, in this case consisting of hundreds of short-cut struct commands like those above, the output was again quite verbose, consisting of structure member dumps of the form: 

crash> < input.file
crash> inode.i_sem f4e52040
  i_sem = {
    count = {
      counter = 1
    },
    sleepers = 0,
    wait = {
      lock = {
        lock = 1
      },
      task_list = {
        next = 0xf4e520ac,
        prev = 0xf4e520ac
      }
    }
  },
crash> inode.i_sem f4e52200
  i_sem = {
    count = {
      counter = 1
    },
    sleepers = 0,
    wait = {
      lock = {
        lock = 1
      },
      task_list = {
        next = 0xf4e5226c,
        prev = 0xf4e5226c
      }
    }
  },
...

However, it was a simple matter of piping the output to grep, and looking for counter values not equal to 1: 

crash> < input.file | grep counter | grep -v "= 1"
      counter = 3
      counter = 3
      counter = 3
      counter = 3
crash>

This turned out to be the smoking gun. Another round of debugging with an instrumented kernel that trapped attempts to free an inode with a semaphore counter of 3 caught the perpetrator in the act.

< Prev Contents Next >