crash-whitepaper-1.0/ 0000755 0001750 0001750 00000000000 10357527750 015057 5 ustar troyhebe troyhebe crash-whitepaper-1.0/.originals/ 0000755 0001750 0001750 00000000000 10357524144 017116 5 ustar troyhebe troyhebe crash-whitepaper-1.0/.originals/problems.html 0000644 0001750 0001750 00000040216 10357524144 021632 0 ustar troyhebe troyhebe
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|||||||||||||||||||||||||||
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|||||||||||
White Paper: Enhanced Console Accessby Michael K. Johnson Copyright © 1999 by Red Hat Software, Inc.
Content
AbstractRed Hat Linux 6.0 solves two problems that are small but annoying for experienced users, but puzzling and even debilitating for users new to Linux or UNIX. The solutions operate without user intervention, but curious users may wish to know how they work—and perennial tinkerers and conscientious systems administrators can manipulate the new features that provide the solutions.
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|||||||||||||||||||||||||||
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
||||||||||||||||||||||||||||||||||||||||||||||
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|||||||||||||||||||||||||||
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|||||||||||
White Paper: Enhanced Console Accessby Michael K. Johnson Copyright © 1999 by Red Hat Software, Inc.
Content
AbstractRed Hat Linux 6.0 solves two problems that are small but annoying for experienced users, but puzzling and even debilitating for users new to Linux or UNIX. The solutions operate without user intervention, but curious users may wish to know how they work—and perennial tinkerers and conscientious systems administrators can manipulate the new features that provide the solutions.
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|||||||||||||||||||||||||||
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
![]() |
![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() |
![]() |
![]() |
|||
![]() |
|||
![]() |
![]() |
![]() |
|
![]() |
![]() ![]() ![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Red Hat Docs > Whitepapers > |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
||||||||||||||||||||||||||||||||||||||||||||||
|
|
Copyright © 2002 Red Hat, Inc. All rights reserved. Search by Google Careers at Red Hat : Legal statement : Privacy statement : Your Account : Contact Red Hat
|
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
Red Hat Advanced Server 2.1 kernel rebuild procedure
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: alias
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: ascii
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: bt
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: btop
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: dev
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: dis
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: eval
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: exit
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: extend
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: files
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: foreach
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: fuser
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: gdb
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: help
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: input
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: irq
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: CRASH_COMMAND_NAME_HERE
|
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: log
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: mach
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: mod
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: mount
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: net
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: output
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: p
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: *
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: ps
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: pte
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: ptob
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: ptov
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: q
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: rd
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: repeat
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: runq
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: search
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: set
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: sig
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: struct
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: swap
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: sym
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: sys
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: task
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: timer
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: union
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: vm
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: vtop
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: waitq
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: whatis
|
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
Contents | ||
![]() |
help page: wr
|
![]() |
||
Contents | ||
![]() |
![]() |
||
Contents | ||
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Build ProcedureStarting with the Red Hat Enterprise Linux 3 release, the crash utility is automatically installed during system installation if the Development Tools package set is selected. However, for all other kernel versions, or if it was not selected during system installation, the binary RPM can be installed, or if desired, the sources re-built and installed. If the crash utility is not pre-installed, and if all dependencies are met on the target system, install the binary RPM like so:
The crash executable will be installed in the /usr/bin directory. Alternatively, the crash source code can be rebuilt. The crash utility's source files come packaged in two typical formats, a source RPM file, or a compressed tar image. So, for example, crash version 3.7-1 can be built from either crash-3.7-1.src.rpm or crash-3.7-1.tar.gz. In either case, the source file layout consists of a top-level directory containing a set of crash-specific files, a compressed tar image containing the full, unmodified, gdb source tree, and a small number of modified gdb files required to merge the two entities. The build procedure does the following:
Building from the source RPMTo build from the source RPM, install the crash-3.7-1.src.rpm, cd to the appropriate SPECS directory, and build the package:
Lastly, install the binary RPM, which copies the crash executable to the /usr/bin directory:
Building from the tar imageTo build from the compressed tar image, simply uncompress/extract the source files, cd into the resultant source directory, and enter make:
The resultant crash executable will be located in the current top-level source directory. Install it in /usr/bin by entering:
|
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | |
![]() |
GNU Free Documentation License Version 1.2, November 2002 Copyright (C) 2000,2001,2002 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. 0. PREAMBLE The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others. This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software. We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference. 1. APPLICABILITY AND DEFINITIONS This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law. A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language. A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them. The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none. The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words. A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque". Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only. The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text. A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition. The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License. 2. VERBATIM COPYING You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3. You may also lend copies, under the same conditions stated above, and you may publicly display copies. 3. COPYING IN QUANTITY If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects. If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages. If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public. It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document. 4. MODIFICATIONS You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version: A. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission. B. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement. C. State on the Title page the name of the publisher of the Modified Version, as the publisher. D. Preserve all the copyright notices of the Document. E. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices. F. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below. G. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice. H. Include an unaltered copy of this License. I. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence. J. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission. K. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein. L. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles. M. Delete any section Entitled "Endorsements". Such a section may not be included in the Modified Version. N. Do not retitle any existing section to be Entitled "Endorsements" or to conflict in title with any Invariant Section. O. Preserve any Warranty Disclaimers. If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles. You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard. You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one. The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version. 5. COMBINING DOCUMENTS You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers. The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work. In the combination, you must combine any sections Entitled "History" in the various original documents, forming one section Entitled "History"; likewise combine any sections Entitled "Acknowledgements", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorsements". 6. COLLECTIONS OF DOCUMENTS You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects. You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document. 7. AGGREGATION WITH INDEPENDENT WORKS A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright resulting from the compilation is not used to limit the legal rights of the compilation's users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document. If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document's Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate. 8. TRANSLATION Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail. If a section in the Document is Entitled "Acknowledgements", "Dedications", or "History", the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title. 9. TERMINATION You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 10. FUTURE REVISIONS OF THIS LICENSE The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/. Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation. |
![]() |
||
< Prev | Contents | |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
The Command SetEach crash command generally falls into one of the following categories: The remainder of this section breaks the command set into categories, and gives a short description of each command in that category. However, for complete details and examples, recall that the crash utility has a self-contained help page for each command; to view the full help page, click on the command name next to its description below. Symbolic Display of Kernel Text or DataThe following commands typically take full advantage of the power of gdb to display kernel data structures symbolically.
System StateThe majority of crash commands come from the following set of "kernel-aware" commands, which delve into various kernel subsystems on a system-wide or per-task basis. The task-specific commands are context-sensitive, meaning that they act upon the current context unless a PID or task address is specified as an argument.
Utility FunctionsThe following commands are a set of useful helper commands serving various purposes, some simple, others quite powerful.
Session Control CommandsThe following commands typcally aid in the efficient running of a crash session.
|
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Crash ContextUpon a successful invocation of a crash session, one of the existing Linux tasks is selected as the current context. It is important to be aware of the current context because several crash commands are "context-sensitive", meaning that the command is executed from the view-point of the current context. Therefore, the output of context-sensitive commands can vary depending upon which context is current. Upon invocation of a crash session, the selection of the current context is based upon the following criteria: On dumpfiles:
On a live system:
The current context selection is shown in the session invocation data. For example, here is a session begun on a dumpfile that was created when an insmod task's attempt to install a module resulted in an "oops" violation:
During runtime, the current context can always be displayed by entering the set command with no arguments:
Changing the Crash ContextThe current context can be changed to a new task via the set command. Either of two "handles" may be used to identify a task, the PID number, or the kernel address of the task's task_struct. For example:
Alternatively, the current context can be set to the task running on a given CPU number, or back to the panicking task. Using the same dumpfile session shown above, in which there is only one CPU, the original context may be restored using the -c CPU-number or the -p ("panic task") options:
Context-Sensitive CommandsIt is important to be aware that several crash commands are context-sensitive. For example, the files command displays data about the open files of a task. If it is issued with no arguments, it displays the open files data of the current context. In this example, the current context happens to be PID 642, the klogd daemon:
However, if the files command is issued with either of the two task handles as an argument, then it will display the open files data of the specified task. In this example, PID 12731 is specified:
This type of context-sensitive behaviour is also exhibited by the vm, bt, sig, set, net and task commands. Unless a PID or task address is specified as an argument, the output will reflect data concerning the current context. Other commands may simply default to the current context. For example, the rd command can read memory from an address that is specified as a user-space address. Since the rd command does not accept a PID or task address as an argument, it would be necessary to be aware that the user-space access will come from the address space of the current context. |
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Command ExtensionsFrequently users wish to add an additional option to an existing crash command, or add a brand new command, in order to address a kernel issue they are debugging or developing. For those reasons, the crash utility was designed with extensibility in mind. There are two ways to add new functionality:
This section consists of a quick guide that describes how to get started using both methods. Adding new code and compiling it into the crash executableThe current set of crash commands can be seen by entering the help command with no arguments:
For each command in the menu above, there is an entry in a data structure in the file global_data.c, which is located in the top-level directory of the crash source code tree:
Each entry consists of the following simple data structure:
The structure members consist of:
For a newly-written command to appear in the help menu, it simply requires a reference to it in the structure above. To test a new command without adding it to the menu, use the hidden "test" command, found in test.c:
The test command contains the basic template used by crash commands to accept dash-arguments, which are fielded by the getopt() routine, while all other command line arguments are fielded in the while loop. To add an option to the test command (or any other existing command), simply fit it into the appropriate argument-gathering mechanism. Or, for that matter, if no arguments are required, put the functionality at the end of the command's function. Here is a trivial example of a change to the cmd_test() function, with the modifications highlighted:
Then re-compile crash by entering make:
Then run the test command with the tree possible argument types:
The easiest way to implement a new command's functionality is to use an existing command as a template. There is a broad range of utility routines that handle argument strings, read memory, access data structures and their members, display data, and so on, that obviate the need to reinvent the wheel to accomplish a task. Look at what other similar commands do, and copy their mechanisms. Creating a shared object library and loading it with extendWhile adding a new command and/or command option in the manner above is useful, it would require the same integration mechanism with each subsequent release of the crash utility. Since that could become tedious, another extension mechanism exists in which share objects containing one or more crash commands can be written, and then dynamically attached to a running crash session, using the extend command. Once loaded, the command(s) in the shared object library automatically appear in the help menu, as if they were compiled into the crash executable. As an quick aid in creating a shared object, the help page for the extend contains an example C program tagged onto the end, which adds a new echo command (which simply echoes back all arguments). The C program piece can be cut and pasted into a file, say extlib.c for example, and then compiled like so:
The resultant extlib.so file may be dynamically linked into crash during runtime using the extend command:
Or, to automatically load the shared library during crash session initialization, put the following string into a .crashrc file located in the current directory, or in the user's $HOME directory:
Here is the help menu once the library is loaded; note the integration of the new echo command:
With this extension mechanism, the most that would be required to use the shared library with subsequent versions of crash would be a simple re-compile of the extlib.c file. |
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Enhanced Console Access |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Builtin HelpReadily available help information is built into the crash utility. During a session, entering the help command with no argument shows the following menu:
Each command has its own man-like help page, which can be viewed by clicking on the command name above. Each help page details the syntax of the command and its available options, a description of the command in general, a description of each option, and a set of examples. During a crash session, a command's help page can be displayed by entering help followed by the command name. So, for example, to get help on how to use the set command:
If for some reason a crash session cannot be invoked, but help information for a particular crash command is desired, the same help page can be displayed from a shell command line using the -h option to crash:
Lastly, help concerning command input and output can be displayed by entering help input or help output during runtime, or crash -h input or crash -h output from a shell command line. |
![]() |
||
< Prev | Contents | Next > |
![]() |
by David Anderson
<anderson@redhat.com>
Copyright © 2003 by Red Hat Software, Inc.
Permission is granted to copy, distribute and/or modify this document under the terms of
the GNU Free Documentation License, Version 1.2 or any later version published by the
Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is included in the section entitled "GNU Free Documentation License".
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
The Red Hat crash analysis utility is loosely based on the SVR4 UNIX crash command, but has been significantly enhanced by completely merging it with the GNU gdb debugger. The marriage of the two effectively combines the kernel-specific nature of the traditional UNIX crash utility with the source code level debugging capabilities of gdb. The utility can be used to investigate:
The crash utility is designed to be independent of Linux version dependencies. When new kernel source code impacts the correct functionality of crash and its command set, the utility will be updated to recognize new kernel code changes while maintaining backwards compatibility with earlier releases.
![]() |
Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Command InputUpon a successful session invocation on a dump file or a live kernel, the crash> prompt will appear. Interactive crash commands are gathered using the GNU readline library, taking advantage of its command line history mechanism, and its vi or emacs command line editing modes. Commands may also be issued to crash from a file. Command Line HistoryThe command line history consists of a numbered list of previously-run commands. The full list of commands may be viewed by entering h at any time. For example:
Commands in the history list may be re-run in the following manners
Command Line EditingThe command line editing mode may be set to either vi (the default) or emacs. The mode may set in the following manners, listed in increasing order of precedence:
Given either editing mode, any previously entered command line can be brought back by entering the mode-specific key-stroke(s), the command line edited using the proper mode, and then run by hitting ENTER. Command Line Input from a FileAn input file consisting of a list of commands may be fed to crash in the following manners:
In all of the three cases above, after the list of commands in the file have completed, the crash> prompt will appear and commands may then be entered interactively (unless one of the file commands happens to be the exit command). Numerical ArgumentsNumerical arguments are typically presumed to be decimal unless the argument contains an a, b, c, d, e or f. In those cases, the preceding 0x is not required. For hexadecimal numbers that do not contain one of those 6 characters, the preceding 0x is required. So, for example, a value of 1 gigabyte would have to be expressed as 0x40000000, whereas 3 gigabytes could be expressed as c0000000. It should be noted that several commands will only accept hexadecimal numerical arguments. For example, the rd ("read") command only accepts hexadecimal addresses. Therefore a read from user address of 0x40017000 could be entered as:
|
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
InvocationWhen crash is run on a dumpfile, at least two arguments are always required:
When crash is run on a live system, /dev/mem is used as the memory image. Therefore, only the kernel object filename is required:
Furthermore, when crash is run on a live system, the vmlinux argument is not required when the kernel object file is located in any of the following locations:
When the vmlinux file is not entered on the command line, a search will be made in all of the directories above until a kernel object file is found that contains a version string that matches the running kernel, as indicated by /proc/version. If a matching kernel is found, then crash may be invoked on a live system simply by entering:
In the examples above, it is presumed that the vmlinux kernel has been built with the -g C flag, which traditionally has not been done by default. To address this requirement, Red Hat Enterprise Linux 3 (RHEL 3) and Red Hat Enterprise Linux 4 (RHEL 4) kernels are now built with -g C flag. The manner of accessing the debug data for RHEL 3 and RHEL 4 kernels is described in the following two sections. Unfortunately, since Red Hat Advanced Server 2.1 kernels are not built with -g, the kernel must be rebuilt; directions for rebuilding Red Hat Advanced Server 2.1 kernels can be found here. RHEL-3 KernelsIn RHEL 3, the vmlinux kernel debug information is stripped and stored in a separate debuginfo file. The stripped vmlinux file in /boot has an embedded link to its associated debuginfo file in /usr/lib/debug/boot, so that the crash utility (and the built-in gdb module) knows where to find it:
The debuginfo files for a specific kernel <release> come from a separate RPM that must be installed for the crash utility to work. For example, the i686 RPM for the examples above would be named kernel-debuginfo-<release>.i686.rpm, and would install the debuginfo file for all three of the kernel flavors. For example, to run crash on a live system, the associated debuginfo package must be installed:
Accordingly, if the running kernel's vmlinux file is in one the search locations above, and its associated debuginfo file is located in the /usr/lib/debug/boot directory or in the current directory from which crash is invoked, no arguments are required to run on a live system:
However, if the linked debuginfo file is not in either of those locations, it can be added to the crash command line along with the vmlinux filename. So, for example, if the debuginfo file was located in /tmp:
For analyzing dumpfiles however, the vmlinux file name must be on the command line along with the dumpfile name, as in the following examples:
or if the debuginfo file is not in the standard location:
RHEL-4 KernelsThe procedure has been made much simpler for RHEL-4 kernels. The kernel is built with the -g flag, and the resultant vmlinux file is stored in the associated debuginfo package. After installing the debuginfo package, the vmlinux file for each kernel flavor of a given RHEL 4 release will be installed in the directory named: /usr/lib/debug/lib/modules/<release><flavor>/vmlinuxwhere for i686 kernels, <flavor> can be either hugemem, smp, or nothing (for uniprocessor kernels). For example:
Once the debuginfo package is installed, crash can be invoked on the live system with no arguments, because the vmlinux file will be found automatically:
To run crash on a dumpfile, however, the appropriate vmlinux file and the dumpfile name must both be on the command line, as in:
Kernels built without -g flagIf the running kernel was not built with the -g C flag, then it is necessary to rebuild a kernel of the same configuration with the -g C flag. The essential change done by this kernel rebuild task is a modification of top-level Makefile of the kernel source tree, such that the CFLAGS definition contains the -g flag. For example, this is the line that must be changed:
by adding the -g flag:
For example, since Red Hat Advanced Server 2.1 kernels are not built with -g, a kernel rebuild is required. For a detailed example of how to rebuild a Red Hat Advanced Server 2.1 kernel with the -g flag, please refer to these directions. Upon rebuilding the kernel, a new vmlinux file will be created that contains the debug data required by crash. However, the symbol values will not match those of the running or dumped kernel. To deal with this inequity, the actual symbol values can be gathered from either the original non-debug vmlinux file or from its associated System.map file. That being the case, two arguments must be supplied to crash to fully describe the running/dumped kernel, the newly-created vmlinux file compiled with -g, as well as a source of the real symbol values. So, for example, if the vmlinux file built with -g were renamed to vmlinux.dbg, the invocation line would look like this on a live system:
The -S argument above is simply an alternative to entering the default /boot/System.map string.
Similarly, when looking at a dumpfile, two arguments are required to describe the dumped kernel, along with the vmcore image:
Again, for a detailed example of how to rebuild a Red Hat Advanced Server 2.1 kernel with the -g flag, refer to these directions. Invocation outputThe arguments may be entered in any order. If the file arguments are not in the current directory, absolute pathnames must be used. When in doubt, simply enter crash -h to get an explanation of the command line arguments:
Given that all invocation arguments are in order, here is an example of a successful invocation on a dumpfile, running a kernel that was built with -g, along with a vmcore dump file was created by the Red Hat Netdump facility:
This next example shows the output when the panicking kernel was not built with -g. In this case, a similar kernel type was built with -g, and the resultant kernel object file was renamed as vmlinux.dbg. Note that there will be a message concerning the patching of gdb data; this indicates that the non-matching symbol values from the vmlinux.dbg are being over-written by the correct symbol values found in the original vmlinux file:
Invocation on a live system looks essentially the same, except that the DUMPFILE will be indicated as /dev/mem. In the following example, no arguments were entered, because the running RHEL 3 kernel was found in the /boot directory, and its associated debuginfo file in the /usr/lib/debug/boot directory. The debuginfo file is listed next to the DEBUGINFO tag:
Invocation ErrorsInvocation errors will cause the crash session to abort upon initialization. Typically they occur as the result of one of the following reasons:
|
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
PrerequisitesThe crash utility has the following prerequisites:
|
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
New Pageyada, yada, yada...
yada, yada, yada... |
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Command Outputcrash commands can often be verbose, and it's helpful to control the output, as well as to be able to scroll backwards to view previous command output. So, by default, command output that would overflow the user's display screen is piped to /usr/bin/less, along with a prompt line that informs the user how to scroll forward, backward, or to quit the command. For example, here is an example of what a ps command might look like:
This default output scrolling behavior can be turned off by entering the following line in a .crashrc file located in either the $HOME or current directories:
During runtime, the following commands (or their respective builtin aliases) can be used to turn the scrolling behavior off, and back on, again:
Alternatively, command output may be redirected to a pipe or to a file using standard shell redirection syntax. For examples:
When a command's output is redirected to a pipe or file, the default /usr/bin/less behavior is turned off for that particular command. Numerical OutputThe default numerical output radix for non-pointer values is decimal, which is most often noticed when using the builtin gdb capability of printing formatted data structures. During runtime, the following commands (or their respective builtin aliases) can be used to toggle the output radix from decimal to hexadecimal, and back again:
Alternatively, the px or pd aliases coerce the "print" command p, to override the current output radix. For example, here the changing value of jiffies on a live system is printed using the current default radix, then in hexadecimal, and lastly in decimal:
|
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Why Crash?A current limitation of the Linux operating system is the lack of a built-in traditional UNIX-like kernel crash dump facility. This has been addressed by the Red Hat Netdump facility, the 2.4-based LKCD (Linux Kernel Crash Dump) kernel patch, and the Mission Critical Linux Mcore kernel patch. But the creation of kernel crash dump files is only half of the picture; a utility is required to be able to recognize the dumpfile format in order to read it, and to offer a useful set of commands to make sense of it. Furthermore, to examine the contents of a live system's kernel internals from user space, the only readily available option has been to use gdb on /proc/kcore. While gdb is an incredibly powerful tool, it is designed to debug user programs, and is not at all "kernel-aware". Consequently, using gdb alone has limited usefulness when looking at kernel memory, essentially constrained to the printing of kernel data structures if the vmlinux file was built with the -g C flag, the disassembly of kernel text, and raw data dumps. As far as kernel crash dump files are concerned, the Red Hat Netdump facility creates dump files that are readable by gdb, but aside from giving it the capability of displaying the panicking task's stack trace, it has the same constraints as when reading /proc/kcore. However, gdb cannot read LKCD or Mcore dump files. That being the state of things, the crash utility was developed as a convenient means to cover all four bases, i.e., the three dumpfile formats as well as live systems. Moreover, it is also designed to be easily enhanced to suit the specific needs of the kernel developers or analysts using it; the builtin command set can easily be extended or enhanced, and external command modules may be written and dynamically attached. |
![]() |
||
< Prev | Contents | Next > |
![]() |
White Paper: Red Hat Crash Utility |
||
![]() |
||
< Prev | Contents | Next > |
![]() |
Crash Usage: A Case StudyThe steps taken to debug a kernel crash dump are not etched in stone, and the crash commands used to debug a kernel issue vary according to the problem exhibited. The section contains of a case study that shows how the capabilities of the crash utility were used to to debug a specific kernel problem. However, before doing so, it should be noted that the following commands are typically the most commonly-used:
A Case Study: "kernel BUG at pipe.c:120!"Upon bringing up a crash session, a great deal of information can be gained just by the invocation data. Here is what what displayed in this particular case:
In this case the PANIC string "kernel BUG at pipe.c:120!" points to the exact kernel source code line at which the panic occurred. Then, getting a backtrace of panicking task is typically the first order of the day:
The backtrace shows that the call to die() was generated by an invalid_op exception. The exception was caused by the BUG() call in the pipe_read() function:
In the code segment above, the pipe_read() code has previously down'd the semaphore of the inode associated with the pipe, giving it exclusive access. It had read all data in the pipe, but still needed more to satisfy the count requested. Finding that there was a writer with more data -- and who was waiting on the semaphore -- it woke up the writer. However, after doing the wakeup, it did a sanity-check on the pipe contents, and found that it was no longer empty -- which is theoretically impossible since it was still holding the semaphore. It appeared that the writer process wrote to the pipe while the reader process still had exclusive access -- somehow overriding the semaphore. Since the semaphore mechanism was seemingly not working, it was first necessary to look at the actual semaphore structure associated with the pipe's inode. This first required looking at the first argument to the pipe_read() function; the whatis command shows that it is a struct file pointer:
Using the bt -f option, each frame in the backtrace is expanded to show all stack data in the frame. Looking at the expansion of the sys_read() frame, we can see that the last thing pushed on the stack before calling pipe_read() was the file pointer address of edf3f740:
The task at hand is finding the inode containing the suspect semaphore from the file structure address. The file structure's f_dentry member points to its dentry structure, whose d_inode member in turn points to the pipe's inode. The struct command can be used to dump the complete contents of a data structure at a given address; by tagging the .member onto the structure name, we can print just the member desired. By following the structure chain, the inode address can be determined like so:
The dump of the semaphore structure above showed the problem: the counter value of 2 is illegal. It should never be greater than 1; in this case a value of 2 allows two successful down operations, i.e., giving two tasks access to the pipe at the same time. (As an aside, determining the inode address above could also be accomplished by using the context-sensitive files command, which dumps the associated file, dentry and inode structure addresses for each open file descriptor of a task. The dumped file descriptor list would contain one with a reference to the file structure at edf3f740, and would also show the associated inode address of f640e740.) Before getting a dumpfile, this same panic had occurred several times. It was erroneously presumed that the problem was in the pipe-handling code, but it was eventually determined not to be the case. By instrumenting a kernel with debug code, the starting counter value of a pipe was found to be 3. Compounding that problem was the fact that the inode slab cache is one of a few special cases that presume that the freed inode's contents are left in a legitimate state so that they do not have to be completely reinitialized with each subsequent reallocation. So when the pipe's inode was created, it received an inode with a bogus counter value. Confirming the existence of bogus inode structures in the slab cache was a multi-stepped procedure. Using the command kmem command to access the inode slab cache, we can get the addresses of all free and currently-allocated inodes. Since there are typically several thousand inodes, the output is extremely verbose, but here is the beginning of it:
In the truncated output above, all of the inode address in the slab cache are dumped; the ones currently in use are surrounded by brackets, the free ones are not. So, for example, the inodes at addresses f4e52040 and f4e52200 are free; the others are not. The full output was piped to a script that pulled out just the free inode addresses (i.e., output lines starting with three spaces), and redirected them into a file. The file was modified to be a crash input file by making each extracted inode address to be the arguments of the struct command, using its short-cut method that allows the dropping of the struct command name; therefore the input file contained hundreds of crash commands of the form:
Note that the struct command would be used by default above, as documented in its help page; if the first command line argument is not a crash or gdb command, but it is the name of a known data structure, it passes the arguments to the struct command. Using the capability of feeding an input file, in this case consisting of hundreds of short-cut struct commands like those above, the output was again quite verbose, consisting of structure member dumps of the form:
However, it was a simple matter of piping the output to grep, and looking for counter values not equal to 1:
This turned out to be the smoking gun. Another round of debugging with an instrumented kernel that trapped attempts to free an inode with a semaphore counter of 3 caught the perpetrator in the act. |
![]() |
||
< Prev | Contents | Next > |
![]() |