grokevt-0.4.1/0000755000175000017500000000000010770556770011357 5ustar timtimgrokevt-0.4.1/INSTALL0000644000175000017500000000302110377135440012372 0ustar timtimPrerequisites ============= In order to install this package, you must have the following already set up on your system: - Python 2.3+ - GNU Make In order to use this software you also need: - RegLookup Earlier versions of Python may work, but they have not yet been tested. To obtain RegLookup, please see the project page at: http://www.sentinelchicken.org/projects/reglookup/ You do not need to have RegLookup installed before installing this software, but you will need to have it installed and in your PATH whenever you run grokevt-builddb(1). Building ======== For most installations, this is as simple as: make which will create the build/ directory with all install-ready items neatly arranged. (If your system does not use GNU Make by default, you'll probably have to use `gmake' instead.) If you wish to change the path of the configuration files, you will want to instead set the ETC_PREFIX variable: make ETC_PREFIX=/path/to/etc This path defaults to '/usr/local/etc'. Installing ========== Installation is typically as simple as: make install If you wish to override any of the installation prefixes (please see the top-level make file for all available paths), you'll need to specify those on the command line as well. By default, everything installs somewhere under '/usr/local'. If you wanted to change this to '/usr', you could do: make PREFIX=/usr install If you just want to override the ETC_PREFIX, you need to override it during build *and* during install. (Sorry, this will be fixed at some point.) grokevt-0.4.1/LICENSE0000644000175000017500000003545110261571706012363 0ustar timtim GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS grokevt-0.4.1/Makefile0000644000175000017500000000235410261601463013004 0ustar timtim# $Id: Makefile 36 2005-07-02 21:12:19Z tim $ # This extracts the major and minor version of the python executable in # the current PATH. PYTHON_VERSION:=`echo 'import sys;\ print "%d.%d" % (sys.version_info[0],sys.version_info[1])' | python` # Installation prefixes. Change to install elsewhere. PREFIX=/usr/local LIB_PREFIX:=$(PREFIX)/lib/python$(PYTHON_VERSION)/site-packages BIN_PREFIX=$(PREFIX)/bin ETC_PREFIX=$(PREFIX)/etc/grokevt DOC_PREFIX=$(PREFIX)/share/doc/grokevt MAN_PREFIX=$(PREFIX)/man ################################################################################ BUILD=$(CURDIR)/build BUILD_BIN=$(BUILD)/bin BUILD_LIB=$(BUILD)/lib BUILD_ETC=$(BUILD)/etc BUILD_DOC=$(BUILD)/doc BUILD_TREE=$(BUILD_BIN) $(BUILD_LIB) $(BUILD_ETC) $(BUILD_DOC) SUB_DIRS=bin lib etc doc .PHONY: $(SUB_DIRS) clean export all: $(BUILD_TREE) $(SUB_DIRS) # XXX: This should be more generalized. install: all mkdir -p $(BIN_PREFIX) mkdir -p $(LIB_PREFIX) mkdir -p $(ETC_PREFIX) mkdir -p $(DOC_PREFIX) cp -r $(BUILD_BIN)/* $(BIN_PREFIX) cp -r $(BUILD_LIB)/* $(LIB_PREFIX) cp -r $(BUILD_ETC)/* $(ETC_PREFIX) cp -r $(BUILD_DOC)/* $(DOC_PREFIX) $(MAKE) -C doc install $(SUB_DIRS): $(MAKE) -C $@ $(BUILD_TREE): mkdir -p $@ clean: rm -rf $(BUILD) grokevt-0.4.1/README0000644000175000017500000000075310257575042012235 0ustar timtimGrokEVT is an collection of scripts built for reading Windows NT(tm) event log files. GrokEVT is released under the GNU GPL, and is implemented in Python. Installation ============ Please see the file INSTALL in this directory. Using ===== A guide on how to get started can be found in: doc/grokevt.txt or in the man page grokevt(7) after installation. Other ===== If you have any other questions, please see the project website: http://www.sentinelchicken.org/projects/grokevt/ grokevt-0.4.1/bin/0000755000175000017500000000000010770556770012127 5ustar timtimgrokevt-0.4.1/bin/Makefile0000644000175000017500000000156410602607641013561 0ustar timtim# $Id: Makefile 105 2007-03-29 01:00:17Z tim $ ################################################################################ ADDLOG=$(BUILD_BIN)/grokevt-addlog BUILDDB=$(BUILD_BIN)/grokevt-builddb DUMPMSGS=$(BUILD_BIN)/grokevt-dumpmsgs FINDLOGS=$(BUILD_BIN)/grokevt-findlogs PARSELOG=$(BUILD_BIN)/grokevt-parselog RIPDLL=$(BUILD_BIN)/grokevt-ripdll FILES=$(ADDLOG) $(BUILDDB) $(DUMPMSGS) $(FINDLOGS) $(PARSELOG) $(RIPDLL) all: $(FILES) $(ADDLOG): grokevt-addlog install -m 0755 grokevt-addlog $(ADDLOG) $(BUILDDB): grokevt-builddb install -m 0755 grokevt-builddb $(BUILDDB) $(DUMPMSGS): grokevt-dumpmsgs install -m 0755 grokevt-dumpmsgs $(DUMPMSGS) $(FINDLOGS): grokevt-findlogs install -m 0755 grokevt-findlogs $(FINDLOGS) $(PARSELOG): grokevt-parselog install -m 0755 grokevt-parselog $(PARSELOG) $(RIPDLL): grokevt-ripdll install -m 0755 grokevt-ripdll $(RIPDLL) grokevt-0.4.1/bin/grokevt-addlog0000755000175000017500000000344110427746170014762 0ustar timtim#!/bin/sh # This script takes a raw Windows event log and adds it to a # previously built database generated by grokevt-builddb. # See the man page for more details. # # Copyright (C) 2006 Timothy D. Morgan # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation version 2 of the # License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # $Id: grokevt-addlog 79 2006-05-08 22:55:52Z tim $ usage() { echo "USAGE:" 1>&2 echo " $0 " 1>&2 echo 1>&2 echo "This script takes a raw Windows event log and adds it to a" 1>&2 echo "previously built database generated by grokevt-builddb." 1>&2 echo "See the man page for more details." 1>&2 } if [ $# -ne 4 ]; then usage exit 1 fi DATABASE_DIR="$1" EVT_FILE="$2" NEW_TYPE="$3" BASE_TYPE="$4" if [ ! -d "$DATABASE_DIR" ]; then echo "ERROR: '$DATABASE_DIR' is not a directory." 1>&2 exit 2 fi if [ ! -r "$EVT_FILE" ]; then echo "ERROR: '$EVT_FILE' could not be read." 1>&2 exit 2 fi if [ -e "${DATABASE_DIR}/logs/${NEW_TYPE}.evt" ]; then echo "ERROR: A '$NEW_TYPE' log already exists." 1>&2 exit 2 fi if [ ! -d "${DATABASE_DIR}/services/${BASE_TYPE}" ]; then echo "ERROR: A '$BASE_TYPE' log does not exist." 1>&2 exit 2 fi cp "$EVT_FILE" "${DATABASE_DIR}/logs/${NEW_TYPE}.evt"\ || ( echo "ERROR: Could not copy log file." 1>&2; exit 3 ) ln -s "$BASE_TYPE" "${DATABASE_DIR}/services/${NEW_TYPE}"\ || ( echo "ERROR: Could not create symlink to BASE_TYPE." 1>&2; exit 3 ) grokevt-0.4.1/bin/grokevt-builddb0000755000175000017500000003307210770535307015137 0ustar timtim#!/usr/bin/env python # This script builds message databases for use by grokevt-parselog. # # Copyright (C) 2005-2008 Timothy D. Morgan # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation version 2 of the # License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # vi:set tabsize=4: # $Id: grokevt-builddb 109 2008-03-20 19:28:07Z tim $ import sys import os import re import popen2 import anydbm import select import grokevt # XXX: should these be changed to absolute paths discovered at install time? CURRENT_DB_VERSION=2 PATH_REGLOOKUP='reglookup' PATH_RIPDLL='grokevt-ripdll' REGPATH_EVENTLOG = None CONTROL_SET_ID_PATH='/Select/Current' missing_dlls = {} def usage(): sys.stderr.write("USAGE:\n") sys.stderr.write(" %s [-v] [-c CSID] \n\n" % os.path.basename(sys.argv[0])) sys.stderr.write("grokevt-builddb builds a database tree based on a\n") sys.stderr.write("single windows system for the purpose of event log\n") sys.stderr.write("conversion. Please see the man page for more\n") sys.stderr.write("information.\n") def unquoteString(s): chunks = re.split("\\\\x([0-9A-F]{2})", s); for i in range(1,len(chunks),2): chunks[i] = "%c" % int(chunks[i], 16) return ''.join(chunks) def windowsPathToUnixPath(path_list, variables, drives): ret_val = path_list.lower() for s in variables.items(): ret_val = ret_val.replace(s[0].lower(), s[1].lower()) ret_val = ret_val.replace("\\", "/") paths = ret_val.split(';') ret_val = [] for p in paths: # Some paths have been found with trailing nulls and garbage. p = p.split('\x00')[0] for s in drives.items(): if p.startswith(s[0].lower()): p = p.replace(s[0].lower(), s[1], 1) ret_val.append(p) return ret_val def regLookupStderrHandler(e, args): sys.stderr.write("WARNING: reglookup reported: %s" % e) def ripdllStderrHandler(e, args): sys.stderr.write(("WARNING: while reading %s and writing %s," +" grokevt-ripdll returned: %s")\ % (args[0], args[1], e)) class procWrapper: child = None files = None stderr_handler = None stderr_handler_args = None exe_name = None def __init__(self, args, stderr_handler, stderr_handler_args=()): self.stderr_handler = stderr_handler self.stderr_handler_args = stderr_handler_args self.exe_name = args[0] self.child = popen2.Popen3(args, 1) #self.child.tochild.close() self.files = [self.child.childerr,self.child.fromchild] def readline(self): l = '' # XXX: could this be simpler? while (l == '') and (len(self.files) > 0): r, w, e = select.select(tuple(self.files),(),tuple(self.files)) for fh in e: fh.close() for fh in self.files: if fh.closed: self.files.remove(fh) for fh in r: if fh.fileno() == self.child.childerr.fileno(): el = fh.readline() if el == '': self.files.remove(fh) else: self.stderr_handler(el, self.stderr_handler_args) else: l = fh.readline() if l == '': self.files.remove(fh) return l def wait(self): while self.readline() != '': continue child_ret = self.child.wait() if child_ret > 0: sys.stderr.write("WARNING: %s returned code: %d. Is it in your path?" % (self.exe_name, child_ret)) return child_ret def buildShortNamesPathsDicts(paths): ret_val1 = {} ret_val2 = {} for p in paths.keys(): if os.path.isfile(p): sn = os.path.basename(p).replace(':','_') ext = '.db' i = 2 while ret_val1.has_key(sn + ext): ext = '_%d.db' % i i += 1 ret_val1[sn+ext] = p ret_val2[p] = sn+ext else: # XXX: this might be handy later #missing_dlls[p] = paths[p] sys.stderr.write("WARNING: %s doesn't exist.\n" % p + "WARNING: (This may affect log output " + "for the following services: %s)\n" % paths[p]) return (ret_val1, ret_val2) def makeOutputDirectories(topdir, logs): dirs = [] for log in logs: dirs.append('%s/services/%s' % (topdir,log)) dirs.append('%s/messages' % topdir) dirs.append('%s/logs' % topdir) for p in dirs: if os.path.exists(p): if not os.path.isdir(p) or not os.access(p, os.W_OK): sys.stderr.write("ERROR: Access denied to '%s'.\n" % p) sys.exit(os.EX_CANTCREAT) else: os.makedirs(p, 0755) # Write DB version vf = file('%s/version' % topdir, 'w+') vf.write('%d\n' % CURRENT_DB_VERSION) vf.close() def writeServiceMapping(topdir, maps): for lt in maps.keys(): for t in maps[lt].keys(): db_file = "%s/services/%s/%s.db"%(topdir,lt,t) db = anydbm.open(db_file, "n", 0644) for si in maps[lt][t].items(): db[si[0]] = si[1] db.sync() db.close() def writeDBFiles(topdir, names2paths): for npi in names2paths.items(): outdb = "%s/messages/%s" % (topdir, npi[0]) db_call = (PATH_RIPDLL, npi[1], outdb) ripdll_proc = procWrapper(db_call, ripdllStderrHandler, (npi[1], outdb)) l = ripdll_proc.readline() while l != '': ripdllStderrHandler(l, (npi[1], outdb)) l = ripdll_proc.readline() ripdll_proc.wait() # Parse command line arguments CONTROL_SET_ID = None print_verbose = 0 next_is_cid = 0 argv_len = len(sys.argv) if argv_len < 3: usage() sys.stderr.write("ERROR: Requires at least 2 arguments.\n") sys.exit(os.EX_USAGE) else: for option in sys.argv[1:argv_len-2]: if next_is_cid: CONTROL_SET_ID = int(option) if CONTROL_SET_ID < 1: usage() sys.stderr.write("ERROR: CONTROL_SET_ID must be positive.\n") sys.exit(os.EX_USAGE) next_is_cid = 0 elif option == '-v': print_verbose = 1 elif option == '-c': next_is_cid = 1 else: usage() sys.stderr.write("ERROR: Unrecognized option '%s'.\n" % option) sys.exit(os.EX_USAGE) if next_is_cid: usage() sys.stderr.write("ERROR: -c requires parameter.\n") sys.exit(os.EX_USAGE) CONFIG_PROFILE=sys.argv[argv_len-2] PATH_OUTPUT=sys.argv[argv_len-1] if print_verbose: sys.stderr.write("INFO: Loading configuration...\n") config = grokevt.grokevtConfig(grokevt.PATH_CONFIG, CONFIG_PROFILE) if print_verbose: sys.stderr.write("INFO: Reading system registry for service information...\n") # Need to determine correct 'CurrentControlSet', if not specified at # command line if not CONTROL_SET_ID: logs_query=[PATH_REGLOOKUP, "-H", "-t", "DWORD", "-p", CONTROL_SET_ID_PATH, config.registry_path] reg_proc = procWrapper(logs_query, regLookupStderrHandler) l = reg_proc.readline() if len(l) == 0: sys.stderr.write("ERROR: Could not automatically " +"determine CONTROL_SET_ID\n") sys.exit(os.EX_IOERR) reg_proc.wait() CONTROL_SET_ID = int(l.split(',')[2], 16) CONTROL_SET_PATH='/ControlSet%.3d' % CONTROL_SET_ID REGPATH_EVENTLOG='%s/Services/Eventlog' % CONTROL_SET_PATH if print_verbose: sys.stderr.write("INFO: Using '%s' as CurrentControlSet path.\n" % CONTROL_SET_PATH) # Next, identify all log types by grabbing all keys under the eventlog key logs_query=[PATH_REGLOOKUP, "-H", "-t", "KEY", "-p", REGPATH_EVENTLOG, config.registry_path] reg_proc = procWrapper(logs_query, regLookupStderrHandler) log_types = {} prefix_len = len(REGPATH_EVENTLOG) l = reg_proc.readline() while l != '': l = l.split(',')[0][prefix_len:].lstrip('/').split('/')[0].strip() if l: # XXX: do log names need to be case-squashed? log_types[unquoteString(l)] = None l = reg_proc.readline() reg_proc.wait() # Now, for each log type, grab the log file path and the list of all # services in the Sources MULTI_SZ files = {} evt_files = {} for lt in log_types.keys(): evt_query = [PATH_REGLOOKUP, "-H", "-t", "EXPAND_SZ", "-p", "%s/%s/File" % (REGPATH_EVENTLOG,lt), config.registry_path] reg_proc = procWrapper(evt_query, regLookupStderrHandler) l = reg_proc.readline() if l == '': if print_verbose: sys.stderr.write('WARNING: Event log file path not found' +' for log type "%s". Removing this type.\n' % lt) del log_types[lt] continue l = l.split(',')[2].strip() evt_files[lt] = windowsPathToUnixPath(unquoteString(l), config.path_vars, config.drive_mapping)[0] reg_proc.wait() logs_query = [PATH_REGLOOKUP, "-H", "-t", "MULTI_SZ", "-p", "%s/%s/Sources" % (REGPATH_EVENTLOG,lt), config.registry_path] reg_proc = procWrapper(logs_query, regLookupStderrHandler) log_types[lt] = {} log_types[lt]["event"] = {} log_types[lt]["category"] = {} log_types[lt]["parameter"] = {} # For each service under this log type, grab the associated event, # category, and parameter message file lists. l = reg_proc.readline() reg_proc.wait() if l != '': services = l.split(',')[2].split('|') for s in services: s = s.strip().lower() log_types[lt]["event"][s] = None log_types[lt]["category"][s] = None log_types[lt]["parameter"][s] = None elif print_verbose: sys.stderr.write('WARNING: No sources found for log type "%s".\n' % lt) # Now grab whole tree under this log type and parse it lt_path = "%s/%s" % (REGPATH_EVENTLOG,lt) lt_path_len = len(lt_path) dlls_query = [PATH_REGLOOKUP, "-H", "-p", lt_path, config.registry_path] reg_proc = procWrapper(dlls_query, regLookupStderrHandler) l = reg_proc.readline() while l != '': fields = l.split(',') # trim unneeded prefix from path fields[0] = fields[0][lt_path_len:].lstrip('/') s = fields[0].split('/')[0].lower() if(fields[1] == 'EXPAND_SZ' and log_types[lt]['event'].has_key(s)): t = None if fields[0].find("EventMessageFile") >= 0: t = "event" elif fields[0].find("CategoryMessageFile") >= 0: t = "category" elif fields[0].find("ParameterMessageFile") >= 0: t = "parameter" if t: log_types[lt][t][s]\ = windowsPathToUnixPath(unquoteString(fields[2].strip()), config.path_vars, config.drive_mapping) # Retain unique list of file names and their associated services for f in log_types[lt][t][s]: if files.has_key(f): files[f] = files[f]+","+s else: files[f] = s l = reg_proc.readline() reg_proc.wait() (names2paths,paths2names) = buildShortNamesPathsDicts(files) # Convert path lists to named path strings. Filter out any unusable resources # and any services that have no remaining resources. for lt in log_types.keys(): for t in log_types[lt].keys(): for s in log_types[lt][t].keys(): if(log_types[lt][t][s]): tmp_str = '' for i in range(0,len(log_types[lt][t][s])): if paths2names.has_key(log_types[lt][t][s][i]): tmp_str += ':' + paths2names[log_types[lt][t][s][i]] log_types[lt][t][s] = tmp_str.lstrip(':') if log_types[lt][t][s] == '': del log_types[lt][t][s] sys.stderr.write("WARNING: service '%s' removed" % s\ +" due to missing message sources.\n") else: del log_types[lt][t][s] if print_verbose: sys.stderr.write("INFO: Writing service mappings...\n") makeOutputDirectories(PATH_OUTPUT, log_types.keys()) writeServiceMapping(PATH_OUTPUT, log_types) if print_verbose: sys.stderr.write("INFO: Writing DLL databases...\n") writeDBFiles(PATH_OUTPUT, names2paths) if print_verbose: sys.stderr.write("INFO: Copying log files...\n") try: for ei in evt_files.items(): o = file("%s/logs/%s.evt" % (PATH_OUTPUT,ei[0]), "w+") i = file(ei[1], "r") buf = i.read(1024) while buf != "": o.write(buf) buf = i.read(1024) o.close() i.close() except Exception, inst: sys.stderr.write("ERROR: %s\n" % inst) sys.stderr.write("ERROR: could not copy all log files.\n") sys.exit(os.EX_IOERR) if print_verbose: sys.stderr.write("INFO: Done.\n") grokevt-0.4.1/bin/grokevt-dumpmsgs0000755000175000017500000000330410602607641015357 0ustar timtim#!/usr/bin/python # This script dumps the contents of message databases built by grokevt-ripdll. # # Copyright (C) 2006-2007 Timothy D. Morgan # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation version 2 of the # License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # vi:set tabsize=4: # $Id: grokevt-dumpmsgs 105 2007-03-29 01:00:17Z tim $ import sys import os import anydbm import grokevt def usage(): sys.stderr.write("USAGE:\n") sys.stderr.write(" %s [ ...]\n\n"\ % os.path.basename(sys.argv[0])) sys.stderr.write("grokevt-dumpmsgs dumps the contents of message databases\n") sys.stderr.write("built previously by grokevt-ripdll.\n") if len(sys.argv) < 2: usage() sys.exit(os.EX_USAGE) for dbfile in sys.argv[1:]: if not os.access(dbfile, os.R_OK): sys.stderr.write("ERROR: DB file could not be read.") sys.exit(1) db = anydbm.open(dbfile, 'r') if grokevt.template_encoding == grokevt.output_encoding: for key in db.keys(): print '%s,%s' % (key, grokevt.quoteString(db[key], '\\,"')) else: for key in db.keys(): msg = db[key].decode(grokevt.template_encoding) msg = grokevt.quoteString(msg.encode(grokevt.output_encoding,'replace'), '\\,"') print '%s,%s' % (key, msg) db.close() grokevt-0.4.1/bin/grokevt-findlogs0000755000175000017500000001233010557251646015335 0ustar timtim#!/usr/bin/python # This script attempts to find log file fragments in raw binary # files, such as memory dumps and disk images. # # Copyright (C) 2006-2007 Timothy D. Morgan # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation version 2 of the # License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # vi:set tabsize=4: # $Id: grokevt-findlogs 97 2007-01-29 01:53:42Z tim $ import sys import os import types import time import struct import grokevt class fileSearcher: f = None largest = 0 bs = 512*1024 strings = None sz = -1 def __init__(self, file_name, strings): self.f = file(BIN_FILE, 'r') self.largest = max(map(len,strings)) self.bs = max(self.largest,self.bs) self.strings = strings def findOne(self, start=0): self.f.seek(start) offset = start buf = self.f.read(self.bs) while buf != '': hits = [] for s in self.strings: h = buf.find(s) if h != -1: hits.append(h) if len(hits) > 0: return offset+min(hits) buf = buf[(self.bs - self.largest):] + self.f.read(self.bs) offset += (self.bs - self.largest) return -1 def read(self, offset, count): self.f.seek(offset) return self.f.read(count) def size(self): if self.sz > 0: return self.sz o = self.f.tell() self.f.seek(0, 2) self.sz = self.f.tell() self.f.seek(o) return self.sz def close(self): self.f.close() def usage(): sys.stderr.write("USAGE:\n") sys.stderr.write(" %s -?\n" % os.path.basename(sys.argv[0])) sys.stderr.write(" %s [-v] [-h] [-H] [-o ] \n\n" % os.path.basename(sys.argv[0])) sys.stderr.write("grokevt-findlogs attempts to find log file fragments in raw\n") sys.stderr.write("binary files, such as memory dumps and disk images.\n") sys.stderr.write("Please see the man page for more information.\n") # Globals influenced by command line options print_header = 1 # XXX: not currently used print_verbose = 0 start_offset = 0 # Parse command line argv_len = len(sys.argv) if (argv_len < 2) or (sys.argv[1] == '-?') or (sys.argv[1] == '--help'): usage() sys.exit(os.EX_USAGE) else: BIN_FILE=sys.argv[argv_len-1] skip_one = 0 for i in range(1,len(sys.argv)-1): if skip_one: skip_one = 0 else: if sys.argv[i] == '-H': print_header = 0 elif sys.argv[i] == '-h': print_header = 1 elif sys.argv[i] == '-v': print_verbose = 1 elif sys.argv[i] == '-o': if i+1 < len(sys.argv): # XXX: might be nice to allow hexadecimal in addition to decimal start_offset = int(sys.argv[i+1]) skip_one = 1 else: usage() sys.stderr.write("ERROR: -o option requires parameter.\n") sys.exit(os.EX_USAGE) else: usage() sys.stderr.write("ERROR: Unrecognized option '%s'.\n" % option) sys.exit(os.EX_USAGE) if not os.access(BIN_FILE, os.R_OK): sys.stderr.write("ERROR: DB file could not be read.") sys.exit(os.EX_OSFILE) searcher = fileSearcher(BIN_FILE, (grokevt.header_log_magic, grokevt.cursor_magic)) if print_header: print "OFFSET,LENGTH,TYPE,MESSAGE_NUM,CREATED,CONTIGUOUS" contig_next = -1 i = searcher.findOne(start_offset) while i != -1: found_one = False if (i + grokevt.min_record_size < searcher.size()): offset = i-4 size1 = struct.unpack("= grokevt.min_record_size) and (end < searcher.size()): size2 = struct.unpack("\n" % command\ +" %s -m \n" % command\ +" %s [-v] [-H] [-h] [-U] [-u] \n\n"\ % command\ +"This program parses a windows event log and prints a\n"\ +"CSV version of the log to stdout. Please see the man\n"\ +"page for more information.\n") # Globals influenced by command line options mode_loglist = 0 mode_meta = 0 print_verbose = 0 print_header = 1 print_unicode = 0 DB_PATH = None LOG = None # Parse command line argv_len = len(sys.argv) if (argv_len < 3) or (sys.argv[1] == '-?') or (sys.argv[1] == '--help'): usage() sys.exit(os.EX_OK) elif sys.argv[1] == '-l': if argv_len == 3: mode_loglist = 1 DB_PATH=sys.argv[2] else: usage() sys.stderr.write("ERROR: Incorrect usage for log list mode.\n") sys.exit(os.EX_USAGE) elif sys.argv[1] == '-m': if argv_len == 4: mode_meta = 1 DB_PATH=sys.argv[2] LOG=sys.argv[3] else: usage() sys.stderr.write("ERROR: Incorrect usage for meta information mode.\n") sys.exit(os.EX_USAGE) else: if (argv_len >= 3): DB_PATH=sys.argv[argv_len-2] LOG=sys.argv[argv_len-1] for option in sys.argv[1:argv_len-2]: if option == '-v': print_verbose = 1 elif option == '-H': print_header = 0 elif option == '-h': print_header = 1 elif option == '-U': print_unicode = 0 elif option == '-u': print_unicode = 1 else: usage() sys.stderr.write("ERROR: Unrecognized option '%s'.\n" % option) sys.exit(os.EX_USAGE) else: usage() sys.stderr.write("ERROR: Incorrect usage for log parse command.\n") sys.exit(os.EX_USAGE) if mode_loglist: try: logs = os.listdir("%s/services" % DB_PATH) for l in logs: print l except Exception, inst: sys.stderr.write("%s\n" % inst) sys.stderr.write("ERROR: Could not list services directory.\n") sys.stderr.write(" Did you run grokevt-builddb first?\n") sys.exit(os.EX_OSFILE) sys.exit(os.EX_OK) try: if print_verbose: sys.stderr.write("INFO: Opening message repository '%s'.\n" % DB_PATH) msg_repo = messageRepository(DB_PATH, LOG) except Exception, inst: sys.stderr.write("%s\n" % inst) sys.stderr.write("ERROR: Could not read message repository.\n") sys.stderr.write(" Did you specify the correct DATABASE_DIR?\n") sys.stderr.write(" Did you run grokevt-builddb first?\n") sys.exit(os.EX_OSFILE) evt_file = None evt_filename = "%s/logs/%s.evt" % (DB_PATH, LOG) try: if print_verbose: sys.stderr.write("INFO: Opening event log file at '%s'.\n" % evt_filename) evt_file = evtFile(evt_filename, msg_repo) except Exception, inst: sys.stderr.write("%s\n" % inst) sys.stderr.write("ERROR: Could not open log file.\n") sys.stderr.write(" Did grokevt-builddb finish without errors?\n") sys.exit(os.EX_OSFILE) evt_size = evt_file.size() # Begin parsing logic if mode_meta: csvwriter = csv.DictWriter(sys.stdout, meta_fields, '', 'ignore') row = {'header_first_off':"Unknown", 'cursor_first_off':"Unknown", 'header_first_num':"Unknown", 'cursor_first_num':"Unknown", 'header_next_off':"Unknown", 'cursor_next_off':"Unknown", 'header_next_num':"Unknown", 'cursor_next_num':"Unknown", 'header_file_size':"Unknown", 'real_file_size':evt_size, 'retention_period':"Unknown", 'flag_dirty':"Unknown", 'flag_wrapped':"Unknown", 'flag_logfull':"Unknown", 'flag_primary':"Unknown"} if evt_file.header: row['header_first_off'] = evt_file.header['first_off'] row['header_first_num'] = evt_file.header['first_num'] row['header_next_off'] = evt_file.header['next_off'] row['header_next_num'] = evt_file.header['next_num'] row['header_file_size'] = evt_file.header['file_size'] row['retention_period'] = evt_file.header['retention'] row['flag_dirty'] = evt_file.header['flag_dirty'] row['flag_wrapped'] = evt_file.header['flag_wrapped'] row['flag_logfull'] = evt_file.header['flag_logfull'] row['flag_primary'] = evt_file.header['flag_primary'] if evt_file.cursor: row['cursor_first_off'] = evt_file.cursor['first_off'] row['cursor_first_num'] = evt_file.cursor['first_num'] row['cursor_next_off'] = evt_file.cursor['next_off'] row['cursor_next_num'] = evt_file.cursor['next_num'] csvwriter.writerow(meta_header) csvwriter.writerow(row) sys.exit(os.EX_OK) csvwriter = csv.DictWriter(sys.stdout, log_fields, '', 'ignore') if print_header: csvwriter.writerow(log_header) if print_verbose: sys.stderr.write("INFO: Now parsing file.\n") if evt_file.header and evt_file.header['flag_dirty']: sys.stderr.write("WARNING: Log file marked as dirty.\n") if (evt_file.header == None) or (evt_file.cursor == None): sys.stderr.write("WARNING: Naive parsing enabled.\n") record_type = None while record_type != 'wrapped-log': # First, try to find the first log record. This will skip over any split # log records at the beginning of the file. record_type = evt_file.guessRecordType() while ((evt_file.guessRecordType()=='unknown') and (evt_file.tell()= evt_size: break # Next walk through the file hoping to stay aligned with proper records # We skip over anything that looks like a header or cursor, and bail out # as soon as we run across a non-record. if record_type in ('log', 'wrapped-log'): # XXX: catch exceptions? rec = evt_file.getLogRecord() for k in rec.keys(): if type(rec[k]) == types.StringType: rec[k] = quoteString(rec[k]) elif type(rec[k]) == types.UnicodeType: if print_unicode: rec[k] = quoteUnicode(rec[k]).encode(output_encoding) else: rec[k] = quoteString(rec[k].encode(output_encoding)) csvwriter.writerow(rec) elif record_type == 'cursor': sys.stderr.write("WARNING: Skipping potential cursor record " +"at offset %d.\n" % evt_file.tell()) evt_file.seek(evt_file.tell()+cursor_size) elif record_type == 'header': sys.stderr.write("WARNING: Skipping potential header record " +"at offset %d.\n" % evt_file.tell()) evt_file.seek(evt_file.tell()+header_size) else: for i in xrange(evt_file.cursor['first_num'],evt_file.cursor['next_num']): # XXX: catch exceptions? rec = evt_file.getLogRecord() for k in rec.keys(): if type(rec[k]) == types.StringType: rec[k] = quoteString(rec[k]) elif type(rec[k]) == types.UnicodeType: if print_unicode: rec[k] = quoteUnicode(rec[k]).encode(output_encoding) else: rec[k] = quoteString(rec[k].encode(output_encoding)) csvwriter.writerow(rec) grokevt-0.4.1/bin/grokevt-ripdll0000755000175000017500000001642110430235270015004 0ustar timtim#!/usr/bin/env python # This script parses a PE formatted file (EXE/DLL) and extracts the # message resources out of the .rsrc section. They are then written to # a database file. # # Original PE header code borrowed from the pymavis project. # For more information, see: http://www.mplayerhq.hu/~arpi/pymavis/ # # Copyright (C) 2005-2006 Timothy D. Morgan # Copyright (C) 2004 A'rpi # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation version 2 of the # License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # vi:set tabsize=4: # $Id: grokevt-ripdll 81 2006-05-10 00:58:32Z tim $ import os import sys import string import struct import traceback import anydbm from grokevt import * # XXX: this whole script needs sanity checks on all data read, and on # length of data read. It may also be best to switch to using mmap # instead of using so many seeks. def usage(): sys.stderr.write("USAGE:\n") sys.stderr.write(" %s \n\n" % os.path.basename(sys.argv[0])) sys.stderr.write("grokevt-ripdll is a tool for extracting message resources") sys.stderr.write(" from a PE-formatted file.\nPlease see the man page for more") sys.stderr.write(" information.\n\n") def unpack(fmt,f): return struct.unpack(fmt,f.read(struct.calcsize(fmt))) def find_name_dir(f,num_dir_names,num_dir_ids): ret_val = None f.seek(f.tell()+8*num_dir_names) for i in range(0,num_dir_ids): (type_id, type_offset) = unpack("=32: sect_name+=c if sect_name == '.rsrc': f.seek(sect_off) (rsrc_flags,date,majver,minver, num_dir_names,num_dir_ids) = unpack("=0x40: try: # new-exe: f.seek(startoff+0x3c) ne_off,=unpack(" $@ $(foreach p,$(MAN_PAGES),man/man$(p).gz): $(foreach p,$(MAN_PAGES),man/man$(p)) gzip -c -9 $(subst .gz,,$@) > $@ && rm $(subst .gz,,$@) release: $(foreach p,$(MAN_PAGES),man/man$(p).gz) release-clean: rm -f $(foreach p,$(MAN_PAGES),man/man$(p)) rm -f $(foreach p,$(MAN_PAGES),man/man$(p).gz) rm -f $(foreach p,$(MAN_PAGES),$(basename $(notdir $(p))).txt) grokevt-0.4.1/doc/devel/0000755000175000017500000000000010770556767013231 5ustar timtimgrokevt-0.4.1/doc/devel/format.txt0000644000175000017500000001273510427472367015262 0ustar timtimWindows Event Log Format Reference ================================== Information on the header and cursor records is courtesy of Andreas Schuster. The log record format is a combination of information from Jamie French and Tim's own guess & check, and additions by Andreas. Please see references.txt for more information. Header Record ============= 0. 32bits, little endian record length in bytes, which is always 0x30 1. char[4] magic number, always "LfLe" 2. 32bits, little endian unknown (always 1, possibly part of a format version number) 3. 32bits, little endian unknown (always 1, possibly part of a format version number) 4. 32bits, little endian offset from beginning of file to the first record 5. 32bits, little endian offset from beginning of file to the next record to be written (or is this really the cursor record?) 6. 32bits, little endian number of the next record to be written 7. 32bits, little endian number of the first record 8. 32bits, little endian filesize in bytes (unreliable?) 9. 32bits, little endian flags: 0x1 (DIRTY): This flag is set upon the first write. In my opinion it is the most important flag. If it is set, you can't rely on any of the other fields provided in the header record. 0x2 (WRAPPED): This flag indicates a wrapped log. That means, the newest event record is at a lower offset than the oldest one. This flag is updated only during shutdown of the service. 0x4 (LOGFULL): This flag is set as soon as a record could not be written because there's not: 1. enough free space left in the file AND 2. the file could not grow any further (no space left on device, MaxSize reached) AND 3. AutoBackupLogFiles is not configured AND 4. Retention does not allow to overwrite the oldest event record. 0x8 (PRIMARY): I doubt that I completely understand this flag. It's set in all configured and active log files I've seen so far. It's cleared in all files written by BackupEventLog(). Maybe this flag is used to disable some checks while opening the file. For sure I'll still have to spend some time playing around with that flag. 10. 32bits, little endian retention period in seconds 11. 32bits, little endian length repeated, always 0x30 Cursor Record ============= 0. 32bits, little endian record length in bytes, which is always 0x28 1. 128bits magic number, which is always: "\x11\x11\x11\x11\x22\x22\x22\x22\x33\x33\x33\x33\x44\x44\x44\x44" 2. 32bits, little endian offset from beginning of file to the first log record 3. 32bits, little endian offset from beginning of file to the next record to be written (or last record in file which has already be written?) 4. uint32, little endian number of the next record to be written 5. uint32, little endian number of the first record 6. 32bits, little endian length repeated, always 0x28 Log Record ========== 0: 32bits, little endian Length of record, in bytes, including these 4 bytes 1: 32bits, fixed value Magic number, with value: 1001100011001100100110001100101b or 0x4c664c65 or "LfLe" 2: 32bits, little endian Message Number 3: 32bits, little endian Date Created UTC/GMT, in seconds since the epoch(unix?). 4: 32bits, little endian Date Written UTC/GMT, in seconds since the epoch(unix?). 5: 32bits, little endian Event ID and Offset High two bytes are the offset into the windoze DLL which holds the messages. The two low order bytes are what is displayed to the user as the Event ID. The number taken as a whole is used to grab the message from the DLL using the windoze FormatMessage() call. 6: 16bits, little endian Event Type One of 5 values. 7: 16bits, little endian String Count The number of strings in the event? Not always accurate? 8: 16bits, little endian Category 9: 16bits UNKNOWN 10: 32bits UNKNOWN 11: 32bits, little endian Strings Offset Pointer to a sequence of NULL terminated UTF-16 strings (NULL == '\x00\x00') used in the message template. This pointer is relative to the very beginning of the record. The end of the strings section appears to be the beginning of the other data section. 12: 32bits, little endian SID string length Set to 0 if it isn't included. 13: 32bits, little endian SID pointer This pointer is relative to the very beginning of the record. 14: 32bits, little endian Other data length Length of supplimentary data included in log. 0 if none exists. 15: 32bits, little endian Other data pointer This pointer is relative to the very beginning of the record. 16: variable length fields 16.1: NULL terminated UTF-16 string (NULL == '\x00\x00') The windoze log source. This value is used to look up (using the registry) what DLLs need to be searched for the proper EventID message template. 16.2: NULL terminated UTF-16 string (NULL == '\x00\x00') Computer Name which generated the log entry. 16.3: General buffer. Everything here is pointed to by the pointers in fields 11, 13, and 15. grokevt-0.4.1/doc/devel/references.txt0000644000175000017500000000272010430235270016064 0ustar timtimEvent Log Format ================ The best reference found to date on the format: Schuster, A. (2005). Windows Eventlogs in der forensischen Analyse. In M. Thorbr?gge (Ed.), Proceedings of the 12th DFN-CERT Workshop "Sicherheit in vernetzten Systemen", Hamburg, March 2005 (pp. D1-D16). ISBN: 3-00-015369-1 And the associated presentation slides: http://www.dfn-cert.de/events/ws/2005/dfncert-ws2005-f4.pdf The info by Jamie French (aka Malik) which inspired the creation of GrokEVT: http://www.whitehats.ca/main/members/Malik/malik_eventlogs/malik_eventlogs.html Helpful chapter from "Windows NT Event Logging" (O'Reilly): http://www.oreilly.com/catalog/winlog/chapter/ch02.html#40421 Other References ================ Micro$oft's PE executable format: http://www.csn.ul.ie/~caolan/publink/winresdump/winresdump/doc/pefile.html http://www.wotsit.org/download.asp?f=pe Language Codes (exist in .rsrc sections of PE files): http://msdn.microsoft.com/library/en-us/intl/nls_238z.asp FormatMessage(): http://msdn.microsoft.com/library/default.asp?url=/library/en-us/debug/base/formatmessage.asp http://msdn.microsoft.com/library/en-us/tools/tools/message_text_files.asp Converting binary SID's to text: http://blogs.msdn.com/oldnewthing/archive/2004/03/15/89753.aspx Event Types: http://technet2.microsoft.com/WindowsServer/en/Library/7e77c2f0-8835-4bea-b972-26edb2aceb3d1033.mspx System Error Codes: http://msdn.microsoft.com/library/en-us/debug/base/system_error_codes.asp grokevt-0.4.1/doc/grokevt-addlog.1.docbook0000644000175000017500000001027610602607641016530 0ustar timtim grokevt-addlog 1 File Conversion Utilities grokevt-addlog A tool for adding a raw event log to an existing GrokEVT database. SYNOPSIS grokevt-addlog database-dir evt-file new-type base-type DESCRIPTION grokevt-addlog takes a raw event log (.evt file) and adds it to a pre-built database generated by grokevt-builddb(1). This new log file will be set up to use the message templates of another log, as determined by the user. This tool is primarily useful for processing deleted logs and log fragments found on a system. While it is possible to use the database generated from one system with the logs of another, this is not recommended for investigations unless no alternatives exist. ARGUMENTS grokevt-addlog uses the following arguments: The base directory for the database generated previously by grokevt-builddb(1). The file to be added to the database. The new log type/name that evt-file will take on. This is the name that will need to be used later with grokevt-parselog(1) to access the new log. This type must not already exist in the database. The existing log type that this new log will be based on. The message templates from this type will be used with the new log when parsing. This type must exist in the current database. BUGS Probably several. This particular script has not been extensively tested. CREDITS Written by Timothy D. Morgan. Copyright (C) 2006-2007 Timothy D. Morgan LICENSE Please see the file "LICENSE" included with this software distribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 2 for more details. SEE ALSO grokevt(7) grokevt-builddb(1) grokevt-dumpmsgs(1) grokevt-findlogs(1) grokevt-parselog(1) grokevt-ripdll(1) grokevt-0.4.1/doc/grokevt-builddb.1.docbook0000644000175000017500000001220010602607641016670 0ustar timtim grokevt-builddb 1 File Conversion Utilities grokevt-builddb Builds a database tree based on a single windows system for the purpose of event log conversion. SYNOPSIS grokevt-builddb [ -c CSID ] config-profile output-dir DESCRIPTION grokevt-builddb uses grokevt-ripdll(1) and reglookup(1) along with information found in configuration files to extract all necessary information from a windows installation for the conversion of event log files. The registry is read to determine the locations of critical DLLs and the event log files themselves. This, and other information out of the registry is stored in a directory structure which acts as a kind of flat-file database. This database can then be used by grokevt-parselog(1) to generate human-readable output. The key to successfully running this utility is proper configuration. Please see grokevt(7) for information on what needs to be configured. ARGUMENTS This is the name of the configuration profiles stored in the global configuration directory under the directory 'systems'. See grokevt(7) for more details on how to properly configure a system profile. The path to the location of the output database. If anything already exists in this directory, it may be overwritten or deleted. OPTIONS This option allows one to explicitly set which ControlSet in the registry is used to extract event log message mappings. If specified, this item must be a positive decimal integer. If unspecified, grokevt-builddb will attempt to determine the best ControlSet by looking at the most recent CurrentControlSet, stored in the system registry under the path '/Select/Current'. Most users should ignore this option unless there is a specific reason why the last CurrentControlSet should not be used. EXAMPLES To generate a database at '~/win2k.grokevt' based on the system configuration profile 'win2k': grokevt-builddb win2k ~/win2k.grokevt To repeat the last command, instead using registry information explicitly from /ControlSet002: grokevt-builddb -c 2 win2k ~/win2k.grokevt BUGS Probably a few. This script has not been extensively tested with some guest platforms. The databases built with this script may not be portable to other systems, depending on the database drivers installed and used in Python. CREDITS Written by Timothy D. Morgan. Copyright (C) 2005-2007 Timothy D. Morgan LICENSE Please see the file "LICENSE" included with this software distribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 2 for more details. SEE ALSO grokevt(7) grokevt-addlog(1) grokevt-dumpmsgs(1) grokevt-findlogs(1) grokevt-parselog(1) grokevt-ripdll(1) reglookup(1) grokevt-0.4.1/doc/grokevt-dumpmsgs.1.docbook0000644000175000017500000000676210561154760017145 0ustar timtim grokevt-dumpmsgs 1 File Conversion Utilities grokevt-dumpmsgs A tool for dumping the contents of message databases built previously by grokevt-ripdll(1). SYNOPSIS grokevt-dumpmsgs message-db1 [message-db2 ...] DESCRIPTION grokevt-dumpmsgs takes one or more message databases previously built with grokevt-ripdll(1) and prints out all entries to stdout. This is mainly a debugging tool, but may be useful for analyzing the message contents of PE files while developing other applications. ARGUMENTS grokevt-dumpmsgs uses the following arguments: If multiple message databases are supplied, entries of all databases are printed to stdout in the order they are provided. OUTOUT grokevt-dumpmsgs prints each database entry out on a single line, in two comma-separated columns. The first column is the message ID, which is in the format: XXXX-YYYYYYYY Here, XXXX represents the message's language code, and the YYYYYYYY represents the message's relative virtual address (RVA) within the message block of the PE file. The second column contains the message itself. Messages containing special characters (such as newlines or commas) are encoded in the same manner that grokevt-parselog(1) encodes them ("\xQQ" where QQ is the hexadecimal value of the character). BUGS Probably several. This particular script has not been extensively tested. CREDITS Written by Timothy D. Morgan. Copyright (C) 2006-2007 Timothy D. Morgan LICENSE Please see the file "LICENSE" included with this software distribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 3 for more details. SEE ALSO grokevt(7) grokevt-addlog(1) grokevt-builddb(1) grokevt-findlogs(1) grokevt-parselog(1) grokevt-ripdll(1) grokevt-0.4.1/doc/grokevt-findlogs.1.docbook0000644000175000017500000001062610602607641017102 0ustar timtim grokevt-findlogs 1 File Conversion Utilities grokevt-findlogs Attempts to find log file fragments in raw binary files, such as memory dumps and disk images. SYNOPSIS grokevt-findlogs -? grokevt-findlogs [-v] [-h] [-H] [-o offset] raw-file DESCRIPTION grokevt-findlogs searches a raw binary file for event log records. It produces a simple comma-separated values (CSV) output to stdout which includes metadata and offsets of any hits. Using the metadata and offset/contiguity information, it should be easy to determine if the hits are false positives or not. ARGUMENTS The binary file to be searched. OPTIONS Prints a basic usage statement. Verbose mode. Prints status messages to stderr, which can be helpful for debugging. (Currently does nothing.) Prints a header row at the top of the CSV output containing labels for each column. (This is the default behavior.) Disables the printing of a header row. This is useful when grokevt-findlogs is used in a script. Begin search at this byte offset within the binary file. BUGS Probably a few. This script has not been extensively tested with some guest platforms. There are likely some speed improvements that could be made. CREDITS Written by Timothy D. Morgan Copyright (C) 2006-2007 Timothy D. Morgan LICENSE Please see the file "LICENSE" included with this software distribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 2 for more details. SEE ALSO grokevt(7) grokevt-addlog(1) grokevt-builddb(1) grokevt-dumpmsgs(1) grokevt-parselog(1) grokevt-ripdll(1) grokevt-0.4.1/doc/grokevt-parselog.1.docbook0000644000175000017500000002015110602607641017103 0ustar timtim grokevt-parselog 1 File Conversion Utilities grokevt-parselog Parse a windows event log and generate human-readable output based on message resources stored in a database. SYNOPSIS grokevt-parselog -?|--help grokevt-parselog -l database-dir grokevt-parselog -m database-dir log-type grokevt-parselog [-v] [-H] [-h] [-U] [-u] database-dir log-type DESCRIPTION grokevt-parselog reads a windows event log (.evt file) and combines that information with messages templates and other resources stored in a pre-generated database. This is then printed to stdout in a comma-separated values (CSV) format. The database must be created by grokevt-builddb(1). ARGUMENTS This is the directory where the database is stored. Currently, the actual log files from the original system are also stored in this directory tree. This is the windows name for the log. By default windows has the following logs: Application Security System But others may have been created by third party software. Use the -l option to print a list of all available log types. (The log names are case-sensitive.) OPTIONS Prints a basic usage statement. Same as -?. Log list mode. Lists the logs available in the specified database. Meta information mode. Lists meta information stored in the header of the specified log file. Does not print any actual log records. (Format of output in this mode is still subject to change.) Verbose mode. Prints status messages to stderr, which can be helpful for debugging. Prints a header row at the top of the CSV output containing labels for each column. (This is the default behavior.) Disables the printing of a header row. This is useful when grokevt-parselog is used in a script. Enables the UTF-8 output of some strings. This can be dangerous on terminals that are not configured to support UTF-8. Disables the use of UTF-8 for output. Unicode strings are instead converted to UTF-8 first, and then any remaining non-ASCII characters are quoted. (This is the default behavior.) EXAMPLES To list all available logs types stored in '~/example.grokevt': grokevt-parselog -l ~/example.grokevt To read the 'Application' log from the database stored in '~/example.grokevt' and print it to stdout: grokevt-parselog ~/example.grokevt Application To read the 'System' log from the database stored in '~/example.grokevt' and print it to stdout without a header, and with verbosity turned on: grokevt-parselog -v -H ~/example.grokevt System BUGS Probably a few. This script has not been extensively tested with some guest platforms. The file event log file format is pretty well understood and implemented, but some diabolical wrapped, dirty, or fragmentary logs may not be correctly parsed. Unicode support is currently limited. Any suggestions on how to better handle unicode output would be appreciated. CREDITS Originally written by Jamie French. Converted to Python and extended by Timothy D. Morgan. Andreas Schuster has contributed greatly to the understanding of the event log format. Copyright (C) 2005-2007 Timothy D. Morgan Copyright (C) 2004 Jamie French LICENSE Please see the file "LICENSE" included with this software distribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 2 for more details. SEE ALSO grokevt(7) grokevt-addlog(1) grokevt-builddb(1) grokevt-dumpmsgs(1) grokevt-findlogs(1) grokevt-ripdll(1) grokevt-0.4.1/doc/grokevt-ripdll.1.docbook0000644000175000017500000000752310561154760016570 0ustar timtim grokevt-ripdll 1 File Conversion Utilities grokevt-ripdll A tool for extracting message resources from a PE-formatted file. SYNOPSIS grokevt-ripdll input-dll output-db DESCRIPTION grokevt-ripdll parses a PE-formatted file (modern .exe and .dll files are examples PE-formatted files) and extracts all message resources. These resources are then stored in a Berkeley-style database file, which maps relative virtual addresses (RVAs) to the message resources themselves. These RVAs are what can be found in a windows event log file (.evt extension) to reference the proper message resource. This utility is not intended to be used directly by end-users. It is used by grokevt-builddb(1) to extract resources from all DLL/EXEs referenced in the registry. ARGUMENTS This is the PE formatted file to extract resources from. (It doesn't need to have a .dll extension, but it is most commonly used on DLLs.) The database file to store the RVA->message mapping in. If this file already exists, it will be overwritten. To extract the entries stored in this database, see grokevt-dumpmsgs(1). BUGS Probably a few. This script has not been extensively tested with some guest platforms or with non-english systems. The documentation used as a reference for PE formatted files was not complete or not completely accurate in places. Much guess-and-check took place. CREDITS Original PE header code borrowed from the pymavis project. For more information, see: http://www.mplayerhq.hu/~arpi/pymavis/ Message resource parsing added by Timothy D. Morgan. Copyright (C) 2005-2007 Timothy D. Morgan Copyright (C) 2004 A'rpi LICENSE Please see the file "LICENSE" included with this software distribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 3 for more details. SEE ALSO grokevt(7) grokevt-addlog(1) grokevt-builddb(1) grokevt-dumpmsgs(1) grokevt-findlogs(1) grokevt-parselog(1) grokevt-0.4.1/doc/grokevt.7.docbook0000644000175000017500000002423510602722407015304 0ustar timtim grokevt 7 File Conversion Utilities GrokEVT a collection of scripts built for reading windows event log files. DESCRIPTION This document serves as an overview to using the GrokEVT suite of tools. Please see the man pages for each tool for specific command-line options. USING Using GrokEVT on a set of windows partitions is a three stage process. One must first mount the partitions and configure the system's profile. Next, grokevt-builddb(1) must be run to extract the necessary information from these partitions. Finally, grokevt-parselog(1) may be run to use this information to convert the logs to something useful. Mounting and configuration: This is the most labor-intensive step of the process. There are several technical reasons why this cannot be simpler, and it is unlikely many will be resolved anytime soon. (Sorry, blame the vendor who produced these logs that you wish to convert.) First mount the partitions used by your windows system. Specifically, mount any partition that has software installed on it. (You can probably get away with not mounting ones with no software, or no software that produces events in the event log.) This includes windows shares, if you have any with software installed on them. The key point when mounting partitions or shares, is that you must mount them with case-insensitive filenames. To do this in Linux, you should be able to use the NTFS option 'posix=0', or with type vfat this appears to be the default. On FreeBSD, one can use the -i option with NTFS and mount_ntfs, but this hasn't been tested. (More research is necessary in this area. Please let me know if case-insensitivity is available on other *NIXes and what the options are.) All of this, of course, depends on your version of mount and related software. See mount(8) for more information. Once you have all necessary filesystems mounted for your windows image, you need to configure GrokEVT so it knows where everything is. If you look in the main configuration directory, (by default at '/usr/local/etc/grokevt') you'll find a sub-directory called 'systems'. In there is a set of system configuration profiles. Each directory under 'systems' represents the configuration for a single windows host. You may create directories here of any name. It is suggested you use the host name of the system you wish to extract logs from. By default, there will be one system configuration there already, named 'example'. I suggest you make a full copy of this, and edit it from there to create your own configuration: # cd /usr/local/etc/grokevt/systems # cp -r example mysystem Now that you have your own configuration, take a look at the files under this directory: path-vars/%SystemRoot% system-registry drives/c: drives/d: Each file represents a single configuration setting. The first line of every file is the setting, all others are ignored. You should start by configuring your drive letters. These drive letters need to map to the *NIX path of the drives. So, if you mounted your C: partition under '/mnt/win/c', then you should do the following: echo /mnt/win/c > drives/c: This will of course, overwrite that file with your actual mount point for the C: drive. Do this for all drives mounted on your windows system. Next, you need to configure your %SystemRoot% path variable. Since the registry often references files in relation to this variable, we need to know where it is on your system. Typically, on W2K this should be 'C:\WINNT'. On other windows systems it is often 'C:\Windows'. This path may be referenced as a full windows path (with drive letters) or as a *NIX path. Finally, you need to set the path of the system registry. This path must be a *NIX path, and not a windows path. It should be located under your %SystemRoot%. For instance, if you mount a W2K machine's C: on '/mnt/win/c', this path will probably be: /mnt/win/c/WINNT/system32/config/system This will vary from system to system, but if you have ever done any forensics, you should be able to find the correct path pretty quickly. Using grokevt-builddb(1): Once you get past the mounting and configuration step, you're in good shape. To extract the necessary information from the system you just mounted, you just need to decide where you want to store the output database. Based on the example paths above, one could run grokevt-builddb(1) like so: grokevt-builddb mysystem /var/db/grokevt/mysystem Or something similar. This step will take some time, as a lot of files need to be parsed. You will likely get some warnings about missing DLLs and possibly service removals. This is normal if software on the windows system failed to clean up the registry correctly when it was un-installed. You might want to record this information though, if later you find you are missing a lot of message templates in your log output. Now that you have finished this step, you have fully extracted all information needed to interpret the event logs. Using grokevt-parselog(1): To run grokevt-parselog(1), you must specify the database directory that you generated with grokevt-builddb(1). Initially, you will want to find out what event logs were extracted by running: grokevt-parselog -l /var/db/grokevt/mysystem This will print out all available event log types, based on information found previously in the system's registry. Then you may view one of those logs simply by dropping the -l option, and adding it's name as the second parameter. For instance, to view the 'System' log: grokevt-parselog /var/db/grokevt/mysystem System See the grokevt-parselog(1) man page for further options. Finding and Parsing Log Fragments: One may find it useful to parse deleted log files or fragments of log files found in a system's RAM dump. A special-purpose tool, grokevt-findlogs(1) is included in this distribution which makes finding log fragments easy. It can find even a single log record by itself with a relatively low rate of false positives. Suppose one has a file, dump.img, which contains an image of a system's RAM, or perhaps the unallocated blocks of a filesystem (as might be produced by The SleuthKit's dls(1)). One could search this dump by running: grokevt-findlogs dump.img This will produce a listing of all hits in the file, which may include header records, cursor records, or log records, offsets and contiguity information to help one eliminate any false positives. Supposing we found a log fragment in this dump that we're interested in, we could simply use dd(1) or some other tool to carve out the relevant data in to a file named fragment.evt. We'd then need to find a way to associate this log with a set of message templates. This is what grokevt-addlog(1) is for. Let's take a guess that the fragment.evt was originally a part of System log. We would add the log into our previously built message database by running: grokevt-addlog /var/db/grokevt/mysystem fragment.evt System-fragment System Finally, we can parse the log by simply specifying the newly imported name with grokevt-parselog(1): grokevt-parselog /var/db/grokevt/mysystem System-fragment If we found that most of the log entries didn't have associated message templates (which would result in lots of warnings to stderr), we would be pretty that fragment.evt really wasn't based on a System log. Simply re-importing it as another log type (and a different name) would let us parse it again as if it were of another type. Please see the grokevt-findlogs(1) and grokevt-addlog(1) man pages for more information. CREDITS This man page written by Timothy D. Morgan LICENSE Please see the file "LICENSE" included with this software distribution. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 2 for more details. SEE ALSO grokevt-addlog(1) grokevt-builddb(1) grokevt-findlogs(1) grokevt-parselog(1) grokevt-ripdll(1) reglookup(1) grokevt-0.4.1/doc/man/0000755000175000017500000000000010770556767012705 5ustar timtimgrokevt-0.4.1/doc/man/man1/0000755000175000017500000000000010770556771013534 5ustar timtimgrokevt-0.4.1/doc/man/man1/grokevt-addlog.1.gz0000644000175000017500000000214510770556771017150 0ustar timtimGgrokevt-addlog.1uUmo6 _ApHvV ۲#l,~\}Ilzȇ|H m:;ᶐ,qPCPZa7YۯAśoXP X%s^t i=6 հFonR>)6p{sFR M+u +'tRX< }9dZA w D뀜ނWZGǏAϹGO_8R樂D&=b7q;P0%QE{C]\!Y1[#&y';2Zp'JǚOYOWdiSeOJA. ݖ;r,BP 44 B)0lt ,i37F6H'5(ԕR(A#dU Vâw6R/AQr|=.T2A1` yI'zhDTF(tyiKT)jjk@O Ѣ{&p]Pj2>J+w^æa (%tse/}װClǽrgIÐ.N;dwGTQGԍhb3N9茡ʻ[=,6:91\ @d)nFnlx? Lh-/bިrHL68S{Yrz`<ΣYHC+A_Q^p]+TGJ<܎X#Fl= oHmGчvJw _ =%ʞ Wx&tkM ^%2 H{qG? .W,%[ m-i.[4Jt9:LՅ}p4]mlmmliy`; S*{őgrokevt-0.4.1/doc/man/man1/grokevt-builddb.1.gz0000644000175000017500000000303410770556771017321 0ustar timtimGgrokevt-builddb.1Vao6_q08-b50t؆ǰtEDD$Q x߾wd>,w{f@#JLoTtּ("ѦvfirO^)G,~Qq5O%>\Y3t8"_DD/I=Ѷyn= no^dO77hI&|ViSң׹Z9<(BTGHR*J[?H0@.SspΫ2cU 2gUzuDab>;dI ނ˽'D=6l #/J\bI;qo\\*׉@)(_5y~]-D`)*dUV4'*wJtUz-B!.jBI0`t3sҫź4A >E-v- Û߄ʬhNzqV9TDUj Ţ3 #ÓiޓټDOan;Ctssֹ2?biaV[))Sv80rѰEA!L&3]R0[wv,Go^3&Z+c}hflLlɷړ Ry~wi /aY-h_[;B;utѽ;Y6OLuzt5yw_Fw>LTfgD6hLiZAZ%RV־QrӾ1{0^vƋouZoN㼤-~||i^L>=hLj=<+Z>\39kNz Q\âMwӺshO4k4mJZUvᦃmk{ i2*R} grokevt-0.4.1/doc/man/man1/grokevt-dumpmsgs.1.gz0000644000175000017500000000222610770556771017555 0ustar timtimGgrokevt-dumpmsgs.1uUmO8_1Nڲ[TJH$=>Z8ve;-_t˕*Z&yyg_ Lj;6I 0(!`e7ZzK^V$`do %o$WVv \EswP7nj]9 .|gg `p#$Vn NHn@)9 ` Nk 61YTZZr`%²е;XufRO&q<0!Yqu#o$Xjp"{y;S05I-($Y? ?WgփmiIEjB~ Y~n3Ma |!ؓJV2’6Tּz__ ^%94TvykI WF;w8[$:n]I<qNpwg**\#zڔTg^(WMY8Γ)$`9o5Ƞ{9@0sY{T=}X]-N8p}]ZyhY5o~))Lpa<qMz?hGTPv~ OY>{܄|$p4,fE=$qr;_-W0%K)o:n Q[d:,ygd7Nm_%-MPztxgrokevt-0.4.1/doc/man/man1/grokevt-findlogs.1.gz0000644000175000017500000000222110770556771017516 0ustar timtimGgrokevt-findlogs.1mUQo6 ~ Kݰi M۹ ۴-T IN?NڬC[ȏ?R}A3YCkG¦RzgǷ?x)\TϞ!lǿ]A8.<6q cDMI4~|=_̣G0h9 C]08bzWԇOܻ[nk4BMMLǫQ)jp^%.2Vm%iʲttwGIRWp;grokevt-0.4.1/doc/man/man1/grokevt-parselog.1.gz0000644000175000017500000000361310770556771017533 0ustar timtimGgrokevt-parselog.1Wko\ "(:"Rg-v]q< `<hJJR({.%J㈏8{fuo"L{޸HT)zb󕾌 g/~ř[~.K/V| vD=/x? now|lVZKJtX`qrϟFvr*3)r+(t ^MS Y)2-|-8 V9QHn3M5qw&qp%cCKh[$ϧJ;e<WKMVJx|˲Koe籩z#R|Q+2$5Ҵ f= o*9@(\9N~k1g<˫Ɓ6UfU˓qHP3NR=Nr<)ެr :Z_-"(C)͎TVUXs{|rKE }Ѱ8y{d+O{$)lu;K/0^!j$ 0d?;̹Y~LJf L.oK8u/Vb/f;"z8l *D@/Ǒ26 5?Pwm42,MQ&evH 2%ֆYQ״cBVtF` ħiiT!oa4|LbL!pi?WϚ5(+EJ޿hͩ7`R(]˭d2ᣇۻдENy9<ʚ\rPQ A&_Kgrokevt-0.4.1/doc/man/man1/grokevt-ripdll.1.gz0000644000175000017500000000241210770556771017201 0ustar timtimGgrokevt-ripdll.1mU]o6}篸0)b:tEQcH-QHʪ_wH9Nŏ{=7ގ(ׅT~EF KZ.'W+=*y,*-[~@ajhY=XM]E9k}/vnd䶨kzO膛>M cVtN kob,jDcrZTjC3u߸q:Ilz;rˍWӰх0"CWkpěxq՞qUceOiA]%Y BI\܋ZGk sJ]%ѧo-ɝ45@l_0s LN36VԻGPBUQ+u: Y;zs,nR|J)*!0/0D,6(kOC**¬4"w{FlpM^.}@r ᡫYWlцB=b#6:8N7C+c㉾C8dC6vcѽUFr*E6u0сY'A']EFЦma"KQaWi5jSK[VqEEϛEg98&mq<0  /'[ñV+ 6GR+zߟY$z5r#|Cb7|ै]N U< XJz;̾0:~>;0տQ՞VŸ/_YT@3̫޽ltpw`{X9NN><_ig$1[=8lF^ŃnT.] ?$S11 dT)I6ͮowm$yvGHaH.=.n Wd7q2t6.>܈Oo8WI2|nIST$r4It-߾yӑ<6hWG"~pJ\ KYTM-N:l5ef]62+:k_M/&ݭM-3Z* PlcV6^[Q5P6ykUkܠX *eV0_uS4HmsinJ-mEf⑥FY,U5CAuɢr65TE[C῅ma*)9xjҬZzW7AD8FeN -][JcN^DIvҔP\uHĵ)Uzֈ Q^BZߧ'/tYO-.yZWϗ|9fjaݩ)]fCE=>[kxQ9xt*NץI [Uݘ1 䊶͓wž]|35֖]O QekoFBҚ]f* L/U7D7,n~Lq(D|)$m(% _AQpV d_s)!ݐbE J=sivLerv(Zr۵My\7PRB eBՄcΎXp$B r:$r*y ݕ:9X 0I)P<JMt`U7KCN# ⱝgQM|> *>챨̌C$q p謭YngINZB]D.N[d3kAMJ3]BWTK2C;ዸڿ 9pa+*R)N"DЛMj&tIBXW=C5h\P9x9yK2Q * lMraUQ-"G-Y~:*jG@U&8Qh0rz'R̿?, e8 Q *t讼__D4,U;NmBF{ 8as@J5xݔy~Ae2uv]@Yz>#;?dÖTvX*J w"\JNdOk1 S4E((n讐Quϳ`ˎo UH6,,bݻthcX{V,1aˬ{F!Cofeej4-lf3/8ڢcKIH1>TlܲYT-79B5TЮ)K& C?Hᒚ%yk+=c՝t.[OtANٮ!~Wozߞ.v6WXb `D44NњLЀ9pjx$aV:o{@ҍ >:;ܐ"Aglat3\X?K6F-&򝪻٤*`| R.}H! pxOw%Y!g٢V߹Ȏ 6#44}涡'u(d0TSp%_c05͍_` A6D&%2)Fu ]R$sBp7o/\bEw a<&ⶊnfn8t#e/v"¶5'?ML>WMXvO)<üB4䕮 7CA.)`]LaDb+E{+rw6_I=xx\|Y[TӪuŎc)gYE?kyeM:ݴw<&#l1 \ɩB!G/)q%Z,{Vl rBC,[m LE^W`Q7W(\̦>aK` (ɺ*n1Gǣ2:ܧǷW"u%`Jx,֋<; ?Ofg|pq;OuK9򧫇ˏ8nr3}!nSy}a6|xw+*{.EnRy᭄oK܉q2Gӫ+yq3;/Cx`&>C `Ι*s2 Ym!grokevt-0.4.1/etc/0000755000175000017500000000000010770556770012132 5ustar timtimgrokevt-0.4.1/etc/Makefile0000644000175000017500000000043410254412450013552 0ustar timtim# $Id: Makefile 9 2005-06-17 00:11:20Z tim $ ################################################################################ SYSTEMS=$(BUILD_ETC)/systems FILES=$(SYSTEMS) all: $(FILES) $(SYSTEMS): rm -rf $(SYSTEMS) cd systems && find . | grep -v '\.svn' | cpio -pmd $(SYSTEMS) grokevt-0.4.1/etc/systems/0000755000175000017500000000000010770556770013641 5ustar timtimgrokevt-0.4.1/etc/systems/example/0000755000175000017500000000000010770556770015274 5ustar timtimgrokevt-0.4.1/etc/systems/example/drives/0000755000175000017500000000000010770556770016570 5ustar timtimgrokevt-0.4.1/etc/systems/example/drives/c:0000644000175000017500000000001310241505531016777 0ustar timtim/mnt/win/c grokevt-0.4.1/etc/systems/example/drives/d:0000644000175000017500000000001310241505531017000 0ustar timtim/mnt/win/d grokevt-0.4.1/etc/systems/example/path-vars/0000755000175000017500000000000010770556770017201 5ustar timtimgrokevt-0.4.1/etc/systems/example/path-vars/%SystemRoot%0000644000175000017500000000001310770555654021360 0ustar timtimc:\Windows grokevt-0.4.1/etc/systems/example/system-registry0000644000175000017500000000005210770555654020406 0ustar timtim/mnt/win/c/WINDOWS/system32/config/system grokevt-0.4.1/lib/0000755000175000017500000000000010770556770012125 5ustar timtimgrokevt-0.4.1/lib/Makefile0000644000175000017500000000044410254412450013546 0ustar timtim# $Id: Makefile 9 2005-06-17 00:11:20Z tim $ ################################################################################ GROKEVT=$(BUILD_LIB)/grokevt.py FILES=$(GROKEVT) all: $(FILES) $(GROKEVT): grokevt.py cp grokevt.py $(GROKEVT) echo 'PATH_CONFIG="$(ETC_PREFIX)"' >> $(GROKEVT) grokevt-0.4.1/lib/grokevt.py0000644000175000017500000005772310537145773014174 0ustar timtim# # Copyright (C) 2005-2006 Timothy D. Morgan # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation version 2 of the # License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # vi:set tabsize=4: # $Id: grokevt.py 88 2006-12-11 03:09:47Z tim $ import sys import os import time import types import re import struct import anydbm ################################################################################ # Constants # This information provided by: # http://technet2.microsoft.com/WindowsServer/en/Library/7e77c2f0-8835-4bea-b972-26edb2aceb3d1033.mspx eventTypeEnum = {0:'Success', 1:'Error', 2:'Warning', 4:'Information', 8:'SuccessAudit', 16:'FailureAudit'} # XXX: this probably never changes, but might be interesting to see if # it is big endian on NT Alpha/MIPS systems. source_encoding = 'utf-16le' # This is what we store message in when ripped from DLLs template_encoding = 'utf-8' # This is what we use if unicode output is enabled output_encoding = 'utf-8' # Log format constants header_log_magic = "\x4c\x66\x4c\x65" header_version = "\x01\x00\x00\x00\x01\x00\x00\x00" cursor_magic ="\x11\x11\x11\x11\x22\x22\x22\x22\x33\x33\x33\x33\x44\x44\x44\x44" header_size = 0x30 cursor_size = 0x28 log_fixed_size = 0x38 min_record_size = min(header_size, cursor_size, log_fixed_size) ################################################################################ # String Formatting Functions # Reference: # http://blogs.msdn.com/oldnewthing/archive/2004/03/15/89753.aspx def binSIDtoASCII(sid_str): auth = ((ord(sid_str[2])<<40) | (ord(sid_str[3])<<32) | (ord(sid_str[4])<<24) | (ord(sid_str[5])<<16) | (ord(sid_str[6])<<8) | ord(sid_str[7])) result = "S-%d-%d" % (ord(sid_str[0]), auth) rest = sid_str[8:] for i in range(0,ord(sid_str[1])): if len(rest) >= 4: next_int = struct.unpack(' 126) or (c in specials): ret_val += ("\\x%.2X" % o) else: ret_val += c return ret_val def quoteUnicode(s, specials=u'\\\r\n'): ret_val = u'' for c in s: if c in specials: ret_val += "\\x%.2X" % ord(c) else: ret_val += c return ret_val # Reference: # http://msdn.microsoft.com/library/en-us/winui/winui/windowsuserinterface/resources/strings/stringreference/stringfunctions/wsprintf.asp # # returns a formatted string. def wsprintf(fmt, vars): # Python's string formatting is very close to wsprintf's. Just a # few types need to be converted to get a close approximation of # proper behavior. # XXX: However, as this isn't 100% compliant with the spec, it # should be re-written as a state-machine to be fully correct. optionals = r'(-?#?0?[0-9]*[.]?[0-9]*)' # S,ls,lS,hs,hS => s py_fmt = re.sub('%'+optionals+'[lh]{0,1}[sS]', r'%\1s', fmt) # lu,li,ld,hu,hi,hd => d py_fmt = re.sub('%'+optionals+'[lh][uid]', r'%\1d', py_fmt) # lc,lC,C => c py_fmt = re.sub('%'+optionals+'l{0,1}[cC]', r'%\1c', py_fmt) # lx => x; lX => X py_fmt = re.sub('%'+optionals+'l([xX])', r'%\1\2', py_fmt) # p => d py_fmt = re.sub('%'+optionals+'p', r'%\1d', py_fmt) return (py_fmt % vars) # References: # http://msdn.microsoft.com/library/en-us/debug/base/formatmessage.asp # http://msdn.microsoft.com/library/en-us/tools/tools/message_text_files.asp # XXX: Does someone know of a better reference for this? The author of # these pages couldn't write themselves out of a wet paper bag. def formatMessage(fmt, vars): # states: # 0: normal text # 1: in escape sequence # 3: in format string state=0 ret_val='' arg_num='' arg_index = None extended_fmt = '' for c in fmt: if state == 0: arg_index = None arg_num = '' if c == '%': state=1 else: ret_val += c elif state == 1: if len(arg_num) == 0: # ASCII digits {1..9} if ord(c) > 0x30 and ord(c) < 0x3a: arg_num = c # This probably isn't exactly correct, but it's better # than cutting off the message early. elif c == '0': ret_val += '\x00' state = 0 elif c == 'b': ret_val += ' ' state = 0 elif c == 't': ret_val += '\x09' state = 0 elif c == 'r': ret_val += '\x0d' state = 0 elif c == '\\': ret_val += '\x0a' state = 0 elif c == 'n': ret_val += '\x0d\x0a' state = 0 else: ret_val += c state = 0 elif len(arg_num) == 1: # ASCII digits {0..9} if ord(c) > 0x2f and ord(c) < 0x3a: arg_index = int(arg_num + c) - 1 else: if arg_index == None: arg_index = int(arg_num) - 1 if c == '!': state = 3 extended_fmt = '%' else: if arg_index < len(vars): if c == '%': ret_val += "%s" % vars[arg_index] state = 1 arg_num = '' arg_index = None else: ret_val += "%s%s" % (vars[arg_index], c) state = 0 else: # arg_num not in vars if c == '%': ret_val += "%%%s" % arg_num state = 1 arg_num = '' arg_index = None else: ret_val += "%%%s%s" % (arg_num, c) state = 0 elif state == 3: if c == '!': state = 0 ret_val += wsprintf(extended_fmt, vars[arg_index]) else: extended_fmt += c return ret_val def guessRecordType(record, wrapped=False): ret_val = 'unknown' # XXX: Can some of these be more strict? if (len(record) == header_size): magic = record[4:4+len(header_log_magic+header_version)] if magic == (header_log_magic+header_version): ret_val = 'header' elif (len(record) == cursor_size): magic = record[4:4+len(cursor_magic)] if(magic == cursor_magic): ret_val = 'cursor' elif (len(record) > log_fixed_size): magic = record[4:4+len(header_log_magic)] if(magic == header_log_magic): if wrapped: ret_val = 'wrapped-log' else: ret_val = 'log' return ret_val ################################################################################ # .evt log files class evtFile: # instance state f = None header = None cursor_offset = None cursor = None # Constructor for evtFile instances # Opens a log file and an associated message repository. # Optionally parses the .evt file's meta records (header and cursor). # # Raises: IOError # Resulting offset: # (error thrown) --> UNDEFINED # (parse_meta == 0) --> 0 # (parse_meta == 1 && missing header) --> 0 # (parse_meta == 1 && missing cursor) --> header_size # (parse_meta == 1 && found cursor) --> self.cursor['first_off'] # def __init__(self, filename, message_repository, parse_meta=1): self.f = file(filename, "rb") self.mr = message_repository if parse_meta: # First parse header if(self.guessRecordType() != 'header'): # XXX: should we search the file for a header and cursor # anyway? How would we trust offsets? sys.stderr.write("WARNING: Could not find header record.\n") self.f.seek(0) return self.header = self.getHeaderRecord() # Next, try to find the cursor self.f.seek(self.header['next_off']) if(self.guessRecordType() != 'cursor'): sys.stderr.write("WARNING: Header does not point "\ +"to cursor record.\n") sys.stderr.write("WARNING: Searching for cursor manually...\n") # XXX: This is kinda ugly. # Perhaps the whole file should be mmapped from the # beginning? self.f.seek(0) s = self.f.read() self.f.seek(0) if s.count(cursor_magic) > 1: sys.stderr.write("WARNING: Multiple cursors may exist."\ +" Attempting to use last one in file.\n") # Search for the cursor magic and attempt to validate the record magic_off = s.rfind(cursor_magic) if magic_off > 3: self.f.seek(magic_off-4) while (self.guessRecordType()!= 'cursor') and (magic_off != -1): magic_off = s.rfind(cursor_magic, magic_off-1) if magic_off > 3: self.f.seek(magic_off-4) s = None if magic_off == -1: sys.stderr.write("WARNING: Could not find cursor record.\n") self.f.seek(header_size) else: self.cursor_offset = magic_off-4 self.f.seek(self.cursor_offset) self.cursor = self.getCursorRecord() self.f.seek(self.cursor['first_off']) else: self.cursor_offset = self.header['next_off'] self.cursor = self.getCursorRecord() self.f.seek(self.cursor['first_off']) def tell(self): return self.f.tell() def seek(self, off, whence=0): self.f.seek(off, whence) def size(self): # XXX: is there a cleaner way to do this? cur_pos = self.f.tell() self.f.seek(0, 2) ret_val = self.f.tell() self.f.seek(cur_pos) return ret_val # Uses heuristics to determine the type of record at the current # file offset. Upon successful return, file offset remains the same. # Opon throwing an exception, offset is unspecified. # # Returns: a string indicating record type. Will be one of: # 'log','wrapped-log','header','cursor','unknown' # Raises: IOError def guessRecordType(self): if not self.f: raise IOError, "Log file not open." wrapped_log = False ret_val = 'unknown' cur_pos = self.f.tell() raw_str = self.f.read(4) if len(raw_str) == 4: (size1,) = struct.unpack('= cursor_size: if (size1 > log_fixed_size and (self.size() < cur_pos+size1)): if (self.size() < cur_pos+log_fixed_size): return ret_val wrapped_log = true self.f.seek(header_size+(cur_pos+size1-self.size()-4)) else: self.f.seek(cur_pos+size1-4) raw_str = self.f.read(4) if len(raw_str) == 4: (size2,) = struct.unpack('> 1 flag_logfull = (flags & 0x4) >> 2 flag_primary = (flags & 0x8) >> 3 ret_val = {'first_off':first_off, 'first_num':first_num, 'next_off':next_off, 'next_num':next_num, 'file_size':file_size, 'retention':retention, 'flag_dirty':flag_dirty, 'flag_wrapped':flag_wrapped, 'flag_logfull':flag_logfull, 'flag_primary':flag_primary} return ret_val # Parses a cursor record starting at the current log file offset # Resulting file offset will be set to the next record on success, # but is undefined if an exception is raised. # # Returns: a dictionary of cursor values # Raises: IOError, EOFError def getCursorRecord(self): fmt = ' 0: strs=rec_str[string_offset-4:data_offset-4].decode( source_encoding,'replace').split(u'\x00') # Grab source and computer fields vstr = variable_str.decode(source_encoding, 'replace').split(u'\x00', 2) source = '' if len(vstr) > 0: source = vstr[0] computer = '' if len(vstr) > 1: computer = vstr[1] vstr = None # Grab SID sid = 'N/A' if sid_len > 0: sid_str = rec_str[sid_offset-4:sid_offset+sid_len-4] sid = binSIDtoASCII(sid_str) # Grab binary data chunk data = '' if data_len > 0: data = rec_str[data_offset-4:data_offset+data_len-4] # Retrieve and process message template event_rva = (long(event_rva_offset) << 16) | event_id message_template = self.mr.getMessageTemplate(source, event_rva) message = '' if message_template: message = formatMessage(message_template, strs) else: sys.stderr.write("WARNING: Missing message"\ +" template for event record #%d. (service: %s)\n" % (msg_num, source)) event_type_str = eventTypeEnum.get(event_type, None) if not event_type_str: event_type_str = "Unknown(0x%.4X)" % event_type # Format fields and return return {'msg_num':msg_num, 'event_type':event_type_str, 'date_created':time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(date_created)), 'date_written':time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(date_written)), 'source':source, 'category':category, 'event_id':event_id, 'event_rva':"0x%s" % event_rva, 'user':sid, 'computer':computer, 'message':message, 'strings':'|'.join(strs).strip('|'), 'data':data} ################################################################################ # Message database wrapper class messageRepository: # XXX: Should this be made configurable, or do event logs implicitly # hard-code the language used at the time of writing a record via the # message ID (RVA)? In other words, does the same message in multiple # languages use the same RVA, just different language tables, or are # the RVAs different for each language (in addition to being in a # different table)? Should we base this on the default language # obtained from the registry? # XXX: The order is basically an arbitrary guess of the likelyhood of # finding logs in each of these languages. A better way of deciding # this is needed. # Codes for (en-US, de-DE, fr-FR, es-ES, Language Neutral) # See: http://msdn.microsoft.com/library/en-us/intl/nls_238z.asp languages = ("0409","0407", "040C", "0C0A", "0000") svc_dbs = {} msg_dbs = {} def __init__(self, topdir, log): # XXX: should these be opened on-demand and cached for repeated use? msg_dir = "%s/messages" % topdir dbs = os.listdir(msg_dir) for db in dbs: db_file = "%s/%s" % (msg_dir,db) self.msg_dbs[db] = anydbm.open(db_file, "r", 0644) log_dir = "%s/services" % topdir for t in ("category", "event", "parameter"): db_file = "%s/%s/%s.db" % (log_dir, log, t) self.svc_dbs[t] = anydbm.open(db_file, "r") def getMessageTemplate(self, service, rva, lang_code=None): if lang_code: langs = (lang_code,) else: langs = self.languages for lang in langs: mdbs = self.svc_dbs["event"].get(service.lower().encode('ascii'), None) if mdbs: for mdb in mdbs.split(':'): ret_val = self.msg_dbs[mdb].get("%s-%.8X"%(lang, rva), None) if ret_val: # Templates shouldn't have any encoding issues. # If they do, we want to know about them, since this # means there's a bug in builddb. return ret_val.decode(template_encoding) return None ################################################################################ # Configuration wrapper class grokevtConfig: path_vars = {} drive_mapping = {} registry_path = '' profile_path = '' def __init__(self, config_dir, profile): # Non-profile-specific configs would be read here (if there were any). self.profile_path = "%s/systems/%s" % (config_dir,profile) # XXX: the os.access() call may not work correctly in suid situations. if not os.path.isdir(self.profile_path)\ or not os.access(self.profile_path, os.R_OK): sys.stderr.write("WARNING: Could not read profile"\ +" directory '%s'.\n" % dir) else: self.registry_path = self.readLineFromFile("%s/system-registry" % self.profile_path) self.path_vars = self.readMappingFromFiles("%s/path-vars" % self.profile_path) self.drive_mapping = self.readMappingFromFiles("%s/drives" % self.profile_path) def readMappingFromFiles(self, dir): ret_val = {} # XXX: the os.access() call may not work correctly in suid situations. if os.path.isdir(dir) and os.access(dir, os.R_OK): for k in os.listdir(dir): l = self.readLineFromFile("%s/%s" % (dir, k)) if l != None: ret_val[k] = l else: sys.stderr.write("WARNING: Could not read configuration"\ +" directory '%s'.\n" % dir) return ret_val def readLineFromFile(self, fp): ret_val = None # XXX: the os.access() call may not work correctly in suid situations. if os.path.isfile(fp): if os.access(fp, os.R_OK): f = file(fp, "r") ret_val = f.readline().rstrip('\n\r') f.close() else: sys.stderr.write("WARNING: Could not read configuration"\ +" file '%s'.\n" % fp) return ret_val # This is only here to aide in debugging. # It will be overridden during a 'make install' below. PATH_CONFIG='/usr/local/etc/grokevt' ################################################################################ ### Below this line are build-time settings. ###