pax_global_header00006660000000000000000000000064121534331030014505gustar00rootroot0000000000000052 comment=045d00847b191da46faf1c1b91368a252412bb17 ninja-1.3.4/000077500000000000000000000000001215343310300126115ustar00rootroot00000000000000ninja-1.3.4/.gitignore000066400000000000000000000003621215343310300146020ustar00rootroot00000000000000*.pyc *.exe *.pdb *.ilk TAGS /build /build.ninja /ninja /build_log_perftest /canon_perftest /hash_collision_bench /ninja_test /parser_perftest /graph.png /doc/manual.html /doc/doxygen /gtest-1.6.0 # Eclipse project files .project .cproject ninja-1.3.4/.travis.yml000066400000000000000000000004221215343310300147200ustar00rootroot00000000000000language: cpp compiler: - gcc - clang before_install: - sudo apt-get update -qq - sudo apt-get install libgtest-dev script: ./bootstrap.py && ./configure.py --with-gtest=/usr/src/gtest && ./ninja ninja_test && ./ninja_test --gtest_filter=-SubprocessTest.SetWithLots ninja-1.3.4/COPYING000066400000000000000000000261361215343310300136540ustar00rootroot00000000000000 Apache License Version 2.0, January 2010 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ninja-1.3.4/HACKING.md000066400000000000000000000136751215343310300142130ustar00rootroot00000000000000## Basic overview `./configure.py` generates the `build.ninja` files used to build ninja. It accepts various flags to adjust build parameters. The primary build target of interest is `ninja`, but when hacking on Ninja your changes should be testable so it's more useful to build and run `ninja_test` when developing. (`./bootstrap.py` creates a bootstrap `ninja` and runs the above process; it's only necessary to run if you don't have a copy of `ninja` to build with.) ### Adjusting build flags Build in "debug" mode while developing (disables optimizations and builds way faster on Windows): ./configure.py --debug To use clang, set `CXX`: CXX=clang++ ./configure.py ## How to successfully make changes to Ninja Github pull requests are convenient for me to merge (I can just click a button and it's all handled server-side), but I'm also comfortable accepting pre-github git patches (via `send-email` etc.). Good pull requests have all of these attributes: * Are scoped to one specific issue * Include a test to demonstrate their correctness * Update the docs where relevant * Match the Ninja coding style (see below) * Don't include a mess of "oops, fix typo" commits These are typically merged without hesitation. If a change is lacking any of the above I usually will ask you to fix it, though there are obvious exceptions (fixing typos in comments don't need tests). I am very wary of changes that increase the complexity of Ninja (in particular, new build file syntax or command-line flags) or increase the maintenance burden of Ninja. Ninja is already successfully in use by hundreds of developers for large projects and it already achieves (most of) the goals I set out for it to do. It's probably best to discuss new feature ideas on the mailing list before I shoot down your patch. ## Testing ### Installing gtest The `ninja_test` binary, containing all the tests, depends on the googletest (gtest) library. * On older Ubuntus it'll install as libraries into `/usr/lib`: apt-get install libgtest * On newer Ubuntus it's only distributed as source apt-get install libgtest-dev ./configure.py --with-gtest=/usr/src/gtest * Otherwise you need to download it, unpack it, and pass `--with-gtest` to `configure.py`. Get it from [its downloads page](http://code.google.com/p/googletest/downloads/list); [this direct download link might work too](http://googletest.googlecode.com/files/gtest-1.6.0.zip). ### Test-driven development Set your build command to ./ninja ninja_test && ./ninja_test --gtest_filter=MyTest.Name now you can repeatedly run that while developing until the tests pass (I frequently set it as my compilation command in Emacs). Remember to build "all" before committing to verify the other source still works! ## Testing performance impact of changes If you have a Chrome build handy, it's a good test case. Otherwise, [the github downoads page](https://github.com/martine/ninja/downloads) has a copy of the Chrome build files (and depfiles). You can untar that, then run path/to/my/ninja chrome and compare that against a baseline Ninja. There's a script at `misc/measure.py` that repeatedly runs a command like the above (to address variance) and summarizes its runtime. E.g. path/to/misc/measure.py path/to/my/ninja chrome For changing the depfile parser, you can also build `parser_perftest` and run that directly on some representative input files. ## Coding guidelines Generally it's the [Google C++ coding style][], but in brief: * Function name are camelcase. * Member methods are camelcase, expect for trivial getters which are underscore separated. * Local variables are underscore separated. * Member variables are underscore separated and suffixed by an extra underscore. * Two spaces indentation. * Opening braces is at the end of line. * Lines are 80 columns maximum. * All source files should have the Google Inc. license header. [Google C++ coding style]: http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml ## Documentation ### Style guidelines * Use `///` for doxygen. * Use `\a` to refer to arguments. * It's not necessary to document each argument, especially when they're relatively self-evident (e.g. in `CanonicalizePath(string* path, string* err)`, the arguments are hopefully obvious) ### Building the manual sudo apt-get install asciidoc --no-install-recommends ./ninja manual ### Building the code documentation sudo apt-get install doxygen ./ninja doxygen ## Building for Windows While developing, it's helpful to copy `ninja.exe` to another name like `n.exe`; otherwise, rebuilds will be unable to write `ninja.exe` because it's locked while in use. ### Via Visual Studio * Install Visual Studio (Express is fine), [Python for Windows][], and (if making changes) googletest (see above instructions) * In a Visual Studio command prompt: `python bootstrap.py` [Python for Windows]: http://www.python.org/getit/windows/ ### Via mingw on Windows (not well supported) * Install mingw, msys, and python * In the mingw shell, put Python in your path, and `python bootstrap.py` * To reconfigure, run `python configure.py` * Remember to strip the resulting executable if size matters to you ### Via mingw on Linux (not well supported) Setup on Ubuntu Lucid: * `sudo apt-get install gcc-mingw32 wine` * `export CC=i586-mingw32msvc-cc CXX=i586-mingw32msvc-c++ AR=i586-mingw32msvc-ar` Setup on Ubuntu Precise: * `sudo apt-get install gcc-mingw-w64-i686 g++-mingw-w64-i686 wine` * `export CC=i686-w64-mingw32-gcc CXX=i686-w64-mingw32-g++ AR=i686-w64-mingw32-ar` Then run: * `./configure.py --platform=mingw --host=linux` * Build `ninja.exe` using a Linux ninja binary: `/path/to/linux/ninja` * Run: `./ninja.exe` (implicitly runs through wine(!)) ### Using Microsoft compilers on Linux (extremely flaky) The trick is to install just the compilers, and not all of Visual Studio, by following [these instructions][win7sdk]. [win7sdk]: http://www.kegel.com/wine/cl-howto-win7sdk.html ninja-1.3.4/README000066400000000000000000000011641215343310300134730ustar00rootroot00000000000000Ninja is a small build system with a focus on speed. http://martine.github.com/ninja/ See the manual -- http://martine.github.com/ninja/manual.html or doc/manual.asciidoc included in the distribution -- for background and more details. To build, run ./bootstrap.py. It first blindly compiles all non-test source files together, then re-builds Ninja using itself. You should end up with a 'ninja' binary in the source root. Run './ninja -h' for help. There is no installation step. The only file of interest to a user is the resulting ninja binary. If you're interested in making changes to Ninja, read HACKING.md first. ninja-1.3.4/RELEASING000066400000000000000000000007761215343310300140570ustar00rootroot00000000000000Notes to myself on all the steps to make for a Ninja release. 1. update src/version.cc with new version (with ".git") 2. git checkout release; git merge master 3. fix version number in src/version.cc (it will likely conflict in the above) 4. fix version in doc/manual.asciidoc 5. rebuild manual, put in place on website 6. commit, tag, push (don't forget to push --tags) 7. construct release notes from prior notes credits: git shortlog -s --no-merges REV.. 8. update home page mention of latest version. ninja-1.3.4/bootstrap.py000077500000000000000000000113551215343310300152100ustar00rootroot00000000000000#!/usr/bin/env python # Copyright 2011 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from __future__ import print_function from optparse import OptionParser import sys import os import glob import errno import shlex import shutil import subprocess import platform_helper os.chdir(os.path.dirname(os.path.abspath(__file__))) parser = OptionParser() parser.add_option('--verbose', action='store_true', help='enable verbose build',) parser.add_option('--x64', action='store_true', help='force 64-bit build (Windows)',) parser.add_option('--platform', help='target platform (' + '/'.join(platform_helper.platforms()) + ')', choices=platform_helper.platforms()) parser.add_option('--force-pselect', action='store_true', help="ppoll() is used by default on Linux and OpenBSD, but older versions might need to use pselect instead",) (options, conf_args) = parser.parse_args() platform = platform_helper.Platform(options.platform) conf_args.append("--platform=" + platform.platform()) def run(*args, **kwargs): returncode = subprocess.call(*args, **kwargs) if returncode != 0: sys.exit(returncode) # Compute system-specific CFLAGS/LDFLAGS as used in both in the below # g++ call as well as in the later configure.py. cflags = os.environ.get('CFLAGS', '').split() ldflags = os.environ.get('LDFLAGS', '').split() if platform.is_freebsd() or platform.is_openbsd(): cflags.append('-I/usr/local/include') ldflags.append('-L/usr/local/lib') print('Building ninja manually...') try: os.mkdir('build') except OSError: e = sys.exc_info()[1] if e.errno != errno.EEXIST: raise sources = [] for src in glob.glob('src/*.cc'): if src.endswith('test.cc') or src.endswith('.in.cc'): continue if src.endswith('bench.cc'): continue filename = os.path.basename(src) if filename == 'browse.cc': # Depends on generated header. continue if platform.is_windows(): if src.endswith('-posix.cc'): continue else: if src.endswith('-win32.cc'): continue sources.append(src) if platform.is_windows(): sources.append('src/getopt.c') if platform.is_msvc(): cl = 'cl' vcdir = os.environ.get('VCINSTALLDIR') if vcdir: if options.x64: cl = os.path.join(vcdir, 'bin', 'x86_amd64', 'cl.exe') if not os.path.exists(cl): cl = os.path.join(vcdir, 'bin', 'amd64', 'cl.exe') else: cl = os.path.join(vcdir, 'bin', 'cl.exe') args = [cl, '/nologo', '/EHsc', '/DNOMINMAX'] else: args = shlex.split(os.environ.get('CXX', 'g++')) cflags.extend(['-Wno-deprecated', '-DNINJA_PYTHON="' + sys.executable + '"', '-DNINJA_BOOTSTRAP']) if platform.is_windows(): cflags.append('-D_WIN32_WINNT=0x0501') if options.x64: cflags.append('-m64') if (platform.is_linux() or platform.is_openbsd()) and not options.force_pselect: cflags.append('-DUSE_PPOLL') if options.force_pselect: conf_args.append("--force-pselect") args.extend(cflags) args.extend(ldflags) binary = 'ninja.bootstrap' if platform.is_windows(): binary = 'ninja.bootstrap.exe' args.extend(sources) if platform.is_msvc(): args.extend(['/link', '/out:' + binary]) else: args.extend(['-o', binary]) if options.verbose: print(' '.join(args)) try: run(args) except: print('Failure running:', args) raise verbose = [] if options.verbose: verbose = ['-v'] if platform.is_windows(): print('Building ninja using itself...') run([sys.executable, 'configure.py'] + conf_args) run(['./' + binary] + verbose) # Copy the new executable over the bootstrap one. shutil.copyfile('ninja.exe', binary) # Clean up. for obj in glob.glob('*.obj'): os.unlink(obj) print(""" Done! Note: to work around Windows file locking, where you can't rebuild an in-use binary, to run ninja after making any changes to build ninja itself you should run ninja.bootstrap instead.""") else: print('Building ninja using itself...') run([sys.executable, 'configure.py'] + conf_args) run(['./' + binary] + verbose) os.unlink(binary) print('Done!') ninja-1.3.4/configure.py000077500000000000000000000354531215343310300151610ustar00rootroot00000000000000#!/usr/bin/env python # # Copyright 2001 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Script that generates the build.ninja for ninja itself. Projects that use ninja themselves should either write a similar script or use a meta-build system that supports Ninja output.""" from __future__ import print_function from optparse import OptionParser import os import sys import platform_helper sys.path.insert(0, 'misc') import ninja_syntax parser = OptionParser() profilers = ['gmon', 'pprof'] parser.add_option('--platform', help='target platform (' + '/'.join(platform_helper.platforms()) + ')', choices=platform_helper.platforms()) parser.add_option('--host', help='host platform (' + '/'.join(platform_helper.platforms()) + ')', choices=platform_helper.platforms()) parser.add_option('--debug', action='store_true', help='enable debugging extras',) parser.add_option('--profile', metavar='TYPE', choices=profilers, help='enable profiling (' + '/'.join(profilers) + ')',) parser.add_option('--with-gtest', metavar='PATH', help='use gtest unpacked in directory PATH') parser.add_option('--with-python', metavar='EXE', help='use EXE as the Python interpreter', default=os.path.basename(sys.executable)) parser.add_option('--force-pselect', action='store_true', help="ppoll() is used by default on Linux and OpenBSD, but older versions might need to use pselect instead",) (options, args) = parser.parse_args() if args: print('ERROR: extra unparsed command-line arguments:', args) sys.exit(1) platform = platform_helper.Platform(options.platform) if options.host: host = platform_helper.Platform(options.host) else: host = platform BUILD_FILENAME = 'build.ninja' buildfile = open(BUILD_FILENAME, 'w') n = ninja_syntax.Writer(buildfile) n.comment('This file is used to build ninja itself.') n.comment('It is generated by ' + os.path.basename(__file__) + '.') n.newline() n.variable('ninja_required_version', '1.3') n.newline() n.comment('The arguments passed to configure.py, for rerunning it.') n.variable('configure_args', ' '.join(sys.argv[1:])) env_keys = set(['CXX', 'AR', 'CFLAGS', 'LDFLAGS']) configure_env = dict((k, os.environ[k]) for k in os.environ if k in env_keys) if configure_env: config_str = ' '.join([k + '=' + configure_env[k] for k in configure_env]) n.variable('configure_env', config_str + '$ ') n.newline() CXX = configure_env.get('CXX', 'g++') objext = '.o' if platform.is_msvc(): CXX = 'cl' objext = '.obj' def src(filename): return os.path.join('src', filename) def built(filename): return os.path.join('$builddir', filename) def doc(filename): return os.path.join('doc', filename) def cc(name, **kwargs): return n.build(built(name + objext), 'cxx', src(name + '.c'), **kwargs) def cxx(name, **kwargs): return n.build(built(name + objext), 'cxx', src(name + '.cc'), **kwargs) def binary(name): if platform.is_windows(): exe = name + '.exe' n.build(name, 'phony', exe) return exe return name n.variable('builddir', 'build') n.variable('cxx', CXX) if platform.is_msvc(): n.variable('ar', 'link') else: n.variable('ar', configure_env.get('AR', 'ar')) if platform.is_msvc(): cflags = ['/nologo', # Don't print startup banner. '/Zi', # Create pdb with debug info. '/W4', # Highest warning level. '/WX', # Warnings as errors. '/wd4530', '/wd4100', '/wd4706', '/wd4512', '/wd4800', '/wd4702', '/wd4819', # Disable warnings about passing "this" during initialization. '/wd4355', '/GR-', # Disable RTTI. # Disable size_t -> int truncation warning. # We never have strings or arrays larger than 2**31. '/wd4267', '/DNOMINMAX', '/D_CRT_SECURE_NO_WARNINGS', '/D_VARIADIC_MAX=10', '/DNINJA_PYTHON="%s"' % options.with_python] ldflags = ['/DEBUG', '/libpath:$builddir'] if not options.debug: cflags += ['/Ox', '/DNDEBUG', '/GL'] ldflags += ['/LTCG', '/OPT:REF', '/OPT:ICF'] else: cflags = ['-g', '-Wall', '-Wextra', '-Wno-deprecated', '-Wno-unused-parameter', '-fno-rtti', '-fno-exceptions', '-fvisibility=hidden', '-pipe', '-Wno-missing-field-initializers', '-DNINJA_PYTHON="%s"' % options.with_python] if options.debug: cflags += ['-D_GLIBCXX_DEBUG', '-D_GLIBCXX_DEBUG_PEDANTIC'] cflags.remove('-fno-rtti') # Needed for above pedanticness. else: cflags += ['-O2', '-DNDEBUG'] if 'clang' in os.path.basename(CXX): cflags += ['-fcolor-diagnostics'] if platform.is_mingw(): cflags += ['-D_WIN32_WINNT=0x0501'] ldflags = ['-L$builddir'] libs = [] if platform.is_mingw(): cflags.remove('-fvisibility=hidden'); ldflags.append('-static') elif platform.is_sunos5(): cflags.remove('-fvisibility=hidden') elif platform.is_msvc(): pass else: if options.profile == 'gmon': cflags.append('-pg') ldflags.append('-pg') elif options.profile == 'pprof': cflags.append('-fno-omit-frame-pointer') libs.extend(['-Wl,--no-as-needed', '-lprofiler']) if (platform.is_linux() or platform.is_openbsd()) and not options.force_pselect: cflags.append('-DUSE_PPOLL') def shell_escape(str): """Escape str such that it's interpreted as a single argument by the shell.""" # This isn't complete, but it's just enough to make NINJA_PYTHON work. if platform.is_windows(): return str if '"' in str: return "'%s'" % str.replace("'", "\\'") return str if 'CFLAGS' in configure_env: cflags.append(configure_env['CFLAGS']) n.variable('cflags', ' '.join(shell_escape(flag) for flag in cflags)) if 'LDFLAGS' in configure_env: ldflags.append(configure_env['LDFLAGS']) n.variable('ldflags', ' '.join(shell_escape(flag) for flag in ldflags)) n.newline() if platform.is_msvc(): n.rule('cxx', command='$cxx /showIncludes $cflags -c $in /Fo$out', description='CXX $out', deps='msvc') else: n.rule('cxx', command='$cxx -MMD -MT $out -MF $out.d $cflags -c $in -o $out', depfile='$out.d', deps='gcc', description='CXX $out') n.newline() if host.is_msvc(): n.rule('ar', command='lib /nologo /ltcg /out:$out $in', description='LIB $out') elif host.is_mingw(): n.rule('ar', command='cmd /c $ar cqs $out.tmp $in && move /Y $out.tmp $out', description='AR $out') else: n.rule('ar', command='rm -f $out && $ar crs $out $in', description='AR $out') n.newline() if platform.is_msvc(): n.rule('link', command='$cxx $in $libs /nologo /link $ldflags /out:$out', description='LINK $out') else: n.rule('link', command='$cxx $ldflags -o $out $in $libs', description='LINK $out') n.newline() objs = [] if not platform.is_windows() and not platform.is_solaris(): n.comment('browse_py.h is used to inline browse.py.') n.rule('inline', command='src/inline.sh $varname < $in > $out', description='INLINE $out') n.build(built('browse_py.h'), 'inline', src('browse.py'), implicit='src/inline.sh', variables=[('varname', 'kBrowsePy')]) n.newline() objs += cxx('browse', order_only=built('browse_py.h')) n.newline() n.comment('the depfile parser and ninja lexers are generated using re2c.') def has_re2c(): import subprocess try: proc = subprocess.Popen(['re2c', '-V'], stdout=subprocess.PIPE) return int(proc.communicate()[0], 10) >= 1103 except OSError: return False if has_re2c(): n.rule('re2c', command='re2c -b -i --no-generation-date -o $out $in', description='RE2C $out') # Generate the .cc files in the source directory so we can check them in. n.build(src('depfile_parser.cc'), 're2c', src('depfile_parser.in.cc')) n.build(src('lexer.cc'), 're2c', src('lexer.in.cc')) else: print("warning: A compatible version of re2c (>= 0.11.3) was not found; " "changes to src/*.in.cc will not affect your build.") n.newline() n.comment('Core source files all build into ninja library.') for name in ['build', 'build_log', 'clean', 'depfile_parser', 'deps_log', 'disk_interface', 'edit_distance', 'eval_env', 'explain', 'graph', 'graphviz', 'lexer', 'line_printer', 'manifest_parser', 'metrics', 'state', 'util', 'version']: objs += cxx(name) if platform.is_windows(): for name in ['subprocess-win32', 'includes_normalize-win32', 'msvc_helper-win32', 'msvc_helper_main-win32']: objs += cxx(name) if platform.is_msvc(): objs += cxx('minidump-win32') objs += cc('getopt') else: objs += cxx('subprocess-posix') if platform.is_msvc(): ninja_lib = n.build(built('ninja.lib'), 'ar', objs) else: ninja_lib = n.build(built('libninja.a'), 'ar', objs) n.newline() if platform.is_msvc(): libs.append('ninja.lib') else: libs.append('-lninja') all_targets = [] n.comment('Main executable is library plus main() function.') objs = cxx('ninja') ninja = n.build(binary('ninja'), 'link', objs, implicit=ninja_lib, variables=[('libs', libs)]) n.newline() all_targets += ninja n.comment('Tests all build into ninja_test executable.') variables = [] test_cflags = cflags + ['-DGTEST_HAS_RTTI=0'] test_ldflags = None test_libs = libs objs = [] if options.with_gtest: path = options.with_gtest gtest_all_incs = '-I%s -I%s' % (path, os.path.join(path, 'include')) if platform.is_msvc(): gtest_cflags = '/nologo /EHsc /Zi /D_VARIADIC_MAX=10 ' + gtest_all_incs else: gtest_cflags = '-fvisibility=hidden ' + gtest_all_incs objs += n.build(built('gtest-all' + objext), 'cxx', os.path.join(path, 'src', 'gtest-all.cc'), variables=[('cflags', gtest_cflags)]) test_cflags.append('-I%s' % os.path.join(path, 'include')) else: # Use gtest from system. if platform.is_msvc(): test_libs.extend(['gtest_main.lib', 'gtest.lib']) else: test_libs.extend(['-lgtest_main', '-lgtest']) n.variable('test_cflags', test_cflags) for name in ['build_log_test', 'build_test', 'clean_test', 'depfile_parser_test', 'deps_log_test', 'disk_interface_test', 'edit_distance_test', 'graph_test', 'lexer_test', 'manifest_parser_test', 'ninja_test', 'state_test', 'subprocess_test', 'test', 'util_test']: objs += cxx(name, variables=[('cflags', '$test_cflags')]) if platform.is_windows(): for name in ['includes_normalize_test', 'msvc_helper_test']: objs += cxx(name, variables=[('cflags', test_cflags)]) if not platform.is_windows(): test_libs.append('-lpthread') ninja_test = n.build(binary('ninja_test'), 'link', objs, implicit=ninja_lib, variables=[('ldflags', test_ldflags), ('libs', test_libs)]) n.newline() all_targets += ninja_test n.comment('Ancillary executables.') objs = cxx('parser_perftest') all_targets += n.build(binary('parser_perftest'), 'link', objs, implicit=ninja_lib, variables=[('libs', libs)]) objs = cxx('build_log_perftest') all_targets += n.build(binary('build_log_perftest'), 'link', objs, implicit=ninja_lib, variables=[('libs', libs)]) objs = cxx('canon_perftest') all_targets += n.build(binary('canon_perftest'), 'link', objs, implicit=ninja_lib, variables=[('libs', libs)]) objs = cxx('hash_collision_bench') all_targets += n.build(binary('hash_collision_bench'), 'link', objs, implicit=ninja_lib, variables=[('libs', libs)]) n.newline() n.comment('Generate a graph using the "graph" tool.') n.rule('gendot', command='./ninja -t graph all > $out') n.rule('gengraph', command='dot -Tpng $in > $out') dot = n.build(built('graph.dot'), 'gendot', ['ninja', 'build.ninja']) n.build('graph.png', 'gengraph', dot) n.newline() n.comment('Generate the manual using asciidoc.') n.rule('asciidoc', command='asciidoc -b docbook -d book -o $out $in', description='ASCIIDOC $out') n.rule('xsltproc', command='xsltproc --nonet doc/docbook.xsl $in > $out', description='XSLTPROC $out') xml = n.build(built('manual.xml'), 'asciidoc', doc('manual.asciidoc')) manual = n.build(doc('manual.html'), 'xsltproc', xml, implicit=doc('style.css')) n.build('manual', 'phony', order_only=manual) n.newline() n.comment('Generate Doxygen.') n.rule('doxygen', command='doxygen $in', description='DOXYGEN $in') n.variable('doxygen_mainpage_generator', src('gen_doxygen_mainpage.sh')) n.rule('doxygen_mainpage', command='$doxygen_mainpage_generator $in > $out', description='DOXYGEN_MAINPAGE $out') mainpage = n.build(built('doxygen_mainpage'), 'doxygen_mainpage', ['README', 'COPYING'], implicit=['$doxygen_mainpage_generator']) n.build('doxygen', 'doxygen', doc('doxygen.config'), implicit=mainpage) n.newline() if not host.is_mingw(): n.comment('Regenerate build files if build script changes.') n.rule('configure', command='${configure_env}%s configure.py $configure_args' % options.with_python, generator=True) n.build('build.ninja', 'configure', implicit=['configure.py', os.path.normpath('misc/ninja_syntax.py')]) n.newline() n.default(ninja) n.newline() if host.is_linux(): n.comment('Packaging') n.rule('rpmbuild', command="misc/packaging/rpmbuild.sh", description='Building rpms..') n.build('rpm', 'rpmbuild') n.newline() n.build('all', 'phony', all_targets) print('wrote %s.' % BUILD_FILENAME) ninja-1.3.4/doc/000077500000000000000000000000001215343310300133565ustar00rootroot00000000000000ninja-1.3.4/doc/docbook.xsl000066400000000000000000000013521215343310300155270ustar00rootroot00000000000000 ]> ul ninja-1.3.4/doc/doxygen.config000066400000000000000000001436321215343310300162330ustar00rootroot00000000000000# Doxyfile 1.4.5 # This file describes the settings to be used by the documentation system # doxygen (www.doxygen.org) for a project # # All text after a hash (#) is considered a comment and will be ignored # The format is: # TAG = value [value, ...] # For lists items can also be appended using: # TAG += value [value, ...] # Values that contain spaces should be placed between quotes (" ") #--------------------------------------------------------------------------- # Project related configuration options #--------------------------------------------------------------------------- # The PROJECT_NAME tag is a single word (or a sequence of words surrounded # by quotes) that should identify the project. PROJECT_NAME = "Ninja" # The PROJECT_NUMBER tag can be used to enter a project or revision number. # This could be handy for archiving the generated documentation or # if some version control system is used. # PROJECT_NUMBER = "0" # The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) # base path where the generated documentation will be put. # If a relative path is entered, it will be relative to the location # where doxygen was started. If left blank the current directory will be used. OUTPUT_DIRECTORY = "doc/doxygen/" # If the CREATE_SUBDIRS tag is set to YES, then doxygen will create # 4096 sub-directories (in 2 levels) under the output directory of each output # format and will distribute the generated files over these directories. # Enabling this option can be useful when feeding doxygen a huge amount of # source files, where putting all generated files in the same directory would # otherwise cause performance problems for the file system. CREATE_SUBDIRS = NO # The OUTPUT_LANGUAGE tag is used to specify the language in which all # documentation generated by doxygen is written. Doxygen will use this # information to generate all constant output in the proper language. # The default language is English, other supported languages are: # Brazilian, Catalan, Chinese, Chinese-Traditional, Croatian, Czech, Danish, # Dutch, Finnish, French, German, Greek, Hungarian, Italian, Japanese, # Japanese-en (Japanese with English messages), Korean, Korean-en, Norwegian, # Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovene, Spanish, # Swedish, and Ukrainian. OUTPUT_LANGUAGE = English # This tag can be used to specify the encoding used in the generated output. # The encoding is not always determined by the language that is chosen, # but also whether or not the output is meant for Windows or non-Windows users. # In case there is a difference, setting the USE_WINDOWS_ENCODING tag to YES # forces the Windows encoding (this is the default for the Windows binary), # whereas setting the tag to NO uses a Unix-style encoding (the default for # all platforms other than Windows). # Obsolet option. #USE_WINDOWS_ENCODING = YES # If the BRIEF_MEMBER_DESC tag is set to YES (the default) Doxygen will # include brief member descriptions after the members that are listed in # the file and class documentation (similar to JavaDoc). # Set to NO to disable this. BRIEF_MEMBER_DESC = YES # If the REPEAT_BRIEF tag is set to YES (the default) Doxygen will prepend # the brief description of a member or function before the detailed description. # Note: if both HIDE_UNDOC_MEMBERS and BRIEF_MEMBER_DESC are set to NO, the # brief descriptions will be completely suppressed. REPEAT_BRIEF = YES # This tag implements a quasi-intelligent brief description abbreviator # that is used to form the text in various listings. Each string # in this list, if found as the leading text of the brief description, will be # stripped from the text and the result after processing the whole list, is # used as the annotated text. Otherwise, the brief description is used as-is. # If left blank, the following values are used ("$name" is automatically # replaced with the name of the entity): "The $name class" "The $name widget" # "The $name file" "is" "provides" "specifies" "contains" # "represents" "a" "an" "the" ABBREVIATE_BRIEF = # If the ALWAYS_DETAILED_SEC and REPEAT_BRIEF tags are both set to YES then # Doxygen will generate a detailed section even if there is only a brief # description. ALWAYS_DETAILED_SEC = NO # If the INLINE_INHERITED_MEMB tag is set to YES, doxygen will show all # inherited members of a class in the documentation of that class as if those # members were ordinary class members. Constructors, destructors and assignment # operators of the base classes will not be shown. INLINE_INHERITED_MEMB = YES # If the FULL_PATH_NAMES tag is set to YES then Doxygen will prepend the full # path before files name in the file list and in the header files. If set # to NO the shortest path that makes the file name unique will be used. FULL_PATH_NAMES = YES # If the FULL_PATH_NAMES tag is set to YES then the STRIP_FROM_PATH tag # can be used to strip a user-defined part of the path. Stripping is # only done if one of the specified strings matches the left-hand part of # the path. The tag can be used to show relative paths in the file list. # If left blank the directory from which doxygen is run is used as the # path to strip. STRIP_FROM_PATH = src # The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of # the path mentioned in the documentation of a class, which tells # the reader which header file to include in order to use a class. # If left blank only the name of the header file containing the class # definition is used. Otherwise one should specify the include paths that # are normally passed to the compiler using the -I flag. STRIP_FROM_INC_PATH = src/ # If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter # (but less readable) file names. This can be useful is your file systems # doesn't support long names like on DOS, Mac, or CD-ROM. SHORT_NAMES = NO # If the JAVADOC_AUTOBRIEF tag is set to YES then Doxygen # will interpret the first line (until the first dot) of a JavaDoc-style # comment as the brief description. If set to NO, the JavaDoc # comments will behave just like the Qt-style comments (thus requiring an # explicit @brief command for a brief description. JAVADOC_AUTOBRIEF = YES # The MULTILINE_CPP_IS_BRIEF tag can be set to YES to make Doxygen # treat a multi-line C++ special comment block (i.e. a block of //! or /// # comments) as a brief description. This used to be the default behaviour. # The new default is to treat a multi-line C++ comment block as a detailed # description. Set this tag to YES if you prefer the old behaviour instead. MULTILINE_CPP_IS_BRIEF = NO # If the DETAILS_AT_TOP tag is set to YES then Doxygen # will output the detailed description near the top, like JavaDoc. # If set to NO, the detailed description appears after the member # documentation. # Has become obsolete. #DETAILS_AT_TOP = NO # If the INHERIT_DOCS tag is set to YES (the default) then an undocumented # member inherits the documentation from any documented member that it # re-implements. INHERIT_DOCS = YES # If the SEPARATE_MEMBER_PAGES tag is set to YES, then doxygen will produce # a new page for each member. If set to NO, the documentation of a member will # be part of the file/class/namespace that contains it. SEPARATE_MEMBER_PAGES = NO # The TAB_SIZE tag can be used to set the number of spaces in a tab. # Doxygen uses this value to replace tabs by spaces in code fragments. TAB_SIZE = 2 # This tag can be used to specify a number of aliases that acts # as commands in the documentation. An alias has the form "name=value". # For example adding "sideeffect=\par Side Effects:\n" will allow you to # put the command \sideeffect (or @sideeffect) in the documentation, which # will result in a user-defined paragraph with heading "Side Effects:". # You can put \n's in the value part of an alias to insert newlines. ALIASES = # Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C # sources only. Doxygen will then generate output that is more tailored for C. # For instance, some of the names that are used will be different. The list # of all members will be omitted, etc. OPTIMIZE_OUTPUT_FOR_C = NO # Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java # sources only. Doxygen will then generate output that is more tailored for Java. # For instance, namespaces will be presented as packages, qualified scopes # will look different, etc. OPTIMIZE_OUTPUT_JAVA = NO # If you use STL classes (i.e. std::string, std::vector, etc.) but do not want to # include (a tag file for) the STL sources as input, then you should # set this tag to YES in order to let doxygen match functions declarations and # definitions whose arguments contain STL classes (e.g. func(std::string); v.s. # func(std::string) {}). This also make the inheritance and collaboration # diagrams that involve STL classes more complete and accurate. # BUILTIN_STL_SUPPORT = NO # If member grouping is used in the documentation and the DISTRIBUTE_GROUP_DOC # tag is set to YES, then doxygen will reuse the documentation of the first # member in the group (if any) for the other members of the group. By default # all members of a group must be documented explicitly. DISTRIBUTE_GROUP_DOC = NO # Set the SUBGROUPING tag to YES (the default) to allow class member groups of # the same type (for instance a group of public functions) to be put as a # subgroup of that type (e.g. under the Public Functions section). Set it to # NO to prevent subgrouping. Alternatively, this can be done per class using # the \nosubgrouping command. SUBGROUPING = YES #--------------------------------------------------------------------------- # Build related configuration options #--------------------------------------------------------------------------- # If the EXTRACT_ALL tag is set to YES doxygen will assume all entities in # documentation are documented, even if no documentation was available. # Private class members and static file members will be hidden unless # the EXTRACT_PRIVATE and EXTRACT_STATIC tags are set to YES EXTRACT_ALL = YES # If the EXTRACT_PRIVATE tag is set to YES all private members of a class # will be included in the documentation. EXTRACT_PRIVATE = YES # If the EXTRACT_STATIC tag is set to YES all static members of a file # will be included in the documentation. EXTRACT_STATIC = YES # If the EXTRACT_LOCAL_CLASSES tag is set to YES classes (and structs) # defined locally in source files will be included in the documentation. # If set to NO only classes defined in header files are included. EXTRACT_LOCAL_CLASSES = YES # This flag is only useful for Objective-C code. When set to YES local # methods, which are defined in the implementation section but not in # the interface are included in the documentation. # If set to NO (the default) only methods in the interface are included. EXTRACT_LOCAL_METHODS = NO # If the HIDE_UNDOC_MEMBERS tag is set to YES, Doxygen will hide all # undocumented members of documented classes, files or namespaces. # If set to NO (the default) these members will be included in the # various overviews, but no documentation section is generated. # This option has no effect if EXTRACT_ALL is enabled. HIDE_UNDOC_MEMBERS = NO # If the HIDE_UNDOC_CLASSES tag is set to YES, Doxygen will hide all # undocumented classes that are normally visible in the class hierarchy. # If set to NO (the default) these classes will be included in the various # overviews. This option has no effect if EXTRACT_ALL is enabled. HIDE_UNDOC_CLASSES = NO # If the HIDE_FRIEND_COMPOUNDS tag is set to YES, Doxygen will hide all # friend (class|struct|union) declarations. # If set to NO (the default) these declarations will be included in the # documentation. HIDE_FRIEND_COMPOUNDS = NO # If the HIDE_IN_BODY_DOCS tag is set to YES, Doxygen will hide any # documentation blocks found inside the body of a function. # If set to NO (the default) these blocks will be appended to the # function's detailed documentation block. HIDE_IN_BODY_DOCS = NO # The INTERNAL_DOCS tag determines if documentation # that is typed after a \internal command is included. If the tag is set # to NO (the default) then the documentation will be excluded. # Set it to YES to include the internal documentation. INTERNAL_DOCS = NO # If the CASE_SENSE_NAMES tag is set to NO then Doxygen will only generate # file names in lower-case letters. If set to YES upper-case letters are also # allowed. This is useful if you have classes or files whose names only differ # in case and if your file system supports case sensitive file names. Windows # and Mac users are advised to set this option to NO. CASE_SENSE_NAMES = YES # If the HIDE_SCOPE_NAMES tag is set to NO (the default) then Doxygen # will show members with their full class and namespace scopes in the # documentation. If set to YES the scope will be hidden. HIDE_SCOPE_NAMES = NO # If the SHOW_INCLUDE_FILES tag is set to YES (the default) then Doxygen # will put a list of the files that are included by a file in the documentation # of that file. SHOW_INCLUDE_FILES = YES # If the INLINE_INFO tag is set to YES (the default) then a tag [inline] # is inserted in the documentation for inline members. INLINE_INFO = YES # If the SORT_MEMBER_DOCS tag is set to YES (the default) then doxygen # will sort the (detailed) documentation of file and class members # alphabetically by member name. If set to NO the members will appear in # declaration order. SORT_MEMBER_DOCS = YES # If the SORT_BRIEF_DOCS tag is set to YES then doxygen will sort the # brief documentation of file, namespace and class members alphabetically # by member name. If set to NO (the default) the members will appear in # declaration order. SORT_BRIEF_DOCS = YES # If the SORT_BY_SCOPE_NAME tag is set to YES, the class list will be # sorted by fully-qualified names, including namespaces. If set to # NO (the default), the class list will be sorted only by class name, # not including the namespace part. # Note: This option is not very useful if HIDE_SCOPE_NAMES is set to YES. # Note: This option applies only to the class list, not to the # alphabetical list. SORT_BY_SCOPE_NAME = NO # The GENERATE_TODOLIST tag can be used to enable (YES) or # disable (NO) the todo list. This list is created by putting \todo # commands in the documentation. GENERATE_TODOLIST = YES # The GENERATE_TESTLIST tag can be used to enable (YES) or # disable (NO) the test list. This list is created by putting \test # commands in the documentation. GENERATE_TESTLIST = YES # The GENERATE_BUGLIST tag can be used to enable (YES) or # disable (NO) the bug list. This list is created by putting \bug # commands in the documentation. GENERATE_BUGLIST = YES # The GENERATE_DEPRECATEDLIST tag can be used to enable (YES) or # disable (NO) the deprecated list. This list is created by putting # \deprecated commands in the documentation. GENERATE_DEPRECATEDLIST= YES # The ENABLED_SECTIONS tag can be used to enable conditional # documentation sections, marked by \if sectionname ... \endif. ENABLED_SECTIONS = # The MAX_INITIALIZER_LINES tag determines the maximum number of lines # the initial value of a variable or define consists of for it to appear in # the documentation. If the initializer consists of more lines than specified # here it will be hidden. Use a value of 0 to hide initializers completely. # The appearance of the initializer of individual variables and defines in the # documentation can be controlled using \showinitializer or \hideinitializer # command in the documentation regardless of this setting. MAX_INITIALIZER_LINES = 30 # Set the SHOW_USED_FILES tag to NO to disable the list of files generated # at the bottom of the documentation of classes and structs. If set to YES the # list will mention the files that were used to generate the documentation. SHOW_USED_FILES = YES # If the sources in your project are distributed over multiple directories # then setting the SHOW_DIRECTORIES tag to YES will show the directory hierarchy # in the documentation. The default is YES. SHOW_DIRECTORIES = YES # The FILE_VERSION_FILTER tag can be used to specify a program or script that # doxygen should invoke to get the current version for each file (typically from the # version control system). Doxygen will invoke the program by executing (via # popen()) the command , where is the value of # the FILE_VERSION_FILTER tag, and is the name of an input file # provided by doxygen. Whatever the program writes to standard output # is used as the file version. See the manual for examples. FILE_VERSION_FILTER = #--------------------------------------------------------------------------- # configuration options related to warning and progress messages #--------------------------------------------------------------------------- # The QUIET tag can be used to turn on/off the messages that are generated # by doxygen. Possible values are YES and NO. If left blank NO is used. QUIET = NO # The WARNINGS tag can be used to turn on/off the warning messages that are # generated by doxygen. Possible values are YES and NO. If left blank # NO is used. WARNINGS = YES # If WARN_IF_UNDOCUMENTED is set to YES, then doxygen will generate warnings # for undocumented members. If EXTRACT_ALL is set to YES then this flag will # automatically be disabled. WARN_IF_UNDOCUMENTED = YES # If WARN_IF_DOC_ERROR is set to YES, doxygen will generate warnings for # potential errors in the documentation, such as not documenting some # parameters in a documented function, or documenting parameters that # don't exist or using markup commands wrongly. WARN_IF_DOC_ERROR = YES # This WARN_NO_PARAMDOC option can be abled to get warnings for # functions that are documented, but have no documentation for their parameters # or return value. If set to NO (the default) doxygen will only warn about # wrong or incomplete parameter documentation, but not about the absence of # documentation. WARN_NO_PARAMDOC = NO # The WARN_FORMAT tag determines the format of the warning messages that # doxygen can produce. The string should contain the $file, $line, and $text # tags, which will be replaced by the file and line number from which the # warning originated and the warning text. Optionally the format may contain # $version, which will be replaced by the version of the file (if it could # be obtained via FILE_VERSION_FILTER) WARN_FORMAT = "$file:$line: $text " # The WARN_LOGFILE tag can be used to specify a file to which warning # and error messages should be written. If left blank the output is written # to stderr. WARN_LOGFILE = #--------------------------------------------------------------------------- # configuration options related to the input files #--------------------------------------------------------------------------- # The INPUT tag can be used to specify the files and/or directories that contain # documented source files. You may enter file names like "myfile.cpp" or # directories like "/usr/src/myproject". Separate the files or directories # with spaces. INPUT = src \ build/doxygen_mainpage # If the value of the INPUT tag contains directories, you can use the # FILE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp # and *.h) to filter out the source-files in the directories. If left # blank the following patterns are tested: # *.c *.cc *.cxx *.cpp *.c++ *.java *.ii *.ixx *.ipp *.i++ *.inl *.h *.hh *.hxx # *.hpp *.h++ *.idl *.odl *.cs *.php *.php3 *.inc *.m *.mm *.py FILE_PATTERNS = *.cc \ *.h # The RECURSIVE tag can be used to turn specify whether or not subdirectories # should be searched for input files as well. Possible values are YES and NO. # If left blank NO is used. RECURSIVE = YES # The EXCLUDE tag can be used to specify files and/or directories that should # excluded from the INPUT source files. This way you can easily exclude a # subdirectory from a directory tree whose root is specified with the INPUT tag. EXCLUDE = # The EXCLUDE_SYMLINKS tag can be used select whether or not files or # directories that are symbolic links (a Unix filesystem feature) are excluded # from the input. EXCLUDE_SYMLINKS = NO # If the value of the INPUT tag contains directories, you can use the # EXCLUDE_PATTERNS tag to specify one or more wildcard patterns to exclude # certain files from those directories. Note that the wildcards are matched # against the file with absolute path, so to exclude all test directories # for example use the pattern */test/* EXCLUDE_PATTERNS = # The EXAMPLE_PATH tag can be used to specify one or more files or # directories that contain example code fragments that are included (see # the \include command). EXAMPLE_PATH = src # If the value of the EXAMPLE_PATH tag contains directories, you can use the # EXAMPLE_PATTERNS tag to specify one or more wildcard pattern (like *.cpp # and *.h) to filter out the source-files in the directories. If left # blank all files are included. EXAMPLE_PATTERNS = *.cpp \ *.cc \ *.h \ *.hh \ INSTALL DEPENDENCIES CHANGELOG LICENSE LGPL # If the EXAMPLE_RECURSIVE tag is set to YES then subdirectories will be # searched for input files to be used with the \include or \dontinclude # commands irrespective of the value of the RECURSIVE tag. # Possible values are YES and NO. If left blank NO is used. EXAMPLE_RECURSIVE = YES # The IMAGE_PATH tag can be used to specify one or more files or # directories that contain image that are included in the documentation (see # the \image command). IMAGE_PATH = src # The INPUT_FILTER tag can be used to specify a program that doxygen should # invoke to filter for each input file. Doxygen will invoke the filter program # by executing (via popen()) the command , where # is the value of the INPUT_FILTER tag, and is the name of an # input file. Doxygen will then use the output that the filter program writes # to standard output. If FILTER_PATTERNS is specified, this tag will be # ignored. INPUT_FILTER = # The FILTER_PATTERNS tag can be used to specify filters on a per file pattern # basis. Doxygen will compare the file name with each pattern and apply the # filter if there is a match. The filters are a list of the form: # pattern=filter (like *.cpp=my_cpp_filter). See INPUT_FILTER for further # info on how filters are used. If FILTER_PATTERNS is empty, INPUT_FILTER # is applied to all files. FILTER_PATTERNS = # If the FILTER_SOURCE_FILES tag is set to YES, the input filter (if set using # INPUT_FILTER) will be used to filter the input files when producing source # files to browse (i.e. when SOURCE_BROWSER is set to YES). FILTER_SOURCE_FILES = NO #--------------------------------------------------------------------------- # configuration options related to source browsing #--------------------------------------------------------------------------- # If the SOURCE_BROWSER tag is set to YES then a list of source files will # be generated. Documented entities will be cross-referenced with these sources. # Note: To get rid of all source code in the generated output, make sure also # VERBATIM_HEADERS is set to NO. SOURCE_BROWSER = YES # Setting the INLINE_SOURCES tag to YES will include the body # of functions and classes directly in the documentation. INLINE_SOURCES = NO # Setting the STRIP_CODE_COMMENTS tag to YES (the default) will instruct # doxygen to hide any special comment blocks from generated source code # fragments. Normal C and C++ comments will always remain visible. STRIP_CODE_COMMENTS = NO # If the REFERENCED_BY_RELATION tag is set to YES (the default) # then for each documented function all documented # functions referencing it will be listed. REFERENCED_BY_RELATION = YES # If the REFERENCES_RELATION tag is set to YES (the default) # then for each documented function all documented entities # called/used by that function will be listed. REFERENCES_RELATION = YES # If the USE_HTAGS tag is set to YES then the references to source code # will point to the HTML generated by the htags(1) tool instead of doxygen # built-in source browser. The htags tool is part of GNU's global source # tagging system (see http://www.gnu.org/software/global/global.html). You # will need version 4.8.6 or higher. USE_HTAGS = NO # If the VERBATIM_HEADERS tag is set to YES (the default) then Doxygen # will generate a verbatim copy of the header file for each class for # which an include is specified. Set to NO to disable this. VERBATIM_HEADERS = YES #--------------------------------------------------------------------------- # configuration options related to the alphabetical class index #--------------------------------------------------------------------------- # If the ALPHABETICAL_INDEX tag is set to YES, an alphabetical index # of all compounds will be generated. Enable this if the project # contains a lot of classes, structs, unions or interfaces. ALPHABETICAL_INDEX = YES # If the alphabetical index is enabled (see ALPHABETICAL_INDEX) then # the COLS_IN_ALPHA_INDEX tag can be used to specify the number of columns # in which this list will be split (can be a number in the range [1..20]) COLS_IN_ALPHA_INDEX = 2 # In case all classes in a project start with a common prefix, all # classes will be put under the same header in the alphabetical index. # The IGNORE_PREFIX tag can be used to specify one or more prefixes that # should be ignored while generating the index headers. IGNORE_PREFIX = #--------------------------------------------------------------------------- # configuration options related to the HTML output #--------------------------------------------------------------------------- # If the GENERATE_HTML tag is set to YES (the default) Doxygen will # generate HTML output. GENERATE_HTML = YES # The HTML_OUTPUT tag is used to specify where the HTML docs will be put. # If a relative path is entered the value of OUTPUT_DIRECTORY will be # put in front of it. If left blank `html' will be used as the default path. HTML_OUTPUT = html # The HTML_FILE_EXTENSION tag can be used to specify the file extension for # each generated HTML page (for example: .htm,.php,.asp). If it is left blank # doxygen will generate files with .html extension. HTML_FILE_EXTENSION = .html # The HTML_HEADER tag can be used to specify a personal HTML header for # each generated HTML page. If it is left blank doxygen will generate a # standard header. HTML_HEADER = # The HTML_FOOTER tag can be used to specify a personal HTML footer for # each generated HTML page. If it is left blank doxygen will generate a # standard footer. HTML_FOOTER = # The HTML_STYLESHEET tag can be used to specify a user-defined cascading # style sheet that is used by each HTML page. It can be used to # fine-tune the look of the HTML output. If the tag is left blank doxygen # will generate a default style sheet. Note that doxygen will try to copy # the style sheet file to the HTML output directory, so don't put your own # stylesheet in the HTML output directory as well, or it will be erased! HTML_STYLESHEET = # If the HTML_ALIGN_MEMBERS tag is set to YES, the members of classes, # files or namespaces will be aligned in HTML using tables. If set to # NO a bullet list will be used. HTML_ALIGN_MEMBERS = YES # If the GENERATE_HTMLHELP tag is set to YES, additional index files # will be generated that can be used as input for tools like the # Microsoft HTML help workshop to generate a compressed HTML help file (.chm) # of the generated HTML documentation. GENERATE_HTMLHELP = YES # If the GENERATE_HTMLHELP tag is set to YES, the CHM_FILE tag can # be used to specify the file name of the resulting .chm file. You # can add a path in front of the file if the result should not be # written to the html output directory. CHM_FILE = # If the GENERATE_HTMLHELP tag is set to YES, the HHC_LOCATION tag can # be used to specify the location (absolute path including file name) of # the HTML help compiler (hhc.exe). If non-empty doxygen will try to run # the HTML help compiler on the generated index.hhp. HHC_LOCATION = # If the GENERATE_HTMLHELP tag is set to YES, the GENERATE_CHI flag # controls if a separate .chi index file is generated (YES) or that # it should be included in the master .chm file (NO). GENERATE_CHI = NO # If the GENERATE_HTMLHELP tag is set to YES, the BINARY_TOC flag # controls whether a binary table of contents is generated (YES) or a # normal table of contents (NO) in the .chm file. BINARY_TOC = NO # The TOC_EXPAND flag can be set to YES to add extra items for group members # to the contents of the HTML help documentation and to the tree view. TOC_EXPAND = NO # The DISABLE_INDEX tag can be used to turn on/off the condensed index at # top of each HTML page. The value NO (the default) enables the index and # the value YES disables it. DISABLE_INDEX = NO # This tag can be used to set the number of enum values (range [1..20]) # that doxygen will group on one line in the generated HTML documentation. ENUM_VALUES_PER_LINE = 4 # If the GENERATE_TREEVIEW tag is set to YES, a side panel will be # generated containing a tree-like index structure (just like the one that # is generated for HTML Help). For this to work a browser that supports # JavaScript, DHTML, CSS and frames is required (for instance Mozilla 1.0+, # Netscape 6.0+, Internet explorer 5.0+, or Konqueror). Windows users are # probably better off using the HTML help feature. GENERATE_TREEVIEW = YES # If the treeview is enabled (see GENERATE_TREEVIEW) then this tag can be # used to set the initial width (in pixels) of the frame in which the tree # is shown. TREEVIEW_WIDTH = 250 #--------------------------------------------------------------------------- # configuration options related to the LaTeX output #--------------------------------------------------------------------------- # If the GENERATE_LATEX tag is set to YES (the default) Doxygen will # generate Latex output. GENERATE_LATEX = NO # The LATEX_OUTPUT tag is used to specify where the LaTeX docs will be put. # If a relative path is entered the value of OUTPUT_DIRECTORY will be # put in front of it. If left blank `latex' will be used as the default path. LATEX_OUTPUT = latex # The LATEX_CMD_NAME tag can be used to specify the LaTeX command name to be # invoked. If left blank `latex' will be used as the default command name. LATEX_CMD_NAME = # The MAKEINDEX_CMD_NAME tag can be used to specify the command name to # generate index for LaTeX. If left blank `makeindex' will be used as the # default command name. MAKEINDEX_CMD_NAME = # If the COMPACT_LATEX tag is set to YES Doxygen generates more compact # LaTeX documents. This may be useful for small projects and may help to # save some trees in general. COMPACT_LATEX = NO # The PAPER_TYPE tag can be used to set the paper type that is used # by the printer. Possible values are: a4, a4wide, letter, legal and # executive. If left blank a4wide will be used. PAPER_TYPE = a4 # The EXTRA_PACKAGES tag can be to specify one or more names of LaTeX # packages that should be included in the LaTeX output. EXTRA_PACKAGES = # The LATEX_HEADER tag can be used to specify a personal LaTeX header for # the generated latex document. The header should contain everything until # the first chapter. If it is left blank doxygen will generate a # standard header. Notice: only use this tag if you know what you are doing! LATEX_HEADER = # If the PDF_HYPERLINKS tag is set to YES, the LaTeX that is generated # is prepared for conversion to pdf (using ps2pdf). The pdf file will # contain links (just like the HTML output) instead of page references # This makes the output suitable for online browsing using a pdf viewer. PDF_HYPERLINKS = YES # If the USE_PDFLATEX tag is set to YES, pdflatex will be used instead of # plain latex in the generated Makefile. Set this option to YES to get a # higher quality PDF documentation. USE_PDFLATEX = YES # If the LATEX_BATCHMODE tag is set to YES, doxygen will add the \\batchmode. # command to the generated LaTeX files. This will instruct LaTeX to keep # running if errors occur, instead of asking the user for help. # This option is also used when generating formulas in HTML. LATEX_BATCHMODE = YES # If LATEX_HIDE_INDICES is set to YES then doxygen will not # include the index chapters (such as File Index, Compound Index, etc.) # in the output. LATEX_HIDE_INDICES = NO #--------------------------------------------------------------------------- # configuration options related to the RTF output #--------------------------------------------------------------------------- # If the GENERATE_RTF tag is set to YES Doxygen will generate RTF output # The RTF output is optimized for Word 97 and may not look very pretty with # other RTF readers or editors. GENERATE_RTF = NO # The RTF_OUTPUT tag is used to specify where the RTF docs will be put. # If a relative path is entered the value of OUTPUT_DIRECTORY will be # put in front of it. If left blank `rtf' will be used as the default path. RTF_OUTPUT = rtf # If the COMPACT_RTF tag is set to YES Doxygen generates more compact # RTF documents. This may be useful for small projects and may help to # save some trees in general. COMPACT_RTF = NO # If the RTF_HYPERLINKS tag is set to YES, the RTF that is generated # will contain hyperlink fields. The RTF file will # contain links (just like the HTML output) instead of page references. # This makes the output suitable for online browsing using WORD or other # programs which support those fields. # Note: wordpad (write) and others do not support links. RTF_HYPERLINKS = NO # Load stylesheet definitions from file. Syntax is similar to doxygen's # config file, i.e. a series of assignments. You only have to provide # replacements, missing definitions are set to their default value. RTF_STYLESHEET_FILE = # Set optional variables used in the generation of an rtf document. # Syntax is similar to doxygen's config file. RTF_EXTENSIONS_FILE = #--------------------------------------------------------------------------- # configuration options related to the man page output #--------------------------------------------------------------------------- # If the GENERATE_MAN tag is set to YES (the default) Doxygen will # generate man pages GENERATE_MAN = NO # The MAN_OUTPUT tag is used to specify where the man pages will be put. # If a relative path is entered the value of OUTPUT_DIRECTORY will be # put in front of it. If left blank `man' will be used as the default path. MAN_OUTPUT = man # The MAN_EXTENSION tag determines the extension that is added to # the generated man pages (default is the subroutine's section .3) MAN_EXTENSION = .3 # If the MAN_LINKS tag is set to YES and Doxygen generates man output, # then it will generate one additional man file for each entity # documented in the real man page(s). These additional files # only source the real man page, but without them the man command # would be unable to find the correct page. The default is NO. MAN_LINKS = NO #--------------------------------------------------------------------------- # configuration options related to the XML output #--------------------------------------------------------------------------- # If the GENERATE_XML tag is set to YES Doxygen will # generate an XML file that captures the structure of # the code including all documentation. GENERATE_XML = NO # The XML_OUTPUT tag is used to specify where the XML pages will be put. # If a relative path is entered the value of OUTPUT_DIRECTORY will be # put in front of it. If left blank `xml' will be used as the default path. XML_OUTPUT = xml # The XML_SCHEMA tag can be used to specify an XML schema, # which can be used by a validating XML parser to check the # syntax of the XML files. XML_SCHEMA = # The XML_DTD tag can be used to specify an XML DTD, # which can be used by a validating XML parser to check the # syntax of the XML files. XML_DTD = # If the XML_PROGRAMLISTING tag is set to YES Doxygen will # dump the program listings (including syntax highlighting # and cross-referencing information) to the XML output. Note that # enabling this will significantly increase the size of the XML output. XML_PROGRAMLISTING = YES #--------------------------------------------------------------------------- # configuration options for the AutoGen Definitions output #--------------------------------------------------------------------------- # If the GENERATE_AUTOGEN_DEF tag is set to YES Doxygen will # generate an AutoGen Definitions (see autogen.sf.net) file # that captures the structure of the code including all # documentation. Note that this feature is still experimental # and incomplete at the moment. GENERATE_AUTOGEN_DEF = NO #--------------------------------------------------------------------------- # configuration options related to the Perl module output #--------------------------------------------------------------------------- # If the GENERATE_PERLMOD tag is set to YES Doxygen will # generate a Perl module file that captures the structure of # the code including all documentation. Note that this # feature is still experimental and incomplete at the # moment. GENERATE_PERLMOD = NO # If the PERLMOD_LATEX tag is set to YES Doxygen will generate # the necessary Makefile rules, Perl scripts and LaTeX code to be able # to generate PDF and DVI output from the Perl module output. PERLMOD_LATEX = NO # If the PERLMOD_PRETTY tag is set to YES the Perl module output will be # nicely formatted so it can be parsed by a human reader. This is useful # if you want to understand what is going on. On the other hand, if this # tag is set to NO the size of the Perl module output will be much smaller # and Perl will parse it just the same. PERLMOD_PRETTY = YES # The names of the make variables in the generated doxyrules.make file # are prefixed with the string contained in PERLMOD_MAKEVAR_PREFIX. # This is useful so different doxyrules.make files included by the same # Makefile don't overwrite each other's variables. PERLMOD_MAKEVAR_PREFIX = #--------------------------------------------------------------------------- # Configuration options related to the preprocessor #--------------------------------------------------------------------------- # If the ENABLE_PREPROCESSING tag is set to YES (the default) Doxygen will # evaluate all C-preprocessor directives found in the sources and include # files. ENABLE_PREPROCESSING = YES # If the MACRO_EXPANSION tag is set to YES Doxygen will expand all macro # names in the source code. If set to NO (the default) only conditional # compilation will be performed. Macro expansion can be done in a controlled # way by setting EXPAND_ONLY_PREDEF to YES. MACRO_EXPANSION = YES # If the EXPAND_ONLY_PREDEF and MACRO_EXPANSION tags are both set to YES # then the macro expansion is limited to the macros specified with the # PREDEFINED and EXPAND_AS_DEFINED tags. EXPAND_ONLY_PREDEF = YES # If the SEARCH_INCLUDES tag is set to YES (the default) the includes files # in the INCLUDE_PATH (see below) will be search if a #include is found. SEARCH_INCLUDES = YES # The INCLUDE_PATH tag can be used to specify one or more directories that # contain include files that are not input files but should be processed by # the preprocessor. INCLUDE_PATH = # You can use the INCLUDE_FILE_PATTERNS tag to specify one or more wildcard # patterns (like *.h and *.hpp) to filter out the header-files in the # directories. If left blank, the patterns specified with FILE_PATTERNS will # be used. INCLUDE_FILE_PATTERNS = # The PREDEFINED tag can be used to specify one or more macro names that # are defined before the preprocessor is started (similar to the -D option of # gcc). The argument of the tag is a list of macros of the form: name # or name=definition (no spaces). If the definition and the = are # omitted =1 is assumed. To prevent a macro definition from being # undefined via #undef or recursively expanded use the := operator # instead of the = operator. PREDEFINED = # If the MACRO_EXPANSION and EXPAND_ONLY_PREDEF tags are set to YES then # this tag can be used to specify a list of macro names that should be expanded. # The macro definition that is found in the sources will be used. # Use the PREDEFINED tag if you want to use a different macro definition. EXPAND_AS_DEFINED = # If the SKIP_FUNCTION_MACROS tag is set to YES (the default) then # doxygen's preprocessor will remove all function-like macros that are alone # on a line, have an all uppercase name, and do not end with a semicolon. Such # function macros are typically used for boiler-plate code, and will confuse # the parser if not removed. SKIP_FUNCTION_MACROS = YES #--------------------------------------------------------------------------- # Configuration::additions related to external references #--------------------------------------------------------------------------- # The TAGFILES option can be used to specify one or more tagfiles. # Optionally an initial location of the external documentation # can be added for each tagfile. The format of a tag file without # this location is as follows: # TAGFILES = file1 file2 ... # Adding location for the tag files is done as follows: # TAGFILES = file1=loc1 "file2 = loc2" ... # where "loc1" and "loc2" can be relative or absolute paths or # URLs. If a location is present for each tag, the installdox tool # does not have to be run to correct the links. # Note that each tag file must have a unique name # (where the name does NOT include the path) # If a tag file is not located in the directory in which doxygen # is run, you must also specify the path to the tagfile here. TAGFILES = # When a file name is specified after GENERATE_TAGFILE, doxygen will create # a tag file that is based on the input files it reads. GENERATE_TAGFILE = doc/doxygen/html/Ninja.TAGFILE # If the ALLEXTERNALS tag is set to YES all external classes will be listed # in the class index. If set to NO only the inherited external classes # will be listed. ALLEXTERNALS = YES # If the EXTERNAL_GROUPS tag is set to YES all external groups will be listed # in the modules index. If set to NO, only the current project's groups will # be listed. EXTERNAL_GROUPS = YES # The PERL_PATH should be the absolute path and name of the perl script # interpreter (i.e. the result of `which perl'). PERL_PATH = /usr/bin/perl #--------------------------------------------------------------------------- # Configuration options related to the dot tool #--------------------------------------------------------------------------- # If the CLASS_DIAGRAMS tag is set to YES (the default) Doxygen will # generate a inheritance diagram (in HTML, RTF and LaTeX) for classes with base # or super classes. Setting the tag to NO turns the diagrams off. Note that # this option is superseded by the HAVE_DOT option below. This is only a # fallback. It is recommended to install and use dot, since it yields more # powerful graphs. CLASS_DIAGRAMS = YES # If set to YES, the inheritance and collaboration graphs will hide # inheritance and usage relations if the target is undocumented # or is not a class. HIDE_UNDOC_RELATIONS = YES # If you set the HAVE_DOT tag to YES then doxygen will assume the dot tool is # available from the path. This tool is part of Graphviz, a graph visualization # toolkit from AT&T and Lucent Bell Labs. The other options in this section # have no effect if this option is set to NO (the default) HAVE_DOT = YES # If the CLASS_GRAPH and HAVE_DOT tags are set to YES then doxygen # will generate a graph for each documented class showing the direct and # indirect inheritance relations. Setting this tag to YES will force the # the CLASS_DIAGRAMS tag to NO. CLASS_GRAPH = YES # If the COLLABORATION_GRAPH and HAVE_DOT tags are set to YES then doxygen # will generate a graph for each documented class showing the direct and # indirect implementation dependencies (inheritance, containment, and # class references variables) of the class with other documented classes. COLLABORATION_GRAPH = NO # If the GROUP_GRAPHS and HAVE_DOT tags are set to YES then doxygen # will generate a graph for groups, showing the direct groups dependencies GROUP_GRAPHS = YES # If the UML_LOOK tag is set to YES doxygen will generate inheritance and # collaboration diagrams in a style similar to the OMG's Unified Modeling # Language. UML_LOOK = NO # UML_LOOK = YES # If set to YES, the inheritance and collaboration graphs will show the # relations between templates and their instances. TEMPLATE_RELATIONS = YES # If the ENABLE_PREPROCESSING, SEARCH_INCLUDES, INCLUDE_GRAPH, and HAVE_DOT # tags are set to YES then doxygen will generate a graph for each documented # file showing the direct and indirect include dependencies of the file with # other documented files. INCLUDE_GRAPH = YES # If the ENABLE_PREPROCESSING, SEARCH_INCLUDES, INCLUDED_BY_GRAPH, and # HAVE_DOT tags are set to YES then doxygen will generate a graph for each # documented header file showing the documented files that directly or # indirectly include this file. INCLUDED_BY_GRAPH = YES # If the CALL_GRAPH and HAVE_DOT tags are set to YES then doxygen will # generate a call dependency graph for every global function or class method. # Note that enabling this option will significantly increase the time of a run. # So in most cases it will be better to enable call graphs for selected # functions only using the \callgraph command. CALL_GRAPH = NO # If the GRAPHICAL_HIERARCHY and HAVE_DOT tags are set to YES then doxygen # will graphical hierarchy of all classes instead of a textual one. GRAPHICAL_HIERARCHY = YES # If the DIRECTORY_GRAPH, SHOW_DIRECTORIES and HAVE_DOT tags are set to YES # then doxygen will show the dependencies a directory has on other directories # in a graphical way. The dependency relations are determined by the #include # relations between the files in the directories. DIRECTORY_GRAPH = YES # The DOT_IMAGE_FORMAT tag can be used to set the image format of the images # generated by dot. Possible values are png, jpg, or gif # If left blank png will be used. DOT_IMAGE_FORMAT = png # The tag DOT_PATH can be used to specify the path where the dot tool can be # found. If left blank, it is assumed the dot tool can be found in the path. DOT_PATH = # The DOTFILE_DIRS tag can be used to specify one or more directories that # contain dot files that are included in the documentation (see the # \dotfile command). DOTFILE_DIRS = # The MAX_DOT_GRAPH_WIDTH tag can be used to set the maximum allowed width # (in pixels) of the graphs generated by dot. If a graph becomes larger than # this value, doxygen will try to truncate the graph, so that it fits within # the specified constraint. Beware that most browsers cannot cope with very # large images. # Obsolet option. #MAX_DOT_GRAPH_WIDTH = 1280 # The MAX_DOT_GRAPH_HEIGHT tag can be used to set the maximum allows height # (in pixels) of the graphs generated by dot. If a graph becomes larger than # this value, doxygen will try to truncate the graph, so that it fits within # the specified constraint. Beware that most browsers cannot cope with very # large images. # Obsolet option. #MAX_DOT_GRAPH_HEIGHT = 1024 # The MAX_DOT_GRAPH_DEPTH tag can be used to set the maximum depth of the # graphs generated by dot. A depth value of 3 means that only nodes reachable # from the root by following a path via at most 3 edges will be shown. Nodes # that lay further from the root node will be omitted. Note that setting this # option to 1 or 2 may greatly reduce the computation time needed for large # code bases. Also note that a graph may be further truncated if the graph's # image dimensions are not sufficient to fit the graph (see MAX_DOT_GRAPH_WIDTH # and MAX_DOT_GRAPH_HEIGHT). If 0 is used for the depth value (the default), # the graph is not depth-constrained. MAX_DOT_GRAPH_DEPTH = 0 # Set the DOT_TRANSPARENT tag to YES to generate images with a transparent # background. This is disabled by default, which results in a white background. # Warning: Depending on the platform used, enabling this option may lead to # badly anti-aliased labels on the edges of a graph (i.e. they become hard to # read). DOT_TRANSPARENT = NO # Set the DOT_MULTI_TARGETS tag to YES allow dot to generate multiple output # files in one run (i.e. multiple -o and -T options on the command line). This # makes dot run faster, but since only newer versions of dot (>1.8.10) # support this, this feature is disabled by default. # JW # DOT_MULTI_TARGETS = NO DOT_MULTI_TARGETS = YES # If the GENERATE_LEGEND tag is set to YES (the default) Doxygen will # generate a legend page explaining the meaning of the various boxes and # arrows in the dot generated graphs. GENERATE_LEGEND = YES # If the DOT_CLEANUP tag is set to YES (the default) Doxygen will # remove the intermediate dot files that are used to generate # the various graphs. DOT_CLEANUP = YES #--------------------------------------------------------------------------- # Configuration::additions related to the search engine #--------------------------------------------------------------------------- # The SEARCHENGINE tag specifies whether or not a search engine should be # used. If set to NO the values of all tags below this one will be ignored. # JW SEARCHENGINE = NO SEARCHENGINE = YES ninja-1.3.4/doc/manual.asciidoc000066400000000000000000001023571215343310300163430ustar00rootroot00000000000000Ninja ===== Evan Martin v1.3.0, May 2013 Introduction ------------ Ninja is yet another build system. It takes as input the interdependencies of files (typically source code and output executables) and orchestrates building them, _quickly_. Ninja joins a sea of other build systems. Its distinguishing goal is to be fast. It is born from http://neugierig.org/software/chromium/notes/2011/02/ninja.html[my work on the Chromium browser project], which has over 30,000 source files and whose other build systems (including one built from custom non-recursive Makefiles) would take ten seconds to start building after changing one file. Ninja is under a second. Philosophical overview ~~~~~~~~~~~~~~~~~~~~~~ Where other build systems are high-level languages, Ninja aims to be an assembler. Build systems get slow when they need to make decisions. When you are in a edit-compile cycle you want it to be as fast as possible -- you want the build system to do the minimum work necessary to figure out what needs to be built immediately. Ninja contains the barest functionality necessary to describe arbitrary dependency graphs. Its lack of syntax makes it impossible to express complex decisions. Instead, Ninja is intended to be used with a separate program generating its input files. The generator program (like the `./configure` found in autotools projects) can analyze system dependencies and make as many decisions as possible up front so that incremental builds stay fast. Going beyond autotools, even build-time decisions like "which compiler flags should I use?" or "should I build a debug or release-mode binary?" belong in the `.ninja` file generator. Design goals ~~~~~~~~~~~~ Here are the design goals of Ninja: * very fast (i.e., instant) incremental builds, even for very large projects. * very little policy about how code is built. Different projects and higher-level build systems have different opinions about how code should be built; for example, should built objects live alongside the sources or should all build output go into a separate directory? Is there an "package" rule that builds a distributable package of the project? Sidestep these decisions by trying to allow either to be implemented, rather than choosing, even if that results in more verbosity. * get dependencies correct, and in particular situations that are difficult to get right with Makefiles (e.g. outputs need an implicit dependency on the command line used to generate them; to build C source code you need to use gcc's `-M` flags for header dependencies). * when convenience and speed are in conflict, prefer speed. Some explicit _non-goals_: * convenient syntax for writing build files by hand. _You should generate your ninja files using another program_. This is how we can sidestep many policy decisions. * built-in rules. _Out of the box, Ninja has no rules for e.g. compiling C code._ * build-time customization of the build. _Options belong in the program that generates the ninja files_. * build-time decision-making ability such as conditionals or search paths. _Making decisions is slow._ To restate, Ninja is faster than other build systems because it is painfully simple. You must tell Ninja exactly what to do when you create your project's `.ninja` files. Comparison to Make ~~~~~~~~~~~~~~~~~~ Ninja is closest in spirit and functionality to Make, relying on simple dependencies between file timestamps. But fundamentally, make has a lot of _features_: suffix rules, functions, built-in rules that e.g. search for RCS files when building source. Make's language was designed to be written by humans. Many projects find make alone adequate for their build problems. In contrast, Ninja has almost no features; just those necessary to get builds correct while punting most complexity to generation of the ninja input files. Ninja by itself is unlikely to be useful for most projects. Here are some of the features Ninja adds to Make. (These sorts of features can often be implemented using more complicated Makefiles, but they are not part of make itself.) * Ninja has special support for discovering extra dependencies at build time, making it easy to get <> correct for C/C++ code. * A build edge may have multiple outputs. * Outputs implicitly depend on the command line that was used to generate them, which means that changing e.g. compilation flags will cause the outputs to rebuild. * Output directories are always implicitly created before running the command that relies on them. * Rules can provide shorter descriptions of the command being run, so you can print e.g. `CC foo.o` instead of a long command line while building. * Builds are always run in parallel, based by default on the number of CPUs your system has. Underspecified build dependencies will result in incorrect builds. * Command output is always buffered. This means commands running in parallel don't interleave their output, and when a command fails we can print its failure output next to the full command line that produced the failure. Using Ninja for your project ---------------------------- Ninja currently works on Unix-like systems and Windows. It's seen the most testing on Linux (and has the best performance there) but it runs fine on Mac OS X and FreeBSD. If your project is small, Ninja's speed impact is likely unnoticeable. (However, even for small projects it sometimes turns out that Ninja's limited syntax forces simpler build rules that result in faster builds.) Another way to say this is that if you're happy with the edit-compile cycle time of your project already then Ninja won't help. There are many other build systems that are more user-friendly or featureful than Ninja itself. For some recommendations: the Ninja author found http://gittup.org/tup/[the tup build system] influential in Ninja's design, and thinks https://github.com/apenwarr/redo[redo]'s design is quite clever. Ninja's benefit comes from using it in conjunction with a smarter meta-build system. http://code.google.com/p/gyp/[gyp]:: The meta-build system used to generate build files for Google Chrome and related projects (v8, node.js). gyp can generate Ninja files for all platforms supported by Chrome. See the http://code.google.com/p/chromium/wiki/NinjaBuild[Chromium Ninja documentation for more details]. http://www.cmake.org/[CMake]:: A widely used meta-build system that can generate Ninja files on Linux as of CMake version 2.8.8. (There is some Mac and Windows support -- http://www.reactos.org[ReactOS] uses Ninja on Windows for their buildbots, but those platforms are not yet officially supported by CMake as the full test suite doesn't pass.) others:: Ninja ought to fit perfectly into other meta-build software like http://industriousone.com/premake[premake]. If you do this work, please let us know! Running Ninja ~~~~~~~~~~~~~ Run `ninja`. By default, it looks for a file named `build.ninja` in the current directory and builds all out-of-date targets. You can specify which targets (files) to build as command line arguments. `ninja -h` prints help output. Many of Ninja's flags intentionally match those of Make; e.g `ninja -C build -j 20` changes into the `build` directory and runs 20 build commands in parallel. (Note that Ninja defaults to running commands in parallel anyway, so typically you don't need to pass `-j`.) Environment variables ~~~~~~~~~~~~~~~~~~~~~ Ninja supports one environment variable to control its behavior: `NINJA_STATUS`, the progress status printed before the rule being run. Several placeholders are available: `%s`:: The number of started edges. `%t`:: The total number of edges that must be run to complete the build. `%p`:: The percentage of started edges. `%r`:: The number of currently running edges. `%u`:: The number of remaining edges to start. `%f`:: The number of finished edges. `%o`:: Overall rate of finished edges per second `%c`:: Current rate of finished edges per second (average over builds specified by `-j` or its default) `%e`:: Elapsed time in seconds. _(Available since Ninja 1.2.)_ `%%`:: A plain `%` character. The default progress status is `"[%s/%t] "` (note the trailing space to separate from the build rule). Another example of possible progress status could be `"[%u/%r/%f] "`. Extra tools ~~~~~~~~~~~ The `-t` flag on the Ninja command line runs some tools that we have found useful during Ninja's development. The current tools are: [horizontal] `query`:: dump the inputs and outputs of a given target. `browse`:: browse the dependency graph in a web browser. Clicking a file focuses the view on that file, showing inputs and outputs. This feature requires a Python installation. `graph`:: output a file in the syntax used by `graphviz`, a automatic graph layout tool. Use it like: + ---- ninja -t graph mytarget | dot -Tpng -ograph.png ---- + In the Ninja source tree, `ninja graph.png` generates an image for Ninja itself. If no target is given generate a graph for all root targets. `targets`:: output a list of targets either by rule or by depth. If used like +ninja -t targets rule _name_+ it prints the list of targets using the given rule to be built. If no rule is given, it prints the source files (the leaves of the graph). If used like +ninja -t targets depth _digit_+ it prints the list of targets in a depth-first manner starting by the root targets (the ones with no outputs). Indentation is used to mark dependencies. If the depth is zero it prints all targets. If no arguments are provided +ninja -t targets depth 1+ is assumed. In this mode targets may be listed several times. If used like this +ninja -t targets all+ it prints all the targets available without indentation and it is faster than the _depth_ mode. `commands`:: given a list of targets, print a list of commands which, if executed in order, may be used to rebuild those targets, assuming that all output files are out of date. `clean`:: remove built files. By default it removes all built files except for those created by the generator. Adding the `-g` flag also removes built files created by the generator (see <>). Additional arguments are targets, which removes the given targets and recursively all files built for them. + If used like +ninja -t clean -r _rules_+ it removes all files built using the given rules. + Files created but not referenced in the graph are not removed. This tool takes in account the +-v+ and the +-n+ options (note that +-n+ implies +-v+). `compdb`:: given a list of rules, each of which is expected to be a C family language compiler rule whose first input is the name of the source file, prints on standard output a compilation database in the http://clang.llvm.org/docs/JSONCompilationDatabase.html[JSON format] expected by the Clang tooling interface. _Available since Ninja 1.2._ Writing your own Ninja files ---------------------------- The remainder of this manual is only useful if you are constructing Ninja files yourself: for example, if you're writing a meta-build system or supporting a new language. Conceptual overview ~~~~~~~~~~~~~~~~~~~ Ninja evaluates a graph of dependencies between files, and runs whichever commands are necessary to make your build target up to date as determined by file modification times. If you are familiar with Make, Ninja is very similar. A build file (default name: `build.ninja`) provides a list of _rules_ -- short names for longer commands, like how to run the compiler -- along with a list of _build_ statements saying how to build files using the rules -- which rule to apply to which inputs to produce which outputs. Conceptually, `build` statements describe the dependency graph of your project, while `rule` statements describe how to generate the files along a given edge of the graph. Syntax example ~~~~~~~~~~~~~~ Here's a basic `.ninja` file that demonstrates most of the syntax. It will be used as an example for the following sections. --------------------------------- cflags = -Wall rule cc command = gcc $cflags -c $in -o $out build foo.o: cc foo.c --------------------------------- Variables ~~~~~~~~~ Despite the non-goal of being convenient to write by hand, to keep build files readable (debuggable), Ninja supports declaring shorter reusable names for strings. A declaration like the following ---------------- cflags = -g ---------------- can be used on the right side of an equals sign, dereferencing it with a dollar sign, like this: ---------------- rule cc command = gcc $cflags -c $in -o $out ---------------- Variables can also be referenced using curly braces like `${in}`. Variables might better be called "bindings", in that a given variable cannot be changed, only shadowed. There is more on how shadowing works later in this document. Rules ~~~~~ Rules declare a short name for a command line. They begin with a line consisting of the `rule` keyword and a name for the rule. Then follows an indented set of `variable = value` lines. The basic example above declares a new rule named `cc`, along with the command to run. In the context of a rule, the `command` variable defines the command to run, `$in` expands to the list of input files (`foo.c`), and `$out` to the output files (`foo.o`) for the command. A full list of special variables is provided in <>. Build statements ~~~~~~~~~~~~~~~~ Build statements declare a relationship between input and output files. They begin with the `build` keyword, and have the format +build _outputs_: _rulename_ _inputs_+. Such a declaration says that all of the output files are derived from the input files. When the output files are missing or when the inputs change, Ninja will run the rule to regenerate the outputs. The basic example above describes how to build `foo.o`, using the `cc` rule. In the scope of a `build` block (including in the evaluation of its associated `rule`), the variable `$in` is the list of inputs and the variable `$out` is the list of outputs. A build statement may be followed by an indented set of `key = value` pairs, much like a rule. These variables will shadow any variables when evaluating the variables in the command. For example: ---------------- cflags = -Wall -Werror rule cc command = gcc $cflags -c $in -o $out # If left unspecified, builds get the outer $cflags. build foo.o: cc foo.c # But you can shadow variables like cflags for a particular build. build special.o: cc special.c cflags = -Wall # The variable was only shadowed for the scope of special.o; # Subsequent build lines get the outer (original) cflags. build bar.o: cc bar.c ---------------- For more discussion of how scoping works, consult <>. If you need more complicated information passed from the build statement to the rule (for example, if the rule needs "the file extension of the first input"), pass that through as an extra variable, like how `cflags` is passed above. If the top-level Ninja file is specified as an output of any build statement and it is out of date, Ninja will rebuild and reload it before building the targets requested by the user. Generating Ninja files from code ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ `misc/ninja_syntax.py` in the Ninja distribution is a tiny Python module to facilitate generating Ninja files. It allows you to make Python calls like `ninja.rule(name='foo', command='bar', depfile='$out.d')` and it will generate the appropriate syntax. Feel free to just inline it into your project's build system if it's useful. More details ------------ The `phony` rule ~~~~~~~~~~~~~~~~ The special rule name `phony` can be used to create aliases for other targets. For example: ---------------- build foo: phony some/file/in/a/faraway/subdir/foo ---------------- This makes `ninja foo` build the longer path. Semantically, the `phony` rule is equivalent to a plain rule where the `command` does nothing, but phony rules are handled specially in that they aren't printed when run, logged (see below), nor do they contribute to the command count printed as part of the build process. `phony` can also be used to create dummy targets for files which may not exist at build time. If a phony build statement is written without any dependencies, the target will be considered out of date if it does not exist. Without a phony build statement, Ninja will report an error if the file does not exist and is required by the build. Default target statements ~~~~~~~~~~~~~~~~~~~~~~~~~ By default, if no targets are specified on the command line, Ninja will build every output that is not named as an input elsewhere. You can override this behavior using a default target statement. A default target statement causes Ninja to build only a given subset of output files if none are specified on the command line. Default target statements begin with the `default` keyword, and have the format +default _targets_+. A default target statement must appear after the build statement that declares the target as an output file. They are cumulative, so multiple statements may be used to extend the list of default targets. For example: ---------------- default foo bar default baz ---------------- This causes Ninja to build the `foo`, `bar` and `baz` targets by default. [[ref_log]] The Ninja log ~~~~~~~~~~~~~ For each built file, Ninja keeps a log of the command used to build it. Using this log Ninja can know when an existing output was built with a different command line than the build files specify (i.e., the command line changed) and knows to rebuild the file. The log file is kept in the build root in a file called `.ninja_log`. If you provide a variable named `builddir` in the outermost scope, `.ninja_log` will be kept in that directory instead. [[ref_versioning]] Version compatibility ~~~~~~~~~~~~~~~~~~~~~ _Available since Ninja 1.2._ Ninja version labels follow the standard major.minor.patch format, where the major version is increased on backwards-incompatible syntax/behavioral changes and the minor version is increased on new behaviors. Your `build.ninja` may declare a variable named `ninja_required_version` that asserts the minimum Ninja version required to use the generated file. For example, ----- ninja_required_version = 1.1 ----- declares that the build file relies on some feature that was introduced in Ninja 1.1 (perhaps the `pool` syntax), and that Ninja 1.1 or greater must be used to build. Unlike other Ninja variables, this version requirement is checked immediately when the variable is encountered in parsing, so it's best to put it at the top of the build file. Ninja always warns if the major versions of Ninja and the `ninja_required_version` don't match; a major version change hasn't come up yet so it's difficult to predict what behavior might be required. [[ref_headers]] C/C++ header dependencies ~~~~~~~~~~~~~~~~~~~~~~~~~ To get C/C++ header dependencies (or any other build dependency that works in a similar way) correct Ninja has some extra functionality. The problem with headers is that the full list of files that a given source file depends on can only be discovered by the compiler: different preprocessor defines and include paths cause different files to be used. Some compilers can emit this information while building, and Ninja can use that to get its dependencies perfect. Consider: if the file has never been compiled, it must be built anyway, generating the header dependencies as a side effect. If any file is later modified (even in a way that changes which headers it depends on) the modification will cause a rebuild as well, keeping the dependencies up to date. When loading these special dependencies, Ninja implicitly adds extra build edges such that it is not an error if the listed dependency is missing. This allows you to delete a header file and rebuild without the build aborting due to a missing input. depfile ^^^^^^^ `gcc` (and other compilers like `clang`) support emitting dependency information in the syntax of a Makefile. (Any command that can write dependencies in this form can be used, not just `gcc`.) To bring this information into Ninja requires cooperation. On the Ninja side, the `depfile` attribute on the `build` must point to a path where this data is written. (Ninja only supports the limited subset of the Makefile syntax emitted by compilers.) Then the command must know to write dependencies into the `depfile` path. Use it like in the following example: ---- rule cc depfile = $out.d command = gcc -MMD -MF $out.d [other gcc flags here] ---- The `-MMD` flag to `gcc` tells it to output header dependencies, and the `-MF` flag tells it where to write them. deps ^^^^ _(Available since Ninja 1.3.)_ It turns out that for large projects (and particularly on Windows, where the file system is slow) loading these dependency files on startup is slow. Ninja 1.3 can instead process dependencies just after they're generated and save a compacted form of the same information in a Ninja-internal database. Ninja supports this processing in two forms. 1. `deps = gcc` specifies that the tool outputs `gcc`-style dependencies in the form of Makefiles. Adding this to the above example will cause Ninja to process the `depfile` immediately after the compilation finishes, then delete the `.d` file (which is only used as a temporary). 2. `deps = msvc` specifies that the tool outputs header dependencies in the form produced by Visual Studio's compiler's http://msdn.microsoft.com/en-us/library/hdkef6tk(v=vs.90).aspx[`/showIncludes` flag]. Briefly, this means the tool outputs specially-formatted lines to its stdout. Ninja then filters these lines from the displayed output. No `depfile` attribute is necessary. + ---- rule cc deps = msvc command = cl /showIncludes -c $in /Fo$out ---- Pools ~~~~~ _Available since Ninja 1.1._ Pools allow you to allocate one or more rules or edges a finite number of concurrent jobs which is more tightly restricted than the default parallelism. This can be useful, for example, to restrict a particular expensive rule (like link steps for huge executables), or to restrict particular build statements which you know perform poorly when run concurrently. Each pool has a `depth` variable which is specified in the build file. The pool is then referred to with the `pool` variable on either a rule or a build statement. No matter what pools you specify, ninja will never run more concurrent jobs than the default parallelism, or the number of jobs specified on the command line (with `-j`). ---------------- # No more than 4 links at a time. pool link_pool depth = 4 # No more than 1 heavy object at a time. pool heavy_object_pool depth = 1 rule link ... pool = link_pool rule cc ... # The link_pool is used here. Only 4 links will run concurrently. build foo.exe: link input.obj # A build statement can be exempted from its rule's pool by setting an # empty pool. This effectively puts the build statement back into the default # pool, which has infinite depth. build other.exe: link input.obj pool = # A build statement can specify a pool directly. # Only one of these builds will run at a time. build heavy_object1.obj: cc heavy_obj1.cc pool = heavy_object_pool build heavy_object2.obj: cc heavy_obj2.cc pool = heavy_object_pool ---------------- Ninja file reference -------------------- A file is a series of declarations. A declaration can be one of: 1. A rule declaration, which begins with +rule _rulename_+, and then has a series of indented lines defining variables. 2. A build edge, which looks like +build _output1_ _output2_: _rulename_ _input1_ _input2_+. + Implicit dependencies may be tacked on the end with +| _dependency1_ _dependency2_+. + Order-only dependencies may be tacked on the end with +|| _dependency1_ _dependency2_+. (See <>.) 3. Variable declarations, which look like +_variable_ = _value_+. 4. Default target statements, which look like +default _target1_ _target2_+. 5. References to more files, which look like +subninja _path_+ or +include _path_+. The difference between these is explained below <>. Lexical syntax ~~~~~~~~~~~~~~ Ninja is mostly encoding agnostic, as long as the bytes Ninja cares about (like slashes in paths) are ASCII. This means e.g. UTF-8 or ISO-8859-1 input files ought to work. (To simplify some code, tabs and carriage returns are currently disallowed; this could be fixed if it really mattered to you.) Comments begin with `#` and extend to the end of the line. Newlines are significant. Statements like `build foo bar` are a set of space-separated tokens that end at the newline. Newlines and spaces within a token must be escaped. There is only one escape character, `$`, and it has the following behaviors: [horizontal] `$` followed by a newline:: escape the newline (continue the current line across a line break). `$` followed by text:: a variable reference. `${varname}`:: alternate syntax for `$varname`. `$` followed by space:: a space. (This is only necessary in lists of paths, where a space would otherwise separate filenames. See below.) `$:` :: a colon. (This is only necessary in `build` lines, where a colon would otherwise terminate the list of outputs.) `$$`:: a literal `$`. A `build` or `default` statement is first parsed as a space-separated list of filenames and then each name is expanded. This means that spaces within a variable will result in spaces in the expanded filename. ---- spaced = foo bar build $spaced/baz other$ file: ... # The above build line has two outputs: "foo bar/baz" and "other file". ---- In a `name = value` statement, whitespace at the beginning of a value is always stripped. Whitespace at the beginning of a line after a line continuation is also stripped. ---- two_words_with_one_space = foo $ bar one_word_with_no_space = foo$ bar ---- Other whitespace is only significant if it's at the beginning of a line. If a line is indented more than the previous one, it's considered part of its parent's scope; if it is indented less than the previous one, it closes the previous scope. [[ref_toplevel]] Top-level variables ~~~~~~~~~~~~~~~~~~~ Two variables are significant when declared in the outermost file scope. `builddir`:: a directory for some Ninja output files. See <>. (You can also store other build output in this directory.) `ninja_required_version`:: the minimum version of Ninja required to process the build correctly. See <>. [[ref_rule]] Rule variables ~~~~~~~~~~~~~~ A `rule` block contains a list of `key = value` declarations that affect the processing of the rule. Here is a full list of special keys. `command` (_required_):: the command line to run. This string (after $variables are expanded) is passed directly to `sh -c` without interpretation by Ninja. Each `rule` may have only one `command` declaration. To specify multiple commands use `&&` (or similar) to concatenate operations. `depfile`:: path to an optional `Makefile` that contains extra _implicit dependencies_ (see <>). This is explicitly to support C/C++ header dependencies; see <>. `deps`:: _(Available since Ninja 1.3.)_ if present, must be one of `gcc` or `msvc` to specify special dependency processing. See <>. The generated database is stored as `.ninja_deps` in the `builddir`, see <>. `description`:: a short description of the command, used to pretty-print the command as it's running. The `-v` flag controls whether to print the full command or its description; if a command fails, the full command line will always be printed before the command's output. `generator`:: if present, specifies that this rule is used to re-invoke the generator program. Files built using `generator` rules are treated specially in two ways: firstly, they will not be rebuilt if the command line changes; and secondly, they are not cleaned by default. `in`:: the shell-quoted space-separated list of files provided as inputs to the build line referencing this `rule`. (`$in` is provided solely for convenience; if you need some subset or variant of this list of files, just construct a new variable with that list and use that instead.) `in_newline`:: the same as `$in` except that multiple inputs are separated by newlines rather than spaces. (For use with `$rspfile_content`; this works around a bug in the MSVC linker where it uses a fixed-size buffer for processing input.) `out`:: the shell-quoted space-separated list of files provided as outputs to the build line referencing this `rule`. `restat`:: if present, causes Ninja to re-stat the command's outputs after execution of the command. Each output whose modification time the command did not change will be treated as though it had never needed to be built. This may cause the output's reverse dependencies to be removed from the list of pending build actions. `rspfile`, `rspfile_content`:: if present (both), Ninja will use a response file for the given command, i.e. write the selected string (`rspfile_content`) to the given file (`rspfile`) before calling the command and delete the file after successful execution of the command. + This is particularly useful on Windows OS, where the maximal length of a command line is limited and response files must be used instead. + Use it like in the following example: + ---- rule link command = link.exe /OUT$out [usual link flags here] @$out.rsp rspfile = $out.rsp rspfile_content = $in build myapp.exe: link a.obj b.obj [possibly many other .obj files] ---- [[ref_dependencies]] Build dependencies ~~~~~~~~~~~~~~~~~~ There are three types of build dependencies which are subtly different. 1. _Explicit dependencies_, as listed in a build line. These are available as the `$in` variable in the rule. Changes in these files cause the output to be rebuilt; if these file are missing and Ninja doesn't know how to build them, the build is aborted. + This is the standard form of dependency to be used for e.g. the source file of a compile command. 2. _Implicit dependencies_, either as picked up from a `depfile` attribute on a rule or from the syntax +| _dep1_ _dep2_+ on the end of a build line. The semantics are identical to explicit dependencies, the only difference is that implicit dependencies don't show up in the `$in` variable. + This is for expressing dependencies that don't show up on the command line of the command; for example, for a rule that runs a script, the script itself should be an implicit dependency, as changes to the script should cause the output to rebuild. + Note that dependencies as loaded through depfiles have slightly different semantics, as described in the <>. 3. _Order-only dependencies_, expressed with the syntax +|| _dep1_ _dep2_+ on the end of a build line. When these are out of date, the output is not rebuilt until they are built, but changes in order-only dependencies alone do not cause the output to be rebuilt. + Order-only dependencies can be useful for bootstrapping dependencies that are only discovered during build time: for example, to generate a header file before starting a subsequent compilation step. (Once the header is used in compilation, a generated dependency file will then express the implicit dependency.) Variable expansion ~~~~~~~~~~~~~~~~~~ Variables are expanded in paths (in a `build` or `default` statement) and on the right side of a `name = value` statement. When a `name = value` statement is evaluated, its right-hand side is expanded immediately (according to the below scoping rules), and from then on `$name` expands to the static string as the result of the expansion. It is never the case that you'll need to "double-escape" a value to prevent it from getting expanded twice. All variables are expanded immediately as they're encountered in parsing, with one important exception: variables in `rule` blocks are expanded when the rule is _used_, not when it is declared. In the following example, the `demo` rule prints "this is a demo of bar". ---- rule demo command = echo "this is a demo of $foo" build out: demo foo = bar ---- [[ref_scope]] Evaluation and scoping ~~~~~~~~~~~~~~~~~~~~~~ Top-level variable declarations are scoped to the file they occur in. The `subninja` keyword, used to include another `.ninja` file, introduces a new scope. The included `subninja` file may use the variables from the parent file, and shadow their values for the file's scope, but it won't affect values of the variables in the parent. To include another `.ninja` file in the current scope, much like a C `#include` statement, use `include` instead of `subninja`. Variable declarations indented in a `build` block are scoped to the `build` block. The full lookup order for a variable expanded in a `build` block (or the `rule` is uses) is: 1. Special built-in variables (`$in`, `$out`). 2. Build-level variables from the `build` block. 3. Rule-level variables from the `rule` block (i.e. `$command`). (Note from the above discussion on expansion that these are expanded "late", and may make use of in-scope bindings like `$in`.) 4. File-level variables from the file that the `build` line was in. 5. Variables from the file that included that file using the `subninja` keyword. ninja-1.3.4/doc/style.css000066400000000000000000000010041215343310300152230ustar00rootroot00000000000000body { margin: 5ex 10ex; max-width: 80ex; line-height: 1.5; font-family: sans-serif; } h1, h2, h3 { font-weight: normal; } pre, code { font-family: x, monospace; } pre { padding: 1ex; background: #eee; border: solid 1px #ddd; min-width: 0; font-size: 90%; } code { color: #007; } div.chapter { margin-top: 4em; border-top: solid 2px black; } .section .title { font-size: 1.3em; } .section .section .title { font-size: 1.2em; } p { margin-top: 0; } ninja-1.3.4/misc/000077500000000000000000000000001215343310300135445ustar00rootroot00000000000000ninja-1.3.4/misc/bash-completion000066400000000000000000000026251215343310300165600ustar00rootroot00000000000000# Copyright 2011 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # Add the following to your .bashrc to tab-complete ninja targets # . path/to/ninja/misc/bash-completion _ninja_target() { local cur targets dir line targets_command OPTIND cur="${COMP_WORDS[COMP_CWORD]}" if [[ "$cur" == "--"* ]]; then # there is currently only one argument that takes -- COMPREPLY=($(compgen -P '--' -W 'version' -- "${cur:2}")) else dir="." line=$(echo ${COMP_LINE} | cut -d" " -f 2-) # filter out all non relevant arguments but keep C for dirs while getopts C:f:j:l:k:nvd:t: opt "${line[@]}"; do case $opt in C) dir="$OPTARG" ;; esac done; targets_command="ninja -C ${dir} -t targets all" targets=$((${targets_command} 2>/dev/null) | awk -F: '{print $1}') COMPREPLY=($(compgen -W "$targets" -- "$cur")) fi return } complete -F _ninja_target ninja ninja-1.3.4/misc/inherited-fds.ninja000066400000000000000000000010411215343310300173060ustar00rootroot00000000000000# This build file prints out a list of open file descriptors in # Ninja subprocesses, to help verify we don't accidentally leak # any. # Because one fd leak was in the code managing multiple subprocesses, # this test brings up multiple subprocesses and then dumps the fd # table of the last one. # Use like: ./ninja -f misc/inherited-fds.ninja rule sleep command = sleep 10000 rule dump command = sleep 1; ls -l /proc/self/fd; exit 1 build all: phony a b c d e build a: sleep build b: sleep build c: sleep build d: sleep build e: dump ninja-1.3.4/misc/long-slow-build.ninja000066400000000000000000000014311215343310300176020ustar00rootroot00000000000000# An input file for running a "slow" build. # Use like: ninja -f misc/long-slow-build.ninja all rule sleep command = sleep 1 description = SLEEP $out build 0: sleep README build 1: sleep README build 2: sleep README build 3: sleep README build 4: sleep README build 5: sleep README build 6: sleep README build 7: sleep README build 8: sleep README build 9: sleep README build 10: sleep 0 build 11: sleep 1 build 12: sleep 2 build 13: sleep 3 build 14: sleep 4 build 15: sleep 5 build 16: sleep 6 build 17: sleep 7 build 18: sleep 8 build 19: sleep 9 build 20: sleep 10 build 21: sleep 11 build 22: sleep 12 build 23: sleep 13 build 24: sleep 14 build 25: sleep 15 build 26: sleep 16 build 27: sleep 17 build 28: sleep 18 build 29: sleep 19 build all: phony 20 21 22 23 24 25 26 27 28 29 ninja-1.3.4/misc/measure.py000077500000000000000000000033321215343310300155630ustar00rootroot00000000000000#!/usr/bin/env python # Copyright 2011 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """measure the runtime of a command by repeatedly running it. """ import time import subprocess import sys devnull = open('/dev/null', 'w') def run(cmd, repeat=10): print 'sampling:', sys.stdout.flush() samples = [] for _ in range(repeat): start = time.time() subprocess.call(cmd, stdout=devnull, stderr=devnull) end = time.time() dt = (end - start) * 1000 print '%dms' % int(dt), sys.stdout.flush() samples.append(dt) print # We're interested in the 'pure' runtime of the code, which is # conceptually the smallest time we'd see if we ran it enough times # such that it got the perfect time slices / disk cache hits. best = min(samples) # Also print how varied the outputs were in an attempt to make it # more obvious if something has gone terribly wrong. err = sum(s - best for s in samples) / float(len(samples)) print 'estimate: %dms (mean err %.1fms)' % (best, err) if __name__ == '__main__': if len(sys.argv) < 2: print 'usage: measure.py command args...' sys.exit(1) run(cmd=sys.argv[1:]) ninja-1.3.4/misc/ninja-mode.el000066400000000000000000000031571215343310300161150ustar00rootroot00000000000000;; Copyright 2011 Google Inc. All Rights Reserved. ;; ;; Licensed under the Apache License, Version 2.0 (the "License"); ;; you may not use this file except in compliance with the License. ;; You may obtain a copy of the License at ;; ;; http://www.apache.org/licenses/LICENSE-2.0 ;; ;; Unless required by applicable law or agreed to in writing, software ;; distributed under the License is distributed on an "AS IS" BASIS, ;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. ;; See the License for the specific language governing permissions and ;; limitations under the License. ;; Simple emacs mode for editing .ninja files. ;; Just some syntax highlighting for now. (setq ninja-keywords (list '("^#.*" . font-lock-comment-face) (cons (concat "^" (regexp-opt '("rule" "build" "subninja" "include" "pool" "default") 'words)) font-lock-keyword-face) '("\\([[:alnum:]_]+\\) =" . (1 font-lock-variable-name-face)) ;; Variable expansion. '("\\($[[:alnum:]_]+\\)" . (1 font-lock-variable-name-face)) ;; Rule names '("rule \\([[:alnum:]_]+\\)" . (1 font-lock-function-name-face)) )) (define-derived-mode ninja-mode fundamental-mode "ninja" (setq comment-start "#") ; Pass extra "t" to turn off syntax-based fontification -- we don't want ; quoted strings highlighted. (setq font-lock-defaults '(ninja-keywords t)) ) (provide 'ninja-mode) ;; Run ninja-mode for files ending in .ninja. ;;;###autoload (add-to-list 'auto-mode-alist '("\\.ninja$" . ninja-mode)) ninja-1.3.4/misc/ninja.vim000066400000000000000000000047731215343310300153730ustar00rootroot00000000000000" ninja build file syntax. " Language: ninja build file as described at " http://martine.github.com/ninja/manual.html " Version: 1.3 " Last Change: 2013/04/16 " Maintainer: Nicolas Weber " Version 1.3 of this script is in the upstream vim repository and will be " included in the next vim release. If you change this, please send your change " upstream. " ninja lexer and parser are at " https://github.com/martine/ninja/blob/master/src/lexer.in.cc " https://github.com/martine/ninja/blob/master/src/manifest_parser.cc if exists("b:current_syntax") finish endif let s:cpo_save = &cpo set cpo&vim syn case match syn match ninjaComment /#.*/ contains=@Spell " Toplevel statements are the ones listed here and " toplevel variable assignments (ident '=' value). " lexer.in.cc, ReadToken() and manifest_parser.cc, Parse() syn match ninjaKeyword "^build\>" syn match ninjaKeyword "^rule\>" syn match ninjaKeyword "^pool\>" syn match ninjaKeyword "^default\>" syn match ninjaKeyword "^include\>" syn match ninjaKeyword "^subninja\>" " Both 'build' and 'rule' begin a variable scope that ends " on the first line without indent. 'rule' allows only a " limited set of magic variables, 'build' allows general " let assignments. " manifest_parser.cc, ParseRule() syn region ninjaRule start="^rule" end="^\ze\S" contains=ALL transparent syn keyword ninjaRuleCommand contained command deps depfile description generator \ pool restat rspfile rspfile_content syn region ninjaPool start="^pool" end="^\ze\S" contains=ALL transparent syn keyword ninjaPoolCommand contained depth " Strings are parsed as follows: " lexer.in.cc, ReadEvalString() " simple_varname = [a-zA-Z0-9_-]+; " varname = [a-zA-Z0-9_.-]+; " $$ -> $ " $\n -> line continuation " '$ ' -> escaped space " $simple_varname -> variable " ${varname} -> variable syn match ninjaWrapLineOperator "\$$" syn match ninjaSimpleVar "\$[a-zA-Z0-9_-]\+" syn match ninjaVar "\${[a-zA-Z0-9_.-]\+}" " operators are: " variable assignment = " rule definition : " implicit dependency | " order-only dependency || syn match ninjaOperator "\(=\|:\||\|||\)\ze\s" hi def link ninjaComment Comment hi def link ninjaKeyword Keyword hi def link ninjaRuleCommand Statement hi def link ninjaPoolCommand Statement hi def link ninjaWrapLineOperator ninjaOperator hi def link ninjaOperator Operator hi def link ninjaSimpleVar ninjaVar hi def link ninjaVar Identifier let b:current_syntax = "ninja" let &cpo = s:cpo_save unlet s:cpo_save ninja-1.3.4/misc/ninja_syntax.py000066400000000000000000000124461215343310300166320ustar00rootroot00000000000000#!/usr/bin/python """Python module for generating .ninja files. Note that this is emphatically not a required piece of Ninja; it's just a helpful utility for build-file-generation systems that already use Python. """ import textwrap import re def escape_path(word): return word.replace('$ ','$$ ').replace(' ','$ ').replace(':', '$:') class Writer(object): def __init__(self, output, width=78): self.output = output self.width = width def newline(self): self.output.write('\n') def comment(self, text): for line in textwrap.wrap(text, self.width - 2): self.output.write('# ' + line + '\n') def variable(self, key, value, indent=0): if value is None: return if isinstance(value, list): value = ' '.join(filter(None, value)) # Filter out empty strings. self._line('%s = %s' % (key, value), indent) def pool(self, name, depth): self._line('pool %s' % name) self.variable('depth', depth, indent=1) def rule(self, name, command, description=None, depfile=None, generator=False, pool=None, restat=False, rspfile=None, rspfile_content=None, deps=None): self._line('rule %s' % name) self.variable('command', command, indent=1) if description: self.variable('description', description, indent=1) if depfile: self.variable('depfile', depfile, indent=1) if generator: self.variable('generator', '1', indent=1) if pool: self.variable('pool', pool, indent=1) if restat: self.variable('restat', '1', indent=1) if rspfile: self.variable('rspfile', rspfile, indent=1) if rspfile_content: self.variable('rspfile_content', rspfile_content, indent=1) if deps: self.variable('deps', deps, indent=1) def build(self, outputs, rule, inputs=None, implicit=None, order_only=None, variables=None): outputs = self._as_list(outputs) all_inputs = self._as_list(inputs)[:] out_outputs = list(map(escape_path, outputs)) all_inputs = list(map(escape_path, all_inputs)) if implicit: implicit = map(escape_path, self._as_list(implicit)) all_inputs.append('|') all_inputs.extend(implicit) if order_only: order_only = map(escape_path, self._as_list(order_only)) all_inputs.append('||') all_inputs.extend(order_only) self._line('build %s: %s' % (' '.join(out_outputs), ' '.join([rule] + all_inputs))) if variables: if isinstance(variables, dict): iterator = iter(variables.items()) else: iterator = iter(variables) for key, val in iterator: self.variable(key, val, indent=1) return outputs def include(self, path): self._line('include %s' % path) def subninja(self, path): self._line('subninja %s' % path) def default(self, paths): self._line('default %s' % ' '.join(self._as_list(paths))) def _count_dollars_before_index(self, s, i): """Returns the number of '$' characters right in front of s[i].""" dollar_count = 0 dollar_index = i - 1 while dollar_index > 0 and s[dollar_index] == '$': dollar_count += 1 dollar_index -= 1 return dollar_count def _line(self, text, indent=0): """Write 'text' word-wrapped at self.width characters.""" leading_space = ' ' * indent while len(leading_space) + len(text) > self.width: # The text is too wide; wrap if possible. # Find the rightmost space that would obey our width constraint and # that's not an escaped space. available_space = self.width - len(leading_space) - len(' $') space = available_space while True: space = text.rfind(' ', 0, space) if space < 0 or \ self._count_dollars_before_index(text, space) % 2 == 0: break if space < 0: # No such space; just use the first unescaped space we can find. space = available_space - 1 while True: space = text.find(' ', space + 1) if space < 0 or \ self._count_dollars_before_index(text, space) % 2 == 0: break if space < 0: # Give up on breaking. break self.output.write(leading_space + text[0:space] + ' $\n') text = text[space+1:] # Subsequent lines are continuations, so indent them. leading_space = ' ' * (indent+2) self.output.write(leading_space + text + '\n') def _as_list(self, input): if input is None: return [] if isinstance(input, list): return input return [input] def escape(string): """Escape a string such that it can be embedded into a Ninja file without further interpretation.""" assert '\n' not in string, 'Ninja syntax does not allow newlines' # We only have one special metacharacter: '$'. return string.replace('$', '$$') ninja-1.3.4/misc/ninja_syntax_test.py000077500000000000000000000122671215343310300176750ustar00rootroot00000000000000#!/usr/bin/env python # Copyright 2011 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import unittest try: from StringIO import StringIO except ImportError: from io import StringIO import ninja_syntax LONGWORD = 'a' * 10 LONGWORDWITHSPACES = 'a'*5 + '$ ' + 'a'*5 INDENT = ' ' class TestLineWordWrap(unittest.TestCase): def setUp(self): self.out = StringIO() self.n = ninja_syntax.Writer(self.out, width=8) def test_single_long_word(self): # We shouldn't wrap a single long word. self.n._line(LONGWORD) self.assertEqual(LONGWORD + '\n', self.out.getvalue()) def test_few_long_words(self): # We should wrap a line where the second word is overlong. self.n._line(' '.join(['x', LONGWORD, 'y'])) self.assertEqual(' $\n'.join(['x', INDENT + LONGWORD, INDENT + 'y']) + '\n', self.out.getvalue()) def test_short_words_indented(self): # Test that indent is taking into acount when breaking subsequent lines. # The second line should not be ' to tree', as that's longer than the # test layout width of 8. self.n._line('line_one to tree') self.assertEqual('''\ line_one $ to $ tree ''', self.out.getvalue()) def test_few_long_words_indented(self): # Check wrapping in the presence of indenting. self.n._line(' '.join(['x', LONGWORD, 'y']), indent=1) self.assertEqual(' $\n'.join([' ' + 'x', ' ' + INDENT + LONGWORD, ' ' + INDENT + 'y']) + '\n', self.out.getvalue()) def test_escaped_spaces(self): self.n._line(' '.join(['x', LONGWORDWITHSPACES, 'y'])) self.assertEqual(' $\n'.join(['x', INDENT + LONGWORDWITHSPACES, INDENT + 'y']) + '\n', self.out.getvalue()) def test_fit_many_words(self): self.n = ninja_syntax.Writer(self.out, width=78) self.n._line('command = cd ../../chrome; python ../tools/grit/grit/format/repack.py ../out/Debug/obj/chrome/chrome_dll.gen/repack/theme_resources_large.pak ../out/Debug/gen/chrome/theme_resources_large.pak', 1) self.assertEqual('''\ command = cd ../../chrome; python ../tools/grit/grit/format/repack.py $ ../out/Debug/obj/chrome/chrome_dll.gen/repack/theme_resources_large.pak $ ../out/Debug/gen/chrome/theme_resources_large.pak ''', self.out.getvalue()) def test_leading_space(self): self.n = ninja_syntax.Writer(self.out, width=14) # force wrapping self.n.variable('foo', ['', '-bar', '-somethinglong'], 0) self.assertEqual('''\ foo = -bar $ -somethinglong ''', self.out.getvalue()) def test_embedded_dollar_dollar(self): self.n = ninja_syntax.Writer(self.out, width=15) # force wrapping self.n.variable('foo', ['a$$b', '-somethinglong'], 0) self.assertEqual('''\ foo = a$$b $ -somethinglong ''', self.out.getvalue()) def test_two_embedded_dollar_dollars(self): self.n = ninja_syntax.Writer(self.out, width=17) # force wrapping self.n.variable('foo', ['a$$b', '-somethinglong'], 0) self.assertEqual('''\ foo = a$$b $ -somethinglong ''', self.out.getvalue()) def test_leading_dollar_dollar(self): self.n = ninja_syntax.Writer(self.out, width=14) # force wrapping self.n.variable('foo', ['$$b', '-somethinglong'], 0) self.assertEqual('''\ foo = $$b $ -somethinglong ''', self.out.getvalue()) def test_trailing_dollar_dollar(self): self.n = ninja_syntax.Writer(self.out, width=14) # force wrapping self.n.variable('foo', ['a$$', '-somethinglong'], 0) self.assertEqual('''\ foo = a$$ $ -somethinglong ''', self.out.getvalue()) class TestBuild(unittest.TestCase): def setUp(self): self.out = StringIO() self.n = ninja_syntax.Writer(self.out) def test_variables_dict(self): self.n.build('out', 'cc', 'in', variables={'name': 'value'}) self.assertEqual('''\ build out: cc in name = value ''', self.out.getvalue()) def test_variables_list(self): self.n.build('out', 'cc', 'in', variables=[('name', 'value')]) self.assertEqual('''\ build out: cc in name = value ''', self.out.getvalue()) if __name__ == '__main__': unittest.main() ninja-1.3.4/misc/packaging/000077500000000000000000000000001215343310300154705ustar00rootroot00000000000000ninja-1.3.4/misc/packaging/ninja.spec000066400000000000000000000023131215343310300174420ustar00rootroot00000000000000Summary: Ninja is a small build system with a focus on speed. Name: ninja Version: %{ver} Release: %{rel}%{?dist} Group: Development/Tools License: Apache 2.0 URL: https://github.com/martine/ninja Source0: %{name}-%{version}-%{rel}.tar.gz BuildRoot: %{_tmppath}/%{name}-%{version}-%{rel} BuildRequires: asciidoc %description Ninja is yet another build system. It takes as input the interdependencies of files (typically source code and output executables) and orchestrates building them, quickly. Ninja joins a sea of other build systems. Its distinguishing goal is to be fast. It is born from my work on the Chromium browser project, which has over 30,000 source files and whose other build systems (including one built from custom non-recursive Makefiles) can take ten seconds to start building after changing one file. Ninja is under a second. %prep %setup -q -n %{name}-%{version}-%{rel} %build echo Building.. ./bootstrap.py ./ninja manual %install mkdir -p %{buildroot}%{_bindir} %{buildroot}%{_docdir} cp -p ninja %{buildroot}%{_bindir}/ %files %defattr(-, root, root) %doc COPYING README doc/manual.html %{_bindir}/* %clean rm -rf %{buildroot} #The changelog is built automatically from Git history %changelog ninja-1.3.4/misc/packaging/rpmbuild.sh000077500000000000000000000020361215343310300176460ustar00rootroot00000000000000#!/bin/bash echo Building ninja RPMs.. GITROOT=$(git rev-parse --show-toplevel) cd $GITROOT VER=1.0 REL=$(git rev-parse --short HEAD)git RPMTOPDIR=$GITROOT/rpm-build echo "Ver: $VER, Release: $REL" # Create tarball mkdir -p $RPMTOPDIR/{SOURCES,SPECS} git archive --format=tar --prefix=ninja-${VER}-${REL}/ HEAD | gzip -c > $RPMTOPDIR/SOURCES/ninja-${VER}-${REL}.tar.gz # Convert git log to RPM's ChangeLog format (shown with rpm -qp --changelog ) sed -e "s/%{ver}/$VER/" -e "s/%{rel}/$REL/" misc/packaging/ninja.spec > $RPMTOPDIR/SPECS/ninja.spec git log --format="* %cd %aN%n- (%h) %s%d%n" --date=local | sed -r 's/[0-9]+:[0-9]+:[0-9]+ //' >> $RPMTOPDIR/SPECS/ninja.spec # Build SRC and binary RPMs rpmbuild --quiet \ --define "_topdir $RPMTOPDIR" \ --define "_rpmdir $PWD" \ --define "_srcrpmdir $PWD" \ --define '_rpmfilename %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm' \ -ba $RPMTOPDIR/SPECS/ninja.spec && rm -rf $RPMTOPDIR && echo Done ninja-1.3.4/misc/zsh-completion000066400000000000000000000014531215343310300164450ustar00rootroot00000000000000# Copyright 2011 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # Add the following to your .zshrc to tab-complete ninja targets # . path/to/ninja/misc/zsh-completion _ninja() { reply=(`(ninja -t targets all 2&>/dev/null) | awk -F: '{print $1}'`) } compctl -K _ninja ninja ninja-1.3.4/platform_helper.py000066400000000000000000000042751215343310300163560ustar00rootroot00000000000000#!/usr/bin/env python # Copyright 2011 Google Inc. # Copyright 2013 Patrick von Reth # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import sys def platforms(): return ['linux', 'darwin', 'freebsd', 'openbsd', 'solaris', 'sunos5', 'mingw', 'msvc', 'gnukfreebsd8'] class Platform( object ): def __init__( self, platform): self._platform = platform if not self._platform is None: return self._platform = sys.platform if self._platform.startswith('linux'): self._platform = 'linux' elif self._platform.startswith('freebsd'): self._platform = 'freebsd' elif self._platform.startswith('gnukfreebsd8'): self._platform = 'freebsd' elif self._platform.startswith('openbsd'): self._platform = 'openbsd' elif self._platform.startswith('solaris'): self._platform = 'solaris' elif self._platform.startswith('mingw'): self._platform = 'mingw' elif self._platform.startswith('win'): self._platform = 'msvc' def platform(self): return self._platform def is_linux(self): return self._platform == 'linux' def is_mingw(self): return self._platform == 'mingw' def is_msvc(self): return self._platform == 'msvc' def is_windows(self): return self.is_mingw() or self.is_msvc() def is_solaris(self): return self._platform == 'solaris' def is_freebsd(self): return self._platform == 'freebsd' def is_openbsd(self): return self._platform == 'openbsd' def is_sunos5(self): return self._platform == 'sunos5' ninja-1.3.4/src/000077500000000000000000000000001215343310300134005ustar00rootroot00000000000000ninja-1.3.4/src/browse.cc000066400000000000000000000034741215343310300152200ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "browse.h" #include #include #include #include "../build/browse_py.h" void RunBrowsePython(State* state, const char* ninja_command, const char* initial_target) { // Fork off a Python process and have it run our code via its stdin. // (Actually the Python process becomes the parent.) int pipefd[2]; if (pipe(pipefd) < 0) { perror("ninja: pipe"); return; } pid_t pid = fork(); if (pid < 0) { perror("ninja: fork"); return; } if (pid > 0) { // Parent. close(pipefd[1]); do { if (dup2(pipefd[0], 0) < 0) { perror("ninja: dup2"); break; } // exec Python, telling it to run the program from stdin. const char* command[] = { NINJA_PYTHON, "-", ninja_command, initial_target, NULL }; execvp(command[0], (char**)command); perror("ninja: execvp"); } while (false); _exit(1); } else { // Child. close(pipefd[0]); // Write the script file into the stdin of the Python process. ssize_t len = write(pipefd[1], kBrowsePy, sizeof(kBrowsePy)); if (len < (ssize_t)sizeof(kBrowsePy)) perror("ninja: write"); close(pipefd[1]); exit(0); } } ninja-1.3.4/src/browse.h000066400000000000000000000020201215343310300150440ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_BROWSE_H_ #define NINJA_BROWSE_H_ struct State; /// Run in "browse" mode, which execs a Python webserver. /// \a ninja_command is the command used to invoke ninja. /// \a initial_target is the first target to load. /// This function does not return if it runs successfully. void RunBrowsePython(State* state, const char* ninja_command, const char* initial_target); #endif // NINJA_BROWSE_H_ ninja-1.3.4/src/browse.py000077500000000000000000000136121215343310300152610ustar00rootroot00000000000000#!/usr/bin/env python # # Copyright 2001 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Simple web server for browsing dependency graph data. This script is inlined into the final executable and spawned by it when needed. """ from __future__ import print_function try: import http.server as httpserver except ImportError: import BaseHTTPServer as httpserver import os import socket import subprocess import sys import webbrowser import urllib2 from collections import namedtuple Node = namedtuple('Node', ['inputs', 'rule', 'target', 'outputs']) # Ideally we'd allow you to navigate to a build edge or a build node, # with appropriate views for each. But there's no way to *name* a build # edge so we can only display nodes. # # For a given node, it has at most one input edge, which has n # different inputs. This becomes node.inputs. (We leave out the # outputs of the input edge due to what follows.) The node can have # multiple dependent output edges. Rather than attempting to display # those, they are summarized by taking the union of all their outputs. # # This means there's no single view that shows you all inputs and outputs # of an edge. But I think it's less confusing than alternatives. def match_strip(line, prefix): if not line.startswith(prefix): return (False, line) return (True, line[len(prefix):]) def parse(text): lines = iter(text.split('\n')) target = None rule = None inputs = [] outputs = [] try: target = next(lines)[:-1] # strip trailing colon line = next(lines) (match, rule) = match_strip(line, ' input: ') if match: (match, line) = match_strip(next(lines), ' ') while match: type = None (match, line) = match_strip(line, '| ') if match: type = 'implicit' (match, line) = match_strip(line, '|| ') if match: type = 'order-only' inputs.append((line, type)) (match, line) = match_strip(next(lines), ' ') match, _ = match_strip(line, ' outputs:') if match: (match, line) = match_strip(next(lines), ' ') while match: outputs.append(line) (match, line) = match_strip(next(lines), ' ') except StopIteration: pass return Node(inputs, rule, target, outputs) def create_page(body): return ''' ''' + body def generate_html(node): document = ['

%s

' % node.target] if node.inputs: document.append('

target is built using rule %s of

' % node.rule) if len(node.inputs) > 0: document.append('
') for input, type in sorted(node.inputs): extra = '' if type: extra = ' (%s)' % type document.append('%s%s
' % (input, input, extra)) document.append('
') if node.outputs: document.append('

dependent edges build:

') document.append('
') for output in sorted(node.outputs): document.append('%s
' % (output, output)) document.append('
') return '\n'.join(document) def ninja_dump(target): proc = subprocess.Popen([sys.argv[1], '-t', 'query', target], stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) return proc.communicate() + (proc.returncode,) class RequestHandler(httpserver.BaseHTTPRequestHandler): def do_GET(self): assert self.path[0] == '/' target = urllib2.unquote(self.path[1:]) if target == '': self.send_response(302) self.send_header('Location', '?' + sys.argv[2]) self.end_headers() return if not target.startswith('?'): self.send_response(404) self.end_headers() return target = target[1:] ninja_output, ninja_error, exit_code = ninja_dump(target) if exit_code == 0: page_body = generate_html(parse(ninja_output.strip())) else: # Relay ninja's error message. page_body = '

%s

' % ninja_error self.send_response(200) self.end_headers() self.wfile.write(create_page(page_body).encode('utf-8')) def log_message(self, format, *args): pass # Swallow console spam. port = 8000 httpd = httpserver.HTTPServer(('',port), RequestHandler) try: hostname = socket.gethostname() print('Web server running on %s:%d, ctl-C to abort...' % (hostname,port) ) print('Web server pid %d' % os.getpid(), file=sys.stderr ) webbrowser.open_new('http://%s:%s' % (hostname, port) ) httpd.serve_forever() except KeyboardInterrupt: print() pass # Swallow console spam. ninja-1.3.4/src/build.cc000066400000000000000000000606741215343310300150230ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "build.h" #include #include #include #include #include #if defined(__SVR4) && defined(__sun) #include #endif #include "build_log.h" #include "depfile_parser.h" #include "deps_log.h" #include "disk_interface.h" #include "graph.h" #include "msvc_helper.h" #include "state.h" #include "subprocess.h" #include "util.h" namespace { /// A CommandRunner that doesn't actually run the commands. struct DryRunCommandRunner : public CommandRunner { virtual ~DryRunCommandRunner() {} // Overridden from CommandRunner: virtual bool CanRunMore(); virtual bool StartCommand(Edge* edge); virtual bool WaitForCommand(Result* result); private: queue finished_; }; bool DryRunCommandRunner::CanRunMore() { return true; } bool DryRunCommandRunner::StartCommand(Edge* edge) { finished_.push(edge); return true; } bool DryRunCommandRunner::WaitForCommand(Result* result) { if (finished_.empty()) return false; result->status = ExitSuccess; result->edge = finished_.front(); finished_.pop(); return true; } } // namespace BuildStatus::BuildStatus(const BuildConfig& config) : config_(config), start_time_millis_(GetTimeMillis()), started_edges_(0), finished_edges_(0), total_edges_(0), progress_status_format_(NULL), overall_rate_(), current_rate_(config.parallelism) { // Don't do anything fancy in verbose mode. if (config_.verbosity != BuildConfig::NORMAL) printer_.set_smart_terminal(false); progress_status_format_ = getenv("NINJA_STATUS"); if (!progress_status_format_) progress_status_format_ = "[%s/%t] "; } void BuildStatus::PlanHasTotalEdges(int total) { total_edges_ = total; } void BuildStatus::BuildEdgeStarted(Edge* edge) { int start_time = (int)(GetTimeMillis() - start_time_millis_); running_edges_.insert(make_pair(edge, start_time)); ++started_edges_; PrintStatus(edge); } void BuildStatus::BuildEdgeFinished(Edge* edge, bool success, const string& output, int* start_time, int* end_time) { int64_t now = GetTimeMillis(); ++finished_edges_; RunningEdgeMap::iterator i = running_edges_.find(edge); *start_time = i->second; *end_time = (int)(now - start_time_millis_); running_edges_.erase(i); if (config_.verbosity == BuildConfig::QUIET) return; if (printer_.is_smart_terminal()) PrintStatus(edge); // Print the command that is spewing before printing its output. if (!success) printer_.PrintOnNewLine("FAILED: " + edge->EvaluateCommand() + "\n"); if (!output.empty()) { // ninja sets stdout and stderr of subprocesses to a pipe, to be able to // check if the output is empty. Some compilers, e.g. clang, check // isatty(stderr) to decide if they should print colored output. // To make it possible to use colored output with ninja, subprocesses should // be run with a flag that forces them to always print color escape codes. // To make sure these escape codes don't show up in a file if ninja's output // is piped to a file, ninja strips ansi escape codes again if it's not // writing to a |smart_terminal_|. // (Launching subprocesses in pseudo ttys doesn't work because there are // only a few hundred available on some systems, and ninja can launch // thousands of parallel compile commands.) // TODO: There should be a flag to disable escape code stripping. string final_output; if (!printer_.is_smart_terminal()) final_output = StripAnsiEscapeCodes(output); else final_output = output; printer_.PrintOnNewLine(final_output); } } void BuildStatus::BuildFinished() { printer_.PrintOnNewLine(""); } string BuildStatus::FormatProgressStatus( const char* progress_status_format) const { string out; char buf[32]; int percent; for (const char* s = progress_status_format; *s != '\0'; ++s) { if (*s == '%') { ++s; switch (*s) { case '%': out.push_back('%'); break; // Started edges. case 's': snprintf(buf, sizeof(buf), "%d", started_edges_); out += buf; break; // Total edges. case 't': snprintf(buf, sizeof(buf), "%d", total_edges_); out += buf; break; // Running edges. case 'r': snprintf(buf, sizeof(buf), "%d", started_edges_ - finished_edges_); out += buf; break; // Unstarted edges. case 'u': snprintf(buf, sizeof(buf), "%d", total_edges_ - started_edges_); out += buf; break; // Finished edges. case 'f': snprintf(buf, sizeof(buf), "%d", finished_edges_); out += buf; break; // Overall finished edges per second. case 'o': overall_rate_.UpdateRate(finished_edges_); snprinfRate(overall_rate_.rate(), buf, "%.1f"); out += buf; break; // Current rate, average over the last '-j' jobs. case 'c': current_rate_.UpdateRate(finished_edges_); snprinfRate(current_rate_.rate(), buf, "%.1f"); out += buf; break; // Percentage case 'p': percent = (100 * started_edges_) / total_edges_; snprintf(buf, sizeof(buf), "%3i%%", percent); out += buf; break; case 'e': { double elapsed = overall_rate_.Elapsed(); snprintf(buf, sizeof(buf), "%.3f", elapsed); out += buf; break; } default: Fatal("unknown placeholder '%%%c' in $NINJA_STATUS", *s); return ""; } } else { out.push_back(*s); } } return out; } void BuildStatus::PrintStatus(Edge* edge) { if (config_.verbosity == BuildConfig::QUIET) return; bool force_full_command = config_.verbosity == BuildConfig::VERBOSE; string to_print = edge->GetBinding("description"); if (to_print.empty() || force_full_command) to_print = edge->GetBinding("command"); if (finished_edges_ == 0) { overall_rate_.Restart(); current_rate_.Restart(); } to_print = FormatProgressStatus(progress_status_format_) + to_print; printer_.Print(to_print, force_full_command ? LinePrinter::FULL : LinePrinter::ELIDE); } Plan::Plan() : command_edges_(0), wanted_edges_(0) {} bool Plan::AddTarget(Node* node, string* err) { vector stack; return AddSubTarget(node, &stack, err); } bool Plan::AddSubTarget(Node* node, vector* stack, string* err) { Edge* edge = node->in_edge(); if (!edge) { // Leaf node. if (node->dirty()) { string referenced; if (!stack->empty()) referenced = ", needed by '" + stack->back()->path() + "',"; *err = "'" + node->path() + "'" + referenced + " missing " "and no known rule to make it"; } return false; } if (CheckDependencyCycle(node, stack, err)) return false; if (edge->outputs_ready()) return false; // Don't need to do anything. // If an entry in want_ does not already exist for edge, create an entry which // maps to false, indicating that we do not want to build this entry itself. pair::iterator, bool> want_ins = want_.insert(make_pair(edge, false)); bool& want = want_ins.first->second; // If we do need to build edge and we haven't already marked it as wanted, // mark it now. if (node->dirty() && !want) { want = true; ++wanted_edges_; if (edge->AllInputsReady()) ScheduleWork(edge); if (!edge->is_phony()) ++command_edges_; } if (!want_ins.second) return true; // We've already processed the inputs. stack->push_back(node); for (vector::iterator i = edge->inputs_.begin(); i != edge->inputs_.end(); ++i) { if (!AddSubTarget(*i, stack, err) && !err->empty()) return false; } assert(stack->back() == node); stack->pop_back(); return true; } bool Plan::CheckDependencyCycle(Node* node, vector* stack, string* err) { vector::reverse_iterator ri = find(stack->rbegin(), stack->rend(), node); if (ri == stack->rend()) return false; // Add this node onto the stack to make it clearer where the loop // is. stack->push_back(node); vector::iterator start = find(stack->begin(), stack->end(), node); *err = "dependency cycle: "; for (vector::iterator i = start; i != stack->end(); ++i) { if (i != start) err->append(" -> "); err->append((*i)->path()); } return true; } Edge* Plan::FindWork() { if (ready_.empty()) return NULL; set::iterator i = ready_.begin(); Edge* edge = *i; ready_.erase(i); return edge; } void Plan::ScheduleWork(Edge* edge) { Pool* pool = edge->pool(); if (pool->ShouldDelayEdge()) { // The graph is not completely clean. Some Nodes have duplicate Out edges. // We need to explicitly ignore these here, otherwise their work will get // scheduled twice (see https://github.com/martine/ninja/pull/519) if (ready_.count(edge)) { return; } pool->DelayEdge(edge); pool->RetrieveReadyEdges(&ready_); } else { pool->EdgeScheduled(*edge); ready_.insert(edge); } } void Plan::ResumeDelayedJobs(Edge* edge) { edge->pool()->EdgeFinished(*edge); edge->pool()->RetrieveReadyEdges(&ready_); } void Plan::EdgeFinished(Edge* edge) { map::iterator i = want_.find(edge); assert(i != want_.end()); if (i->second) --wanted_edges_; want_.erase(i); edge->outputs_ready_ = true; // See if this job frees up any delayed jobs ResumeDelayedJobs(edge); // Check off any nodes we were waiting for with this edge. for (vector::iterator i = edge->outputs_.begin(); i != edge->outputs_.end(); ++i) { NodeFinished(*i); } } void Plan::NodeFinished(Node* node) { // See if we we want any edges from this node. for (vector::const_iterator i = node->out_edges().begin(); i != node->out_edges().end(); ++i) { map::iterator want_i = want_.find(*i); if (want_i == want_.end()) continue; // See if the edge is now ready. if ((*i)->AllInputsReady()) { if (want_i->second) { ScheduleWork(*i); } else { // We do not need to build this edge, but we might need to build one of // its dependents. EdgeFinished(*i); } } } } void Plan::CleanNode(DependencyScan* scan, Node* node) { node->set_dirty(false); for (vector::const_iterator ei = node->out_edges().begin(); ei != node->out_edges().end(); ++ei) { // Don't process edges that we don't actually want. map::iterator want_i = want_.find(*ei); if (want_i == want_.end() || !want_i->second) continue; // If all non-order-only inputs for this edge are now clean, // we might have changed the dirty state of the outputs. vector::iterator begin = (*ei)->inputs_.begin(), end = (*ei)->inputs_.end() - (*ei)->order_only_deps_; if (find_if(begin, end, mem_fun(&Node::dirty)) == end) { // Recompute most_recent_input and command. Node* most_recent_input = NULL; for (vector::iterator ni = begin; ni != end; ++ni) { if (!most_recent_input || (*ni)->mtime() > most_recent_input->mtime()) most_recent_input = *ni; } string command = (*ei)->EvaluateCommand(true); // Now, recompute the dirty state of each output. bool all_outputs_clean = true; for (vector::iterator ni = (*ei)->outputs_.begin(); ni != (*ei)->outputs_.end(); ++ni) { if (!(*ni)->dirty()) continue; if (scan->RecomputeOutputDirty(*ei, most_recent_input, 0, command, *ni)) { (*ni)->MarkDirty(); all_outputs_clean = false; } else { CleanNode(scan, *ni); } } // If we cleaned all outputs, mark the node as not wanted. if (all_outputs_clean) { want_i->second = false; --wanted_edges_; if (!(*ei)->is_phony()) --command_edges_; } } } } void Plan::Dump() { printf("pending: %d\n", (int)want_.size()); for (map::iterator i = want_.begin(); i != want_.end(); ++i) { if (i->second) printf("want "); i->first->Dump(); } printf("ready: %d\n", (int)ready_.size()); } struct RealCommandRunner : public CommandRunner { explicit RealCommandRunner(const BuildConfig& config) : config_(config) {} virtual ~RealCommandRunner() {} virtual bool CanRunMore(); virtual bool StartCommand(Edge* edge); virtual bool WaitForCommand(Result* result); virtual vector GetActiveEdges(); virtual void Abort(); const BuildConfig& config_; SubprocessSet subprocs_; map subproc_to_edge_; }; vector RealCommandRunner::GetActiveEdges() { vector edges; for (map::iterator i = subproc_to_edge_.begin(); i != subproc_to_edge_.end(); ++i) edges.push_back(i->second); return edges; } void RealCommandRunner::Abort() { subprocs_.Clear(); } bool RealCommandRunner::CanRunMore() { return ((int)subprocs_.running_.size()) < config_.parallelism && ((subprocs_.running_.empty() || config_.max_load_average <= 0.0f) || GetLoadAverage() < config_.max_load_average); } bool RealCommandRunner::StartCommand(Edge* edge) { string command = edge->EvaluateCommand(); Subprocess* subproc = subprocs_.Add(command); if (!subproc) return false; subproc_to_edge_.insert(make_pair(subproc, edge)); return true; } bool RealCommandRunner::WaitForCommand(Result* result) { Subprocess* subproc; while ((subproc = subprocs_.NextFinished()) == NULL) { bool interrupted = subprocs_.DoWork(); if (interrupted) return false; } result->status = subproc->Finish(); result->output = subproc->GetOutput(); map::iterator i = subproc_to_edge_.find(subproc); result->edge = i->second; subproc_to_edge_.erase(i); delete subproc; return true; } Builder::Builder(State* state, const BuildConfig& config, BuildLog* build_log, DepsLog* deps_log, DiskInterface* disk_interface) : state_(state), config_(config), disk_interface_(disk_interface), scan_(state, build_log, deps_log, disk_interface) { status_ = new BuildStatus(config); } Builder::~Builder() { Cleanup(); } void Builder::Cleanup() { if (command_runner_.get()) { vector active_edges = command_runner_->GetActiveEdges(); command_runner_->Abort(); for (vector::iterator i = active_edges.begin(); i != active_edges.end(); ++i) { string depfile = (*i)->GetBinding("depfile"); for (vector::iterator ni = (*i)->outputs_.begin(); ni != (*i)->outputs_.end(); ++ni) { // Only delete this output if it was actually modified. This is // important for things like the generator where we don't want to // delete the manifest file if we can avoid it. But if the rule // uses a depfile, always delete. (Consider the case where we // need to rebuild an output because of a modified header file // mentioned in a depfile, and the command touches its depfile // but is interrupted before it touches its output file.) if (!depfile.empty() || (*ni)->mtime() != disk_interface_->Stat((*ni)->path())) { disk_interface_->RemoveFile((*ni)->path()); } } if (!depfile.empty()) disk_interface_->RemoveFile(depfile); } } } Node* Builder::AddTarget(const string& name, string* err) { Node* node = state_->LookupNode(name); if (!node) { *err = "unknown target: '" + name + "'"; return NULL; } if (!AddTarget(node, err)) return NULL; return node; } bool Builder::AddTarget(Node* node, string* err) { node->StatIfNecessary(disk_interface_); if (Edge* in_edge = node->in_edge()) { if (!scan_.RecomputeDirty(in_edge, err)) return false; if (in_edge->outputs_ready()) return true; // Nothing to do. } if (!plan_.AddTarget(node, err)) return false; return true; } bool Builder::AlreadyUpToDate() const { return !plan_.more_to_do(); } bool Builder::Build(string* err) { assert(!AlreadyUpToDate()); status_->PlanHasTotalEdges(plan_.command_edge_count()); int pending_commands = 0; int failures_allowed = config_.failures_allowed; // Set up the command runner if we haven't done so already. if (!command_runner_.get()) { if (config_.dry_run) command_runner_.reset(new DryRunCommandRunner); else command_runner_.reset(new RealCommandRunner(config_)); } // This main loop runs the entire build process. // It is structured like this: // First, we attempt to start as many commands as allowed by the // command runner. // Second, we attempt to wait for / reap the next finished command. while (plan_.more_to_do()) { // See if we can start any more commands. if (failures_allowed && command_runner_->CanRunMore()) { if (Edge* edge = plan_.FindWork()) { if (!StartEdge(edge, err)) { status_->BuildFinished(); return false; } if (edge->is_phony()) { plan_.EdgeFinished(edge); } else { ++pending_commands; } // We made some progress; go back to the main loop. continue; } } // See if we can reap any finished commands. if (pending_commands) { CommandRunner::Result result; if (!command_runner_->WaitForCommand(&result) || result.status == ExitInterrupted) { status_->BuildFinished(); *err = "interrupted by user"; return false; } --pending_commands; FinishCommand(&result); if (!result.success()) { if (failures_allowed) failures_allowed--; } // We made some progress; start the main loop over. continue; } // If we get here, we cannot make any more progress. status_->BuildFinished(); if (failures_allowed == 0) { if (config_.failures_allowed > 1) *err = "subcommands failed"; else *err = "subcommand failed"; } else if (failures_allowed < config_.failures_allowed) *err = "cannot make progress due to previous errors"; else *err = "stuck [this is a bug]"; return false; } status_->BuildFinished(); return true; } bool Builder::StartEdge(Edge* edge, string* err) { METRIC_RECORD("StartEdge"); if (edge->is_phony()) return true; status_->BuildEdgeStarted(edge); // Create directories necessary for outputs. // XXX: this will block; do we care? for (vector::iterator i = edge->outputs_.begin(); i != edge->outputs_.end(); ++i) { if (!disk_interface_->MakeDirs((*i)->path())) return false; } // Create response file, if needed // XXX: this may also block; do we care? string rspfile = edge->GetBinding("rspfile"); if (!rspfile.empty()) { string content = edge->GetBinding("rspfile_content"); if (!disk_interface_->WriteFile(rspfile, content)) return false; } // start command computing and run it if (!command_runner_->StartCommand(edge)) { err->assign("command '" + edge->EvaluateCommand() + "' failed."); return false; } return true; } void Builder::FinishCommand(CommandRunner::Result* result) { METRIC_RECORD("FinishCommand"); Edge* edge = result->edge; // First try to extract dependencies from the result, if any. // This must happen first as it filters the command output (we want // to filter /showIncludes output, even on compile failure) and // extraction itself can fail, which makes the command fail from a // build perspective. vector deps_nodes; string deps_type = edge->GetBinding("deps"); if (!deps_type.empty()) { string extract_err; if (!ExtractDeps(result, deps_type, &deps_nodes, &extract_err) && result->success()) { if (!result->output.empty()) result->output.append("\n"); result->output.append(extract_err); result->status = ExitFailure; } } int start_time, end_time; status_->BuildEdgeFinished(edge, result->success(), result->output, &start_time, &end_time); // The rest of this function only applies to successful commands. if (!result->success()) return; // Restat the edge outputs, if necessary. TimeStamp restat_mtime = 0; if (edge->GetBindingBool("restat") && !config_.dry_run) { bool node_cleaned = false; for (vector::iterator i = edge->outputs_.begin(); i != edge->outputs_.end(); ++i) { TimeStamp new_mtime = disk_interface_->Stat((*i)->path()); if ((*i)->mtime() == new_mtime) { // The rule command did not change the output. Propagate the clean // state through the build graph. // Note that this also applies to nonexistent outputs (mtime == 0). plan_.CleanNode(&scan_, *i); node_cleaned = true; } } if (node_cleaned) { // If any output was cleaned, find the most recent mtime of any // (existing) non-order-only input or the depfile. for (vector::iterator i = edge->inputs_.begin(); i != edge->inputs_.end() - edge->order_only_deps_; ++i) { TimeStamp input_mtime = disk_interface_->Stat((*i)->path()); if (input_mtime > restat_mtime) restat_mtime = input_mtime; } string depfile = edge->GetBinding("depfile"); if (restat_mtime != 0 && !depfile.empty()) { TimeStamp depfile_mtime = disk_interface_->Stat(depfile); if (depfile_mtime > restat_mtime) restat_mtime = depfile_mtime; } // The total number of edges in the plan may have changed as a result // of a restat. status_->PlanHasTotalEdges(plan_.command_edge_count()); } } plan_.EdgeFinished(edge); // Delete any left over response file. string rspfile = edge->GetBinding("rspfile"); if (!rspfile.empty()) disk_interface_->RemoveFile(rspfile); if (scan_.build_log()) { scan_.build_log()->RecordCommand(edge, start_time, end_time, restat_mtime); } if (!deps_type.empty() && !config_.dry_run) { assert(edge->outputs_.size() == 1 && "should have been rejected by parser"); Node* out = edge->outputs_[0]; TimeStamp deps_mtime = disk_interface_->Stat(out->path()); scan_.deps_log()->RecordDeps(out, deps_mtime, deps_nodes); } } bool Builder::ExtractDeps(CommandRunner::Result* result, const string& deps_type, vector* deps_nodes, string* err) { #ifdef _WIN32 if (deps_type == "msvc") { CLParser parser; result->output = parser.Parse(result->output); for (set::iterator i = parser.includes_.begin(); i != parser.includes_.end(); ++i) { deps_nodes->push_back(state_->GetNode(*i)); } } else #endif if (deps_type == "gcc") { string depfile = result->edge->GetBinding("depfile"); if (depfile.empty()) { *err = string("edge with deps=gcc but no depfile makes no sense"); return false; } string content = disk_interface_->ReadFile(depfile, err); if (!err->empty()) return false; if (content.empty()) return true; DepfileParser deps; if (!deps.Parse(&content, err)) return false; // XXX check depfile matches expected output. deps_nodes->reserve(deps.ins_.size()); for (vector::iterator i = deps.ins_.begin(); i != deps.ins_.end(); ++i) { if (!CanonicalizePath(const_cast(i->str_), &i->len_, err)) return false; deps_nodes->push_back(state_->GetNode(*i)); } if (disk_interface_->RemoveFile(depfile) < 0) { *err = string("deleting depfile: ") + strerror(errno) + string("\n"); return false; } } else { Fatal("unknown deps type '%s'", deps_type.c_str()); } return true; } ninja-1.3.4/src/build.h000066400000000000000000000203701215343310300146520ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_BUILD_H_ #define NINJA_BUILD_H_ #include #include #include #include #include #include #include #include "graph.h" // XXX needed for DependencyScan; should rearrange. #include "exit_status.h" #include "line_printer.h" #include "metrics.h" #include "util.h" // int64_t struct BuildLog; struct BuildStatus; struct DiskInterface; struct Edge; struct Node; struct State; /// Plan stores the state of a build plan: what we intend to build, /// which steps we're ready to execute. struct Plan { Plan(); /// Add a target to our plan (including all its dependencies). /// Returns false if we don't need to build this target; may /// fill in |err| with an error message if there's a problem. bool AddTarget(Node* node, string* err); // Pop a ready edge off the queue of edges to build. // Returns NULL if there's no work to do. Edge* FindWork(); /// Returns true if there's more work to be done. bool more_to_do() const { return wanted_edges_; } /// Dumps the current state of the plan. void Dump(); /// Mark an edge as done building. Used internally and by /// tests. void EdgeFinished(Edge* edge); /// Clean the given node during the build. void CleanNode(DependencyScan* scan, Node* node); /// Number of edges with commands to run. int command_edge_count() const { return command_edges_; } private: bool AddSubTarget(Node* node, vector* stack, string* err); bool CheckDependencyCycle(Node* node, vector* stack, string* err); void NodeFinished(Node* node); /// Submits a ready edge as a candidate for execution. /// The edge may be delayed from running, for example if it's a member of a /// currently-full pool. void ScheduleWork(Edge* edge); /// Allows jobs blocking on |edge| to potentially resume. /// For example, if |edge| is a member of a pool, calling this may schedule /// previously pending jobs in that pool. void ResumeDelayedJobs(Edge* edge); /// Keep track of which edges we want to build in this plan. If this map does /// not contain an entry for an edge, we do not want to build the entry or its /// dependents. If an entry maps to false, we do not want to build it, but we /// might want to build one of its dependents. If the entry maps to true, we /// want to build it. map want_; set ready_; /// Total number of edges that have commands (not phony). int command_edges_; /// Total remaining number of wanted edges. int wanted_edges_; }; /// CommandRunner is an interface that wraps running the build /// subcommands. This allows tests to abstract out running commands. /// RealCommandRunner is an implementation that actually runs commands. struct CommandRunner { virtual ~CommandRunner() {} virtual bool CanRunMore() = 0; virtual bool StartCommand(Edge* edge) = 0; /// The result of waiting for a command. struct Result { Result() : edge(NULL) {} Edge* edge; ExitStatus status; string output; bool success() const { return status == ExitSuccess; } }; /// Wait for a command to complete, or return false if interrupted. virtual bool WaitForCommand(Result* result) = 0; virtual vector GetActiveEdges() { return vector(); } virtual void Abort() {} }; /// Options (e.g. verbosity, parallelism) passed to a build. struct BuildConfig { BuildConfig() : verbosity(NORMAL), dry_run(false), parallelism(1), failures_allowed(1), max_load_average(-0.0f) {} enum Verbosity { NORMAL, QUIET, // No output -- used when testing. VERBOSE }; Verbosity verbosity; bool dry_run; int parallelism; int failures_allowed; /// The maximum load average we must not exceed. A negative value /// means that we do not have any limit. double max_load_average; }; /// Builder wraps the build process: starting commands, updating status. struct Builder { Builder(State* state, const BuildConfig& config, BuildLog* build_log, DepsLog* deps_log, DiskInterface* disk_interface); ~Builder(); /// Clean up after interrupted commands by deleting output files. void Cleanup(); Node* AddTarget(const string& name, string* err); /// Add a target to the build, scanning dependencies. /// @return false on error. bool AddTarget(Node* target, string* err); /// Returns true if the build targets are already up to date. bool AlreadyUpToDate() const; /// Run the build. Returns false on error. /// It is an error to call this function when AlreadyUpToDate() is true. bool Build(string* err); bool StartEdge(Edge* edge, string* err); void FinishCommand(CommandRunner::Result* result); /// Used for tests. void SetBuildLog(BuildLog* log) { scan_.set_build_log(log); } State* state_; const BuildConfig& config_; Plan plan_; auto_ptr command_runner_; BuildStatus* status_; private: bool ExtractDeps(CommandRunner::Result* result, const string& deps_type, vector* deps_nodes, string* err); DiskInterface* disk_interface_; DependencyScan scan_; // Unimplemented copy ctor and operator= ensure we don't copy the auto_ptr. Builder(const Builder &other); // DO NOT IMPLEMENT void operator=(const Builder &other); // DO NOT IMPLEMENT }; /// Tracks the status of a build: completion fraction, printing updates. struct BuildStatus { explicit BuildStatus(const BuildConfig& config); void PlanHasTotalEdges(int total); void BuildEdgeStarted(Edge* edge); void BuildEdgeFinished(Edge* edge, bool success, const string& output, int* start_time, int* end_time); void BuildFinished(); /// Format the progress status string by replacing the placeholders. /// See the user manual for more information about the available /// placeholders. /// @param progress_status_format The format of the progress status. string FormatProgressStatus(const char* progress_status_format) const; private: void PrintStatus(Edge* edge); const BuildConfig& config_; /// Time the build started. int64_t start_time_millis_; int started_edges_, finished_edges_, total_edges_; /// Map of running edge to time the edge started running. typedef map RunningEdgeMap; RunningEdgeMap running_edges_; /// Prints progress output. LinePrinter printer_; /// The custom progress status format to use. const char* progress_status_format_; template void snprinfRate(double rate, char(&buf)[S], const char* format) const { if (rate == -1) snprintf(buf, S, "?"); else snprintf(buf, S, format, rate); } struct RateInfo { RateInfo() : rate_(-1) {} void Restart() { stopwatch_.Restart(); } double Elapsed() const { return stopwatch_.Elapsed(); } double rate() { return rate_; } void UpdateRate(int edges) { if (edges && stopwatch_.Elapsed()) rate_ = edges / stopwatch_.Elapsed(); } private: double rate_; Stopwatch stopwatch_; }; struct SlidingRateInfo { SlidingRateInfo(int n) : rate_(-1), N(n), last_update_(-1) {} void Restart() { stopwatch_.Restart(); } double rate() { return rate_; } void UpdateRate(int update_hint) { if (update_hint == last_update_) return; last_update_ = update_hint; if (times_.size() == N) times_.pop(); times_.push(stopwatch_.Elapsed()); if (times_.back() != times_.front()) rate_ = times_.size() / (times_.back() - times_.front()); } private: double rate_; Stopwatch stopwatch_; const size_t N; queue times_; int last_update_; }; mutable RateInfo overall_rate_; mutable SlidingRateInfo current_rate_; }; #endif // NINJA_BUILD_H_ ninja-1.3.4/src/build_log.cc000066400000000000000000000244351215343310300156570ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "build_log.h" #include #include #include #ifndef _WIN32 #define __STDC_FORMAT_MACROS #include #include #endif #include "build.h" #include "graph.h" #include "metrics.h" #include "util.h" // Implementation details: // Each run's log appends to the log file. // To load, we run through all log entries in series, throwing away // older runs. // Once the number of redundant entries exceeds a threshold, we write // out a new file and replace the existing one with it. namespace { const char kFileSignature[] = "# ninja log v%d\n"; const int kOldestSupportedVersion = 4; const int kCurrentVersion = 5; // 64bit MurmurHash2, by Austin Appleby #if defined(_MSC_VER) #define BIG_CONSTANT(x) (x) #else // defined(_MSC_VER) #define BIG_CONSTANT(x) (x##LLU) #endif // !defined(_MSC_VER) inline uint64_t MurmurHash64A(const void* key, size_t len) { static const uint64_t seed = 0xDECAFBADDECAFBADull; const uint64_t m = BIG_CONSTANT(0xc6a4a7935bd1e995); const int r = 47; uint64_t h = seed ^ (len * m); const uint64_t * data = (const uint64_t *)key; const uint64_t * end = data + (len/8); while (data != end) { uint64_t k = *data++; k *= m; k ^= k >> r; k *= m; h ^= k; h *= m; } const unsigned char* data2 = (const unsigned char*)data; switch (len & 7) { case 7: h ^= uint64_t(data2[6]) << 48; case 6: h ^= uint64_t(data2[5]) << 40; case 5: h ^= uint64_t(data2[4]) << 32; case 4: h ^= uint64_t(data2[3]) << 24; case 3: h ^= uint64_t(data2[2]) << 16; case 2: h ^= uint64_t(data2[1]) << 8; case 1: h ^= uint64_t(data2[0]); h *= m; }; h ^= h >> r; h *= m; h ^= h >> r; return h; } #undef BIG_CONSTANT } // namespace // static uint64_t BuildLog::LogEntry::HashCommand(StringPiece command) { return MurmurHash64A(command.str_, command.len_); } BuildLog::LogEntry::LogEntry(const string& output) : output(output) {} BuildLog::LogEntry::LogEntry(const string& output, uint64_t command_hash, int start_time, int end_time, TimeStamp restat_mtime) : output(output), command_hash(command_hash), start_time(start_time), end_time(end_time), restat_mtime(restat_mtime) {} BuildLog::BuildLog() : log_file_(NULL), needs_recompaction_(false) {} BuildLog::~BuildLog() { Close(); } bool BuildLog::OpenForWrite(const string& path, string* err) { if (needs_recompaction_) { Close(); if (!Recompact(path, err)) return false; } log_file_ = fopen(path.c_str(), "ab"); if (!log_file_) { *err = strerror(errno); return false; } setvbuf(log_file_, NULL, _IOLBF, BUFSIZ); SetCloseOnExec(fileno(log_file_)); // Opening a file in append mode doesn't set the file pointer to the file's // end on Windows. Do that explicitly. fseek(log_file_, 0, SEEK_END); if (ftell(log_file_) == 0) { if (fprintf(log_file_, kFileSignature, kCurrentVersion) < 0) { *err = strerror(errno); return false; } } return true; } void BuildLog::RecordCommand(Edge* edge, int start_time, int end_time, TimeStamp restat_mtime) { string command = edge->EvaluateCommand(true); uint64_t command_hash = LogEntry::HashCommand(command); for (vector::iterator out = edge->outputs_.begin(); out != edge->outputs_.end(); ++out) { const string& path = (*out)->path(); Entries::iterator i = entries_.find(path); LogEntry* log_entry; if (i != entries_.end()) { log_entry = i->second; } else { log_entry = new LogEntry(path); entries_.insert(Entries::value_type(log_entry->output, log_entry)); } log_entry->command_hash = command_hash; log_entry->start_time = start_time; log_entry->end_time = end_time; log_entry->restat_mtime = restat_mtime; if (log_file_) WriteEntry(log_file_, *log_entry); } } void BuildLog::Close() { if (log_file_) fclose(log_file_); log_file_ = NULL; } struct LineReader { explicit LineReader(FILE* file) : file_(file), buf_end_(buf_), line_start_(buf_), line_end_(NULL) { memset(buf_, 0, sizeof(buf_)); } // Reads a \n-terminated line from the file passed to the constructor. // On return, *line_start points to the beginning of the next line, and // *line_end points to the \n at the end of the line. If no newline is seen // in a fixed buffer size, *line_end is set to NULL. Returns false on EOF. bool ReadLine(char** line_start, char** line_end) { if (line_start_ >= buf_end_ || !line_end_) { // Buffer empty, refill. size_t size_read = fread(buf_, 1, sizeof(buf_), file_); if (!size_read) return false; line_start_ = buf_; buf_end_ = buf_ + size_read; } else { // Advance to next line in buffer. line_start_ = line_end_ + 1; } line_end_ = (char*)memchr(line_start_, '\n', buf_end_ - line_start_); if (!line_end_) { // No newline. Move rest of data to start of buffer, fill rest. size_t already_consumed = line_start_ - buf_; size_t size_rest = (buf_end_ - buf_) - already_consumed; memmove(buf_, line_start_, size_rest); size_t read = fread(buf_ + size_rest, 1, sizeof(buf_) - size_rest, file_); buf_end_ = buf_ + size_rest + read; line_start_ = buf_; line_end_ = (char*)memchr(line_start_, '\n', buf_end_ - line_start_); } *line_start = line_start_; *line_end = line_end_; return true; } private: FILE* file_; char buf_[256 << 10]; char* buf_end_; // Points one past the last valid byte in |buf_|. char* line_start_; // Points at the next \n in buf_ after line_start, or NULL. char* line_end_; }; bool BuildLog::Load(const string& path, string* err) { METRIC_RECORD(".ninja_log load"); FILE* file = fopen(path.c_str(), "r"); if (!file) { if (errno == ENOENT) return true; *err = strerror(errno); return false; } int log_version = 0; int unique_entry_count = 0; int total_entry_count = 0; LineReader reader(file); char* line_start = 0; char* line_end = 0; while (reader.ReadLine(&line_start, &line_end)) { if (!log_version) { sscanf(line_start, kFileSignature, &log_version); if (log_version < kOldestSupportedVersion) { *err = ("build log version invalid, perhaps due to being too old; " "starting over"); fclose(file); unlink(path.c_str()); // Don't report this as a failure. An empty build log will cause // us to rebuild the outputs anyway. return true; } } // If no newline was found in this chunk, read the next. if (!line_end) continue; const char kFieldSeparator = '\t'; char* start = line_start; char* end = (char*)memchr(start, kFieldSeparator, line_end - start); if (!end) continue; *end = 0; int start_time = 0, end_time = 0; TimeStamp restat_mtime = 0; start_time = atoi(start); start = end + 1; end = (char*)memchr(start, kFieldSeparator, line_end - start); if (!end) continue; *end = 0; end_time = atoi(start); start = end + 1; end = (char*)memchr(start, kFieldSeparator, line_end - start); if (!end) continue; *end = 0; restat_mtime = atol(start); start = end + 1; end = (char*)memchr(start, kFieldSeparator, line_end - start); if (!end) continue; string output = string(start, end - start); start = end + 1; end = line_end; LogEntry* entry; Entries::iterator i = entries_.find(output); if (i != entries_.end()) { entry = i->second; } else { entry = new LogEntry(output); entries_.insert(Entries::value_type(entry->output, entry)); ++unique_entry_count; } ++total_entry_count; entry->start_time = start_time; entry->end_time = end_time; entry->restat_mtime = restat_mtime; if (log_version >= 5) { char c = *end; *end = '\0'; entry->command_hash = (uint64_t)strtoull(start, NULL, 16); *end = c; } else { entry->command_hash = LogEntry::HashCommand(StringPiece(start, end - start)); } } fclose(file); if (!line_start) { return true; // file was empty } // Decide whether it's time to rebuild the log: // - if we're upgrading versions // - if it's getting large int kMinCompactionEntryCount = 100; int kCompactionRatio = 3; if (log_version < kCurrentVersion) { needs_recompaction_ = true; } else if (total_entry_count > kMinCompactionEntryCount && total_entry_count > unique_entry_count * kCompactionRatio) { needs_recompaction_ = true; } return true; } BuildLog::LogEntry* BuildLog::LookupByOutput(const string& path) { Entries::iterator i = entries_.find(path); if (i != entries_.end()) return i->second; return NULL; } void BuildLog::WriteEntry(FILE* f, const LogEntry& entry) { fprintf(f, "%d\t%d\t%d\t%s\t%" PRIx64 "\n", entry.start_time, entry.end_time, entry.restat_mtime, entry.output.c_str(), entry.command_hash); } bool BuildLog::Recompact(const string& path, string* err) { METRIC_RECORD(".ninja_log recompact"); printf("Recompacting log...\n"); string temp_path = path + ".recompact"; FILE* f = fopen(temp_path.c_str(), "wb"); if (!f) { *err = strerror(errno); return false; } if (fprintf(f, kFileSignature, kCurrentVersion) < 0) { *err = strerror(errno); fclose(f); return false; } for (Entries::iterator i = entries_.begin(); i != entries_.end(); ++i) { WriteEntry(f, *i->second); } fclose(f); if (unlink(path.c_str()) < 0) { *err = strerror(errno); return false; } if (rename(temp_path.c_str(), path.c_str()) < 0) { *err = strerror(errno); return false; } return true; } ninja-1.3.4/src/build_log.h000066400000000000000000000050061215343310300155120ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_BUILD_LOG_H_ #define NINJA_BUILD_LOG_H_ #include #include using namespace std; #include "hash_map.h" #include "timestamp.h" #include "util.h" // uint64_t struct Edge; /// Store a log of every command ran for every build. /// It has a few uses: /// /// 1) (hashes of) command lines for existing output files, so we know /// when we need to rebuild due to the command changing /// 2) timing information, perhaps for generating reports /// 3) restat information struct BuildLog { BuildLog(); ~BuildLog(); bool OpenForWrite(const string& path, string* err); void RecordCommand(Edge* edge, int start_time, int end_time, TimeStamp restat_mtime = 0); void Close(); /// Load the on-disk log. bool Load(const string& path, string* err); struct LogEntry { string output; uint64_t command_hash; int start_time; int end_time; TimeStamp restat_mtime; static uint64_t HashCommand(StringPiece command); // Used by tests. bool operator==(const LogEntry& o) { return output == o.output && command_hash == o.command_hash && start_time == o.start_time && end_time == o.end_time && restat_mtime == o.restat_mtime; } explicit LogEntry(const string& output); LogEntry(const string& output, uint64_t command_hash, int start_time, int end_time, TimeStamp restat_mtime); }; /// Lookup a previously-run command by its output path. LogEntry* LookupByOutput(const string& path); /// Serialize an entry into a log file. void WriteEntry(FILE* f, const LogEntry& entry); /// Rewrite the known log entries, throwing away old data. bool Recompact(const string& path, string* err); typedef ExternalStringHashMap::Type Entries; const Entries& entries() const { return entries_; } private: Entries entries_; FILE* log_file_; bool needs_recompaction_; }; #endif // NINJA_BUILD_LOG_H_ ninja-1.3.4/src/build_log_perftest.cc000066400000000000000000000075231215343310300175720ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include #include "build_log.h" #include "graph.h" #include "manifest_parser.h" #include "state.h" #include "util.h" #include "metrics.h" #ifndef _WIN32 #include #endif const char kTestFilename[] = "BuildLogPerfTest-tempfile"; bool WriteTestData(string* err) { BuildLog log; if (!log.OpenForWrite(kTestFilename, err)) return false; /* A histogram of command lengths in chromium. For example, 407 builds, 1.4% of all builds, had commands longer than 32 bytes but shorter than 64. 32 407 1.4% 64 183 0.6% 128 1461 5.1% 256 791 2.8% 512 1314 4.6% 1024 6114 21.3% 2048 11759 41.0% 4096 2056 7.2% 8192 4567 15.9% 16384 13 0.0% 32768 4 0.0% 65536 5 0.0% The average command length is 4.1 kB and there were 28674 commands in total, which makes for a total log size of ~120 MB (also counting output filenames). Based on this, write 30000 many 4 kB long command lines. */ // ManifestParser is the only object allowed to create Rules. const size_t kRuleSize = 4000; string long_rule_command = "gcc "; for (int i = 0; long_rule_command.size() < kRuleSize; ++i) { char buf[80]; sprintf(buf, "-I../../and/arbitrary/but/fairly/long/path/suffixed/%d ", i); long_rule_command += buf; } long_rule_command += "$in -o $out\n"; State state; ManifestParser parser(&state, NULL); if (!parser.ParseTest("rule cxx\n command = " + long_rule_command, err)) return false; // Create build edges. Using ManifestParser is as fast as using the State api // for edge creation, so just use that. const int kNumCommands = 30000; string build_rules; for (int i = 0; i < kNumCommands; ++i) { char buf[80]; sprintf(buf, "build input%d.o: cxx input%d.cc\n", i, i); build_rules += buf; } if (!parser.ParseTest(build_rules, err)) return false; for (int i = 0; i < kNumCommands; ++i) { log.RecordCommand(state.edges_[i], /*start_time=*/100 * i, /*end_time=*/100 * i + 1, /*restat_mtime=*/0); } return true; } int main() { vector times; string err; if (!WriteTestData(&err)) { fprintf(stderr, "Failed to write test data: %s\n", err.c_str()); return 1; } { // Read once to warm up disk cache. BuildLog log; if (!log.Load(kTestFilename, &err)) { fprintf(stderr, "Failed to read test data: %s\n", err.c_str()); return 1; } } const int kNumRepetitions = 5; for (int i = 0; i < kNumRepetitions; ++i) { int64_t start = GetTimeMillis(); BuildLog log; if (!log.Load(kTestFilename, &err)) { fprintf(stderr, "Failed to read test data: %s\n", err.c_str()); return 1; } int delta = (int)(GetTimeMillis() - start); printf("%dms\n", delta); times.push_back(delta); } int min = times[0]; int max = times[0]; float total = 0; for (size_t i = 0; i < times.size(); ++i) { total += times[i]; if (times[i] < min) min = times[i]; else if (times[i] > max) max = times[i]; } printf("min %dms max %dms avg %.1fms\n", min, max, total / times.size()); unlink(kTestFilename); return 0; } ninja-1.3.4/src/build_log_test.cc000066400000000000000000000164361215343310300167200ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "build_log.h" #include "util.h" #include "test.h" #ifdef _WIN32 #include #include #else #include #include #include #endif namespace { const char kTestFilename[] = "BuildLogTest-tempfile"; struct BuildLogTest : public StateTestWithBuiltinRules { virtual void SetUp() { // In case a crashing test left a stale file behind. unlink(kTestFilename); } virtual void TearDown() { unlink(kTestFilename); } }; TEST_F(BuildLogTest, WriteRead) { AssertParse(&state_, "build out: cat mid\n" "build mid: cat in\n"); BuildLog log1; string err; EXPECT_TRUE(log1.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); log1.RecordCommand(state_.edges_[0], 15, 18); log1.RecordCommand(state_.edges_[1], 20, 25); log1.Close(); BuildLog log2; EXPECT_TRUE(log2.Load(kTestFilename, &err)); ASSERT_EQ("", err); ASSERT_EQ(2u, log1.entries().size()); ASSERT_EQ(2u, log2.entries().size()); BuildLog::LogEntry* e1 = log1.LookupByOutput("out"); ASSERT_TRUE(e1); BuildLog::LogEntry* e2 = log2.LookupByOutput("out"); ASSERT_TRUE(e2); ASSERT_TRUE(*e1 == *e2); ASSERT_EQ(15, e1->start_time); ASSERT_EQ("out", e1->output); } TEST_F(BuildLogTest, FirstWriteAddsSignature) { const char kExpectedVersion[] = "# ninja log vX\n"; const size_t kVersionPos = strlen(kExpectedVersion) - 2; // Points at 'X'. BuildLog log; string contents, err; EXPECT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); log.Close(); ASSERT_EQ(0, ReadFile(kTestFilename, &contents, &err)); ASSERT_EQ("", err); if (contents.size() >= kVersionPos) contents[kVersionPos] = 'X'; EXPECT_EQ(kExpectedVersion, contents); // Opening the file anew shouldn't add a second version string. EXPECT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); log.Close(); contents.clear(); ASSERT_EQ(0, ReadFile(kTestFilename, &contents, &err)); ASSERT_EQ("", err); if (contents.size() >= kVersionPos) contents[kVersionPos] = 'X'; EXPECT_EQ(kExpectedVersion, contents); } TEST_F(BuildLogTest, DoubleEntry) { FILE* f = fopen(kTestFilename, "wb"); fprintf(f, "# ninja log v4\n"); fprintf(f, "0\t1\t2\tout\tcommand abc\n"); fprintf(f, "3\t4\t5\tout\tcommand def\n"); fclose(f); string err; BuildLog log; EXPECT_TRUE(log.Load(kTestFilename, &err)); ASSERT_EQ("", err); BuildLog::LogEntry* e = log.LookupByOutput("out"); ASSERT_TRUE(e); ASSERT_NO_FATAL_FAILURE(AssertHash("command def", e->command_hash)); } TEST_F(BuildLogTest, Truncate) { AssertParse(&state_, "build out: cat mid\n" "build mid: cat in\n"); BuildLog log1; string err; EXPECT_TRUE(log1.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); log1.RecordCommand(state_.edges_[0], 15, 18); log1.RecordCommand(state_.edges_[1], 20, 25); log1.Close(); struct stat statbuf; ASSERT_EQ(0, stat(kTestFilename, &statbuf)); ASSERT_GT(statbuf.st_size, 0); // For all possible truncations of the input file, assert that we don't // crash when parsing. for (off_t size = statbuf.st_size; size > 0; --size) { BuildLog log2; string err; EXPECT_TRUE(log2.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); log2.RecordCommand(state_.edges_[0], 15, 18); log2.RecordCommand(state_.edges_[1], 20, 25); log2.Close(); ASSERT_TRUE(Truncate(kTestFilename, size, &err)); BuildLog log3; err.clear(); ASSERT_TRUE(log3.Load(kTestFilename, &err) || !err.empty()); } } TEST_F(BuildLogTest, ObsoleteOldVersion) { FILE* f = fopen(kTestFilename, "wb"); fprintf(f, "# ninja log v3\n"); fprintf(f, "123 456 0 out command\n"); fclose(f); string err; BuildLog log; EXPECT_TRUE(log.Load(kTestFilename, &err)); ASSERT_NE(err.find("version"), string::npos); } TEST_F(BuildLogTest, SpacesInOutputV4) { FILE* f = fopen(kTestFilename, "wb"); fprintf(f, "# ninja log v4\n"); fprintf(f, "123\t456\t456\tout with space\tcommand\n"); fclose(f); string err; BuildLog log; EXPECT_TRUE(log.Load(kTestFilename, &err)); ASSERT_EQ("", err); BuildLog::LogEntry* e = log.LookupByOutput("out with space"); ASSERT_TRUE(e); ASSERT_EQ(123, e->start_time); ASSERT_EQ(456, e->end_time); ASSERT_EQ(456, e->restat_mtime); ASSERT_NO_FATAL_FAILURE(AssertHash("command", e->command_hash)); } TEST_F(BuildLogTest, DuplicateVersionHeader) { // Old versions of ninja accidentally wrote multiple version headers to the // build log on Windows. This shouldn't crash, and the second version header // should be ignored. FILE* f = fopen(kTestFilename, "wb"); fprintf(f, "# ninja log v4\n"); fprintf(f, "123\t456\t456\tout\tcommand\n"); fprintf(f, "# ninja log v4\n"); fprintf(f, "456\t789\t789\tout2\tcommand2\n"); fclose(f); string err; BuildLog log; EXPECT_TRUE(log.Load(kTestFilename, &err)); ASSERT_EQ("", err); BuildLog::LogEntry* e = log.LookupByOutput("out"); ASSERT_TRUE(e); ASSERT_EQ(123, e->start_time); ASSERT_EQ(456, e->end_time); ASSERT_EQ(456, e->restat_mtime); ASSERT_NO_FATAL_FAILURE(AssertHash("command", e->command_hash)); e = log.LookupByOutput("out2"); ASSERT_TRUE(e); ASSERT_EQ(456, e->start_time); ASSERT_EQ(789, e->end_time); ASSERT_EQ(789, e->restat_mtime); ASSERT_NO_FATAL_FAILURE(AssertHash("command2", e->command_hash)); } TEST_F(BuildLogTest, VeryLongInputLine) { // Ninja's build log buffer is currently 256kB. Lines longer than that are // silently ignored, but don't affect parsing of other lines. FILE* f = fopen(kTestFilename, "wb"); fprintf(f, "# ninja log v4\n"); fprintf(f, "123\t456\t456\tout\tcommand start"); for (size_t i = 0; i < (512 << 10) / strlen(" more_command"); ++i) fputs(" more_command", f); fprintf(f, "\n"); fprintf(f, "456\t789\t789\tout2\tcommand2\n"); fclose(f); string err; BuildLog log; EXPECT_TRUE(log.Load(kTestFilename, &err)); ASSERT_EQ("", err); BuildLog::LogEntry* e = log.LookupByOutput("out"); ASSERT_EQ(NULL, e); e = log.LookupByOutput("out2"); ASSERT_TRUE(e); ASSERT_EQ(456, e->start_time); ASSERT_EQ(789, e->end_time); ASSERT_EQ(789, e->restat_mtime); ASSERT_NO_FATAL_FAILURE(AssertHash("command2", e->command_hash)); } TEST_F(BuildLogTest, MultiTargetEdge) { AssertParse(&state_, "build out out.d: cat\n"); BuildLog log; log.RecordCommand(state_.edges_[0], 21, 22); ASSERT_EQ(2u, log.entries().size()); BuildLog::LogEntry* e1 = log.LookupByOutput("out"); ASSERT_TRUE(e1); BuildLog::LogEntry* e2 = log.LookupByOutput("out.d"); ASSERT_TRUE(e2); ASSERT_EQ("out", e1->output); ASSERT_EQ("out.d", e2->output); ASSERT_EQ(21, e1->start_time); ASSERT_EQ(21, e2->start_time); ASSERT_EQ(22, e2->end_time); ASSERT_EQ(22, e2->end_time); } } // anonymous namespace ninja-1.3.4/src/build_test.cc000066400000000000000000001352531215343310300160560ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "build.h" #include "build_log.h" #include "deps_log.h" #include "graph.h" #include "test.h" /// Fixture for tests involving Plan. // Though Plan doesn't use State, it's useful to have one around // to create Nodes and Edges. struct PlanTest : public StateTestWithBuiltinRules { Plan plan_; /// Because FindWork does not return Edges in any sort of predictable order, // provide a means to get available Edges in order and in a format which is // easy to write tests around. void FindWorkSorted(deque* ret, int count) { struct CompareEdgesByOutput { static bool cmp(const Edge* a, const Edge* b) { return a->outputs_[0]->path() < b->outputs_[0]->path(); } }; for (int i = 0; i < count; ++i) { ASSERT_TRUE(plan_.more_to_do()); Edge* edge = plan_.FindWork(); ASSERT_TRUE(edge); ret->push_back(edge); } ASSERT_FALSE(plan_.FindWork()); sort(ret->begin(), ret->end(), CompareEdgesByOutput::cmp); } }; TEST_F(PlanTest, Basic) { AssertParse(&state_, "build out: cat mid\n" "build mid: cat in\n"); GetNode("mid")->MarkDirty(); GetNode("out")->MarkDirty(); string err; EXPECT_TRUE(plan_.AddTarget(GetNode("out"), &err)); ASSERT_EQ("", err); ASSERT_TRUE(plan_.more_to_do()); Edge* edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_EQ("in", edge->inputs_[0]->path()); ASSERT_EQ("mid", edge->outputs_[0]->path()); ASSERT_FALSE(plan_.FindWork()); plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_EQ("mid", edge->inputs_[0]->path()); ASSERT_EQ("out", edge->outputs_[0]->path()); plan_.EdgeFinished(edge); ASSERT_FALSE(plan_.more_to_do()); edge = plan_.FindWork(); ASSERT_EQ(0, edge); } // Test that two outputs from one rule can be handled as inputs to the next. TEST_F(PlanTest, DoubleOutputDirect) { AssertParse(&state_, "build out: cat mid1 mid2\n" "build mid1 mid2: cat in\n"); GetNode("mid1")->MarkDirty(); GetNode("mid2")->MarkDirty(); GetNode("out")->MarkDirty(); string err; EXPECT_TRUE(plan_.AddTarget(GetNode("out"), &err)); ASSERT_EQ("", err); ASSERT_TRUE(plan_.more_to_do()); Edge* edge; edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat in plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat mid1 mid2 plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_FALSE(edge); // done } // Test that two outputs from one rule can eventually be routed to another. TEST_F(PlanTest, DoubleOutputIndirect) { AssertParse(&state_, "build out: cat b1 b2\n" "build b1: cat a1\n" "build b2: cat a2\n" "build a1 a2: cat in\n"); GetNode("a1")->MarkDirty(); GetNode("a2")->MarkDirty(); GetNode("b1")->MarkDirty(); GetNode("b2")->MarkDirty(); GetNode("out")->MarkDirty(); string err; EXPECT_TRUE(plan_.AddTarget(GetNode("out"), &err)); ASSERT_EQ("", err); ASSERT_TRUE(plan_.more_to_do()); Edge* edge; edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat in plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat a1 plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat a2 plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat b1 b2 plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_FALSE(edge); // done } // Test that two edges from one output can both execute. TEST_F(PlanTest, DoubleDependent) { AssertParse(&state_, "build out: cat a1 a2\n" "build a1: cat mid\n" "build a2: cat mid\n" "build mid: cat in\n"); GetNode("mid")->MarkDirty(); GetNode("a1")->MarkDirty(); GetNode("a2")->MarkDirty(); GetNode("out")->MarkDirty(); string err; EXPECT_TRUE(plan_.AddTarget(GetNode("out"), &err)); ASSERT_EQ("", err); ASSERT_TRUE(plan_.more_to_do()); Edge* edge; edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat in plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat mid plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat mid plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); // cat a1 a2 plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_FALSE(edge); // done } TEST_F(PlanTest, DependencyCycle) { AssertParse(&state_, "build out: cat mid\n" "build mid: cat in\n" "build in: cat pre\n" "build pre: cat out\n"); GetNode("out")->MarkDirty(); GetNode("mid")->MarkDirty(); GetNode("in")->MarkDirty(); GetNode("pre")->MarkDirty(); string err; EXPECT_FALSE(plan_.AddTarget(GetNode("out"), &err)); ASSERT_EQ("dependency cycle: out -> mid -> in -> pre -> out", err); } TEST_F(PlanTest, PoolWithDepthOne) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "pool foobar\n" " depth = 1\n" "rule poolcat\n" " command = cat $in > $out\n" " pool = foobar\n" "build out1: poolcat in\n" "build out2: poolcat in\n")); GetNode("out1")->MarkDirty(); GetNode("out2")->MarkDirty(); string err; EXPECT_TRUE(plan_.AddTarget(GetNode("out1"), &err)); ASSERT_EQ("", err); EXPECT_TRUE(plan_.AddTarget(GetNode("out2"), &err)); ASSERT_EQ("", err); ASSERT_TRUE(plan_.more_to_do()); Edge* edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_EQ("in", edge->inputs_[0]->path()); ASSERT_EQ("out1", edge->outputs_[0]->path()); // This will be false since poolcat is serialized ASSERT_FALSE(plan_.FindWork()); plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_EQ("in", edge->inputs_[0]->path()); ASSERT_EQ("out2", edge->outputs_[0]->path()); ASSERT_FALSE(plan_.FindWork()); plan_.EdgeFinished(edge); ASSERT_FALSE(plan_.more_to_do()); edge = plan_.FindWork(); ASSERT_EQ(0, edge); } TEST_F(PlanTest, PoolsWithDepthTwo) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "pool foobar\n" " depth = 2\n" "pool bazbin\n" " depth = 2\n" "rule foocat\n" " command = cat $in > $out\n" " pool = foobar\n" "rule bazcat\n" " command = cat $in > $out\n" " pool = bazbin\n" "build out1: foocat in\n" "build out2: foocat in\n" "build out3: foocat in\n" "build outb1: bazcat in\n" "build outb2: bazcat in\n" "build outb3: bazcat in\n" " pool =\n" "build allTheThings: cat out1 out2 out3 outb1 outb2 outb3\n" )); // Mark all the out* nodes dirty for (int i = 0; i < 3; ++i) { GetNode("out" + string(1, '1' + static_cast(i)))->MarkDirty(); GetNode("outb" + string(1, '1' + static_cast(i)))->MarkDirty(); } GetNode("allTheThings")->MarkDirty(); string err; EXPECT_TRUE(plan_.AddTarget(GetNode("allTheThings"), &err)); ASSERT_EQ("", err); deque edges; FindWorkSorted(&edges, 5); for (int i = 0; i < 4; ++i) { Edge *edge = edges[i]; ASSERT_EQ("in", edge->inputs_[0]->path()); string base_name(i < 2 ? "out" : "outb"); ASSERT_EQ(base_name + string(1, '1' + (i % 2)), edge->outputs_[0]->path()); } // outb3 is exempt because it has an empty pool Edge* edge = edges[4]; ASSERT_TRUE(edge); ASSERT_EQ("in", edge->inputs_[0]->path()); ASSERT_EQ("outb3", edge->outputs_[0]->path()); // finish out1 plan_.EdgeFinished(edges.front()); edges.pop_front(); // out3 should be available Edge* out3 = plan_.FindWork(); ASSERT_TRUE(out3); ASSERT_EQ("in", out3->inputs_[0]->path()); ASSERT_EQ("out3", out3->outputs_[0]->path()); ASSERT_FALSE(plan_.FindWork()); plan_.EdgeFinished(out3); ASSERT_FALSE(plan_.FindWork()); for (deque::iterator it = edges.begin(); it != edges.end(); ++it) { plan_.EdgeFinished(*it); } Edge* last = plan_.FindWork(); ASSERT_TRUE(last); ASSERT_EQ("allTheThings", last->outputs_[0]->path()); plan_.EdgeFinished(last); ASSERT_FALSE(plan_.more_to_do()); ASSERT_FALSE(plan_.FindWork()); } TEST_F(PlanTest, PoolWithRedundantEdges) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "pool compile\n" " depth = 1\n" "rule gen_foo\n" " command = touch foo.cpp\n" "rule gen_bar\n" " command = touch bar.cpp\n" "rule echo\n" " command = echo $out > $out\n" "build foo.cpp.obj: echo foo.cpp || foo.cpp\n" " pool = compile\n" "build bar.cpp.obj: echo bar.cpp || bar.cpp\n" " pool = compile\n" "build libfoo.a: echo foo.cpp.obj bar.cpp.obj\n" "build foo.cpp: gen_foo\n" "build bar.cpp: gen_bar\n" "build all: phony libfoo.a\n")); GetNode("foo.cpp")->MarkDirty(); GetNode("foo.cpp.obj")->MarkDirty(); GetNode("bar.cpp")->MarkDirty(); GetNode("bar.cpp.obj")->MarkDirty(); GetNode("libfoo.a")->MarkDirty(); GetNode("all")->MarkDirty(); string err; EXPECT_TRUE(plan_.AddTarget(GetNode("all"), &err)); ASSERT_EQ("", err); ASSERT_TRUE(plan_.more_to_do()); Edge* edge = NULL; deque initial_edges; FindWorkSorted(&initial_edges, 2); edge = initial_edges[1]; // Foo first ASSERT_EQ("foo.cpp", edge->outputs_[0]->path()); plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_FALSE(plan_.FindWork()); ASSERT_EQ("foo.cpp", edge->inputs_[0]->path()); ASSERT_EQ("foo.cpp", edge->inputs_[1]->path()); ASSERT_EQ("foo.cpp.obj", edge->outputs_[0]->path()); plan_.EdgeFinished(edge); edge = initial_edges[0]; // Now for bar ASSERT_EQ("bar.cpp", edge->outputs_[0]->path()); plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_FALSE(plan_.FindWork()); ASSERT_EQ("bar.cpp", edge->inputs_[0]->path()); ASSERT_EQ("bar.cpp", edge->inputs_[1]->path()); ASSERT_EQ("bar.cpp.obj", edge->outputs_[0]->path()); plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_FALSE(plan_.FindWork()); ASSERT_EQ("foo.cpp.obj", edge->inputs_[0]->path()); ASSERT_EQ("bar.cpp.obj", edge->inputs_[1]->path()); ASSERT_EQ("libfoo.a", edge->outputs_[0]->path()); plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_TRUE(edge); ASSERT_FALSE(plan_.FindWork()); ASSERT_EQ("libfoo.a", edge->inputs_[0]->path()); ASSERT_EQ("all", edge->outputs_[0]->path()); plan_.EdgeFinished(edge); edge = plan_.FindWork(); ASSERT_FALSE(edge); ASSERT_FALSE(plan_.more_to_do()); } /// Fake implementation of CommandRunner, useful for tests. struct FakeCommandRunner : public CommandRunner { explicit FakeCommandRunner(VirtualFileSystem* fs) : last_command_(NULL), fs_(fs) {} // CommandRunner impl virtual bool CanRunMore(); virtual bool StartCommand(Edge* edge); virtual bool WaitForCommand(Result* result); virtual vector GetActiveEdges(); virtual void Abort(); vector commands_ran_; Edge* last_command_; VirtualFileSystem* fs_; }; struct BuildTest : public StateTestWithBuiltinRules { BuildTest() : config_(MakeConfig()), command_runner_(&fs_), builder_(&state_, config_, NULL, NULL, &fs_), status_(config_) { } virtual void SetUp() { StateTestWithBuiltinRules::SetUp(); builder_.command_runner_.reset(&command_runner_); AssertParse(&state_, "build cat1: cat in1\n" "build cat2: cat in1 in2\n" "build cat12: cat cat1 cat2\n"); fs_.Create("in1", ""); fs_.Create("in2", ""); } ~BuildTest() { builder_.command_runner_.release(); } // Mark a path dirty. void Dirty(const string& path); BuildConfig MakeConfig() { BuildConfig config; config.verbosity = BuildConfig::QUIET; return config; } BuildConfig config_; FakeCommandRunner command_runner_; VirtualFileSystem fs_; Builder builder_; BuildStatus status_; }; bool FakeCommandRunner::CanRunMore() { // Only run one at a time. return last_command_ == NULL; } bool FakeCommandRunner::StartCommand(Edge* edge) { assert(!last_command_); commands_ran_.push_back(edge->EvaluateCommand()); if (edge->rule().name() == "cat" || edge->rule().name() == "cat_rsp" || edge->rule().name() == "cc" || edge->rule().name() == "touch" || edge->rule().name() == "touch-interrupt") { for (vector::iterator out = edge->outputs_.begin(); out != edge->outputs_.end(); ++out) { fs_->Create((*out)->path(), ""); } } else if (edge->rule().name() == "true" || edge->rule().name() == "fail" || edge->rule().name() == "interrupt") { // Don't do anything. } else { printf("unknown command\n"); return false; } last_command_ = edge; return true; } bool FakeCommandRunner::WaitForCommand(Result* result) { if (!last_command_) return false; Edge* edge = last_command_; result->edge = edge; if (edge->rule().name() == "interrupt" || edge->rule().name() == "touch-interrupt") { result->status = ExitInterrupted; return true; } if (edge->rule().name() == "fail") result->status = ExitFailure; else result->status = ExitSuccess; last_command_ = NULL; return true; } vector FakeCommandRunner::GetActiveEdges() { vector edges; if (last_command_) edges.push_back(last_command_); return edges; } void FakeCommandRunner::Abort() { last_command_ = NULL; } void BuildTest::Dirty(const string& path) { Node* node = GetNode(path); node->MarkDirty(); // If it's an input file, mark that we've already stat()ed it and // it's missing. if (!node->in_edge()) node->MarkMissing(); } TEST_F(BuildTest, NoWork) { string err; EXPECT_TRUE(builder_.AlreadyUpToDate()); } TEST_F(BuildTest, OneStep) { // Given a dirty target with one ready input, // we should rebuild the target. Dirty("cat1"); string err; EXPECT_TRUE(builder_.AddTarget("cat1", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); EXPECT_EQ("cat in1 > cat1", command_runner_.commands_ran_[0]); } TEST_F(BuildTest, OneStep2) { // Given a target with one dirty input, // we should rebuild the target. Dirty("cat1"); string err; EXPECT_TRUE(builder_.AddTarget("cat1", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); EXPECT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); EXPECT_EQ("cat in1 > cat1", command_runner_.commands_ran_[0]); } TEST_F(BuildTest, TwoStep) { string err; EXPECT_TRUE(builder_.AddTarget("cat12", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); EXPECT_EQ("", err); ASSERT_EQ(3u, command_runner_.commands_ran_.size()); // Depending on how the pointers work out, we could've ran // the first two commands in either order. EXPECT_TRUE((command_runner_.commands_ran_[0] == "cat in1 > cat1" && command_runner_.commands_ran_[1] == "cat in1 in2 > cat2") || (command_runner_.commands_ran_[1] == "cat in1 > cat1" && command_runner_.commands_ran_[0] == "cat in1 in2 > cat2")); EXPECT_EQ("cat cat1 cat2 > cat12", command_runner_.commands_ran_[2]); fs_.Tick(); // Modifying in2 requires rebuilding one intermediate file // and the final file. fs_.Create("in2", ""); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("cat12", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(5u, command_runner_.commands_ran_.size()); EXPECT_EQ("cat in1 in2 > cat2", command_runner_.commands_ran_[3]); EXPECT_EQ("cat cat1 cat2 > cat12", command_runner_.commands_ran_[4]); } TEST_F(BuildTest, TwoOutputs) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule touch\n" " command = touch $out\n" "build out1 out2: touch in.txt\n")); fs_.Create("in.txt", ""); string err; EXPECT_TRUE(builder_.AddTarget("out1", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); EXPECT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); EXPECT_EQ("touch out1 out2", command_runner_.commands_ran_[0]); } // Test case from // https://github.com/martine/ninja/issues/148 TEST_F(BuildTest, MultiOutIn) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule touch\n" " command = touch $out\n" "build in1 otherfile: touch in\n" "build out: touch in | in1\n")); fs_.Create("in", ""); fs_.Tick(); fs_.Create("in1", ""); string err; EXPECT_TRUE(builder_.AddTarget("out", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); EXPECT_EQ("", err); } TEST_F(BuildTest, Chain) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build c2: cat c1\n" "build c3: cat c2\n" "build c4: cat c3\n" "build c5: cat c4\n")); fs_.Create("c1", ""); string err; EXPECT_TRUE(builder_.AddTarget("c5", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); EXPECT_EQ("", err); ASSERT_EQ(4u, command_runner_.commands_ran_.size()); err.clear(); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("c5", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.AlreadyUpToDate()); fs_.Tick(); fs_.Create("c3", ""); err.clear(); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("c5", &err)); ASSERT_EQ("", err); EXPECT_FALSE(builder_.AlreadyUpToDate()); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(2u, command_runner_.commands_ran_.size()); // 3->4, 4->5 } TEST_F(BuildTest, MissingInput) { // Input is referenced by build file, but no rule for it. string err; Dirty("in1"); EXPECT_FALSE(builder_.AddTarget("cat1", &err)); EXPECT_EQ("'in1', needed by 'cat1', missing and no known rule to make it", err); } TEST_F(BuildTest, MissingTarget) { // Target is not referenced by build file. string err; EXPECT_FALSE(builder_.AddTarget("meow", &err)); EXPECT_EQ("unknown target: 'meow'", err); } TEST_F(BuildTest, MakeDirs) { string err; #ifdef _WIN32 ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build subdir\\dir2\\file: cat in1\n")); EXPECT_TRUE(builder_.AddTarget("subdir\\dir2\\file", &err)); #else ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build subdir/dir2/file: cat in1\n")); EXPECT_TRUE(builder_.AddTarget("subdir/dir2/file", &err)); #endif EXPECT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(2u, fs_.directories_made_.size()); EXPECT_EQ("subdir", fs_.directories_made_[0]); #ifdef _WIN32 EXPECT_EQ("subdir\\dir2", fs_.directories_made_[1]); #else EXPECT_EQ("subdir/dir2", fs_.directories_made_[1]); #endif } TEST_F(BuildTest, DepFileMissing) { string err; ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n command = cc $in\n depfile = $out.d\n" "build foo.o: cc foo.c\n")); fs_.Create("foo.c", ""); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); ASSERT_EQ("", err); ASSERT_EQ(1u, fs_.files_read_.size()); EXPECT_EQ("foo.o.d", fs_.files_read_[0]); } TEST_F(BuildTest, DepFileOK) { string err; int orig_edges = state_.edges_.size(); ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n command = cc $in\n depfile = $out.d\n" "build foo.o: cc foo.c\n")); Edge* edge = state_.edges_.back(); fs_.Create("foo.c", ""); GetNode("bar.h")->MarkDirty(); // Mark bar.h as missing. fs_.Create("foo.o.d", "foo.o: blah.h bar.h\n"); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); ASSERT_EQ("", err); ASSERT_EQ(1u, fs_.files_read_.size()); EXPECT_EQ("foo.o.d", fs_.files_read_[0]); // Expect three new edges: one generating foo.o, and two more from // loading the depfile. ASSERT_EQ(orig_edges + 3, (int)state_.edges_.size()); // Expect our edge to now have three inputs: foo.c and two headers. ASSERT_EQ(3u, edge->inputs_.size()); // Expect the command line we generate to only use the original input. ASSERT_EQ("cc foo.c", edge->EvaluateCommand()); } TEST_F(BuildTest, DepFileParseError) { string err; ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n command = cc $in\n depfile = $out.d\n" "build foo.o: cc foo.c\n")); fs_.Create("foo.c", ""); fs_.Create("foo.o.d", "randomtext\n"); EXPECT_FALSE(builder_.AddTarget("foo.o", &err)); EXPECT_EQ("expected depfile 'foo.o.d' to mention 'foo.o', got 'randomtext'", err); } TEST_F(BuildTest, OrderOnlyDeps) { string err; ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n command = cc $in\n depfile = $out.d\n" "build foo.o: cc foo.c || otherfile\n")); Edge* edge = state_.edges_.back(); fs_.Create("foo.c", ""); fs_.Create("otherfile", ""); fs_.Create("foo.o.d", "foo.o: blah.h bar.h\n"); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); ASSERT_EQ("", err); // One explicit, two implicit, one order only. ASSERT_EQ(4u, edge->inputs_.size()); EXPECT_EQ(2, edge->implicit_deps_); EXPECT_EQ(1, edge->order_only_deps_); // Verify the inputs are in the order we expect // (explicit then implicit then orderonly). EXPECT_EQ("foo.c", edge->inputs_[0]->path()); EXPECT_EQ("blah.h", edge->inputs_[1]->path()); EXPECT_EQ("bar.h", edge->inputs_[2]->path()); EXPECT_EQ("otherfile", edge->inputs_[3]->path()); // Expect the command line we generate to only use the original input. ASSERT_EQ("cc foo.c", edge->EvaluateCommand()); // explicit dep dirty, expect a rebuild. EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); fs_.Tick(); // Recreate the depfile, as it should have been deleted by the build. fs_.Create("foo.o.d", "foo.o: blah.h bar.h\n"); // implicit dep dirty, expect a rebuild. fs_.Create("blah.h", ""); fs_.Create("bar.h", ""); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); fs_.Tick(); // Recreate the depfile, as it should have been deleted by the build. fs_.Create("foo.o.d", "foo.o: blah.h bar.h\n"); // order only dep dirty, no rebuild. fs_.Create("otherfile", ""); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); EXPECT_EQ("", err); EXPECT_TRUE(builder_.AlreadyUpToDate()); // implicit dep missing, expect rebuild. fs_.RemoveFile("bar.h"); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); } TEST_F(BuildTest, RebuildOrderOnlyDeps) { string err; ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n command = cc $in\n" "rule true\n command = true\n" "build oo.h: cc oo.h.in\n" "build foo.o: cc foo.c || oo.h\n")); fs_.Create("foo.c", ""); fs_.Create("oo.h.in", ""); // foo.o and order-only dep dirty, build both. EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(2u, command_runner_.commands_ran_.size()); // all clean, no rebuild. command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); EXPECT_EQ("", err); EXPECT_TRUE(builder_.AlreadyUpToDate()); // order-only dep missing, build it only. fs_.RemoveFile("oo.h"); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); ASSERT_EQ("cc oo.h.in", command_runner_.commands_ran_[0]); fs_.Tick(); // order-only dep dirty, build it only. fs_.Create("oo.h.in", ""); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("foo.o", &err)); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); ASSERT_EQ("cc oo.h.in", command_runner_.commands_ran_[0]); } TEST_F(BuildTest, Phony) { string err; ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat bar.cc\n" "build all: phony out\n")); fs_.Create("bar.cc", ""); EXPECT_TRUE(builder_.AddTarget("all", &err)); ASSERT_EQ("", err); // Only one command to run, because phony runs no command. EXPECT_FALSE(builder_.AlreadyUpToDate()); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); } TEST_F(BuildTest, PhonyNoWork) { string err; ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat bar.cc\n" "build all: phony out\n")); fs_.Create("bar.cc", ""); fs_.Create("out", ""); EXPECT_TRUE(builder_.AddTarget("all", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.AlreadyUpToDate()); } TEST_F(BuildTest, Fail) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule fail\n" " command = fail\n" "build out1: fail\n")); string err; EXPECT_TRUE(builder_.AddTarget("out1", &err)); ASSERT_EQ("", err); EXPECT_FALSE(builder_.Build(&err)); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); ASSERT_EQ("subcommand failed", err); } TEST_F(BuildTest, SwallowFailures) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule fail\n" " command = fail\n" "build out1: fail\n" "build out2: fail\n" "build out3: fail\n" "build all: phony out1 out2 out3\n")); // Swallow two failures, die on the third. config_.failures_allowed = 3; string err; EXPECT_TRUE(builder_.AddTarget("all", &err)); ASSERT_EQ("", err); EXPECT_FALSE(builder_.Build(&err)); ASSERT_EQ(3u, command_runner_.commands_ran_.size()); ASSERT_EQ("subcommands failed", err); } TEST_F(BuildTest, SwallowFailuresLimit) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule fail\n" " command = fail\n" "build out1: fail\n" "build out2: fail\n" "build out3: fail\n" "build final: cat out1 out2 out3\n")); // Swallow ten failures; we should stop before building final. config_.failures_allowed = 11; string err; EXPECT_TRUE(builder_.AddTarget("final", &err)); ASSERT_EQ("", err); EXPECT_FALSE(builder_.Build(&err)); ASSERT_EQ(3u, command_runner_.commands_ran_.size()); ASSERT_EQ("cannot make progress due to previous errors", err); } struct BuildWithLogTest : public BuildTest { BuildWithLogTest() { builder_.SetBuildLog(&build_log_); } BuildLog build_log_; }; TEST_F(BuildWithLogTest, NotInLogButOnDisk) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n" " command = cc\n" "build out1: cc in\n")); // Create input/output that would be considered up to date when // not considering the command line hash. fs_.Create("in", ""); fs_.Create("out1", ""); string err; // Because it's not in the log, it should not be up-to-date until // we build again. EXPECT_TRUE(builder_.AddTarget("out1", &err)); EXPECT_FALSE(builder_.AlreadyUpToDate()); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("out1", &err)); EXPECT_TRUE(builder_.Build(&err)); EXPECT_TRUE(builder_.AlreadyUpToDate()); } TEST_F(BuildWithLogTest, RestatTest) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule true\n" " command = true\n" " restat = 1\n" "rule cc\n" " command = cc\n" " restat = 1\n" "build out1: cc in\n" "build out2: true out1\n" "build out3: cat out2\n")); fs_.Create("out1", ""); fs_.Create("out2", ""); fs_.Create("out3", ""); fs_.Tick(); fs_.Create("in", ""); // Do a pre-build so that there's commands in the log for the outputs, // otherwise, the lack of an entry in the build log will cause out3 to rebuild // regardless of restat. string err; EXPECT_TRUE(builder_.AddTarget("out3", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); command_runner_.commands_ran_.clear(); state_.Reset(); fs_.Tick(); fs_.Create("in", ""); // "cc" touches out1, so we should build out2. But because "true" does not // touch out2, we should cancel the build of out3. EXPECT_TRUE(builder_.AddTarget("out3", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(2u, command_runner_.commands_ran_.size()); // If we run again, it should be a no-op, because the build log has recorded // that we've already built out2 with an input timestamp of 2 (from out1). command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("out3", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.AlreadyUpToDate()); fs_.Tick(); fs_.Create("in", ""); // The build log entry should not, however, prevent us from rebuilding out2 // if out1 changes. command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("out3", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(2u, command_runner_.commands_ran_.size()); } TEST_F(BuildWithLogTest, RestatMissingFile) { // If a restat rule doesn't create its output, and the output didn't // exist before the rule was run, consider that behavior equivalent // to a rule that doesn't modify its existent output file. ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule true\n" " command = true\n" " restat = 1\n" "rule cc\n" " command = cc\n" "build out1: true in\n" "build out2: cc out1\n")); fs_.Create("in", ""); fs_.Create("out2", ""); // Do a pre-build so that there's commands in the log for the outputs, // otherwise, the lack of an entry in the build log will cause out2 to rebuild // regardless of restat. string err; EXPECT_TRUE(builder_.AddTarget("out2", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ("", err); command_runner_.commands_ran_.clear(); state_.Reset(); fs_.Tick(); fs_.Create("in", ""); fs_.Create("out2", ""); // Run a build, expect only the first command to run. // It doesn't touch its output (due to being the "true" command), so // we shouldn't run the dependent build. EXPECT_TRUE(builder_.AddTarget("out2", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); } // Test scenario, in which an input file is removed, but output isn't changed // https://github.com/martine/ninja/issues/295 TEST_F(BuildWithLogTest, RestatMissingInput) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule true\n" " command = true\n" " depfile = $out.d\n" " restat = 1\n" "rule cc\n" " command = cc\n" "build out1: true in\n" "build out2: cc out1\n")); // Create all necessary files fs_.Create("in", ""); // The implicit dependencies and the depfile itself // are newer than the output TimeStamp restat_mtime = fs_.Tick(); fs_.Create("out1.d", "out1: will.be.deleted restat.file\n"); fs_.Create("will.be.deleted", ""); fs_.Create("restat.file", ""); // Run the build, out1 and out2 get built string err; EXPECT_TRUE(builder_.AddTarget("out2", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(2u, command_runner_.commands_ran_.size()); // See that an entry in the logfile is created, capturing // the right mtime BuildLog::LogEntry * log_entry = build_log_.LookupByOutput("out1"); ASSERT_TRUE(NULL != log_entry); ASSERT_EQ(restat_mtime, log_entry->restat_mtime); // Now remove a file, referenced from depfile, so that target becomes // dirty, but the output does not change fs_.RemoveFile("will.be.deleted"); // Trigger the build again - only out1 gets built command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("out2", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); // Check that the logfile entry remains correctly set log_entry = build_log_.LookupByOutput("out1"); ASSERT_TRUE(NULL != log_entry); ASSERT_EQ(restat_mtime, log_entry->restat_mtime); } struct BuildDryRun : public BuildWithLogTest { BuildDryRun() { config_.dry_run = true; } }; TEST_F(BuildDryRun, AllCommandsShown) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule true\n" " command = true\n" " restat = 1\n" "rule cc\n" " command = cc\n" " restat = 1\n" "build out1: cc in\n" "build out2: true out1\n" "build out3: cat out2\n")); fs_.Create("out1", ""); fs_.Create("out2", ""); fs_.Create("out3", ""); fs_.Tick(); fs_.Create("in", ""); // "cc" touches out1, so we should build out2. But because "true" does not // touch out2, we should cancel the build of out3. string err; EXPECT_TRUE(builder_.AddTarget("out3", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(3u, command_runner_.commands_ran_.size()); } // Test that RSP files are created when & where appropriate and deleted after // successful execution. TEST_F(BuildTest, RspFileSuccess) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cat_rsp\n" " command = cat $rspfile > $out\n" " rspfile = $rspfile\n" " rspfile_content = $long_command\n" "build out1: cat in\n" "build out2: cat_rsp in\n" " rspfile = out2.rsp\n" " long_command = Some very long command\n")); fs_.Create("out1", ""); fs_.Create("out2", ""); fs_.Create("out3", ""); fs_.Tick(); fs_.Create("in", ""); string err; EXPECT_TRUE(builder_.AddTarget("out1", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.AddTarget("out2", &err)); ASSERT_EQ("", err); size_t files_created = fs_.files_created_.size(); size_t files_removed = fs_.files_removed_.size(); EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(2u, command_runner_.commands_ran_.size()); // cat + cat_rsp // The RSP file was created ASSERT_EQ(files_created + 1, fs_.files_created_.size()); ASSERT_EQ(1u, fs_.files_created_.count("out2.rsp")); // The RSP file was removed ASSERT_EQ(files_removed + 1, fs_.files_removed_.size()); ASSERT_EQ(1u, fs_.files_removed_.count("out2.rsp")); } // Test that RSP file is created but not removed for commands, which fail TEST_F(BuildTest, RspFileFailure) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule fail\n" " command = fail\n" " rspfile = $rspfile\n" " rspfile_content = $long_command\n" "build out: fail in\n" " rspfile = out.rsp\n" " long_command = Another very long command\n")); fs_.Create("out", ""); fs_.Tick(); fs_.Create("in", ""); string err; EXPECT_TRUE(builder_.AddTarget("out", &err)); ASSERT_EQ("", err); size_t files_created = fs_.files_created_.size(); size_t files_removed = fs_.files_removed_.size(); EXPECT_FALSE(builder_.Build(&err)); ASSERT_EQ("subcommand failed", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); // The RSP file was created ASSERT_EQ(files_created + 1, fs_.files_created_.size()); ASSERT_EQ(1u, fs_.files_created_.count("out.rsp")); // The RSP file was NOT removed ASSERT_EQ(files_removed, fs_.files_removed_.size()); ASSERT_EQ(0u, fs_.files_removed_.count("out.rsp")); // The RSP file contains what it should ASSERT_EQ("Another very long command", fs_.files_["out.rsp"].contents); } // Test that contens of the RSP file behaves like a regular part of // command line, i.e. triggers a rebuild if changed TEST_F(BuildWithLogTest, RspFileCmdLineChange) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cat_rsp\n" " command = cat $rspfile > $out\n" " rspfile = $rspfile\n" " rspfile_content = $long_command\n" "build out: cat_rsp in\n" " rspfile = out.rsp\n" " long_command = Original very long command\n")); fs_.Create("out", ""); fs_.Tick(); fs_.Create("in", ""); string err; EXPECT_TRUE(builder_.AddTarget("out", &err)); ASSERT_EQ("", err); // 1. Build for the 1st time (-> populate log) EXPECT_TRUE(builder_.Build(&err)); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); // 2. Build again (no change) command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("out", &err)); EXPECT_EQ("", err); ASSERT_TRUE(builder_.AlreadyUpToDate()); // 3. Alter the entry in the logfile // (to simulate a change in the command line between 2 builds) BuildLog::LogEntry * log_entry = build_log_.LookupByOutput("out"); ASSERT_TRUE(NULL != log_entry); ASSERT_NO_FATAL_FAILURE(AssertHash( "cat out.rsp > out;rspfile=Original very long command", log_entry->command_hash)); log_entry->command_hash++; // Change the command hash to something else. // Now expect the target to be rebuilt command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("out", &err)); EXPECT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); EXPECT_EQ(1u, command_runner_.commands_ran_.size()); } TEST_F(BuildTest, InterruptCleanup) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule interrupt\n" " command = interrupt\n" "rule touch-interrupt\n" " command = touch-interrupt\n" "build out1: interrupt in1\n" "build out2: touch-interrupt in2\n")); fs_.Create("out1", ""); fs_.Create("out2", ""); fs_.Tick(); fs_.Create("in1", ""); fs_.Create("in2", ""); // An untouched output of an interrupted command should be retained. string err; EXPECT_TRUE(builder_.AddTarget("out1", &err)); EXPECT_EQ("", err); EXPECT_FALSE(builder_.Build(&err)); EXPECT_EQ("interrupted by user", err); builder_.Cleanup(); EXPECT_GT(fs_.Stat("out1"), 0); err = ""; // A touched output of an interrupted command should be deleted. EXPECT_TRUE(builder_.AddTarget("out2", &err)); EXPECT_EQ("", err); EXPECT_FALSE(builder_.Build(&err)); EXPECT_EQ("interrupted by user", err); builder_.Cleanup(); EXPECT_EQ(0, fs_.Stat("out2")); } TEST_F(BuildTest, PhonyWithNoInputs) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build nonexistent: phony\n" "build out1: cat || nonexistent\n" "build out2: cat nonexistent\n")); fs_.Create("out1", ""); fs_.Create("out2", ""); // out1 should be up to date even though its input is dirty, because its // order-only dependency has nothing to do. string err; EXPECT_TRUE(builder_.AddTarget("out1", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.AlreadyUpToDate()); // out2 should still be out of date though, because its input is dirty. err.clear(); command_runner_.commands_ran_.clear(); state_.Reset(); EXPECT_TRUE(builder_.AddTarget("out2", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder_.Build(&err)); EXPECT_EQ("", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); } TEST_F(BuildTest, DepsGccWithEmptyDepfileErrorsOut) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n" " command = cc\n" " deps = gcc\n" "build out: cc\n")); Dirty("out"); string err; EXPECT_TRUE(builder_.AddTarget("out", &err)); ASSERT_EQ("", err); EXPECT_FALSE(builder_.AlreadyUpToDate()); EXPECT_FALSE(builder_.Build(&err)); ASSERT_EQ("subcommand failed", err); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); } TEST_F(BuildTest, StatusFormatReplacePlaceholder) { EXPECT_EQ("[%/s0/t0/r0/u0/f0]", status_.FormatProgressStatus("[%%/s%s/t%t/r%r/u%u/f%f]")); } TEST_F(BuildTest, FailedDepsParse) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build bad_deps.o: cat in1\n" " deps = gcc\n" " depfile = in1.d\n")); string err; EXPECT_TRUE(builder_.AddTarget("bad_deps.o", &err)); ASSERT_EQ("", err); // These deps will fail to parse, as they should only have one // path to the left of the colon. fs_.Create("in1.d", "AAA BBB"); EXPECT_FALSE(builder_.Build(&err)); EXPECT_EQ("subcommand failed", err); } /// Tests of builds involving deps logs necessarily must span /// multiple builds. We reuse methods on BuildTest but not the /// builder_ it sets up, because we want pristine objects for /// each build. struct BuildWithDepsLogTest : public BuildTest { BuildWithDepsLogTest() {} virtual void SetUp() { BuildTest::SetUp(); temp_dir_.CreateAndEnter("BuildWithDepsLogTest"); } virtual void TearDown() { temp_dir_.Cleanup(); } ScopedTempDir temp_dir_; /// Shadow parent class builder_ so we don't accidentally use it. void* builder_; }; /// Run a straightforwad build where the deps log is used. TEST_F(BuildWithDepsLogTest, Straightforward) { string err; // Note: in1 was created by the superclass SetUp(). const char* manifest = "build out: cat in1\n" " deps = gcc\n" " depfile = in1.d\n"; { State state; ASSERT_NO_FATAL_FAILURE(AddCatRule(&state)); ASSERT_NO_FATAL_FAILURE(AssertParse(&state, manifest)); // Run the build once, everything should be ok. DepsLog deps_log; ASSERT_TRUE(deps_log.OpenForWrite("ninja_deps", &err)); ASSERT_EQ("", err); Builder builder(&state, config_, NULL, &deps_log, &fs_); builder.command_runner_.reset(&command_runner_); EXPECT_TRUE(builder.AddTarget("out", &err)); ASSERT_EQ("", err); fs_.Create("in1.d", "out: in2"); EXPECT_TRUE(builder.Build(&err)); EXPECT_EQ("", err); // The deps file should have been removed. EXPECT_EQ(0, fs_.Stat("in1.d")); // Recreate it for the next step. fs_.Create("in1.d", "out: in2"); deps_log.Close(); builder.command_runner_.release(); } { State state; ASSERT_NO_FATAL_FAILURE(AddCatRule(&state)); ASSERT_NO_FATAL_FAILURE(AssertParse(&state, manifest)); // Touch the file only mentioned in the deps. fs_.Tick(); fs_.Create("in2", ""); // Run the build again. DepsLog deps_log; ASSERT_TRUE(deps_log.Load("ninja_deps", &state, &err)); ASSERT_TRUE(deps_log.OpenForWrite("ninja_deps", &err)); Builder builder(&state, config_, NULL, &deps_log, &fs_); builder.command_runner_.reset(&command_runner_); command_runner_.commands_ran_.clear(); EXPECT_TRUE(builder.AddTarget("out", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder.Build(&err)); EXPECT_EQ("", err); // We should have rebuilt the output due to in2 being // out of date. EXPECT_EQ(1u, command_runner_.commands_ran_.size()); builder.command_runner_.release(); } } /// Verify that obsolete dependency info causes a rebuild. /// 1) Run a successful build where everything has time t, record deps. /// 2) Move input/output to time t+1 -- despite files in alignment, /// should still need to rebuild due to deps at older time. TEST_F(BuildWithDepsLogTest, ObsoleteDeps) { string err; // Note: in1 was created by the superclass SetUp(). const char* manifest = "build out: cat in1\n" " deps = gcc\n" " depfile = in1.d\n"; { // Run an ordinary build that gathers dependencies. fs_.Create("in1", ""); fs_.Create("in1.d", "out: "); State state; ASSERT_NO_FATAL_FAILURE(AddCatRule(&state)); ASSERT_NO_FATAL_FAILURE(AssertParse(&state, manifest)); // Run the build once, everything should be ok. DepsLog deps_log; ASSERT_TRUE(deps_log.OpenForWrite("ninja_deps", &err)); ASSERT_EQ("", err); Builder builder(&state, config_, NULL, &deps_log, &fs_); builder.command_runner_.reset(&command_runner_); EXPECT_TRUE(builder.AddTarget("out", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder.Build(&err)); EXPECT_EQ("", err); deps_log.Close(); builder.command_runner_.release(); } // Push all files one tick forward so that only the deps are out // of date. fs_.Tick(); fs_.Create("in1", ""); fs_.Create("out", ""); // The deps file should have been removed, so no need to timestamp it. EXPECT_EQ(0, fs_.Stat("in1.d")); { State state; ASSERT_NO_FATAL_FAILURE(AddCatRule(&state)); ASSERT_NO_FATAL_FAILURE(AssertParse(&state, manifest)); DepsLog deps_log; ASSERT_TRUE(deps_log.Load("ninja_deps", &state, &err)); ASSERT_TRUE(deps_log.OpenForWrite("ninja_deps", &err)); Builder builder(&state, config_, NULL, &deps_log, &fs_); builder.command_runner_.reset(&command_runner_); command_runner_.commands_ran_.clear(); EXPECT_TRUE(builder.AddTarget("out", &err)); ASSERT_EQ("", err); // Recreate the deps file here because the build expects them to exist. fs_.Create("in1.d", "out: "); EXPECT_TRUE(builder.Build(&err)); EXPECT_EQ("", err); // We should have rebuilt the output due to the deps being // out of date. EXPECT_EQ(1u, command_runner_.commands_ran_.size()); builder.command_runner_.release(); } } TEST_F(BuildWithDepsLogTest, DepsIgnoredInDryRun) { const char* manifest = "build out: cat in1\n" " deps = gcc\n" " depfile = in1.d\n"; fs_.Create("out", ""); fs_.Tick(); fs_.Create("in1", ""); State state; ASSERT_NO_FATAL_FAILURE(AddCatRule(&state)); ASSERT_NO_FATAL_FAILURE(AssertParse(&state, manifest)); // The deps log is NULL in dry runs. config_.dry_run = true; Builder builder(&state, config_, NULL, NULL, &fs_); builder.command_runner_.reset(&command_runner_); command_runner_.commands_ran_.clear(); string err; EXPECT_TRUE(builder.AddTarget("out", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder.Build(&err)); ASSERT_EQ(1u, command_runner_.commands_ran_.size()); builder.command_runner_.release(); } /// Check that a restat rule generating a header cancels compilations correctly. TEST_F(BuildWithDepsLogTest, RestatDepfileDependency) { string err; // Note: in1 was created by the superclass SetUp(). const char* manifest = "rule true\n" " command = true\n" // Would be "write if out-of-date" in reality. " restat = 1\n" "build header.h: true header.in\n" "build out: cat in1\n" " deps = gcc\n" " depfile = in1.d\n"; { State state; ASSERT_NO_FATAL_FAILURE(AddCatRule(&state)); ASSERT_NO_FATAL_FAILURE(AssertParse(&state, manifest)); // Run the build once, everything should be ok. DepsLog deps_log; ASSERT_TRUE(deps_log.OpenForWrite("ninja_deps", &err)); ASSERT_EQ("", err); Builder builder(&state, config_, NULL, &deps_log, &fs_); builder.command_runner_.reset(&command_runner_); EXPECT_TRUE(builder.AddTarget("out", &err)); ASSERT_EQ("", err); fs_.Create("in1.d", "out: header.h"); EXPECT_TRUE(builder.Build(&err)); EXPECT_EQ("", err); deps_log.Close(); builder.command_runner_.release(); } { State state; ASSERT_NO_FATAL_FAILURE(AddCatRule(&state)); ASSERT_NO_FATAL_FAILURE(AssertParse(&state, manifest)); // Touch the input of the restat rule. fs_.Tick(); fs_.Create("header.in", ""); // Run the build again. DepsLog deps_log; ASSERT_TRUE(deps_log.Load("ninja_deps", &state, &err)); ASSERT_TRUE(deps_log.OpenForWrite("ninja_deps", &err)); Builder builder(&state, config_, NULL, &deps_log, &fs_); builder.command_runner_.reset(&command_runner_); command_runner_.commands_ran_.clear(); EXPECT_TRUE(builder.AddTarget("out", &err)); ASSERT_EQ("", err); EXPECT_TRUE(builder.Build(&err)); EXPECT_EQ("", err); // Rule "true" should have run again, but the build of "out" should have // been cancelled due to restat propagating through the depfile header. EXPECT_EQ(1u, command_runner_.commands_ran_.size()); builder.command_runner_.release(); } } ninja-1.3.4/src/canon_perftest.cc000066400000000000000000000027751215343310300167340ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include #include "util.h" #include "metrics.h" const char kPath[] = "../../third_party/WebKit/Source/WebCore/" "platform/leveldb/LevelDBWriteBatch.cpp"; int main() { vector times; string err; char buf[200]; size_t len = strlen(kPath); strcpy(buf, kPath); for (int j = 0; j < 5; ++j) { const int kNumRepetitions = 2000000; int64_t start = GetTimeMillis(); for (int i = 0; i < kNumRepetitions; ++i) { CanonicalizePath(buf, &len, &err); } int delta = (int)(GetTimeMillis() - start); times.push_back(delta); } int min = times[0]; int max = times[0]; float total = 0; for (size_t i = 0; i < times.size(); ++i) { total += times[i]; if (times[i] < min) min = times[i]; else if (times[i] > max) max = times[i]; } printf("min %dms max %dms avg %.1fms\n", min, max, total / times.size()); } ninja-1.3.4/src/clean.cc000066400000000000000000000136301215343310300147740ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "clean.h" #include #include #include "disk_interface.h" #include "graph.h" #include "state.h" #include "util.h" Cleaner::Cleaner(State* state, const BuildConfig& config) : state_(state), config_(config), removed_(), cleaned_(), cleaned_files_count_(0), disk_interface_(new RealDiskInterface), status_(0) { } Cleaner::Cleaner(State* state, const BuildConfig& config, DiskInterface* disk_interface) : state_(state), config_(config), removed_(), cleaned_(), cleaned_files_count_(0), disk_interface_(disk_interface), status_(0) { } int Cleaner::RemoveFile(const string& path) { return disk_interface_->RemoveFile(path); } bool Cleaner::FileExists(const string& path) { return disk_interface_->Stat(path) > 0; } void Cleaner::Report(const string& path) { ++cleaned_files_count_; if (IsVerbose()) printf("Remove %s\n", path.c_str()); } void Cleaner::Remove(const string& path) { if (!IsAlreadyRemoved(path)) { removed_.insert(path); if (config_.dry_run) { if (FileExists(path)) Report(path); } else { int ret = RemoveFile(path); if (ret == 0) Report(path); else if (ret == -1) status_ = 1; } } } bool Cleaner::IsAlreadyRemoved(const string& path) { set::iterator i = removed_.find(path); return (i != removed_.end()); } void Cleaner::RemoveEdgeFiles(Edge* edge) { string depfile = edge->GetBinding("depfile"); if (!depfile.empty()) Remove(depfile); string rspfile = edge->GetBinding("rspfile"); if (!rspfile.empty()) Remove(rspfile); } void Cleaner::PrintHeader() { if (config_.verbosity == BuildConfig::QUIET) return; printf("Cleaning..."); if (IsVerbose()) printf("\n"); else printf(" "); } void Cleaner::PrintFooter() { if (config_.verbosity == BuildConfig::QUIET) return; printf("%d files.\n", cleaned_files_count_); } int Cleaner::CleanAll(bool generator) { Reset(); PrintHeader(); for (vector::iterator e = state_->edges_.begin(); e != state_->edges_.end(); ++e) { // Do not try to remove phony targets if ((*e)->is_phony()) continue; // Do not remove generator's files unless generator specified. if (!generator && (*e)->GetBindingBool("generator")) continue; for (vector::iterator out_node = (*e)->outputs_.begin(); out_node != (*e)->outputs_.end(); ++out_node) { Remove((*out_node)->path()); } RemoveEdgeFiles(*e); } PrintFooter(); return status_; } void Cleaner::DoCleanTarget(Node* target) { if (Edge* e = target->in_edge()) { // Do not try to remove phony targets if (!e->is_phony()) { Remove(target->path()); RemoveEdgeFiles(e); } for (vector::iterator n = e->inputs_.begin(); n != e->inputs_.end(); ++n) { Node* next = *n; // call DoCleanTarget recursively if this node has not been visited if (cleaned_.count(next) == 0) { DoCleanTarget(next); } } } // mark this target to be cleaned already cleaned_.insert(target); } int Cleaner::CleanTarget(Node* target) { assert(target); Reset(); PrintHeader(); DoCleanTarget(target); PrintFooter(); return status_; } int Cleaner::CleanTarget(const char* target) { assert(target); Reset(); Node* node = state_->LookupNode(target); if (node) { CleanTarget(node); } else { Error("unknown target '%s'", target); status_ = 1; } return status_; } int Cleaner::CleanTargets(int target_count, char* targets[]) { Reset(); PrintHeader(); for (int i = 0; i < target_count; ++i) { const char* target_name = targets[i]; Node* target = state_->LookupNode(target_name); if (target) { if (IsVerbose()) printf("Target %s\n", target_name); DoCleanTarget(target); } else { Error("unknown target '%s'", target_name); status_ = 1; } } PrintFooter(); return status_; } void Cleaner::DoCleanRule(const Rule* rule) { assert(rule); for (vector::iterator e = state_->edges_.begin(); e != state_->edges_.end(); ++e) { if ((*e)->rule().name() == rule->name()) { for (vector::iterator out_node = (*e)->outputs_.begin(); out_node != (*e)->outputs_.end(); ++out_node) { Remove((*out_node)->path()); RemoveEdgeFiles(*e); } } } } int Cleaner::CleanRule(const Rule* rule) { assert(rule); Reset(); PrintHeader(); DoCleanRule(rule); PrintFooter(); return status_; } int Cleaner::CleanRule(const char* rule) { assert(rule); Reset(); const Rule* r = state_->LookupRule(rule); if (r) { CleanRule(r); } else { Error("unknown rule '%s'", rule); status_ = 1; } return status_; } int Cleaner::CleanRules(int rule_count, char* rules[]) { assert(rules); Reset(); PrintHeader(); for (int i = 0; i < rule_count; ++i) { const char* rule_name = rules[i]; const Rule* rule = state_->LookupRule(rule_name); if (rule) { if (IsVerbose()) printf("Rule %s\n", rule_name); DoCleanRule(rule); } else { Error("unknown rule '%s'", rule_name); status_ = 1; } } PrintFooter(); return status_; } void Cleaner::Reset() { status_ = 0; cleaned_files_count_ = 0; removed_.clear(); cleaned_.clear(); } ninja-1.3.4/src/clean.h000066400000000000000000000065661215343310300146500ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_CLEAN_H_ #define NINJA_CLEAN_H_ #include #include #include "build.h" using namespace std; struct State; struct Node; struct Rule; struct DiskInterface; struct Cleaner { /// Build a cleaner object with a real disk interface. Cleaner(State* state, const BuildConfig& config); /// Build a cleaner object with the given @a disk_interface /// (Useful for testing). Cleaner(State* state, const BuildConfig& config, DiskInterface* disk_interface); /// Clean the given @a target and all the file built for it. /// @return non-zero if an error occurs. int CleanTarget(Node* target); /// Clean the given target @a target. /// @return non-zero if an error occurs. int CleanTarget(const char* target); /// Clean the given target @a targets. /// @return non-zero if an error occurs. int CleanTargets(int target_count, char* targets[]); /// Clean all built files, except for files created by generator rules. /// @param generator If set, also clean files created by generator rules. /// @return non-zero if an error occurs. int CleanAll(bool generator = false); /// Clean all the file built with the given rule @a rule. /// @return non-zero if an error occurs. int CleanRule(const Rule* rule); /// Clean the file produced by the given @a rule. /// @return non-zero if an error occurs. int CleanRule(const char* rule); /// Clean the file produced by the given @a rules. /// @return non-zero if an error occurs. int CleanRules(int rule_count, char* rules[]); /// @return the number of file cleaned. int cleaned_files_count() const { return cleaned_files_count_; } /// @return whether the cleaner is in verbose mode. bool IsVerbose() const { return (config_.verbosity != BuildConfig::QUIET && (config_.verbosity == BuildConfig::VERBOSE || config_.dry_run)); } private: /// Remove the file @a path. /// @return whether the file has been removed. int RemoveFile(const string& path); /// @returns whether the file @a path exists. bool FileExists(const string& path); void Report(const string& path); /// Remove the given @a path file only if it has not been already removed. void Remove(const string& path); /// @return whether the given @a path has already been removed. bool IsAlreadyRemoved(const string& path); /// Remove the depfile and rspfile for an Edge. void RemoveEdgeFiles(Edge* edge); /// Helper recursive method for CleanTarget(). void DoCleanTarget(Node* target); void PrintHeader(); void PrintFooter(); void DoCleanRule(const Rule* rule); void Reset(); State* state_; const BuildConfig& config_; set removed_; set cleaned_; int cleaned_files_count_; DiskInterface* disk_interface_; int status_; }; #endif // NINJA_CLEAN_H_ ninja-1.3.4/src/clean_test.cc000066400000000000000000000244501215343310300160350ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "clean.h" #include "build.h" #include "test.h" struct CleanTest : public StateTestWithBuiltinRules { VirtualFileSystem fs_; BuildConfig config_; virtual void SetUp() { config_.verbosity = BuildConfig::QUIET; } }; TEST_F(CleanTest, CleanAll) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build in1: cat src1\n" "build out1: cat in1\n" "build in2: cat src2\n" "build out2: cat in2\n")); fs_.Create("in1", ""); fs_.Create("out1", ""); fs_.Create("in2", ""); fs_.Create("out2", ""); Cleaner cleaner(&state_, config_, &fs_); ASSERT_EQ(0, cleaner.cleaned_files_count()); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(4, cleaner.cleaned_files_count()); EXPECT_EQ(4u, fs_.files_removed_.size()); // Check they are removed. EXPECT_EQ(0, fs_.Stat("in1")); EXPECT_EQ(0, fs_.Stat("out1")); EXPECT_EQ(0, fs_.Stat("in2")); EXPECT_EQ(0, fs_.Stat("out2")); fs_.files_removed_.clear(); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(0, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanAllDryRun) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build in1: cat src1\n" "build out1: cat in1\n" "build in2: cat src2\n" "build out2: cat in2\n")); fs_.Create("in1", ""); fs_.Create("out1", ""); fs_.Create("in2", ""); fs_.Create("out2", ""); config_.dry_run = true; Cleaner cleaner(&state_, config_, &fs_); ASSERT_EQ(0, cleaner.cleaned_files_count()); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(4, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); // Check they are not removed. EXPECT_NE(0, fs_.Stat("in1")); EXPECT_NE(0, fs_.Stat("out1")); EXPECT_NE(0, fs_.Stat("in2")); EXPECT_NE(0, fs_.Stat("out2")); fs_.files_removed_.clear(); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(4, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanTarget) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build in1: cat src1\n" "build out1: cat in1\n" "build in2: cat src2\n" "build out2: cat in2\n")); fs_.Create("in1", ""); fs_.Create("out1", ""); fs_.Create("in2", ""); fs_.Create("out2", ""); Cleaner cleaner(&state_, config_, &fs_); ASSERT_EQ(0, cleaner.cleaned_files_count()); ASSERT_EQ(0, cleaner.CleanTarget("out1")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(2u, fs_.files_removed_.size()); // Check they are removed. EXPECT_EQ(0, fs_.Stat("in1")); EXPECT_EQ(0, fs_.Stat("out1")); EXPECT_NE(0, fs_.Stat("in2")); EXPECT_NE(0, fs_.Stat("out2")); fs_.files_removed_.clear(); ASSERT_EQ(0, cleaner.CleanTarget("out1")); EXPECT_EQ(0, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanTargetDryRun) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build in1: cat src1\n" "build out1: cat in1\n" "build in2: cat src2\n" "build out2: cat in2\n")); fs_.Create("in1", ""); fs_.Create("out1", ""); fs_.Create("in2", ""); fs_.Create("out2", ""); config_.dry_run = true; Cleaner cleaner(&state_, config_, &fs_); ASSERT_EQ(0, cleaner.cleaned_files_count()); ASSERT_EQ(0, cleaner.CleanTarget("out1")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); // Check they are removed. EXPECT_NE(0, fs_.Stat("in1")); EXPECT_NE(0, fs_.Stat("out1")); EXPECT_NE(0, fs_.Stat("in2")); EXPECT_NE(0, fs_.Stat("out2")); fs_.files_removed_.clear(); ASSERT_EQ(0, cleaner.CleanTarget("out1")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanRule) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cat_e\n" " command = cat -e $in > $out\n" "build in1: cat_e src1\n" "build out1: cat in1\n" "build in2: cat_e src2\n" "build out2: cat in2\n")); fs_.Create("in1", ""); fs_.Create("out1", ""); fs_.Create("in2", ""); fs_.Create("out2", ""); Cleaner cleaner(&state_, config_, &fs_); ASSERT_EQ(0, cleaner.cleaned_files_count()); ASSERT_EQ(0, cleaner.CleanRule("cat_e")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(2u, fs_.files_removed_.size()); // Check they are removed. EXPECT_EQ(0, fs_.Stat("in1")); EXPECT_NE(0, fs_.Stat("out1")); EXPECT_EQ(0, fs_.Stat("in2")); EXPECT_NE(0, fs_.Stat("out2")); fs_.files_removed_.clear(); ASSERT_EQ(0, cleaner.CleanRule("cat_e")); EXPECT_EQ(0, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanRuleDryRun) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cat_e\n" " command = cat -e $in > $out\n" "build in1: cat_e src1\n" "build out1: cat in1\n" "build in2: cat_e src2\n" "build out2: cat in2\n")); fs_.Create("in1", ""); fs_.Create("out1", ""); fs_.Create("in2", ""); fs_.Create("out2", ""); config_.dry_run = true; Cleaner cleaner(&state_, config_, &fs_); ASSERT_EQ(0, cleaner.cleaned_files_count()); ASSERT_EQ(0, cleaner.CleanRule("cat_e")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); // Check they are removed. EXPECT_NE(0, fs_.Stat("in1")); EXPECT_NE(0, fs_.Stat("out1")); EXPECT_NE(0, fs_.Stat("in2")); EXPECT_NE(0, fs_.Stat("out2")); fs_.files_removed_.clear(); ASSERT_EQ(0, cleaner.CleanRule("cat_e")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(0u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanRuleGenerator) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule regen\n" " command = cat $in > $out\n" " generator = 1\n" "build out1: cat in1\n" "build out2: regen in2\n")); fs_.Create("out1", ""); fs_.Create("out2", ""); Cleaner cleaner(&state_, config_, &fs_); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(1, cleaner.cleaned_files_count()); EXPECT_EQ(1u, fs_.files_removed_.size()); fs_.Create("out1", ""); EXPECT_EQ(0, cleaner.CleanAll(/*generator=*/true)); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(2u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanDepFile) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n" " command = cc $in > $out\n" " depfile = $out.d\n" "build out1: cc in1\n")); fs_.Create("out1", ""); fs_.Create("out1.d", ""); Cleaner cleaner(&state_, config_, &fs_); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(2u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanDepFileOnCleanTarget) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n" " command = cc $in > $out\n" " depfile = $out.d\n" "build out1: cc in1\n")); fs_.Create("out1", ""); fs_.Create("out1.d", ""); Cleaner cleaner(&state_, config_, &fs_); EXPECT_EQ(0, cleaner.CleanTarget("out1")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(2u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanDepFileOnCleanRule) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n" " command = cc $in > $out\n" " depfile = $out.d\n" "build out1: cc in1\n")); fs_.Create("out1", ""); fs_.Create("out1.d", ""); Cleaner cleaner(&state_, config_, &fs_); EXPECT_EQ(0, cleaner.CleanRule("cc")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(2u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanRspFile) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cc\n" " command = cc $in > $out\n" " rspfile = $rspfile\n" " rspfile_content=$in\n" "build out1: cc in1\n" " rspfile = cc1.rsp\n" " rspfile_content=$in\n")); fs_.Create("out1", ""); fs_.Create("cc1.rsp", ""); Cleaner cleaner(&state_, config_, &fs_); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(2u, fs_.files_removed_.size()); } TEST_F(CleanTest, CleanRsp) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule cat_rsp \n" " command = cat $rspfile > $out\n" " rspfile = $rspfile\n" " rspfile_content = $in\n" "build in1: cat src1\n" "build out1: cat in1\n" "build in2: cat_rsp src2\n" " rspfile=in2.rsp\n" " rspfile_content=$in\n" "build out2: cat_rsp in2\n" " rspfile=out2.rsp\n" " rspfile_content=$in\n")); fs_.Create("in1", ""); fs_.Create("out1", ""); fs_.Create("in2.rsp", ""); fs_.Create("out2.rsp", ""); fs_.Create("in2", ""); fs_.Create("out2", ""); Cleaner cleaner(&state_, config_, &fs_); ASSERT_EQ(0, cleaner.cleaned_files_count()); ASSERT_EQ(0, cleaner.CleanTarget("out1")); EXPECT_EQ(2, cleaner.cleaned_files_count()); ASSERT_EQ(0, cleaner.CleanTarget("in2")); EXPECT_EQ(2, cleaner.cleaned_files_count()); ASSERT_EQ(0, cleaner.CleanRule("cat_rsp")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_EQ(6u, fs_.files_removed_.size()); // Check they are removed. EXPECT_EQ(0, fs_.Stat("in1")); EXPECT_EQ(0, fs_.Stat("out1")); EXPECT_EQ(0, fs_.Stat("in2")); EXPECT_EQ(0, fs_.Stat("out2")); EXPECT_EQ(0, fs_.Stat("in2.rsp")); EXPECT_EQ(0, fs_.Stat("out2.rsp")); fs_.files_removed_.clear(); } TEST_F(CleanTest, CleanFailure) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build dir: cat src1\n")); fs_.MakeDir("dir"); Cleaner cleaner(&state_, config_, &fs_); EXPECT_NE(0, cleaner.CleanAll()); } TEST_F(CleanTest, CleanPhony) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build phony: phony t1 t2\n" "build t1: cat\n" "build t2: cat\n")); fs_.Create("phony", ""); fs_.Create("t1", ""); fs_.Create("t2", ""); // Check that CleanAll does not remove "phony". Cleaner cleaner(&state_, config_, &fs_); EXPECT_EQ(0, cleaner.CleanAll()); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_NE(0, fs_.Stat("phony")); fs_.Create("t1", ""); fs_.Create("t2", ""); // Check that CleanTarget does not remove "phony". EXPECT_EQ(0, cleaner.CleanTarget("phony")); EXPECT_EQ(2, cleaner.cleaned_files_count()); EXPECT_NE(0, fs_.Stat("phony")); } ninja-1.3.4/src/depfile_parser.cc000066400000000000000000000147751215343310300167110ustar00rootroot00000000000000/* Generated by re2c 0.13.5 */ // Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "depfile_parser.h" // A note on backslashes in Makefiles, from reading the docs: // Backslash-newline is the line continuation character. // Backslash-# escapes a # (otherwise meaningful as a comment start). // Backslash-% escapes a % (otherwise meaningful as a special). // Finally, quoting the GNU manual, "Backslashes that are not in danger // of quoting ‘%’ characters go unmolested." // How do you end a line with a backslash? The netbsd Make docs suggest // reading the result of a shell command echoing a backslash! // // Rather than implement all of above, we do a simpler thing here: // Backslashes escape a set of characters (see "escapes" defined below), // otherwise they are passed through verbatim. // If anyone actually has depfiles that rely on the more complicated // behavior we can adjust this. bool DepfileParser::Parse(string* content, string* err) { // in: current parser input point. // end: end of input. // parsing_targets: whether we are parsing targets or dependencies. char* in = &(*content)[0]; char* end = in + content->size(); bool parsing_targets = true; while (in < end) { // out: current output point (typically same as in, but can fall behind // as we de-escape backslashes). char* out = in; // filename: start of the current parsed filename. char* filename = out; for (;;) { // start: beginning of the current parsed span. const char* start = in; { char yych; static const unsigned char yybm[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 0, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 0, 0, 0, 0, 128, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 0, 0, 0, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, }; yych = *in; if (yych <= '[') { if (yych <= '$') { if (yych <= 0x00) goto yy7; if (yych <= ' ') goto yy9; if (yych <= '#') goto yy6; goto yy4; } else { if (yych <= '=') goto yy6; if (yych <= '?') goto yy9; if (yych <= 'Z') goto yy6; goto yy9; } } else { if (yych <= '`') { if (yych <= '\\') goto yy2; if (yych == '_') goto yy6; goto yy9; } else { if (yych <= 'z') goto yy6; if (yych == '~') goto yy6; goto yy9; } } yy2: ++in; if ((yych = *in) <= '#') { if (yych <= '\n') { if (yych <= 0x00) goto yy3; if (yych <= '\t') goto yy14; } else { if (yych == ' ') goto yy16; if (yych <= '"') goto yy14; goto yy16; } } else { if (yych <= 'Z') { if (yych == '*') goto yy16; goto yy14; } else { if (yych <= '\\') goto yy16; if (yych == '|') goto yy16; goto yy14; } } yy3: { // For any other character (e.g. whitespace), swallow it here, // allowing the outer logic to loop around again. break; } yy4: ++in; if ((yych = *in) == '$') goto yy12; goto yy11; yy5: { // Got a span of plain text. int len = (int)(in - start); // Need to shift it over if we're overwriting backslashes. if (out < start) memmove(out, start, len); out += len; continue; } yy6: yych = *++in; goto yy11; yy7: ++in; { break; } yy9: yych = *++in; goto yy3; yy10: ++in; yych = *in; yy11: if (yybm[0+yych] & 128) { goto yy10; } goto yy5; yy12: ++in; if (yybm[0+(yych = *in)] & 128) { goto yy10; } { // De-escape dollar character. *out++ = '$'; continue; } yy14: ++in; { // Let backslash before other characters through verbatim. *out++ = '\\'; *out++ = yych; continue; } yy16: ++in; { // De-escape backslashed character. *out++ = yych; continue; } } } int len = (int)(out - filename); const bool is_target = parsing_targets; if (len > 0 && filename[len - 1] == ':') { len--; // Strip off trailing colon, if any. parsing_targets = false; } if (len == 0) continue; if (!is_target) { ins_.push_back(StringPiece(filename, len)); } else if (!out_.str_) { out_ = StringPiece(filename, len); } else if (out_ != StringPiece(filename, len)) { *err = "depfile has multiple output paths."; return false; } } return true; } ninja-1.3.4/src/depfile_parser.h000066400000000000000000000021661215343310300165420ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_DEPFILE_PARSER_H_ #define NINJA_DEPFILE_PARSER_H_ #include #include using namespace std; #include "string_piece.h" /// Parser for the dependency information emitted by gcc's -M flags. struct DepfileParser { /// Parse an input file. Input must be NUL-terminated. /// Warning: may mutate the content in-place and parsed StringPieces are /// pointers within it. bool Parse(string* content, string* err); StringPiece out_; vector ins_; }; #endif // NINJA_DEPFILE_PARSER_H_ ninja-1.3.4/src/depfile_parser.in.cc000066400000000000000000000072231215343310300173040ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "depfile_parser.h" // A note on backslashes in Makefiles, from reading the docs: // Backslash-newline is the line continuation character. // Backslash-# escapes a # (otherwise meaningful as a comment start). // Backslash-% escapes a % (otherwise meaningful as a special). // Finally, quoting the GNU manual, "Backslashes that are not in danger // of quoting ‘%’ characters go unmolested." // How do you end a line with a backslash? The netbsd Make docs suggest // reading the result of a shell command echoing a backslash! // // Rather than implement all of above, we do a simpler thing here: // Backslashes escape a set of characters (see "escapes" defined below), // otherwise they are passed through verbatim. // If anyone actually has depfiles that rely on the more complicated // behavior we can adjust this. bool DepfileParser::Parse(string* content, string* err) { // in: current parser input point. // end: end of input. // parsing_targets: whether we are parsing targets or dependencies. char* in = &(*content)[0]; char* end = in + content->size(); bool parsing_targets = true; while (in < end) { // out: current output point (typically same as in, but can fall behind // as we de-escape backslashes). char* out = in; // filename: start of the current parsed filename. char* filename = out; for (;;) { // start: beginning of the current parsed span. const char* start = in; /*!re2c re2c:define:YYCTYPE = "char"; re2c:define:YYCURSOR = in; re2c:define:YYLIMIT = end; re2c:yyfill:enable = 0; re2c:indent:top = 2; re2c:indent:string = " "; nul = "\000"; escape = [ \\#*[|]; '\\' escape { // De-escape backslashed character. *out++ = yych; continue; } '$$' { // De-escape dollar character. *out++ = '$'; continue; } '\\' [^\000\n] { // Let backslash before other characters through verbatim. *out++ = '\\'; *out++ = yych; continue; } [a-zA-Z0-9+,/_:.~()@=-!]+ { // Got a span of plain text. int len = (int)(in - start); // Need to shift it over if we're overwriting backslashes. if (out < start) memmove(out, start, len); out += len; continue; } nul { break; } [^] { // For any other character (e.g. whitespace), swallow it here, // allowing the outer logic to loop around again. break; } */ } int len = (int)(out - filename); const bool is_target = parsing_targets; if (len > 0 && filename[len - 1] == ':') { len--; // Strip off trailing colon, if any. parsing_targets = false; } if (len == 0) continue; if (!is_target) { ins_.push_back(StringPiece(filename, len)); } else if (!out_.str_) { out_ = StringPiece(filename, len); } else if (out_ != StringPiece(filename, len)) { *err = "depfile has multiple output paths."; return false; } } return true; } ninja-1.3.4/src/depfile_parser_test.cc000066400000000000000000000076311215343310300177410ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "depfile_parser.h" #include struct DepfileParserTest : public testing::Test { bool Parse(const char* input, string* err); DepfileParser parser_; string input_; }; bool DepfileParserTest::Parse(const char* input, string* err) { input_ = input; return parser_.Parse(&input_, err); } TEST_F(DepfileParserTest, Basic) { string err; EXPECT_TRUE(Parse( "build/ninja.o: ninja.cc ninja.h eval_env.h manifest_parser.h\n", &err)); ASSERT_EQ("", err); EXPECT_EQ("build/ninja.o", parser_.out_.AsString()); EXPECT_EQ(4u, parser_.ins_.size()); } TEST_F(DepfileParserTest, EarlyNewlineAndWhitespace) { string err; EXPECT_TRUE(Parse( " \\\n" " out: in\n", &err)); ASSERT_EQ("", err); } TEST_F(DepfileParserTest, Continuation) { string err; EXPECT_TRUE(Parse( "foo.o: \\\n" " bar.h baz.h\n", &err)); ASSERT_EQ("", err); EXPECT_EQ("foo.o", parser_.out_.AsString()); EXPECT_EQ(2u, parser_.ins_.size()); } TEST_F(DepfileParserTest, BackSlashes) { string err; EXPECT_TRUE(Parse( "Project\\Dir\\Build\\Release8\\Foo\\Foo.res : \\\n" " Dir\\Library\\Foo.rc \\\n" " Dir\\Library\\Version\\Bar.h \\\n" " Dir\\Library\\Foo.ico \\\n" " Project\\Thing\\Bar.tlb \\\n", &err)); ASSERT_EQ("", err); EXPECT_EQ("Project\\Dir\\Build\\Release8\\Foo\\Foo.res", parser_.out_.AsString()); EXPECT_EQ(4u, parser_.ins_.size()); } TEST_F(DepfileParserTest, Spaces) { string err; EXPECT_TRUE(Parse( "a\\ bc\\ def: a\\ b c d", &err)); ASSERT_EQ("", err); EXPECT_EQ("a bc def", parser_.out_.AsString()); ASSERT_EQ(3u, parser_.ins_.size()); EXPECT_EQ("a b", parser_.ins_[0].AsString()); EXPECT_EQ("c", parser_.ins_[1].AsString()); EXPECT_EQ("d", parser_.ins_[2].AsString()); } TEST_F(DepfileParserTest, Escapes) { // Put backslashes before a variety of characters, see which ones make // it through. string err; EXPECT_TRUE(Parse( "\\!\\@\\#$$\\%\\^\\&\\\\", &err)); ASSERT_EQ("", err); EXPECT_EQ("\\!\\@#$\\%\\^\\&\\", parser_.out_.AsString()); ASSERT_EQ(0u, parser_.ins_.size()); } TEST_F(DepfileParserTest, SpecialChars) { // See filenames like istreambuf.iterator_op!= in // https://github.com/google/libcxx/tree/master/test/iterators/stream.iterators/istreambuf.iterator/ string err; EXPECT_TRUE(Parse( "C:/Program\\ Files\\ (x86)/Microsoft\\ crtdefs.h: \n" " en@quot.header~ t+t-x!=1", &err)); ASSERT_EQ("", err); EXPECT_EQ("C:/Program Files (x86)/Microsoft crtdefs.h", parser_.out_.AsString()); ASSERT_EQ(2u, parser_.ins_.size()); EXPECT_EQ("en@quot.header~", parser_.ins_[0].AsString()); EXPECT_EQ("t+t-x!=1", parser_.ins_[1].AsString()); } TEST_F(DepfileParserTest, UnifyMultipleOutputs) { // check that multiple duplicate targets are properly unified string err; EXPECT_TRUE(Parse("foo foo: x y z", &err)); ASSERT_EQ(parser_.out_.AsString(), "foo"); ASSERT_EQ(parser_.ins_.size(), 3u); EXPECT_EQ("x", parser_.ins_[0].AsString()); EXPECT_EQ("y", parser_.ins_[1].AsString()); EXPECT_EQ("z", parser_.ins_[2].AsString()); } TEST_F(DepfileParserTest, RejectMultipleDifferentOutputs) { // check that multiple different outputs are rejected by the parser string err; EXPECT_FALSE(Parse("foo bar: x y z", &err)); } ninja-1.3.4/src/deps_log.cc000066400000000000000000000206631215343310300155120ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "deps_log.h" #include #include #include #include #ifndef _WIN32 #include #endif #include "graph.h" #include "metrics.h" #include "state.h" #include "util.h" // The version is stored as 4 bytes after the signature and also serves as a // byte order mark. Signature and version combined are 16 bytes long. const char kFileSignature[] = "# ninjadeps\n"; const int kCurrentVersion = 1; DepsLog::~DepsLog() { Close(); } bool DepsLog::OpenForWrite(const string& path, string* err) { if (needs_recompaction_) { Close(); if (!Recompact(path, err)) return false; } file_ = fopen(path.c_str(), "ab"); if (!file_) { *err = strerror(errno); return false; } SetCloseOnExec(fileno(file_)); // Opening a file in append mode doesn't set the file pointer to the file's // end on Windows. Do that explicitly. fseek(file_, 0, SEEK_END); if (ftell(file_) == 0) { if (fwrite(kFileSignature, sizeof(kFileSignature) - 1, 1, file_) < 1) { *err = strerror(errno); return false; } if (fwrite(&kCurrentVersion, 4, 1, file_) < 1) { *err = strerror(errno); return false; } } return true; } bool DepsLog::RecordDeps(Node* node, TimeStamp mtime, const vector& nodes) { return RecordDeps(node, mtime, nodes.size(), nodes.empty() ? NULL : (Node**)&nodes.front()); } bool DepsLog::RecordDeps(Node* node, TimeStamp mtime, int node_count, Node** nodes) { // Track whether there's any new data to be recorded. bool made_change = false; // Assign ids to all nodes that are missing one. if (node->id() < 0) { RecordId(node); made_change = true; } for (int i = 0; i < node_count; ++i) { if (nodes[i]->id() < 0) { RecordId(nodes[i]); made_change = true; } } // See if the new data is different than the existing data, if any. if (!made_change) { Deps* deps = GetDeps(node); if (!deps || deps->mtime != mtime || deps->node_count != node_count) { made_change = true; } else { for (int i = 0; i < node_count; ++i) { if (deps->nodes[i] != nodes[i]) { made_change = true; break; } } } } // Don't write anything if there's no new info. if (!made_change) return true; // Update on-disk representation. uint16_t size = 4 * (1 + 1 + (uint16_t)node_count); size |= 0x8000; // Deps record: set high bit. fwrite(&size, 2, 1, file_); int id = node->id(); fwrite(&id, 4, 1, file_); int timestamp = mtime; fwrite(×tamp, 4, 1, file_); for (int i = 0; i < node_count; ++i) { id = nodes[i]->id(); fwrite(&id, 4, 1, file_); } // Update in-memory representation. Deps* deps = new Deps(mtime, node_count); for (int i = 0; i < node_count; ++i) deps->nodes[i] = nodes[i]; UpdateDeps(node->id(), deps); return true; } void DepsLog::Close() { if (file_) fclose(file_); file_ = NULL; } bool DepsLog::Load(const string& path, State* state, string* err) { METRIC_RECORD(".ninja_deps load"); char buf[32 << 10]; FILE* f = fopen(path.c_str(), "rb"); if (!f) { if (errno == ENOENT) return true; *err = strerror(errno); return false; } bool valid_header = true; int version = 0; if (!fgets(buf, sizeof(buf), f) || fread(&version, 4, 1, f) < 1) valid_header = false; if (!valid_header || strcmp(buf, kFileSignature) != 0 || version != kCurrentVersion) { *err = "bad deps log signature or version; starting over"; fclose(f); unlink(path.c_str()); // Don't report this as a failure. An empty deps log will cause // us to rebuild the outputs anyway. return true; } long offset; bool read_failed = false; int unique_dep_record_count = 0; int total_dep_record_count = 0; for (;;) { offset = ftell(f); uint16_t size; if (fread(&size, 2, 1, f) < 1) { if (!feof(f)) read_failed = true; break; } bool is_deps = (size >> 15) != 0; size = size & 0x7FFF; if (fread(buf, size, 1, f) < 1) { read_failed = true; break; } if (is_deps) { assert(size % 4 == 0); int* deps_data = reinterpret_cast(buf); int out_id = deps_data[0]; int mtime = deps_data[1]; deps_data += 2; int deps_count = (size / 4) - 2; Deps* deps = new Deps(mtime, deps_count); for (int i = 0; i < deps_count; ++i) { assert(deps_data[i] < (int)nodes_.size()); assert(nodes_[deps_data[i]]); deps->nodes[i] = nodes_[deps_data[i]]; } total_dep_record_count++; if (!UpdateDeps(out_id, deps)) ++unique_dep_record_count; } else { StringPiece path(buf, size); Node* node = state->GetNode(path); assert(node->id() < 0); node->set_id(nodes_.size()); nodes_.push_back(node); } } if (read_failed) { // An error occurred while loading; try to recover by truncating the // file to the last fully-read record. if (ferror(f)) { *err = strerror(ferror(f)); } else { *err = "premature end of file"; } fclose(f); if (!Truncate(path.c_str(), offset, err)) return false; // The truncate succeeded; we'll just report the load error as a // warning because the build can proceed. *err += "; recovering"; return true; } fclose(f); // Rebuild the log if there are too many dead records. int kMinCompactionEntryCount = 1000; int kCompactionRatio = 3; if (total_dep_record_count > kMinCompactionEntryCount && total_dep_record_count > unique_dep_record_count * kCompactionRatio) { needs_recompaction_ = true; } return true; } DepsLog::Deps* DepsLog::GetDeps(Node* node) { // Abort if the node has no id (never referenced in the deps) or if // there's no deps recorded for the node. if (node->id() < 0 || node->id() >= (int)deps_.size()) return NULL; return deps_[node->id()]; } bool DepsLog::Recompact(const string& path, string* err) { METRIC_RECORD(".ninja_deps recompact"); printf("Recompacting deps...\n"); string temp_path = path + ".recompact"; // OpenForWrite() opens for append. Make sure it's not appending to a // left-over file from a previous recompaction attempt that crashed somehow. unlink(temp_path.c_str()); DepsLog new_log; if (!new_log.OpenForWrite(temp_path, err)) return false; // Clear all known ids so that new ones can be reassigned. The new indices // will refer to the ordering in new_log, not in the current log. for (vector::iterator i = nodes_.begin(); i != nodes_.end(); ++i) (*i)->set_id(-1); // Write out all deps again. for (int old_id = 0; old_id < (int)deps_.size(); ++old_id) { Deps* deps = deps_[old_id]; if (!deps) continue; // If nodes_[old_id] is a leaf, it has no deps. if (!new_log.RecordDeps(nodes_[old_id], deps->mtime, deps->node_count, deps->nodes)) { new_log.Close(); return false; } } new_log.Close(); // All nodes now have ids that refer to new_log, so steal its data. deps_.swap(new_log.deps_); nodes_.swap(new_log.nodes_); if (unlink(path.c_str()) < 0) { *err = strerror(errno); return false; } if (rename(temp_path.c_str(), path.c_str()) < 0) { *err = strerror(errno); return false; } return true; } bool DepsLog::UpdateDeps(int out_id, Deps* deps) { if (out_id >= (int)deps_.size()) deps_.resize(out_id + 1); bool delete_old = deps_[out_id] != NULL; if (delete_old) delete deps_[out_id]; deps_[out_id] = deps; return delete_old; } bool DepsLog::RecordId(Node* node) { uint16_t size = (uint16_t)node->path().size(); fwrite(&size, 2, 1, file_); fwrite(node->path().data(), node->path().size(), 1, file_); node->set_id(nodes_.size()); nodes_.push_back(node); return true; } ninja-1.3.4/src/deps_log.h000066400000000000000000000101241215343310300153430ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_DEPS_LOG_H_ #define NINJA_DEPS_LOG_H_ #include #include using namespace std; #include #include "timestamp.h" struct Node; struct State; /// As build commands run they can output extra dependency information /// (e.g. header dependencies for C source) dynamically. DepsLog collects /// that information at build time and uses it for subsequent builds. /// /// The on-disk format is based on two primary design constraints: /// - it must be written to as a stream (during the build, which may be /// interrupted); /// - it can be read all at once on startup. (Alternative designs, where /// it contains indexing information, were considered and discarded as /// too complicated to implement; if the file is small than reading it /// fully on startup is acceptable.) /// Here are some stats from the Windows Chrome dependency files, to /// help guide the design space. The total text in the files sums to /// 90mb so some compression is warranted to keep load-time fast. /// There's about 10k files worth of dependencies that reference about /// 40k total paths totalling 2mb of unique strings. /// /// Based on these stats, here's the current design. /// The file is structured as version header followed by a sequence of records. /// Each record is either a path string or a dependency list. /// Numbering the path strings in file order gives them dense integer ids. /// A dependency list maps an output id to a list of input ids. /// /// Concretely, a record is: /// two bytes record length, high bit indicates record type /// (implies max record length 32k) /// path records contain just the string name of the path /// dependency records are an array of 4-byte integers /// [output path id, output path mtime, input path id, input path id...] /// (The mtime is compared against the on-disk output path mtime /// to verify the stored data is up-to-date.) /// If two records reference the same output the latter one in the file /// wins, allowing updates to just be appended to the file. A separate /// repacking step can run occasionally to remove dead records. struct DepsLog { DepsLog() : needs_recompaction_(false), file_(NULL) {} ~DepsLog(); // Writing (build-time) interface. bool OpenForWrite(const string& path, string* err); bool RecordDeps(Node* node, TimeStamp mtime, const vector& nodes); bool RecordDeps(Node* node, TimeStamp mtime, int node_count, Node** nodes); void Close(); // Reading (startup-time) interface. struct Deps { Deps(int mtime, int node_count) : mtime(mtime), node_count(node_count), nodes(new Node*[node_count]) {} ~Deps() { delete [] nodes; } int mtime; int node_count; Node** nodes; }; bool Load(const string& path, State* state, string* err); Deps* GetDeps(Node* node); /// Rewrite the known log entries, throwing away old data. bool Recompact(const string& path, string* err); /// Used for tests. const vector& nodes() const { return nodes_; } const vector& deps() const { return deps_; } private: // Updates the in-memory representation. Takes ownership of |deps|. // Returns true if a prior deps record was deleted. bool UpdateDeps(int out_id, Deps* deps); // Write a node name record, assigning it an id. bool RecordId(Node* node); bool needs_recompaction_; FILE* file_; /// Maps id -> Node. vector nodes_; /// Maps id -> deps of that id. vector deps_; friend struct DepsLogTest; }; #endif // NINJA_DEPS_LOG_H_ ninja-1.3.4/src/deps_log_test.cc000066400000000000000000000252521215343310300165500ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "deps_log.h" #include "graph.h" #include "util.h" #include "test.h" namespace { const char kTestFilename[] = "DepsLogTest-tempfile"; struct DepsLogTest : public testing::Test { virtual void SetUp() { // In case a crashing test left a stale file behind. unlink(kTestFilename); } virtual void TearDown() { unlink(kTestFilename); } }; TEST_F(DepsLogTest, WriteRead) { State state1; DepsLog log1; string err; EXPECT_TRUE(log1.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); { vector deps; deps.push_back(state1.GetNode("foo.h")); deps.push_back(state1.GetNode("bar.h")); log1.RecordDeps(state1.GetNode("out.o"), 1, deps); deps.clear(); deps.push_back(state1.GetNode("foo.h")); deps.push_back(state1.GetNode("bar2.h")); log1.RecordDeps(state1.GetNode("out2.o"), 2, deps); DepsLog::Deps* log_deps = log1.GetDeps(state1.GetNode("out.o")); ASSERT_TRUE(log_deps); ASSERT_EQ(1, log_deps->mtime); ASSERT_EQ(2, log_deps->node_count); ASSERT_EQ("foo.h", log_deps->nodes[0]->path()); ASSERT_EQ("bar.h", log_deps->nodes[1]->path()); } log1.Close(); State state2; DepsLog log2; EXPECT_TRUE(log2.Load(kTestFilename, &state2, &err)); ASSERT_EQ("", err); ASSERT_EQ(log1.nodes().size(), log2.nodes().size()); for (int i = 0; i < (int)log1.nodes().size(); ++i) { Node* node1 = log1.nodes()[i]; Node* node2 = log2.nodes()[i]; ASSERT_EQ(i, node1->id()); ASSERT_EQ(node1->id(), node2->id()); } // Spot-check the entries in log2. DepsLog::Deps* log_deps = log2.GetDeps(state2.GetNode("out2.o")); ASSERT_TRUE(log_deps); ASSERT_EQ(2, log_deps->mtime); ASSERT_EQ(2, log_deps->node_count); ASSERT_EQ("foo.h", log_deps->nodes[0]->path()); ASSERT_EQ("bar2.h", log_deps->nodes[1]->path()); } // Verify that adding the same deps twice doesn't grow the file. TEST_F(DepsLogTest, DoubleEntry) { // Write some deps to the file and grab its size. int file_size; { State state; DepsLog log; string err; EXPECT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); vector deps; deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar.h")); log.RecordDeps(state.GetNode("out.o"), 1, deps); log.Close(); struct stat st; ASSERT_EQ(0, stat(kTestFilename, &st)); file_size = (int)st.st_size; ASSERT_GT(file_size, 0); } // Now reload the file, and readd the same deps. { State state; DepsLog log; string err; EXPECT_TRUE(log.Load(kTestFilename, &state, &err)); EXPECT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); vector deps; deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar.h")); log.RecordDeps(state.GetNode("out.o"), 1, deps); log.Close(); struct stat st; ASSERT_EQ(0, stat(kTestFilename, &st)); int file_size_2 = (int)st.st_size; ASSERT_EQ(file_size, file_size_2); } } // Verify that adding the new deps works and can be compacted away. TEST_F(DepsLogTest, Recompact) { // Write some deps to the file and grab its size. int file_size; { State state; DepsLog log; string err; ASSERT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); vector deps; deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar.h")); log.RecordDeps(state.GetNode("out.o"), 1, deps); deps.clear(); deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("baz.h")); log.RecordDeps(state.GetNode("other_out.o"), 1, deps); log.Close(); struct stat st; ASSERT_EQ(0, stat(kTestFilename, &st)); file_size = (int)st.st_size; ASSERT_GT(file_size, 0); } // Now reload the file, and add slighly different deps. int file_size_2; { State state; DepsLog log; string err; ASSERT_TRUE(log.Load(kTestFilename, &state, &err)); ASSERT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); vector deps; deps.push_back(state.GetNode("foo.h")); log.RecordDeps(state.GetNode("out.o"), 1, deps); log.Close(); struct stat st; ASSERT_EQ(0, stat(kTestFilename, &st)); file_size_2 = (int)st.st_size; // The file should grow to record the new deps. ASSERT_GT(file_size_2, file_size); } // Now reload the file, verify the new deps have replaced the old, then // recompact. { State state; DepsLog log; string err; ASSERT_TRUE(log.Load(kTestFilename, &state, &err)); Node* out = state.GetNode("out.o"); DepsLog::Deps* deps = log.GetDeps(out); ASSERT_TRUE(deps); ASSERT_EQ(1, deps->mtime); ASSERT_EQ(1, deps->node_count); ASSERT_EQ("foo.h", deps->nodes[0]->path()); Node* other_out = state.GetNode("other_out.o"); deps = log.GetDeps(other_out); ASSERT_TRUE(deps); ASSERT_EQ(1, deps->mtime); ASSERT_EQ(2, deps->node_count); ASSERT_EQ("foo.h", deps->nodes[0]->path()); ASSERT_EQ("baz.h", deps->nodes[1]->path()); ASSERT_TRUE(log.Recompact(kTestFilename, &err)); // The in-memory deps graph should still be valid after recompaction. deps = log.GetDeps(out); ASSERT_TRUE(deps); ASSERT_EQ(1, deps->mtime); ASSERT_EQ(1, deps->node_count); ASSERT_EQ("foo.h", deps->nodes[0]->path()); ASSERT_EQ(out, log.nodes()[out->id()]); deps = log.GetDeps(other_out); ASSERT_TRUE(deps); ASSERT_EQ(1, deps->mtime); ASSERT_EQ(2, deps->node_count); ASSERT_EQ("foo.h", deps->nodes[0]->path()); ASSERT_EQ("baz.h", deps->nodes[1]->path()); ASSERT_EQ(other_out, log.nodes()[other_out->id()]); // The file should have shrunk a bit for the smaller deps. struct stat st; ASSERT_EQ(0, stat(kTestFilename, &st)); int file_size_3 = (int)st.st_size; ASSERT_LT(file_size_3, file_size_2); } } // Verify that invalid file headers cause a new build. TEST_F(DepsLogTest, InvalidHeader) { const char *kInvalidHeaders[] = { "", // Empty file. "# ninjad", // Truncated first line. "# ninjadeps\n", // No version int. "# ninjadeps\n\001\002", // Truncated version int. "# ninjadeps\n\001\002\003\004" // Invalid version int. }; for (size_t i = 0; i < sizeof(kInvalidHeaders) / sizeof(kInvalidHeaders[0]); ++i) { FILE* deps_log = fopen(kTestFilename, "wb"); ASSERT_TRUE(deps_log != NULL); ASSERT_EQ( strlen(kInvalidHeaders[i]), fwrite(kInvalidHeaders[i], 1, strlen(kInvalidHeaders[i]), deps_log)); ASSERT_EQ(0 ,fclose(deps_log)); string err; DepsLog log; State state; ASSERT_TRUE(log.Load(kTestFilename, &state, &err)); EXPECT_EQ("bad deps log signature or version; starting over", err); } } // Simulate what happens when loading a truncated log file. TEST_F(DepsLogTest, Truncated) { // Create a file with some entries. { State state; DepsLog log; string err; EXPECT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); vector deps; deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar.h")); log.RecordDeps(state.GetNode("out.o"), 1, deps); deps.clear(); deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar2.h")); log.RecordDeps(state.GetNode("out2.o"), 2, deps); log.Close(); } // Get the file size. struct stat st; ASSERT_EQ(0, stat(kTestFilename, &st)); // Try reloading at truncated sizes. // Track how many nodes/deps were found; they should decrease with // smaller sizes. int node_count = 5; int deps_count = 2; for (int size = (int)st.st_size; size > 0; --size) { string err; ASSERT_TRUE(Truncate(kTestFilename, size, &err)); State state; DepsLog log; EXPECT_TRUE(log.Load(kTestFilename, &state, &err)); if (!err.empty()) { // At some point the log will be so short as to be unparseable. break; } ASSERT_GE(node_count, (int)log.nodes().size()); node_count = log.nodes().size(); // Count how many non-NULL deps entries there are. int new_deps_count = 0; for (vector::const_iterator i = log.deps().begin(); i != log.deps().end(); ++i) { if (*i) ++new_deps_count; } ASSERT_GE(deps_count, new_deps_count); deps_count = new_deps_count; } } // Run the truncation-recovery logic. TEST_F(DepsLogTest, TruncatedRecovery) { // Create a file with some entries. { State state; DepsLog log; string err; EXPECT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); vector deps; deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar.h")); log.RecordDeps(state.GetNode("out.o"), 1, deps); deps.clear(); deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar2.h")); log.RecordDeps(state.GetNode("out2.o"), 2, deps); log.Close(); } // Shorten the file, corrupting the last record. struct stat st; ASSERT_EQ(0, stat(kTestFilename, &st)); string err; ASSERT_TRUE(Truncate(kTestFilename, st.st_size - 2, &err)); // Load the file again, add an entry. { State state; DepsLog log; string err; EXPECT_TRUE(log.Load(kTestFilename, &state, &err)); ASSERT_EQ("premature end of file; recovering", err); err.clear(); // The truncated entry should've been discarded. EXPECT_EQ(NULL, log.GetDeps(state.GetNode("out2.o"))); EXPECT_TRUE(log.OpenForWrite(kTestFilename, &err)); ASSERT_EQ("", err); // Add a new entry. vector deps; deps.push_back(state.GetNode("foo.h")); deps.push_back(state.GetNode("bar2.h")); log.RecordDeps(state.GetNode("out2.o"), 3, deps); log.Close(); } // Load the file a third time to verify appending after a mangled // entry doesn't break things. { State state; DepsLog log; string err; EXPECT_TRUE(log.Load(kTestFilename, &state, &err)); // The truncated entry should exist. DepsLog::Deps* deps = log.GetDeps(state.GetNode("out2.o")); ASSERT_TRUE(deps); } } } // anonymous namespace ninja-1.3.4/src/disk_interface.cc000066400000000000000000000114031215343310300166600ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "disk_interface.h" #include #include #include #include #include #ifdef _WIN32 #include #include // _mkdir #endif #include "util.h" namespace { string DirName(const string& path) { #ifdef _WIN32 const char kPathSeparator = '\\'; #else const char kPathSeparator = '/'; #endif string::size_type slash_pos = path.rfind(kPathSeparator); if (slash_pos == string::npos) return string(); // Nothing to do. while (slash_pos > 0 && path[slash_pos - 1] == kPathSeparator) --slash_pos; return path.substr(0, slash_pos); } int MakeDir(const string& path) { #ifdef _WIN32 return _mkdir(path.c_str()); #else return mkdir(path.c_str(), 0777); #endif } } // namespace // DiskInterface --------------------------------------------------------------- bool DiskInterface::MakeDirs(const string& path) { string dir = DirName(path); if (dir.empty()) return true; // Reached root; assume it's there. TimeStamp mtime = Stat(dir); if (mtime < 0) return false; // Error. if (mtime > 0) return true; // Exists already; we're done. // Directory doesn't exist. Try creating its parent first. bool success = MakeDirs(dir); if (!success) return false; return MakeDir(dir); } // RealDiskInterface ----------------------------------------------------------- TimeStamp RealDiskInterface::Stat(const string& path) { #ifdef _WIN32 // MSDN: "Naming Files, Paths, and Namespaces" // http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx if (!path.empty() && path[0] != '\\' && path.size() > MAX_PATH) { if (!quiet_) { Error("Stat(%s): Filename longer than %i characters", path.c_str(), MAX_PATH); } return -1; } WIN32_FILE_ATTRIBUTE_DATA attrs; if (!GetFileAttributesEx(path.c_str(), GetFileExInfoStandard, &attrs)) { DWORD err = GetLastError(); if (err == ERROR_FILE_NOT_FOUND || err == ERROR_PATH_NOT_FOUND) return 0; if (!quiet_) { Error("GetFileAttributesEx(%s): %s", path.c_str(), GetLastErrorString().c_str()); } return -1; } const FILETIME& filetime = attrs.ftLastWriteTime; // FILETIME is in 100-nanosecond increments since the Windows epoch. // We don't much care about epoch correctness but we do want the // resulting value to fit in an integer. uint64_t mtime = ((uint64_t)filetime.dwHighDateTime << 32) | ((uint64_t)filetime.dwLowDateTime); mtime /= 1000000000LL / 100; // 100ns -> s. mtime -= 12622770400LL; // 1600 epoch -> 2000 epoch (subtract 400 years). return (TimeStamp)mtime; #else struct stat st; if (stat(path.c_str(), &st) < 0) { if (errno == ENOENT || errno == ENOTDIR) return 0; if (!quiet_) { Error("stat(%s): %s", path.c_str(), strerror(errno)); } return -1; } return st.st_mtime; #endif } bool RealDiskInterface::WriteFile(const string& path, const string& contents) { FILE * fp = fopen(path.c_str(), "w"); if (fp == NULL) { Error("WriteFile(%s): Unable to create file. %s", path.c_str(), strerror(errno)); return false; } if (fwrite(contents.data(), 1, contents.length(), fp) < contents.length()) { Error("WriteFile(%s): Unable to write to the file. %s", path.c_str(), strerror(errno)); fclose(fp); return false; } if (fclose(fp) == EOF) { Error("WriteFile(%s): Unable to close the file. %s", path.c_str(), strerror(errno)); return false; } return true; } bool RealDiskInterface::MakeDir(const string& path) { if (::MakeDir(path) < 0) { Error("mkdir(%s): %s", path.c_str(), strerror(errno)); return false; } return true; } string RealDiskInterface::ReadFile(const string& path, string* err) { string contents; int ret = ::ReadFile(path, &contents, err); if (ret == -ENOENT) { // Swallow ENOENT. err->clear(); } return contents; } int RealDiskInterface::RemoveFile(const string& path) { if (remove(path.c_str()) < 0) { switch (errno) { case ENOENT: return 1; default: Error("remove(%s): %s", path.c_str(), strerror(errno)); return -1; } } else { return 0; } } ninja-1.3.4/src/disk_interface.h000066400000000000000000000046671215343310300165400ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_DISK_INTERFACE_H_ #define NINJA_DISK_INTERFACE_H_ #include using namespace std; #include "timestamp.h" /// Interface for accessing the disk. /// /// Abstract so it can be mocked out for tests. The real implementation /// is RealDiskInterface. struct DiskInterface { virtual ~DiskInterface() {} /// stat() a file, returning the mtime, or 0 if missing and -1 on /// other errors. virtual TimeStamp Stat(const string& path) = 0; /// Create a directory, returning false on failure. virtual bool MakeDir(const string& path) = 0; /// Create a file, with the specified name and contents /// Returns true on success, false on failure virtual bool WriteFile(const string& path, const string& contents) = 0; /// Read a file to a string. Fill in |err| on error. virtual string ReadFile(const string& path, string* err) = 0; /// Remove the file named @a path. It behaves like 'rm -f path' so no errors /// are reported if it does not exists. /// @returns 0 if the file has been removed, /// 1 if the file does not exist, and /// -1 if an error occurs. virtual int RemoveFile(const string& path) = 0; /// Create all the parent directories for path; like mkdir -p /// `basename path`. bool MakeDirs(const string& path); }; /// Implementation of DiskInterface that actually hits the disk. struct RealDiskInterface : public DiskInterface { RealDiskInterface() : quiet_(false) {} virtual ~RealDiskInterface() {} virtual TimeStamp Stat(const string& path); virtual bool MakeDir(const string& path); virtual bool WriteFile(const string& path, const string& contents); virtual string ReadFile(const string& path, string* err); virtual int RemoveFile(const string& path); /// Whether to print on errors. Used to make a test quieter. bool quiet_; }; #endif // NINJA_DISK_INTERFACE_H_ ninja-1.3.4/src/disk_interface_test.cc000066400000000000000000000124141215343310300177220ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #ifdef _WIN32 #include #include #endif #include "disk_interface.h" #include "graph.h" #include "test.h" namespace { struct DiskInterfaceTest : public testing::Test { virtual void SetUp() { // These tests do real disk accesses, so create a temp dir. temp_dir_.CreateAndEnter("Ninja-DiskInterfaceTest"); } virtual void TearDown() { temp_dir_.Cleanup(); } bool Touch(const char* path) { FILE *f = fopen(path, "w"); if (!f) return false; return fclose(f) == 0; } ScopedTempDir temp_dir_; RealDiskInterface disk_; }; TEST_F(DiskInterfaceTest, StatMissingFile) { EXPECT_EQ(0, disk_.Stat("nosuchfile")); // On Windows, the errno for a file in a nonexistent directory // is different. EXPECT_EQ(0, disk_.Stat("nosuchdir/nosuchfile")); // On POSIX systems, the errno is different if a component of the // path prefix is not a directory. ASSERT_TRUE(Touch("notadir")); EXPECT_EQ(0, disk_.Stat("notadir/nosuchfile")); } TEST_F(DiskInterfaceTest, StatBadPath) { disk_.quiet_ = true; #ifdef _WIN32 string bad_path("cc:\\foo"); EXPECT_EQ(-1, disk_.Stat(bad_path)); #else string too_long_name(512, 'x'); EXPECT_EQ(-1, disk_.Stat(too_long_name)); #endif disk_.quiet_ = false; } TEST_F(DiskInterfaceTest, StatExistingFile) { ASSERT_TRUE(Touch("file")); EXPECT_GT(disk_.Stat("file"), 1); } TEST_F(DiskInterfaceTest, ReadFile) { string err; EXPECT_EQ("", disk_.ReadFile("foobar", &err)); EXPECT_EQ("", err); const char* kTestFile = "testfile"; FILE* f = fopen(kTestFile, "wb"); ASSERT_TRUE(f); const char* kTestContent = "test content\nok"; fprintf(f, "%s", kTestContent); ASSERT_EQ(0, fclose(f)); EXPECT_EQ(kTestContent, disk_.ReadFile(kTestFile, &err)); EXPECT_EQ("", err); } TEST_F(DiskInterfaceTest, MakeDirs) { EXPECT_TRUE(disk_.MakeDirs("path/with/double//slash/")); } TEST_F(DiskInterfaceTest, RemoveFile) { const char* kFileName = "file-to-remove"; ASSERT_TRUE(Touch(kFileName)); EXPECT_EQ(0, disk_.RemoveFile(kFileName)); EXPECT_EQ(1, disk_.RemoveFile(kFileName)); EXPECT_EQ(1, disk_.RemoveFile("does not exist")); } struct StatTest : public StateTestWithBuiltinRules, public DiskInterface { StatTest() : scan_(&state_, NULL, NULL, this) {} // DiskInterface implementation. virtual TimeStamp Stat(const string& path); virtual bool WriteFile(const string& path, const string& contents) { assert(false); return true; } virtual bool MakeDir(const string& path) { assert(false); return false; } virtual string ReadFile(const string& path, string* err) { assert(false); return ""; } virtual int RemoveFile(const string& path) { assert(false); return 0; } DependencyScan scan_; map mtimes_; vector stats_; }; TimeStamp StatTest::Stat(const string& path) { stats_.push_back(path); map::iterator i = mtimes_.find(path); if (i == mtimes_.end()) return 0; // File not found. return i->second; } TEST_F(StatTest, Simple) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat in\n")); Node* out = GetNode("out"); out->Stat(this); ASSERT_EQ(1u, stats_.size()); scan_.RecomputeDirty(out->in_edge(), NULL); ASSERT_EQ(2u, stats_.size()); ASSERT_EQ("out", stats_[0]); ASSERT_EQ("in", stats_[1]); } TEST_F(StatTest, TwoStep) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat mid\n" "build mid: cat in\n")); Node* out = GetNode("out"); out->Stat(this); ASSERT_EQ(1u, stats_.size()); scan_.RecomputeDirty(out->in_edge(), NULL); ASSERT_EQ(3u, stats_.size()); ASSERT_EQ("out", stats_[0]); ASSERT_TRUE(GetNode("out")->dirty()); ASSERT_EQ("mid", stats_[1]); ASSERT_TRUE(GetNode("mid")->dirty()); ASSERT_EQ("in", stats_[2]); } TEST_F(StatTest, Tree) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat mid1 mid2\n" "build mid1: cat in11 in12\n" "build mid2: cat in21 in22\n")); Node* out = GetNode("out"); out->Stat(this); ASSERT_EQ(1u, stats_.size()); scan_.RecomputeDirty(out->in_edge(), NULL); ASSERT_EQ(1u + 6u, stats_.size()); ASSERT_EQ("mid1", stats_[1]); ASSERT_TRUE(GetNode("mid1")->dirty()); ASSERT_EQ("in11", stats_[2]); } TEST_F(StatTest, Middle) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat mid\n" "build mid: cat in\n")); mtimes_["in"] = 1; mtimes_["mid"] = 0; // missing mtimes_["out"] = 1; Node* out = GetNode("out"); out->Stat(this); ASSERT_EQ(1u, stats_.size()); scan_.RecomputeDirty(out->in_edge(), NULL); ASSERT_FALSE(GetNode("in")->dirty()); ASSERT_TRUE(GetNode("mid")->dirty()); ASSERT_TRUE(GetNode("out")->dirty()); } } // namespace ninja-1.3.4/src/edit_distance.cc000066400000000000000000000040521215343310300165070ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "edit_distance.h" #include int EditDistance(const StringPiece& s1, const StringPiece& s2, bool allow_replacements, int max_edit_distance) { // The algorithm implemented below is the "classic" // dynamic-programming algorithm for computing the Levenshtein // distance, which is described here: // // http://en.wikipedia.org/wiki/Levenshtein_distance // // Although the algorithm is typically described using an m x n // array, only two rows are used at a time, so this implemenation // just keeps two separate vectors for those two rows. int m = s1.len_; int n = s2.len_; vector previous(n + 1); vector current(n + 1); for (int i = 0; i <= n; ++i) previous[i] = i; for (int y = 1; y <= m; ++y) { current[0] = y; int best_this_row = current[0]; for (int x = 1; x <= n; ++x) { if (allow_replacements) { current[x] = min(previous[x-1] + (s1.str_[y-1] == s2.str_[x-1] ? 0 : 1), min(current[x-1], previous[x])+1); } else { if (s1.str_[y-1] == s2.str_[x-1]) current[x] = previous[x-1]; else current[x] = min(current[x-1], previous[x]) + 1; } best_this_row = min(best_this_row, current[x]); } if (max_edit_distance && best_this_row > max_edit_distance) return max_edit_distance + 1; current.swap(previous); } return previous[n]; } ninja-1.3.4/src/edit_distance.h000066400000000000000000000016151215343310300163530ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_EDIT_DISTANCE_H_ #define NINJA_EDIT_DISTANCE_H_ #include "string_piece.h" int EditDistance(const StringPiece& s1, const StringPiece& s2, bool allow_replacements = true, int max_edit_distance = 0); #endif // NINJA_EDIT_DISTANCE_H_ ninja-1.3.4/src/edit_distance_test.cc000066400000000000000000000033311215343310300175450ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "edit_distance.h" #include "test.h" TEST(EditDistanceTest, TestEmpty) { EXPECT_EQ(5, EditDistance("", "ninja")); EXPECT_EQ(5, EditDistance("ninja", "")); EXPECT_EQ(0, EditDistance("", "")); } TEST(EditDistanceTest, TestMaxDistance) { const bool allow_replacements = true; for (int max_distance = 1; max_distance < 7; ++max_distance) { EXPECT_EQ(max_distance + 1, EditDistance("abcdefghijklmnop", "ponmlkjihgfedcba", allow_replacements, max_distance)); } } TEST(EditDistanceTest, TestAllowReplacements) { bool allow_replacements = true; EXPECT_EQ(1, EditDistance("ninja", "njnja", allow_replacements)); EXPECT_EQ(1, EditDistance("njnja", "ninja", allow_replacements)); allow_replacements = false; EXPECT_EQ(2, EditDistance("ninja", "njnja", allow_replacements)); EXPECT_EQ(2, EditDistance("njnja", "ninja", allow_replacements)); } TEST(EditDistanceTest, TestBasics) { EXPECT_EQ(0, EditDistance("browser_tests", "browser_tests")); EXPECT_EQ(1, EditDistance("browser_test", "browser_tests")); EXPECT_EQ(1, EditDistance("browser_tests", "browser_test")); } ninja-1.3.4/src/eval_env.cc000066400000000000000000000044501215343310300155110ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "eval_env.h" string BindingEnv::LookupVariable(const string& var) { map::iterator i = bindings_.find(var); if (i != bindings_.end()) return i->second; if (parent_) return parent_->LookupVariable(var); return ""; } void BindingEnv::AddBinding(const string& key, const string& val) { bindings_[key] = val; } string BindingEnv::LookupWithFallback(const string& var, const EvalString* eval, Env* env) { map::iterator i = bindings_.find(var); if (i != bindings_.end()) return i->second; if (eval) return eval->Evaluate(env); if (parent_) return parent_->LookupVariable(var); return ""; } string EvalString::Evaluate(Env* env) const { string result; for (TokenList::const_iterator i = parsed_.begin(); i != parsed_.end(); ++i) { if (i->second == RAW) result.append(i->first); else result.append(env->LookupVariable(i->first)); } return result; } void EvalString::AddText(StringPiece text) { // Add it to the end of an existing RAW token if possible. if (!parsed_.empty() && parsed_.back().second == RAW) { parsed_.back().first.append(text.str_, text.len_); } else { parsed_.push_back(make_pair(text.AsString(), RAW)); } } void EvalString::AddSpecial(StringPiece text) { parsed_.push_back(make_pair(text.AsString(), SPECIAL)); } string EvalString::Serialize() const { string result; for (TokenList::const_iterator i = parsed_.begin(); i != parsed_.end(); ++i) { result.append("["); if (i->second == SPECIAL) result.append("$"); result.append(i->first); result.append("]"); } return result; } ninja-1.3.4/src/eval_env.h000066400000000000000000000045201215343310300153510ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_EVAL_ENV_H_ #define NINJA_EVAL_ENV_H_ #include #include #include using namespace std; #include "string_piece.h" struct EvalString; /// An interface for a scope for variable (e.g. "$foo") lookups. struct Env { virtual ~Env() {} virtual string LookupVariable(const string& var) = 0; }; /// An Env which contains a mapping of variables to values /// as well as a pointer to a parent scope. struct BindingEnv : public Env { BindingEnv() : parent_(NULL) {} explicit BindingEnv(Env* parent) : parent_(parent) {} virtual ~BindingEnv() {} virtual string LookupVariable(const string& var); void AddBinding(const string& key, const string& val); /// This is tricky. Edges want lookup scope to go in this order: /// 1) value set on edge itself (edge_->env_) /// 2) value set on rule, with expansion in the edge's scope /// 3) value set on enclosing scope of edge (edge_->env_->parent_) /// This function takes as parameters the necessary info to do (2). string LookupWithFallback(const string& var, const EvalString* eval, Env* env); private: map bindings_; Env* parent_; }; /// A tokenized string that contains variable references. /// Can be evaluated relative to an Env. struct EvalString { string Evaluate(Env* env) const; void Clear() { parsed_.clear(); } bool empty() const { return parsed_.empty(); } void AddText(StringPiece text); void AddSpecial(StringPiece text); /// Construct a human-readable representation of the parsed state, /// for use in tests. string Serialize() const; private: enum TokenType { RAW, SPECIAL }; typedef vector > TokenList; TokenList parsed_; }; #endif // NINJA_EVAL_ENV_H_ ninja-1.3.4/src/exit_status.h000066400000000000000000000014031215343310300161230ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_EXIT_STATUS_H_ #define NINJA_EXIT_STATUS_H_ enum ExitStatus { ExitSuccess, ExitFailure, ExitInterrupted }; #endif // NINJA_EXIT_STATUS_H_ ninja-1.3.4/src/explain.cc000066400000000000000000000011751215343310300153530ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. bool g_explaining = false; ninja-1.3.4/src/explain.h000066400000000000000000000017001215343310300152070ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_EXPLAIN_H_ #define NINJA_EXPLAIN_H_ #include #define EXPLAIN(fmt, ...) { \ if (g_explaining) \ fprintf(stderr, "ninja explain: " fmt "\n", __VA_ARGS__); \ } extern bool g_explaining; #endif // NINJA_EXPLAIN_H_ ninja-1.3.4/src/gen_doxygen_mainpage.sh000077500000000000000000000027341215343310300201140ustar00rootroot00000000000000#!/bin/sh # Copyright 2011 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. set -o errexit set -o nounset STATUS=0 # Print each of its arguments on stderr (one per line) prefixed by the # basename of this script. stderr() { local me=$(basename "$0") local i for i do echo >&2 "$me: $i" done } # Print each of its arguments on stderr (one per line) prefixed by the # basename of this script and 'error'. error() { local i for i do stderr "error: $i" done STATUS=1 } generate_header() { cat <&2 "usage: $0 inputs..." exit 1 fi generate_header for i in "$@" do include_file "$i" done generate_footer exit $STATUS ninja-1.3.4/src/getopt.c000066400000000000000000000302401215343310300150450ustar00rootroot00000000000000/**************************************************************************** getopt.c - Read command line options AUTHOR: Gregory Pietsch CREATED Fri Jan 10 21:13:05 1997 DESCRIPTION: The getopt() function parses the command line arguments. Its arguments argc and argv are the argument count and array as passed to the main() function on program invocation. The argument optstring is a list of available option characters. If such a character is followed by a colon (`:'), the option takes an argument, which is placed in optarg. If such a character is followed by two colons, the option takes an optional argument, which is placed in optarg. If the option does not take an argument, optarg is NULL. The external variable optind is the index of the next array element of argv to be processed; it communicates from one call to the next which element to process. The getopt_long() function works like getopt() except that it also accepts long options started by two dashes `--'. If these take values, it is either in the form --arg=value or --arg value It takes the additional arguments longopts which is a pointer to the first element of an array of type GETOPT_LONG_OPTION_T. The last element of the array has to be filled with NULL for the name field. The longind pointer points to the index of the current long option relative to longopts if it is non-NULL. The getopt() function returns the option character if the option was found successfully, `:' if there was a missing parameter for one of the options, `?' for an unknown option character, and EOF for the end of the option list. The getopt_long() function's return value is described in the header file. The function getopt_long_only() is identical to getopt_long(), except that a plus sign `+' can introduce long options as well as `--'. The following describes how to deal with options that follow non-option argv-elements. If the caller did not specify anything, the default is REQUIRE_ORDER if the environment variable POSIXLY_CORRECT is defined, PERMUTE otherwise. REQUIRE_ORDER means don't recognize them as options; stop option processing when the first non-option is seen. This is what Unix does. This mode of operation is selected by either setting the environment variable POSIXLY_CORRECT, or using `+' as the first character of the optstring parameter. PERMUTE is the default. We permute the contents of ARGV as we scan, so that eventually all the non-options are at the end. This allows options to be given in any order, even with programs that were not written to expect this. RETURN_IN_ORDER is an option available to programs that were written to expect options and other argv-elements in any order and that care about the ordering of the two. We describe each non-option argv-element as if it were the argument of an option with character code 1. Using `-' as the first character of the optstring parameter selects this mode of operation. The special argument `--' forces an end of option-scanning regardless of the value of ordering. In the case of RETURN_IN_ORDER, only `--' can cause getopt() and friends to return EOF with optind != argc. COPYRIGHT NOTICE AND DISCLAIMER: Copyright (C) 1997 Gregory Pietsch This file and the accompanying getopt.h header file are hereby placed in the public domain without restrictions. Just give the author credit, don't claim you wrote it or prevent anyone else from using it. Gregory Pietsch's current e-mail address: gpietsch@comcast.net ****************************************************************************/ /* include files */ #include #include #include #ifndef GETOPT_H #include "getopt.h" #endif /* macros */ /* types */ typedef enum GETOPT_ORDERING_T { PERMUTE, RETURN_IN_ORDER, REQUIRE_ORDER } GETOPT_ORDERING_T; /* globally-defined variables */ char *optarg = NULL; int optind = 0; int opterr = 1; int optopt = '?'; /* functions */ /* reverse_argv_elements: reverses num elements starting at argv */ static void reverse_argv_elements (char **argv, int num) { int i; char *tmp; for (i = 0; i < (num >> 1); i++) { tmp = argv[i]; argv[i] = argv[num - i - 1]; argv[num - i - 1] = tmp; } } /* permute: swap two blocks of argv-elements given their lengths */ static void permute (char **argv, int len1, int len2) { reverse_argv_elements (argv, len1); reverse_argv_elements (argv, len1 + len2); reverse_argv_elements (argv, len2); } /* is_option: is this argv-element an option or the end of the option list? */ static int is_option (char *argv_element, int only) { return ((argv_element == NULL) || (argv_element[0] == '-') || (only && argv_element[0] == '+')); } /* getopt_internal: the function that does all the dirty work */ static int getopt_internal (int argc, char **argv, char *shortopts, GETOPT_LONG_OPTION_T * longopts, int *longind, int only) { GETOPT_ORDERING_T ordering = PERMUTE; static size_t optwhere = 0; size_t permute_from = 0; int num_nonopts = 0; int optindex = 0; size_t match_chars = 0; char *possible_arg = NULL; int longopt_match = -1; int has_arg = -1; char *cp = NULL; int arg_next = 0; /* first, deal with silly parameters and easy stuff */ if (argc == 0 || argv == NULL || (shortopts == NULL && longopts == NULL)) return (optopt = '?'); if (optind >= argc || argv[optind] == NULL) return EOF; if (strcmp (argv[optind], "--") == 0) { optind++; return EOF; } /* if this is our first time through */ if (optind == 0) optind = optwhere = 1; /* define ordering */ if (shortopts != NULL && (*shortopts == '-' || *shortopts == '+')) { ordering = (*shortopts == '-') ? RETURN_IN_ORDER : REQUIRE_ORDER; shortopts++; } else ordering = (getenv ("POSIXLY_CORRECT") != NULL) ? REQUIRE_ORDER : PERMUTE; /* * based on ordering, find our next option, if we're at the beginning of * one */ if (optwhere == 1) { switch (ordering) { case PERMUTE: permute_from = optind; num_nonopts = 0; while (!is_option (argv[optind], only)) { optind++; num_nonopts++; } if (argv[optind] == NULL) { /* no more options */ optind = permute_from; return EOF; } else if (strcmp (argv[optind], "--") == 0) { /* no more options, but have to get `--' out of the way */ permute (argv + permute_from, num_nonopts, 1); optind = permute_from + 1; return EOF; } break; case RETURN_IN_ORDER: if (!is_option (argv[optind], only)) { optarg = argv[optind++]; return (optopt = 1); } break; case REQUIRE_ORDER: if (!is_option (argv[optind], only)) return EOF; break; } } /* we've got an option, so parse it */ /* first, is it a long option? */ if (longopts != NULL && (memcmp (argv[optind], "--", 2) == 0 || (only && argv[optind][0] == '+')) && optwhere == 1) { /* handle long options */ if (memcmp (argv[optind], "--", 2) == 0) optwhere = 2; longopt_match = -1; possible_arg = strchr (argv[optind] + optwhere, '='); if (possible_arg == NULL) { /* no =, so next argv might be arg */ match_chars = strlen (argv[optind]); possible_arg = argv[optind] + match_chars; match_chars = match_chars - optwhere; } else match_chars = (possible_arg - argv[optind]) - optwhere; for (optindex = 0; longopts[optindex].name != NULL; optindex++) { if (memcmp (argv[optind] + optwhere, longopts[optindex].name, match_chars) == 0) { /* do we have an exact match? */ if (match_chars == strlen (longopts[optindex].name)) { longopt_match = optindex; break; } /* do any characters match? */ else { if (longopt_match < 0) longopt_match = optindex; else { /* we have ambiguous options */ if (opterr) fprintf (stderr, "%s: option `%s' is ambiguous " "(could be `--%s' or `--%s')\n", argv[0], argv[optind], longopts[longopt_match].name, longopts[optindex].name); return (optopt = '?'); } } } } if (longopt_match >= 0) has_arg = longopts[longopt_match].has_arg; } /* if we didn't find a long option, is it a short option? */ if (longopt_match < 0 && shortopts != NULL) { cp = strchr (shortopts, argv[optind][optwhere]); if (cp == NULL) { /* couldn't find option in shortopts */ if (opterr) fprintf (stderr, "%s: invalid option -- `-%c'\n", argv[0], argv[optind][optwhere]); optwhere++; if (argv[optind][optwhere] == '\0') { optind++; optwhere = 1; } return (optopt = '?'); } has_arg = ((cp[1] == ':') ? ((cp[2] == ':') ? OPTIONAL_ARG : REQUIRED_ARG) : no_argument); possible_arg = argv[optind] + optwhere + 1; optopt = *cp; } /* get argument and reset optwhere */ arg_next = 0; switch (has_arg) { case OPTIONAL_ARG: if (*possible_arg == '=') possible_arg++; if (*possible_arg != '\0') { optarg = possible_arg; optwhere = 1; } else optarg = NULL; break; case REQUIRED_ARG: if (*possible_arg == '=') possible_arg++; if (*possible_arg != '\0') { optarg = possible_arg; optwhere = 1; } else if (optind + 1 >= argc) { if (opterr) { fprintf (stderr, "%s: argument required for option `", argv[0]); if (longopt_match >= 0) fprintf (stderr, "--%s'\n", longopts[longopt_match].name); else fprintf (stderr, "-%c'\n", *cp); } optind++; return (optopt = ':'); } else { optarg = argv[optind + 1]; arg_next = 1; optwhere = 1; } break; case no_argument: if (longopt_match < 0) { optwhere++; if (argv[optind][optwhere] == '\0') optwhere = 1; } else optwhere = 1; optarg = NULL; break; } /* do we have to permute or otherwise modify optind? */ if (ordering == PERMUTE && optwhere == 1 && num_nonopts != 0) { permute (argv + permute_from, num_nonopts, 1 + arg_next); optind = permute_from + 1 + arg_next; } else if (optwhere == 1) optind = optind + 1 + arg_next; /* finally return */ if (longopt_match >= 0) { if (longind != NULL) *longind = longopt_match; if (longopts[longopt_match].flag != NULL) { *(longopts[longopt_match].flag) = longopts[longopt_match].val; return 0; } else return longopts[longopt_match].val; } else return optopt; } int getopt (int argc, char **argv, char *optstring) { return getopt_internal (argc, argv, optstring, NULL, NULL, 0); } int getopt_long (int argc, char **argv, const char *shortopts, const GETOPT_LONG_OPTION_T * longopts, int *longind) { return getopt_internal (argc, argv, (char*)shortopts, (GETOPT_LONG_OPTION_T*)longopts, longind, 0); } int getopt_long_only (int argc, char **argv, const char *shortopts, const GETOPT_LONG_OPTION_T * longopts, int *longind) { return getopt_internal (argc, argv, (char*)shortopts, (GETOPT_LONG_OPTION_T*)longopts, longind, 1); } /* end of file GETOPT.C */ ninja-1.3.4/src/getopt.h000066400000000000000000000032251215343310300150550ustar00rootroot00000000000000#ifndef GETOPT_H #define GETOPT_H /* include files needed by this include file */ /* macros defined by this include file */ #define no_argument 0 #define REQUIRED_ARG 1 #define OPTIONAL_ARG 2 /* types defined by this include file */ /* GETOPT_LONG_OPTION_T: The type of long option */ typedef struct GETOPT_LONG_OPTION_T { const char *name; /* the name of the long option */ int has_arg; /* one of the above macros */ int *flag; /* determines if getopt_long() returns a * value for a long option; if it is * non-NULL, 0 is returned as a function * value and the value of val is stored in * the area pointed to by flag. Otherwise, * val is returned. */ int val; /* determines the value to return if flag is * NULL. */ } GETOPT_LONG_OPTION_T; typedef GETOPT_LONG_OPTION_T option; #ifdef __cplusplus extern "C" { #endif /* externally-defined variables */ extern char *optarg; extern int optind; extern int opterr; extern int optopt; /* function prototypes */ int getopt (int argc, char **argv, char *optstring); int getopt_long (int argc, char **argv, const char *shortopts, const GETOPT_LONG_OPTION_T * longopts, int *longind); int getopt_long_only (int argc, char **argv, const char *shortopts, const GETOPT_LONG_OPTION_T * longopts, int *longind); #ifdef __cplusplus }; #endif #endif /* GETOPT_H */ /* END OF FILE getopt.h */ ninja-1.3.4/src/graph.cc000066400000000000000000000336231215343310300150170ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "graph.h" #include #include #include "build_log.h" #include "depfile_parser.h" #include "deps_log.h" #include "disk_interface.h" #include "explain.h" #include "manifest_parser.h" #include "metrics.h" #include "state.h" #include "util.h" bool Node::Stat(DiskInterface* disk_interface) { METRIC_RECORD("node stat"); mtime_ = disk_interface->Stat(path_); return mtime_ > 0; } void Rule::AddBinding(const string& key, const EvalString& val) { bindings_[key] = val; } const EvalString* Rule::GetBinding(const string& key) const { map::const_iterator i = bindings_.find(key); if (i == bindings_.end()) return NULL; return &i->second; } // static bool Rule::IsReservedBinding(const string& var) { return var == "command" || var == "depfile" || var == "description" || var == "deps" || var == "generator" || var == "pool" || var == "restat" || var == "rspfile" || var == "rspfile_content"; } bool DependencyScan::RecomputeDirty(Edge* edge, string* err) { bool dirty = false; edge->outputs_ready_ = true; TimeStamp deps_mtime = 0; if (!dep_loader_.LoadDeps(edge, &deps_mtime, err)) { if (!err->empty()) return false; // Failed to load dependency info: rebuild to regenerate it. dirty = true; } // Visit all inputs; we're dirty if any of the inputs are dirty. Node* most_recent_input = NULL; for (vector::iterator i = edge->inputs_.begin(); i != edge->inputs_.end(); ++i) { if ((*i)->StatIfNecessary(disk_interface_)) { if (Edge* in_edge = (*i)->in_edge()) { if (!RecomputeDirty(in_edge, err)) return false; } else { // This input has no in-edge; it is dirty if it is missing. if (!(*i)->exists()) EXPLAIN("%s has no in-edge and is missing", (*i)->path().c_str()); (*i)->set_dirty(!(*i)->exists()); } } // If an input is not ready, neither are our outputs. if (Edge* in_edge = (*i)->in_edge()) { if (!in_edge->outputs_ready_) edge->outputs_ready_ = false; } if (!edge->is_order_only(i - edge->inputs_.begin())) { // If a regular input is dirty (or missing), we're dirty. // Otherwise consider mtime. if ((*i)->dirty()) { EXPLAIN("%s is dirty", (*i)->path().c_str()); dirty = true; } else { if (!most_recent_input || (*i)->mtime() > most_recent_input->mtime()) { most_recent_input = *i; } } } } // We may also be dirty due to output state: missing outputs, out of // date outputs, etc. Visit all outputs and determine whether they're dirty. if (!dirty) { string command = edge->EvaluateCommand(true); for (vector::iterator i = edge->outputs_.begin(); i != edge->outputs_.end(); ++i) { (*i)->StatIfNecessary(disk_interface_); if (RecomputeOutputDirty(edge, most_recent_input, deps_mtime, command, *i)) { dirty = true; break; } } } // Finally, visit each output to mark off that we've visited it, and update // their dirty state if necessary. for (vector::iterator i = edge->outputs_.begin(); i != edge->outputs_.end(); ++i) { (*i)->StatIfNecessary(disk_interface_); if (dirty) (*i)->MarkDirty(); } // If an edge is dirty, its outputs are normally not ready. (It's // possible to be clean but still not be ready in the presence of // order-only inputs.) // But phony edges with no inputs have nothing to do, so are always // ready. if (dirty && !(edge->is_phony() && edge->inputs_.empty())) edge->outputs_ready_ = false; return true; } bool DependencyScan::RecomputeOutputDirty(Edge* edge, Node* most_recent_input, TimeStamp deps_mtime, const string& command, Node* output) { if (edge->is_phony()) { // Phony edges don't write any output. Outputs are only dirty if // there are no inputs and we're missing the output. return edge->inputs_.empty() && !output->exists(); } BuildLog::LogEntry* entry = 0; // Dirty if we're missing the output. if (!output->exists()) { EXPLAIN("output %s doesn't exist", output->path().c_str()); return true; } // Dirty if the output is older than the input. if (most_recent_input && output->mtime() < most_recent_input->mtime()) { TimeStamp output_mtime = output->mtime(); // If this is a restat rule, we may have cleaned the output with a restat // rule in a previous run and stored the most recent input mtime in the // build log. Use that mtime instead, so that the file will only be // considered dirty if an input was modified since the previous run. bool used_restat = false; if (edge->GetBindingBool("restat") && build_log() && (entry = build_log()->LookupByOutput(output->path()))) { output_mtime = entry->restat_mtime; used_restat = true; } if (output_mtime < most_recent_input->mtime()) { EXPLAIN("%soutput %s older than most recent input %s " "(%d vs %d)", used_restat ? "restat of " : "", output->path().c_str(), most_recent_input->path().c_str(), output_mtime, most_recent_input->mtime()); return true; } } // Dirty if the output is newer than the deps. if (deps_mtime && output->mtime() > deps_mtime) { EXPLAIN("stored deps info out of date for for %s (%d vs %d)", output->path().c_str(), deps_mtime, output->mtime()); return true; } // May also be dirty due to the command changing since the last build. // But if this is a generator rule, the command changing does not make us // dirty. if (!edge->GetBindingBool("generator") && build_log()) { if (entry || (entry = build_log()->LookupByOutput(output->path()))) { if (BuildLog::LogEntry::HashCommand(command) != entry->command_hash) { EXPLAIN("command line changed for %s", output->path().c_str()); return true; } } if (!entry) { EXPLAIN("command line not found in log for %s", output->path().c_str()); return true; } } return false; } bool Edge::AllInputsReady() const { for (vector::const_iterator i = inputs_.begin(); i != inputs_.end(); ++i) { if ((*i)->in_edge() && !(*i)->in_edge()->outputs_ready()) return false; } return true; } /// An Env for an Edge, providing $in and $out. struct EdgeEnv : public Env { explicit EdgeEnv(Edge* edge) : edge_(edge) {} virtual string LookupVariable(const string& var); /// Given a span of Nodes, construct a list of paths suitable for a command /// line. string MakePathList(vector::iterator begin, vector::iterator end, char sep); Edge* edge_; }; string EdgeEnv::LookupVariable(const string& var) { if (var == "in" || var == "in_newline") { int explicit_deps_count = edge_->inputs_.size() - edge_->implicit_deps_ - edge_->order_only_deps_; return MakePathList(edge_->inputs_.begin(), edge_->inputs_.begin() + explicit_deps_count, var == "in" ? ' ' : '\n'); } else if (var == "out") { return MakePathList(edge_->outputs_.begin(), edge_->outputs_.end(), ' '); } // See notes on BindingEnv::LookupWithFallback. const EvalString* eval = edge_->rule_->GetBinding(var); return edge_->env_->LookupWithFallback(var, eval, this); } string EdgeEnv::MakePathList(vector::iterator begin, vector::iterator end, char sep) { string result; for (vector::iterator i = begin; i != end; ++i) { if (!result.empty()) result.push_back(sep); const string& path = (*i)->path(); if (path.find(" ") != string::npos) { result.append("\""); result.append(path); result.append("\""); } else { result.append(path); } } return result; } string Edge::EvaluateCommand(bool incl_rsp_file) { string command = GetBinding("command"); if (incl_rsp_file) { string rspfile_content = GetBinding("rspfile_content"); if (!rspfile_content.empty()) command += ";rspfile=" + rspfile_content; } return command; } string Edge::GetBinding(const string& key) { EdgeEnv env(this); return env.LookupVariable(key); } bool Edge::GetBindingBool(const string& key) { return !GetBinding(key).empty(); } void Edge::Dump(const char* prefix) const { printf("%s[ ", prefix); for (vector::const_iterator i = inputs_.begin(); i != inputs_.end() && *i != NULL; ++i) { printf("%s ", (*i)->path().c_str()); } printf("--%s-> ", rule_->name().c_str()); for (vector::const_iterator i = outputs_.begin(); i != outputs_.end() && *i != NULL; ++i) { printf("%s ", (*i)->path().c_str()); } if (pool_) { if (!pool_->name().empty()) { printf("(in pool '%s')", pool_->name().c_str()); } } else { printf("(null pool?)"); } printf("] 0x%p\n", this); } bool Edge::is_phony() const { return rule_ == &State::kPhonyRule; } void Node::Dump(const char* prefix) const { printf("%s <%s 0x%p> mtime: %d%s, (:%s), ", prefix, path().c_str(), this, mtime(), mtime() ? "" : " (:missing)", dirty() ? " dirty" : " clean"); if (in_edge()) { in_edge()->Dump("in-edge: "); } else { printf("no in-edge\n"); } printf(" out edges:\n"); for (vector::const_iterator e = out_edges().begin(); e != out_edges().end() && *e != NULL; ++e) { (*e)->Dump(" +- "); } } bool ImplicitDepLoader::LoadDeps(Edge* edge, TimeStamp* mtime, string* err) { string deps_type = edge->GetBinding("deps"); if (!deps_type.empty()) { if (!LoadDepsFromLog(edge, mtime, err)) { if (!err->empty()) return false; EXPLAIN("deps for %s are missing", edge->outputs_[0]->path().c_str()); return false; } return true; } string depfile = edge->GetBinding("depfile"); if (!depfile.empty()) { if (!LoadDepFile(edge, depfile, err)) { if (!err->empty()) return false; EXPLAIN("depfile '%s' is missing", depfile.c_str()); return false; } return true; } // No deps to load. return true; } bool ImplicitDepLoader::LoadDepFile(Edge* edge, const string& path, string* err) { METRIC_RECORD("depfile load"); string content = disk_interface_->ReadFile(path, err); if (!err->empty()) { *err = "loading '" + path + "': " + *err; return false; } // On a missing depfile: return false and empty *err. if (content.empty()) return false; DepfileParser depfile; string depfile_err; if (!depfile.Parse(&content, &depfile_err)) { *err = path + ": " + depfile_err; return false; } // Check that this depfile matches the edge's output. Node* first_output = edge->outputs_[0]; StringPiece opath = StringPiece(first_output->path()); if (opath != depfile.out_) { *err = "expected depfile '" + path + "' to mention '" + first_output->path() + "', got '" + depfile.out_.AsString() + "'"; return false; } // Preallocate space in edge->inputs_ to be filled in below. vector::iterator implicit_dep = PreallocateSpace(edge, depfile.ins_.size()); // Add all its in-edges. for (vector::iterator i = depfile.ins_.begin(); i != depfile.ins_.end(); ++i, ++implicit_dep) { if (!CanonicalizePath(const_cast(i->str_), &i->len_, err)) return false; Node* node = state_->GetNode(*i); *implicit_dep = node; node->AddOutEdge(edge); CreatePhonyInEdge(node); } return true; } bool ImplicitDepLoader::LoadDepsFromLog(Edge* edge, TimeStamp* deps_mtime, string* err) { DepsLog::Deps* deps = deps_log_->GetDeps(edge->outputs_[0]); if (!deps) return false; *deps_mtime = deps->mtime; vector::iterator implicit_dep = PreallocateSpace(edge, deps->node_count); for (int i = 0; i < deps->node_count; ++i, ++implicit_dep) { *implicit_dep = deps->nodes[i]; deps->nodes[i]->AddOutEdge(edge); CreatePhonyInEdge(*implicit_dep); } return true; } vector::iterator ImplicitDepLoader::PreallocateSpace(Edge* edge, int count) { edge->inputs_.insert(edge->inputs_.end() - edge->order_only_deps_, (size_t)count, 0); edge->implicit_deps_ += count; return edge->inputs_.end() - edge->order_only_deps_ - count; } void ImplicitDepLoader::CreatePhonyInEdge(Node* node) { if (node->in_edge()) return; Edge* phony_edge = state_->AddEdge(&State::kPhonyRule); node->set_in_edge(phony_edge); phony_edge->outputs_.push_back(node); // RecomputeDirty might not be called for phony_edge if a previous call // to RecomputeDirty had caused the file to be stat'ed. Because previous // invocations of RecomputeDirty would have seen this node without an // input edge (and therefore ready), we have to set outputs_ready_ to true // to avoid a potential stuck build. If we do call RecomputeDirty for // this node, it will simply set outputs_ready_ to the correct value. phony_edge->outputs_ready_ = true; } ninja-1.3.4/src/graph.h000066400000000000000000000177611215343310300146660ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_GRAPH_H_ #define NINJA_GRAPH_H_ #include #include using namespace std; #include "eval_env.h" #include "timestamp.h" struct BuildLog; struct DiskInterface; struct DepsLog; struct Edge; struct Node; struct Pool; struct State; /// Information about a node in the dependency graph: the file, whether /// it's dirty, mtime, etc. struct Node { explicit Node(const string& path) : path_(path), mtime_(-1), dirty_(false), in_edge_(NULL), id_(-1) {} /// Return true if the file exists (mtime_ got a value). bool Stat(DiskInterface* disk_interface); /// Return true if we needed to stat. bool StatIfNecessary(DiskInterface* disk_interface) { if (status_known()) return false; Stat(disk_interface); return true; } /// Mark as not-yet-stat()ed and not dirty. void ResetState() { mtime_ = -1; dirty_ = false; } /// Mark the Node as already-stat()ed and missing. void MarkMissing() { mtime_ = 0; } bool exists() const { return mtime_ != 0; } bool status_known() const { return mtime_ != -1; } const string& path() const { return path_; } TimeStamp mtime() const { return mtime_; } bool dirty() const { return dirty_; } void set_dirty(bool dirty) { dirty_ = dirty; } void MarkDirty() { dirty_ = true; } Edge* in_edge() const { return in_edge_; } void set_in_edge(Edge* edge) { in_edge_ = edge; } int id() const { return id_; } void set_id(int id) { id_ = id; } const vector& out_edges() const { return out_edges_; } void AddOutEdge(Edge* edge) { out_edges_.push_back(edge); } void Dump(const char* prefix="") const; private: string path_; /// Possible values of mtime_: /// -1: file hasn't been examined /// 0: we looked, and file doesn't exist /// >0: actual file's mtime TimeStamp mtime_; /// Dirty is true when the underlying file is out-of-date. /// But note that Edge::outputs_ready_ is also used in judging which /// edges to build. bool dirty_; /// The Edge that produces this Node, or NULL when there is no /// known edge to produce it. Edge* in_edge_; /// All Edges that use this Node as an input. vector out_edges_; /// A dense integer id for the node, assigned and used by DepsLog. int id_; }; /// An invokable build command and associated metadata (description, etc.). struct Rule { explicit Rule(const string& name) : name_(name) {} const string& name() const { return name_; } typedef map Bindings; void AddBinding(const string& key, const EvalString& val); static bool IsReservedBinding(const string& var); const EvalString* GetBinding(const string& key) const; private: // Allow the parsers to reach into this object and fill out its fields. friend struct ManifestParser; string name_; map bindings_; }; /// An edge in the dependency graph; links between Nodes using Rules. struct Edge { Edge() : rule_(NULL), env_(NULL), outputs_ready_(false), implicit_deps_(0), order_only_deps_(0) {} /// Return true if all inputs' in-edges are ready. bool AllInputsReady() const; /// Expand all variables in a command and return it as a string. /// If incl_rsp_file is enabled, the string will also contain the /// full contents of a response file (if applicable) string EvaluateCommand(bool incl_rsp_file = false); string GetBinding(const string& key); bool GetBindingBool(const string& key); void Dump(const char* prefix="") const; const Rule* rule_; Pool* pool_; vector inputs_; vector outputs_; BindingEnv* env_; bool outputs_ready_; const Rule& rule() const { return *rule_; } Pool* pool() const { return pool_; } int weight() const { return 1; } bool outputs_ready() const { return outputs_ready_; } // There are three types of inputs. // 1) explicit deps, which show up as $in on the command line; // 2) implicit deps, which the target depends on implicitly (e.g. C headers), // and changes in them cause the target to rebuild; // 3) order-only deps, which are needed before the target builds but which // don't cause the target to rebuild. // These are stored in inputs_ in that order, and we keep counts of // #2 and #3 when we need to access the various subsets. int implicit_deps_; int order_only_deps_; bool is_implicit(size_t index) { return index >= inputs_.size() - order_only_deps_ - implicit_deps_ && !is_order_only(index); } bool is_order_only(size_t index) { return index >= inputs_.size() - order_only_deps_; } bool is_phony() const; }; /// ImplicitDepLoader loads implicit dependencies, as referenced via the /// "depfile" attribute in build files. struct ImplicitDepLoader { ImplicitDepLoader(State* state, DepsLog* deps_log, DiskInterface* disk_interface) : state_(state), disk_interface_(disk_interface), deps_log_(deps_log) {} /// Load implicit dependencies for \a edge. May fill in \a mtime with /// the timestamp of the loaded information. /// @return false on error (without filling \a err if info is just missing). bool LoadDeps(Edge* edge, TimeStamp* mtime, string* err); DepsLog* deps_log() const { return deps_log_; } private: /// Load implicit dependencies for \a edge from a depfile attribute. /// @return false on error (without filling \a err if info is just missing). bool LoadDepFile(Edge* edge, const string& path, string* err); /// Load implicit dependencies for \a edge from the DepsLog. /// @return false on error (without filling \a err if info is just missing). bool LoadDepsFromLog(Edge* edge, TimeStamp* mtime, string* err); /// Preallocate \a count spaces in the input array on \a edge, returning /// an iterator pointing at the first new space. vector::iterator PreallocateSpace(Edge* edge, int count); /// If we don't have a edge that generates this input already, /// create one; this makes us not abort if the input is missing, /// but instead will rebuild in that circumstance. void CreatePhonyInEdge(Node* node); State* state_; DiskInterface* disk_interface_; DepsLog* deps_log_; }; /// DependencyScan manages the process of scanning the files in a graph /// and updating the dirty/outputs_ready state of all the nodes and edges. struct DependencyScan { DependencyScan(State* state, BuildLog* build_log, DepsLog* deps_log, DiskInterface* disk_interface) : build_log_(build_log), disk_interface_(disk_interface), dep_loader_(state, deps_log, disk_interface) {} /// Examine inputs, outputs, and command lines to judge whether an edge /// needs to be re-run, and update outputs_ready_ and each outputs' |dirty_| /// state accordingly. /// Returns false on failure. bool RecomputeDirty(Edge* edge, string* err); /// Recompute whether a given single output should be marked dirty. /// Returns true if so. bool RecomputeOutputDirty(Edge* edge, Node* most_recent_input, TimeStamp deps_mtime, const string& command, Node* output); BuildLog* build_log() const { return build_log_; } void set_build_log(BuildLog* log) { build_log_ = log; } DepsLog* deps_log() const { return dep_loader_.deps_log(); } private: BuildLog* build_log_; DiskInterface* disk_interface_; ImplicitDepLoader dep_loader_; }; #endif // NINJA_GRAPH_H_ ninja-1.3.4/src/graph_test.cc000066400000000000000000000150051215343310300160500ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "graph.h" #include "test.h" struct GraphTest : public StateTestWithBuiltinRules { GraphTest() : scan_(&state_, NULL, NULL, &fs_) {} VirtualFileSystem fs_; DependencyScan scan_; }; TEST_F(GraphTest, MissingImplicit) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat in | implicit\n")); fs_.Create("in", ""); fs_.Create("out", ""); Edge* edge = GetNode("out")->in_edge(); string err; EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); // A missing implicit dep *should* make the output dirty. // (In fact, a build will fail.) // This is a change from prior semantics of ninja. EXPECT_TRUE(GetNode("out")->dirty()); } TEST_F(GraphTest, ModifiedImplicit) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out: cat in | implicit\n")); fs_.Create("in", ""); fs_.Create("out", ""); fs_.Tick(); fs_.Create("implicit", ""); Edge* edge = GetNode("out")->in_edge(); string err; EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); // A modified implicit dep should make the output dirty. EXPECT_TRUE(GetNode("out")->dirty()); } TEST_F(GraphTest, FunkyMakefilePath) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule catdep\n" " depfile = $out.d\n" " command = cat $in > $out\n" "build out.o: catdep foo.cc\n")); fs_.Create("foo.cc", ""); fs_.Create("out.o.d", "out.o: ./foo/../implicit.h\n"); fs_.Create("out.o", ""); fs_.Tick(); fs_.Create("implicit.h", ""); Edge* edge = GetNode("out.o")->in_edge(); string err; EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); // implicit.h has changed, though our depfile refers to it with a // non-canonical path; we should still find it. EXPECT_TRUE(GetNode("out.o")->dirty()); } TEST_F(GraphTest, ExplicitImplicit) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule catdep\n" " depfile = $out.d\n" " command = cat $in > $out\n" "build implicit.h: cat data\n" "build out.o: catdep foo.cc || implicit.h\n")); fs_.Create("implicit.h", ""); fs_.Create("foo.cc", ""); fs_.Create("out.o.d", "out.o: implicit.h\n"); fs_.Create("out.o", ""); fs_.Tick(); fs_.Create("data", ""); Edge* edge = GetNode("out.o")->in_edge(); string err; EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); // We have both an implicit and an explicit dep on implicit.h. // The implicit dep should "win" (in the sense that it should cause // the output to be dirty). EXPECT_TRUE(GetNode("out.o")->dirty()); } TEST_F(GraphTest, PathWithCurrentDirectory) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule catdep\n" " depfile = $out.d\n" " command = cat $in > $out\n" "build ./out.o: catdep ./foo.cc\n")); fs_.Create("foo.cc", ""); fs_.Create("out.o.d", "out.o: foo.cc\n"); fs_.Create("out.o", ""); Edge* edge = GetNode("out.o")->in_edge(); string err; EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); EXPECT_FALSE(GetNode("out.o")->dirty()); } TEST_F(GraphTest, RootNodes) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build out1: cat in1\n" "build mid1: cat in1\n" "build out2: cat mid1\n" "build out3 out4: cat mid1\n")); string err; vector root_nodes = state_.RootNodes(&err); EXPECT_EQ(4u, root_nodes.size()); for (size_t i = 0; i < root_nodes.size(); ++i) { string name = root_nodes[i]->path(); EXPECT_EQ("out", name.substr(0, 3)); } } TEST_F(GraphTest, VarInOutQuoteSpaces) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "build a$ b: cat nospace with$ space nospace2\n")); Edge* edge = GetNode("a b")->in_edge(); EXPECT_EQ("cat nospace \"with space\" nospace2 > \"a b\"", edge->EvaluateCommand()); } // Regression test for https://github.com/martine/ninja/issues/380 TEST_F(GraphTest, DepfileWithCanonicalizablePath) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule catdep\n" " depfile = $out.d\n" " command = cat $in > $out\n" "build ./out.o: catdep ./foo.cc\n")); fs_.Create("foo.cc", ""); fs_.Create("out.o.d", "out.o: bar/../foo.cc\n"); fs_.Create("out.o", ""); Edge* edge = GetNode("out.o")->in_edge(); string err; EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); EXPECT_FALSE(GetNode("out.o")->dirty()); } // Regression test for https://github.com/martine/ninja/issues/404 TEST_F(GraphTest, DepfileRemoved) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule catdep\n" " depfile = $out.d\n" " command = cat $in > $out\n" "build ./out.o: catdep ./foo.cc\n")); fs_.Create("foo.h", ""); fs_.Create("foo.cc", ""); fs_.Tick(); fs_.Create("out.o.d", "out.o: foo.h\n"); fs_.Create("out.o", ""); Edge* edge = GetNode("out.o")->in_edge(); string err; EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); EXPECT_FALSE(GetNode("out.o")->dirty()); state_.Reset(); fs_.RemoveFile("out.o.d"); EXPECT_TRUE(scan_.RecomputeDirty(edge, &err)); ASSERT_EQ("", err); EXPECT_TRUE(GetNode("out.o")->dirty()); } // Check that rule-level variables are in scope for eval. TEST_F(GraphTest, RuleVariablesInScope) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule r\n" " depfile = x\n" " command = depfile is $depfile\n" "build out: r in\n")); Edge* edge = GetNode("out")->in_edge(); EXPECT_EQ("depfile is x", edge->EvaluateCommand()); } // Check that build statements can override rule builtins like depfile. TEST_F(GraphTest, DepfileOverride) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule r\n" " depfile = x\n" " command = unused\n" "build out: r in\n" " depfile = y\n")); Edge* edge = GetNode("out")->in_edge(); EXPECT_EQ("y", edge->GetBinding("depfile")); } // Check that overridden values show up in expansion of rule-level bindings. TEST_F(GraphTest, DepfileOverrideParent) { ASSERT_NO_FATAL_FAILURE(AssertParse(&state_, "rule r\n" " depfile = x\n" " command = depfile is $depfile\n" "build out: r in\n" " depfile = y\n")); Edge* edge = GetNode("out")->in_edge(); EXPECT_EQ("depfile is y", edge->GetBinding("command")); } ninja-1.3.4/src/graphviz.cc000066400000000000000000000044671215343310300155540ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "graphviz.h" #include #include "graph.h" void GraphViz::AddTarget(Node* node) { if (visited_nodes_.find(node) != visited_nodes_.end()) return; printf("\"%p\" [label=\"%s\"]\n", node, node->path().c_str()); visited_nodes_.insert(node); Edge* edge = node->in_edge(); if (!edge) { // Leaf node. // Draw as a rect? return; } if (visited_edges_.find(edge) != visited_edges_.end()) return; visited_edges_.insert(edge); if (edge->inputs_.size() == 1 && edge->outputs_.size() == 1) { // Can draw simply. // Note extra space before label text -- this is cosmetic and feels // like a graphviz bug. printf("\"%p\" -> \"%p\" [label=\" %s\"]\n", edge->inputs_[0], edge->outputs_[0], edge->rule_->name().c_str()); } else { printf("\"%p\" [label=\"%s\", shape=ellipse]\n", edge, edge->rule_->name().c_str()); for (vector::iterator out = edge->outputs_.begin(); out != edge->outputs_.end(); ++out) { printf("\"%p\" -> \"%p\"\n", edge, *out); } for (vector::iterator in = edge->inputs_.begin(); in != edge->inputs_.end(); ++in) { const char* order_only = ""; if (edge->is_order_only(in - edge->inputs_.begin())) order_only = " style=dotted"; printf("\"%p\" -> \"%p\" [arrowhead=none%s]\n", (*in), edge, order_only); } } for (vector::iterator in = edge->inputs_.begin(); in != edge->inputs_.end(); ++in) { AddTarget(*in); } } void GraphViz::Start() { printf("digraph ninja {\n"); printf("rankdir=\"LR\"\n"); printf("node [fontsize=10, shape=box, height=0.25]\n"); printf("edge [fontsize=10]\n"); } void GraphViz::Finish() { printf("}\n"); } ninja-1.3.4/src/graphviz.h000066400000000000000000000017001215343310300154010ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_GRAPHVIZ_H_ #define NINJA_GRAPHVIZ_H_ #include using namespace std; struct Node; struct Edge; /// Runs the process of creating GraphViz .dot file output. struct GraphViz { void Start(); void AddTarget(Node* node); void Finish(); set visited_nodes_; set visited_edges_; }; #endif // NINJA_GRAPHVIZ_H_ ninja-1.3.4/src/hash_collision_bench.cc000066400000000000000000000034711215343310300200510ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "build_log.h" #include using namespace std; #include int random(int low, int high) { return int(low + (rand() / double(RAND_MAX)) * (high - low) + 0.5); } void RandomCommand(char** s) { int len = random(5, 100); *s = new char[len]; for (int i = 0; i < len; ++i) (*s)[i] = (char)random(32, 127); } int main() { const int N = 20 * 1000 * 1000; // Leak these, else 10% of the runtime is spent destroying strings. char** commands = new char*[N]; pair* hashes = new pair[N]; srand((int)time(NULL)); for (int i = 0; i < N; ++i) { RandomCommand(&commands[i]); hashes[i] = make_pair(BuildLog::LogEntry::HashCommand(commands[i]), i); } sort(hashes, hashes + N); int num_collisions = 0; for (int i = 1; i < N; ++i) { if (hashes[i - 1].first == hashes[i].first) { if (strcmp(commands[hashes[i - 1].second], commands[hashes[i].second]) != 0) { printf("collision!\n string 1: '%s'\n string 2: '%s'\n", commands[hashes[i - 1].second], commands[hashes[i].second]); num_collisions++; } } } printf("\n\n%d collisions after %d runs\n", num_collisions, N); } ninja-1.3.4/src/hash_map.h000066400000000000000000000051111215343310300153270ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_MAP_H_ #define NINJA_MAP_H_ #include "string_piece.h" // MurmurHash2, by Austin Appleby static inline unsigned int MurmurHash2(const void* key, size_t len) { static const unsigned int seed = 0xDECAFBAD; const unsigned int m = 0x5bd1e995; const int r = 24; unsigned int h = seed ^ len; const unsigned char * data = (const unsigned char *)key; while (len >= 4) { unsigned int k = *(unsigned int *)data; k *= m; k ^= k >> r; k *= m; h *= m; h ^= k; data += 4; len -= 4; } switch (len) { case 3: h ^= data[2] << 16; case 2: h ^= data[1] << 8; case 1: h ^= data[0]; h *= m; }; h ^= h >> 13; h *= m; h ^= h >> 15; return h; } #ifdef _MSC_VER #include using stdext::hash_map; using stdext::hash_compare; struct StringPieceCmp : public hash_compare { size_t operator()(const StringPiece& key) const { return MurmurHash2(key.str_, key.len_); } bool operator()(const StringPiece& a, const StringPiece& b) const { int cmp = strncmp(a.str_, b.str_, min(a.len_, b.len_)); if (cmp < 0) { return true; } else if (cmp > 0) { return false; } else { return a.len_ < b.len_; } } }; #else #include using __gnu_cxx::hash_map; namespace __gnu_cxx { template<> struct hash { size_t operator()(const std::string& s) const { return hash()(s.c_str()); } }; template<> struct hash { size_t operator()(StringPiece key) const { return MurmurHash2(key.str_, key.len_); } }; } #endif /// A template for hash_maps keyed by a StringPiece whose string is /// owned externally (typically by the values). Use like: /// ExternalStringHash::Type foos; to make foos into a hash /// mapping StringPiece => Foo*. template struct ExternalStringHashMap { #ifdef _MSC_VER typedef hash_map Type; #else typedef hash_map Type; #endif }; #endif // NINJA_MAP_H_ ninja-1.3.4/src/includes_normalize-win32.cc000066400000000000000000000066531215343310300205470ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "includes_normalize.h" #include "string_piece.h" #include "util.h" #include #include #include #include namespace { /// Return true if paths a and b are on the same Windows drive. bool SameDrive(StringPiece a, StringPiece b) { char a_absolute[_MAX_PATH]; char b_absolute[_MAX_PATH]; GetFullPathName(a.AsString().c_str(), sizeof(a_absolute), a_absolute, NULL); GetFullPathName(b.AsString().c_str(), sizeof(b_absolute), b_absolute, NULL); char a_drive[_MAX_DIR]; char b_drive[_MAX_DIR]; _splitpath(a_absolute, a_drive, NULL, NULL, NULL); _splitpath(b_absolute, b_drive, NULL, NULL, NULL); return _stricmp(a_drive, b_drive) == 0; } } // anonymous namespace string IncludesNormalize::Join(const vector& list, char sep) { string ret; for (size_t i = 0; i < list.size(); ++i) { ret += list[i]; if (i != list.size() - 1) ret += sep; } return ret; } vector IncludesNormalize::Split(const string& input, char sep) { vector elems; stringstream ss(input); string item; while (getline(ss, item, sep)) elems.push_back(item); return elems; } string IncludesNormalize::ToLower(const string& s) { string ret; transform(s.begin(), s.end(), back_inserter(ret), ::tolower); return ret; } string IncludesNormalize::AbsPath(StringPiece s) { char result[_MAX_PATH]; GetFullPathName(s.AsString().c_str(), sizeof(result), result, NULL); return result; } string IncludesNormalize::Relativize(StringPiece path, const string& start) { vector start_list = Split(AbsPath(start), '\\'); vector path_list = Split(AbsPath(path), '\\'); int i; for (i = 0; i < static_cast(min(start_list.size(), path_list.size())); ++i) { if (ToLower(start_list[i]) != ToLower(path_list[i])) break; } vector rel_list; for (int j = 0; j < static_cast(start_list.size() - i); ++j) rel_list.push_back(".."); for (int j = i; j < static_cast(path_list.size()); ++j) rel_list.push_back(path_list[j]); if (rel_list.size() == 0) return "."; return Join(rel_list, '\\'); } string IncludesNormalize::Normalize(const string& input, const char* relative_to) { char copy[_MAX_PATH]; size_t len = input.size(); strncpy(copy, input.c_str(), input.size() + 1); for (size_t j = 0; j < len; ++j) if (copy[j] == '/') copy[j] = '\\'; string err; if (!CanonicalizePath(copy, &len, &err)) { Warning("couldn't canonicalize '%s: %s\n", input.c_str(), err.c_str()); } string curdir; if (!relative_to) { curdir = AbsPath("."); relative_to = curdir.c_str(); } StringPiece partially_fixed(copy, len); if (!SameDrive(partially_fixed, relative_to)) return partially_fixed.AsString(); return Relativize(partially_fixed, relative_to); } ninja-1.3.4/src/includes_normalize.h000066400000000000000000000026571215343310300174510ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include using namespace std; struct StringPiece; /// Utility functions for normalizing include paths on Windows. /// TODO: this likely duplicates functionality of CanonicalizePath; refactor. struct IncludesNormalize { // Internal utilities made available for testing, maybe useful otherwise. static string Join(const vector& list, char sep); static vector Split(const string& input, char sep); static string ToLower(const string& s); static string AbsPath(StringPiece s); static string Relativize(StringPiece path, const string& start); /// Normalize by fixing slashes style, fixing redundant .. and . and makes the /// path relative to |relative_to|. Case is normalized to lowercase on /// Windows too. static string Normalize(const string& input, const char* relative_to); }; ninja-1.3.4/src/includes_normalize_test.cc000066400000000000000000000077331215343310300206460ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "includes_normalize.h" #include #include "test.h" #include "util.h" TEST(IncludesNormalize, Simple) { EXPECT_EQ("b", IncludesNormalize::Normalize("a\\..\\b", NULL)); EXPECT_EQ("b", IncludesNormalize::Normalize("a\\../b", NULL)); EXPECT_EQ("a\\b", IncludesNormalize::Normalize("a\\.\\b", NULL)); EXPECT_EQ("a\\b", IncludesNormalize::Normalize("a\\./b", NULL)); } namespace { string GetCurDir() { char buf[_MAX_PATH]; _getcwd(buf, sizeof(buf)); vector parts = IncludesNormalize::Split(string(buf), '\\'); return parts[parts.size() - 1]; } } // namespace TEST(IncludesNormalize, WithRelative) { string currentdir = IncludesNormalize::ToLower(GetCurDir()); EXPECT_EQ("c", IncludesNormalize::Normalize("a/b/c", "a/b")); EXPECT_EQ("a", IncludesNormalize::Normalize(IncludesNormalize::AbsPath("a"), NULL)); EXPECT_EQ(string("..\\") + currentdir + string("\\a"), IncludesNormalize::Normalize("a", "../b")); EXPECT_EQ(string("..\\") + currentdir + string("\\a\\b"), IncludesNormalize::Normalize("a/b", "../c")); EXPECT_EQ("..\\..\\a", IncludesNormalize::Normalize("a", "b/c")); EXPECT_EQ(".", IncludesNormalize::Normalize("a", "a")); } TEST(IncludesNormalize, Case) { EXPECT_EQ("b", IncludesNormalize::Normalize("Abc\\..\\b", NULL)); EXPECT_EQ("BdEf", IncludesNormalize::Normalize("Abc\\..\\BdEf", NULL)); EXPECT_EQ("A\\b", IncludesNormalize::Normalize("A\\.\\b", NULL)); EXPECT_EQ("a\\b", IncludesNormalize::Normalize("a\\./b", NULL)); EXPECT_EQ("A\\B", IncludesNormalize::Normalize("A\\.\\B", NULL)); EXPECT_EQ("A\\B", IncludesNormalize::Normalize("A\\./B", NULL)); } TEST(IncludesNormalize, Join) { vector x; EXPECT_EQ("", IncludesNormalize::Join(x, ':')); x.push_back("alpha"); EXPECT_EQ("alpha", IncludesNormalize::Join(x, ':')); x.push_back("beta"); x.push_back("gamma"); EXPECT_EQ("alpha:beta:gamma", IncludesNormalize::Join(x, ':')); } TEST(IncludesNormalize, Split) { EXPECT_EQ("", IncludesNormalize::Join(IncludesNormalize::Split("", '/'), ':')); EXPECT_EQ("a", IncludesNormalize::Join(IncludesNormalize::Split("a", '/'), ':')); EXPECT_EQ("a:b:c", IncludesNormalize::Join( IncludesNormalize::Split("a/b/c", '/'), ':')); } TEST(IncludesNormalize, ToLower) { EXPECT_EQ("", IncludesNormalize::ToLower("")); EXPECT_EQ("stuff", IncludesNormalize::ToLower("Stuff")); EXPECT_EQ("stuff and things", IncludesNormalize::ToLower("Stuff AND thINGS")); EXPECT_EQ("stuff 3and thin43gs", IncludesNormalize::ToLower("Stuff 3AND thIN43GS")); } TEST(IncludesNormalize, DifferentDrive) { EXPECT_EQ("stuff.h", IncludesNormalize::Normalize("p:\\vs08\\stuff.h", "p:\\vs08")); EXPECT_EQ("stuff.h", IncludesNormalize::Normalize("P:\\Vs08\\stuff.h", "p:\\vs08")); EXPECT_EQ("p:\\vs08\\stuff.h", IncludesNormalize::Normalize("p:\\vs08\\stuff.h", "c:\\vs08")); EXPECT_EQ("P:\\vs08\\stufF.h", IncludesNormalize::Normalize("P:\\vs08\\stufF.h", "D:\\stuff/things")); EXPECT_EQ("P:\\vs08\\stuff.h", IncludesNormalize::Normalize("P:/vs08\\stuff.h", "D:\\stuff/things")); // TODO: this fails; fix it. //EXPECT_EQ("P:\\wee\\stuff.h", // IncludesNormalize::Normalize("P:/vs08\\../wee\\stuff.h", "D:\\stuff/things")); } ninja-1.3.4/src/inline.sh000077500000000000000000000016011215343310300152130ustar00rootroot00000000000000#!/bin/sh # # Copyright 2001 Google Inc. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # This quick script converts a text file into an #include-able header. # It expects the name of the variable as its first argument, and reads # stdin and writes stdout. varname="$1" echo "const char $varname[] =" od -t x1 -A n -v | sed -e 's| ||g; s|..|\\x&|g; s|^|"|; s|$|"|' echo ";" ninja-1.3.4/src/lexer.cc000066400000000000000000000420701215343310300150310ustar00rootroot00000000000000/* Generated by re2c 0.13.5 */ // Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "lexer.h" #include #include "eval_env.h" #include "util.h" bool Lexer::Error(const string& message, string* err) { // Compute line/column. int line = 1; const char* context = input_.str_; for (const char* p = input_.str_; p < last_token_; ++p) { if (*p == '\n') { ++line; context = p + 1; } } int col = last_token_ ? (int)(last_token_ - context) : 0; char buf[1024]; snprintf(buf, sizeof(buf), "%s:%d: ", filename_.AsString().c_str(), line); *err = buf; *err += message + "\n"; // Add some context to the message. const int kTruncateColumn = 72; if (col > 0 && col < kTruncateColumn) { int len; bool truncated = true; for (len = 0; len < kTruncateColumn; ++len) { if (context[len] == 0 || context[len] == '\n') { truncated = false; break; } } *err += string(context, len); if (truncated) *err += "..."; *err += "\n"; *err += string(col, ' '); *err += "^ near here"; } return false; } Lexer::Lexer(const char* input) { Start("input", input); } void Lexer::Start(StringPiece filename, StringPiece input) { filename_ = filename; input_ = input; ofs_ = input_.str_; last_token_ = NULL; } const char* Lexer::TokenName(Token t) { switch (t) { case ERROR: return "lexing error"; case BUILD: return "'build'"; case COLON: return "':'"; case DEFAULT: return "'default'"; case EQUALS: return "'='"; case IDENT: return "identifier"; case INCLUDE: return "'include'"; case INDENT: return "indent"; case NEWLINE: return "newline"; case PIPE2: return "'||'"; case PIPE: return "'|'"; case POOL: return "'pool'"; case RULE: return "'rule'"; case SUBNINJA: return "'subninja'"; case TEOF: return "eof"; } return NULL; // not reached } const char* Lexer::TokenErrorHint(Token expected) { switch (expected) { case COLON: return " ($ also escapes ':')"; default: return ""; } } string Lexer::DescribeLastError() { if (last_token_) { switch (last_token_[0]) { case '\r': return "carriage returns are not allowed, use newlines"; case '\t': return "tabs are not allowed, use spaces"; } } return "lexing error"; } void Lexer::UnreadToken() { ofs_ = last_token_; } Lexer::Token Lexer::ReadToken() { const char* p = ofs_; const char* q; const char* start; Lexer::Token token; for (;;) { start = p; { unsigned char yych; unsigned int yyaccept = 0; static const unsigned char yybm[] = { 0, 64, 64, 64, 64, 64, 64, 64, 64, 64, 0, 64, 64, 0, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 192, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 96, 96, 64, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 64, 64, 64, 64, 64, 64, 64, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 64, 64, 64, 64, 96, 64, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 96, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, 64, }; yych = *p; if (yych <= '^') { if (yych <= ',') { if (yych <= 0x1F) { if (yych <= 0x00) goto yy22; if (yych == '\n') goto yy6; goto yy24; } else { if (yych <= ' ') goto yy2; if (yych == '#') goto yy4; goto yy24; } } else { if (yych <= ':') { if (yych == '/') goto yy24; if (yych <= '9') goto yy21; goto yy15; } else { if (yych <= '=') { if (yych <= '<') goto yy24; goto yy13; } else { if (yych <= '@') goto yy24; if (yych <= 'Z') goto yy21; goto yy24; } } } } else { if (yych <= 'i') { if (yych <= 'b') { if (yych == '`') goto yy24; if (yych <= 'a') goto yy21; goto yy8; } else { if (yych == 'd') goto yy12; if (yych <= 'h') goto yy21; goto yy19; } } else { if (yych <= 'r') { if (yych == 'p') goto yy10; if (yych <= 'q') goto yy21; goto yy11; } else { if (yych <= 'z') { if (yych <= 's') goto yy20; goto yy21; } else { if (yych == '|') goto yy17; goto yy24; } } } } yy2: yyaccept = 0; yych = *(q = ++p); goto yy70; yy3: { token = INDENT; break; } yy4: yyaccept = 1; yych = *(q = ++p); if (yych <= 0x00) goto yy5; if (yych != '\r') goto yy65; yy5: { token = ERROR; break; } yy6: ++p; yy7: { token = NEWLINE; break; } yy8: ++p; if ((yych = *p) == 'u') goto yy59; goto yy26; yy9: { token = IDENT; break; } yy10: yych = *++p; if (yych == 'o') goto yy55; goto yy26; yy11: yych = *++p; if (yych == 'u') goto yy51; goto yy26; yy12: yych = *++p; if (yych == 'e') goto yy44; goto yy26; yy13: ++p; { token = EQUALS; break; } yy15: ++p; { token = COLON; break; } yy17: ++p; if ((yych = *p) == '|') goto yy42; { token = PIPE; break; } yy19: yych = *++p; if (yych == 'n') goto yy35; goto yy26; yy20: yych = *++p; if (yych == 'u') goto yy27; goto yy26; yy21: yych = *++p; goto yy26; yy22: ++p; { token = TEOF; break; } yy24: yych = *++p; goto yy5; yy25: ++p; yych = *p; yy26: if (yybm[0+yych] & 32) { goto yy25; } goto yy9; yy27: yych = *++p; if (yych != 'b') goto yy26; yych = *++p; if (yych != 'n') goto yy26; yych = *++p; if (yych != 'i') goto yy26; yych = *++p; if (yych != 'n') goto yy26; yych = *++p; if (yych != 'j') goto yy26; yych = *++p; if (yych != 'a') goto yy26; ++p; if (yybm[0+(yych = *p)] & 32) { goto yy25; } { token = SUBNINJA; break; } yy35: yych = *++p; if (yych != 'c') goto yy26; yych = *++p; if (yych != 'l') goto yy26; yych = *++p; if (yych != 'u') goto yy26; yych = *++p; if (yych != 'd') goto yy26; yych = *++p; if (yych != 'e') goto yy26; ++p; if (yybm[0+(yych = *p)] & 32) { goto yy25; } { token = INCLUDE; break; } yy42: ++p; { token = PIPE2; break; } yy44: yych = *++p; if (yych != 'f') goto yy26; yych = *++p; if (yych != 'a') goto yy26; yych = *++p; if (yych != 'u') goto yy26; yych = *++p; if (yych != 'l') goto yy26; yych = *++p; if (yych != 't') goto yy26; ++p; if (yybm[0+(yych = *p)] & 32) { goto yy25; } { token = DEFAULT; break; } yy51: yych = *++p; if (yych != 'l') goto yy26; yych = *++p; if (yych != 'e') goto yy26; ++p; if (yybm[0+(yych = *p)] & 32) { goto yy25; } { token = RULE; break; } yy55: yych = *++p; if (yych != 'o') goto yy26; yych = *++p; if (yych != 'l') goto yy26; ++p; if (yybm[0+(yych = *p)] & 32) { goto yy25; } { token = POOL; break; } yy59: yych = *++p; if (yych != 'i') goto yy26; yych = *++p; if (yych != 'l') goto yy26; yych = *++p; if (yych != 'd') goto yy26; ++p; if (yybm[0+(yych = *p)] & 32) { goto yy25; } { token = BUILD; break; } yy64: ++p; yych = *p; yy65: if (yybm[0+yych] & 64) { goto yy64; } if (yych <= 0x00) goto yy66; if (yych <= '\f') goto yy67; yy66: p = q; if (yyaccept <= 0) { goto yy3; } else { goto yy5; } yy67: ++p; { continue; } yy69: yyaccept = 0; q = ++p; yych = *p; yy70: if (yybm[0+yych] & 128) { goto yy69; } if (yych == '\n') goto yy71; if (yych == '#') goto yy64; goto yy3; yy71: ++p; yych = *p; goto yy7; } } last_token_ = start; ofs_ = p; if (token != NEWLINE && token != TEOF) EatWhitespace(); return token; } bool Lexer::PeekToken(Token token) { Token t = ReadToken(); if (t == token) return true; UnreadToken(); return false; } void Lexer::EatWhitespace() { const char* p = ofs_; for (;;) { ofs_ = p; { unsigned char yych; static const unsigned char yybm[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, }; yych = *p; if (yych <= ' ') { if (yych <= 0x00) goto yy78; if (yych <= 0x1F) goto yy80; } else { if (yych == '$') goto yy76; goto yy80; } ++p; yych = *p; goto yy84; yy75: { continue; } yy76: ++p; if ((yych = *p) == '\n') goto yy81; yy77: { break; } yy78: ++p; { break; } yy80: yych = *++p; goto yy77; yy81: ++p; { continue; } yy83: ++p; yych = *p; yy84: if (yybm[0+yych] & 128) { goto yy83; } goto yy75; } } } bool Lexer::ReadIdent(string* out) { const char* p = ofs_; for (;;) { const char* start = p; { unsigned char yych; static const unsigned char yybm[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128, 128, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 0, 0, 0, 0, 0, 0, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 0, 0, 0, 0, 128, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, }; yych = *p; if (yych <= '@') { if (yych <= '.') { if (yych <= ',') goto yy89; } else { if (yych <= '/') goto yy89; if (yych >= ':') goto yy89; } } else { if (yych <= '_') { if (yych <= 'Z') goto yy87; if (yych <= '^') goto yy89; } else { if (yych <= '`') goto yy89; if (yych >= '{') goto yy89; } } yy87: ++p; yych = *p; goto yy92; yy88: { out->assign(start, p - start); break; } yy89: ++p; { return false; } yy91: ++p; yych = *p; yy92: if (yybm[0+yych] & 128) { goto yy91; } goto yy88; } } ofs_ = p; EatWhitespace(); return true; } bool Lexer::ReadEvalString(EvalString* eval, bool path, string* err) { const char* p = ofs_; const char* q; const char* start; for (;;) { start = p; { unsigned char yych; static const unsigned char yybm[] = { 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 0, 128, 128, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 16, 128, 128, 128, 0, 128, 128, 128, 128, 128, 128, 128, 128, 224, 160, 128, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 0, 128, 128, 128, 128, 128, 128, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 128, 128, 128, 128, 224, 128, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 224, 128, 0, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, 128, }; yych = *p; if (yych <= ' ') { if (yych <= '\n') { if (yych <= 0x00) goto yy101; if (yych >= '\n') goto yy97; } else { if (yych == '\r') goto yy103; if (yych >= ' ') goto yy97; } } else { if (yych <= '9') { if (yych == '$') goto yy99; } else { if (yych <= ':') goto yy97; if (yych == '|') goto yy97; } } ++p; yych = *p; goto yy126; yy96: { eval->AddText(StringPiece(start, p - start)); continue; } yy97: ++p; { if (path) { p = start; break; } else { if (*start == '\n') break; eval->AddText(StringPiece(start, 1)); continue; } } yy99: ++p; if ((yych = *p) <= '/') { if (yych <= ' ') { if (yych == '\n') goto yy115; if (yych <= 0x1F) goto yy104; goto yy106; } else { if (yych <= '$') { if (yych <= '#') goto yy104; goto yy108; } else { if (yych == '-') goto yy110; goto yy104; } } } else { if (yych <= '^') { if (yych <= ':') { if (yych <= '9') goto yy110; goto yy112; } else { if (yych <= '@') goto yy104; if (yych <= 'Z') goto yy110; goto yy104; } } else { if (yych <= '`') { if (yych <= '_') goto yy110; goto yy104; } else { if (yych <= 'z') goto yy110; if (yych <= '{') goto yy114; goto yy104; } } } yy100: { last_token_ = start; return Error(DescribeLastError(), err); } yy101: ++p; { last_token_ = start; return Error("unexpected EOF", err); } yy103: yych = *++p; goto yy100; yy104: ++p; yy105: { last_token_ = start; return Error("bad $-escape (literal $ must be written as $$)", err); } yy106: ++p; { eval->AddText(StringPiece(" ", 1)); continue; } yy108: ++p; { eval->AddText(StringPiece("$", 1)); continue; } yy110: ++p; yych = *p; goto yy124; yy111: { eval->AddSpecial(StringPiece(start + 1, p - start - 1)); continue; } yy112: ++p; { eval->AddText(StringPiece(":", 1)); continue; } yy114: yych = *(q = ++p); if (yybm[0+yych] & 32) { goto yy118; } goto yy105; yy115: ++p; yych = *p; if (yybm[0+yych] & 16) { goto yy115; } { continue; } yy118: ++p; yych = *p; if (yybm[0+yych] & 32) { goto yy118; } if (yych == '}') goto yy121; p = q; goto yy105; yy121: ++p; { eval->AddSpecial(StringPiece(start + 2, p - start - 3)); continue; } yy123: ++p; yych = *p; yy124: if (yybm[0+yych] & 64) { goto yy123; } goto yy111; yy125: ++p; yych = *p; yy126: if (yybm[0+yych] & 128) { goto yy125; } goto yy96; } } last_token_ = start; ofs_ = p; if (path) EatWhitespace(); // Non-path strings end in newlines, so there's no whitespace to eat. return true; } ninja-1.3.4/src/lexer.h000066400000000000000000000052721215343310300146760ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_LEXER_H_ #define NINJA_LEXER_H_ #include "string_piece.h" // Windows may #define ERROR. #ifdef ERROR #undef ERROR #endif struct EvalString; struct Lexer { Lexer() {} /// Helper ctor useful for tests. explicit Lexer(const char* input); enum Token { ERROR, BUILD, COLON, DEFAULT, EQUALS, IDENT, INCLUDE, INDENT, NEWLINE, PIPE, PIPE2, POOL, RULE, SUBNINJA, TEOF, }; /// Return a human-readable form of a token, used in error messages. static const char* TokenName(Token t); /// Return a human-readable token hint, used in error messages. static const char* TokenErrorHint(Token expected); /// If the last token read was an ERROR token, provide more info /// or the empty string. string DescribeLastError(); /// Start parsing some input. void Start(StringPiece filename, StringPiece input); /// Read a Token from the Token enum. Token ReadToken(); /// Rewind to the last read Token. void UnreadToken(); /// If the next token is \a token, read it and return true. bool PeekToken(Token token); /// Read a simple identifier (a rule or variable name). /// Returns false if a name can't be read. bool ReadIdent(string* out); /// Read a path (complete with $escapes). /// Returns false only on error, returned path may be empty if a delimiter /// (space, newline) is hit. bool ReadPath(EvalString* path, string* err) { return ReadEvalString(path, true, err); } /// Read the value side of a var = value line (complete with $escapes). /// Returns false only on error. bool ReadVarValue(EvalString* value, string* err) { return ReadEvalString(value, false, err); } /// Construct an error message with context. bool Error(const string& message, string* err); private: /// Skip past whitespace (called after each read token/ident/etc.). void EatWhitespace(); /// Read a $-escaped string. bool ReadEvalString(EvalString* eval, bool path, string* err); StringPiece filename_; StringPiece input_; const char* ofs_; const char* last_token_; }; #endif // NINJA_LEXER_H_ ninja-1.3.4/src/lexer.in.cc000066400000000000000000000140201215343310300154300ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "lexer.h" #include #include "eval_env.h" #include "util.h" bool Lexer::Error(const string& message, string* err) { // Compute line/column. int line = 1; const char* context = input_.str_; for (const char* p = input_.str_; p < last_token_; ++p) { if (*p == '\n') { ++line; context = p + 1; } } int col = last_token_ ? (int)(last_token_ - context) : 0; char buf[1024]; snprintf(buf, sizeof(buf), "%s:%d: ", filename_.AsString().c_str(), line); *err = buf; *err += message + "\n"; // Add some context to the message. const int kTruncateColumn = 72; if (col > 0 && col < kTruncateColumn) { int len; bool truncated = true; for (len = 0; len < kTruncateColumn; ++len) { if (context[len] == 0 || context[len] == '\n') { truncated = false; break; } } *err += string(context, len); if (truncated) *err += "..."; *err += "\n"; *err += string(col, ' '); *err += "^ near here"; } return false; } Lexer::Lexer(const char* input) { Start("input", input); } void Lexer::Start(StringPiece filename, StringPiece input) { filename_ = filename; input_ = input; ofs_ = input_.str_; last_token_ = NULL; } const char* Lexer::TokenName(Token t) { switch (t) { case ERROR: return "lexing error"; case BUILD: return "'build'"; case COLON: return "':'"; case DEFAULT: return "'default'"; case EQUALS: return "'='"; case IDENT: return "identifier"; case INCLUDE: return "'include'"; case INDENT: return "indent"; case NEWLINE: return "newline"; case PIPE2: return "'||'"; case PIPE: return "'|'"; case POOL: return "'pool'"; case RULE: return "'rule'"; case SUBNINJA: return "'subninja'"; case TEOF: return "eof"; } return NULL; // not reached } const char* Lexer::TokenErrorHint(Token expected) { switch (expected) { case COLON: return " ($ also escapes ':')"; default: return ""; } } string Lexer::DescribeLastError() { if (last_token_) { switch (last_token_[0]) { case '\r': return "carriage returns are not allowed, use newlines"; case '\t': return "tabs are not allowed, use spaces"; } } return "lexing error"; } void Lexer::UnreadToken() { ofs_ = last_token_; } Lexer::Token Lexer::ReadToken() { const char* p = ofs_; const char* q; const char* start; Lexer::Token token; for (;;) { start = p; /*!re2c re2c:define:YYCTYPE = "unsigned char"; re2c:define:YYCURSOR = p; re2c:define:YYMARKER = q; re2c:yyfill:enable = 0; nul = "\000"; simple_varname = [a-zA-Z0-9_-]+; varname = [a-zA-Z0-9_.-]+; [ ]*"#"[^\000\r\n]*"\n" { continue; } [ ]*[\n] { token = NEWLINE; break; } [ ]+ { token = INDENT; break; } "build" { token = BUILD; break; } "pool" { token = POOL; break; } "rule" { token = RULE; break; } "default" { token = DEFAULT; break; } "=" { token = EQUALS; break; } ":" { token = COLON; break; } "||" { token = PIPE2; break; } "|" { token = PIPE; break; } "include" { token = INCLUDE; break; } "subninja" { token = SUBNINJA; break; } varname { token = IDENT; break; } nul { token = TEOF; break; } [^] { token = ERROR; break; } */ } last_token_ = start; ofs_ = p; if (token != NEWLINE && token != TEOF) EatWhitespace(); return token; } bool Lexer::PeekToken(Token token) { Token t = ReadToken(); if (t == token) return true; UnreadToken(); return false; } void Lexer::EatWhitespace() { const char* p = ofs_; for (;;) { ofs_ = p; /*!re2c [ ]+ { continue; } "$\n" { continue; } nul { break; } [^] { break; } */ } } bool Lexer::ReadIdent(string* out) { const char* p = ofs_; for (;;) { const char* start = p; /*!re2c varname { out->assign(start, p - start); break; } [^] { return false; } */ } ofs_ = p; EatWhitespace(); return true; } bool Lexer::ReadEvalString(EvalString* eval, bool path, string* err) { const char* p = ofs_; const char* q; const char* start; for (;;) { start = p; /*!re2c [^$ :\r\n|\000]+ { eval->AddText(StringPiece(start, p - start)); continue; } [ :|\n] { if (path) { p = start; break; } else { if (*start == '\n') break; eval->AddText(StringPiece(start, 1)); continue; } } "$$" { eval->AddText(StringPiece("$", 1)); continue; } "$ " { eval->AddText(StringPiece(" ", 1)); continue; } "$\n"[ ]* { continue; } "${"varname"}" { eval->AddSpecial(StringPiece(start + 2, p - start - 3)); continue; } "$"simple_varname { eval->AddSpecial(StringPiece(start + 1, p - start - 1)); continue; } "$:" { eval->AddText(StringPiece(":", 1)); continue; } "$". { last_token_ = start; return Error("bad $-escape (literal $ must be written as $$)", err); } nul { last_token_ = start; return Error("unexpected EOF", err); } [^] { last_token_ = start; return Error(DescribeLastError(), err); } */ } last_token_ = start; ofs_ = p; if (path) EatWhitespace(); // Non-path strings end in newlines, so there's no whitespace to eat. return true; } ninja-1.3.4/src/lexer_test.cc000066400000000000000000000054121215343310300160670ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "lexer.h" #include #include "eval_env.h" TEST(Lexer, ReadVarValue) { Lexer lexer("plain text $var $VaR ${x}\n"); EvalString eval; string err; EXPECT_TRUE(lexer.ReadVarValue(&eval, &err)); EXPECT_EQ("", err); EXPECT_EQ("[plain text ][$var][ ][$VaR][ ][$x]", eval.Serialize()); } TEST(Lexer, ReadEvalStringEscapes) { Lexer lexer("$ $$ab c$: $\ncde\n"); EvalString eval; string err; EXPECT_TRUE(lexer.ReadVarValue(&eval, &err)); EXPECT_EQ("", err); EXPECT_EQ("[ $ab c: cde]", eval.Serialize()); } TEST(Lexer, ReadIdent) { Lexer lexer("foo baR baz_123 foo-bar"); string ident; EXPECT_TRUE(lexer.ReadIdent(&ident)); EXPECT_EQ("foo", ident); EXPECT_TRUE(lexer.ReadIdent(&ident)); EXPECT_EQ("baR", ident); EXPECT_TRUE(lexer.ReadIdent(&ident)); EXPECT_EQ("baz_123", ident); EXPECT_TRUE(lexer.ReadIdent(&ident)); EXPECT_EQ("foo-bar", ident); } TEST(Lexer, ReadIdentCurlies) { // Verify that ReadIdent includes dots in the name, // but in an expansion $bar.dots stops at the dot. Lexer lexer("foo.dots $bar.dots ${bar.dots}\n"); string ident; EXPECT_TRUE(lexer.ReadIdent(&ident)); EXPECT_EQ("foo.dots", ident); EvalString eval; string err; EXPECT_TRUE(lexer.ReadVarValue(&eval, &err)); EXPECT_EQ("", err); EXPECT_EQ("[$bar][.dots ][$bar.dots]", eval.Serialize()); } TEST(Lexer, Error) { Lexer lexer("foo$\nbad $"); EvalString eval; string err; ASSERT_FALSE(lexer.ReadVarValue(&eval, &err)); EXPECT_EQ("input:2: bad $-escape (literal $ must be written as $$)\n" "bad $\n" " ^ near here" , err); } TEST(Lexer, CommentEOF) { // Verify we don't run off the end of the string when the EOF is // mid-comment. Lexer lexer("# foo"); Lexer::Token token = lexer.ReadToken(); EXPECT_EQ(Lexer::ERROR, token); } TEST(Lexer, Tabs) { // Verify we print a useful error on a disallowed character. Lexer lexer(" \tfoobar"); Lexer::Token token = lexer.ReadToken(); EXPECT_EQ(Lexer::INDENT, token); token = lexer.ReadToken(); EXPECT_EQ(Lexer::ERROR, token); EXPECT_EQ("tabs are not allowed, use spaces", lexer.DescribeLastError()); } ninja-1.3.4/src/line_printer.cc000066400000000000000000000070311215343310300164020ustar00rootroot00000000000000// Copyright 2013 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "line_printer.h" #include #include #ifdef _WIN32 #include #else #include #include #include #endif #include "util.h" LinePrinter::LinePrinter() : have_blank_line_(true) { #ifndef _WIN32 const char* term = getenv("TERM"); smart_terminal_ = isatty(1) && term && string(term) != "dumb"; #else // Disable output buffer. It'd be nice to use line buffering but // MSDN says: "For some systems, [_IOLBF] provides line // buffering. However, for Win32, the behavior is the same as _IOFBF // - Full Buffering." setvbuf(stdout, NULL, _IONBF, 0); console_ = GetStdHandle(STD_OUTPUT_HANDLE); CONSOLE_SCREEN_BUFFER_INFO csbi; smart_terminal_ = GetConsoleScreenBufferInfo(console_, &csbi); #endif } void LinePrinter::Print(string to_print, LineType type) { #ifdef _WIN32 CONSOLE_SCREEN_BUFFER_INFO csbi; GetConsoleScreenBufferInfo(console_, &csbi); #endif if (smart_terminal_) { #ifndef _WIN32 printf("\r"); // Print over previous line, if any. #else csbi.dwCursorPosition.X = 0; SetConsoleCursorPosition(console_, csbi.dwCursorPosition); #endif } if (smart_terminal_ && type == ELIDE) { #ifdef _WIN32 // Don't use the full width or console will move to next line. size_t width = static_cast(csbi.dwSize.X) - 1; to_print = ElideMiddle(to_print, width); // We don't want to have the cursor spamming back and forth, so // use WriteConsoleOutput instead which updates the contents of // the buffer, but doesn't move the cursor position. GetConsoleScreenBufferInfo(console_, &csbi); COORD buf_size = { csbi.dwSize.X, 1 }; COORD zero_zero = { 0, 0 }; SMALL_RECT target = { csbi.dwCursorPosition.X, csbi.dwCursorPosition.Y, static_cast(csbi.dwCursorPosition.X + csbi.dwSize.X - 1), csbi.dwCursorPosition.Y }; CHAR_INFO* char_data = new CHAR_INFO[csbi.dwSize.X]; memset(char_data, 0, sizeof(CHAR_INFO) * csbi.dwSize.X); for (int i = 0; i < csbi.dwSize.X; ++i) { char_data[i].Char.AsciiChar = ' '; char_data[i].Attributes = csbi.wAttributes; } for (size_t i = 0; i < to_print.size(); ++i) char_data[i].Char.AsciiChar = to_print[i]; WriteConsoleOutput(console_, char_data, buf_size, zero_zero, &target); delete[] char_data; #else // Limit output to width of the terminal if provided so we don't cause // line-wrapping. winsize size; if ((ioctl(0, TIOCGWINSZ, &size) == 0) && size.ws_col) { to_print = ElideMiddle(to_print, size.ws_col); } printf("%s", to_print.c_str()); printf("\x1B[K"); // Clear to end of line. fflush(stdout); #endif have_blank_line_ = false; } else { printf("%s\n", to_print.c_str()); } } void LinePrinter::PrintOnNewLine(const string& to_print) { if (!have_blank_line_) printf("\n"); printf("%s", to_print.c_str()); have_blank_line_ = to_print.empty() || *to_print.rbegin() == '\n'; } ninja-1.3.4/src/line_printer.h000066400000000000000000000030051215343310300162410ustar00rootroot00000000000000// Copyright 2013 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_LINE_PRINTER_H_ #define NINJA_LINE_PRINTER_H_ #include using namespace std; /// Prints lines of text, possibly overprinting previously printed lines /// if the terminal supports it. struct LinePrinter { LinePrinter(); bool is_smart_terminal() const { return smart_terminal_; } void set_smart_terminal(bool smart) { smart_terminal_ = smart; } enum LineType { FULL, ELIDE }; /// Overprints the current line. If type is ELIDE, elides to_print to fit on /// one line. void Print(string to_print, LineType type); /// Prints a string on a new line, not overprinting previous output. void PrintOnNewLine(const string& to_print); private: /// Whether we can do fancy terminal control codes. bool smart_terminal_; /// Whether the caret is at the beginning of a blank line. bool have_blank_line_; #ifdef _WIN32 void* console_; #endif }; #endif // NINJA_LINE_PRINTER_H_ ninja-1.3.4/src/manifest_parser.cc000066400000000000000000000232531215343310300170760ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "manifest_parser.h" #include #include #include "graph.h" #include "metrics.h" #include "state.h" #include "util.h" #include "version.h" ManifestParser::ManifestParser(State* state, FileReader* file_reader) : state_(state), file_reader_(file_reader) { env_ = &state->bindings_; } bool ManifestParser::Load(const string& filename, string* err) { string contents; string read_err; if (!file_reader_->ReadFile(filename, &contents, &read_err)) { *err = "loading '" + filename + "': " + read_err; return false; } contents.resize(contents.size() + 10); return Parse(filename, contents, err); } bool ManifestParser::Parse(const string& filename, const string& input, string* err) { METRIC_RECORD(".ninja parse"); lexer_.Start(filename, input); for (;;) { Lexer::Token token = lexer_.ReadToken(); switch (token) { case Lexer::POOL: if (!ParsePool(err)) return false; break; case Lexer::BUILD: if (!ParseEdge(err)) return false; break; case Lexer::RULE: if (!ParseRule(err)) return false; break; case Lexer::DEFAULT: if (!ParseDefault(err)) return false; break; case Lexer::IDENT: { lexer_.UnreadToken(); string name; EvalString let_value; if (!ParseLet(&name, &let_value, err)) return false; string value = let_value.Evaluate(env_); // Check ninja_required_version immediately so we can exit // before encountering any syntactic surprises. if (name == "ninja_required_version") CheckNinjaVersion(value); env_->AddBinding(name, value); break; } case Lexer::INCLUDE: if (!ParseFileInclude(false, err)) return false; break; case Lexer::SUBNINJA: if (!ParseFileInclude(true, err)) return false; break; case Lexer::ERROR: { return lexer_.Error(lexer_.DescribeLastError(), err); } case Lexer::TEOF: return true; case Lexer::NEWLINE: break; default: return lexer_.Error(string("unexpected ") + Lexer::TokenName(token), err); } } return false; // not reached } bool ManifestParser::ParsePool(string* err) { string name; if (!lexer_.ReadIdent(&name)) return lexer_.Error("expected pool name", err); if (!ExpectToken(Lexer::NEWLINE, err)) return false; if (state_->LookupPool(name) != NULL) return lexer_.Error("duplicate pool '" + name + "'", err); int depth = -1; while (lexer_.PeekToken(Lexer::INDENT)) { string key; EvalString value; if (!ParseLet(&key, &value, err)) return false; if (key == "depth") { string depth_string = value.Evaluate(env_); depth = atol(depth_string.c_str()); if (depth < 0) return lexer_.Error("invalid pool depth", err); } else { return lexer_.Error("unexpected variable '" + key + "'", err); } } if (depth < 0) return lexer_.Error("expected 'depth =' line", err); state_->AddPool(new Pool(name, depth)); return true; } bool ManifestParser::ParseRule(string* err) { string name; if (!lexer_.ReadIdent(&name)) return lexer_.Error("expected rule name", err); if (!ExpectToken(Lexer::NEWLINE, err)) return false; if (state_->LookupRule(name) != NULL) { *err = "duplicate rule '" + name + "'"; return false; } Rule* rule = new Rule(name); // XXX scoped_ptr while (lexer_.PeekToken(Lexer::INDENT)) { string key; EvalString value; if (!ParseLet(&key, &value, err)) return false; if (Rule::IsReservedBinding(key)) { rule->AddBinding(key, value); } else { // Die on other keyvals for now; revisit if we want to add a // scope here. return lexer_.Error("unexpected variable '" + key + "'", err); } } if (rule->bindings_["rspfile"].empty() != rule->bindings_["rspfile_content"].empty()) { return lexer_.Error("rspfile and rspfile_content need to be " "both specified", err); } if (rule->bindings_["command"].empty()) return lexer_.Error("expected 'command =' line", err); state_->AddRule(rule); return true; } bool ManifestParser::ParseLet(string* key, EvalString* value, string* err) { if (!lexer_.ReadIdent(key)) return false; if (!ExpectToken(Lexer::EQUALS, err)) return false; if (!lexer_.ReadVarValue(value, err)) return false; return true; } bool ManifestParser::ParseDefault(string* err) { EvalString eval; if (!lexer_.ReadPath(&eval, err)) return false; if (eval.empty()) return lexer_.Error("expected target name", err); do { string path = eval.Evaluate(env_); string path_err; if (!CanonicalizePath(&path, &path_err)) return lexer_.Error(path_err, err); if (!state_->AddDefault(path, &path_err)) return lexer_.Error(path_err, err); eval.Clear(); if (!lexer_.ReadPath(&eval, err)) return false; } while (!eval.empty()); if (!ExpectToken(Lexer::NEWLINE, err)) return false; return true; } bool ManifestParser::ParseEdge(string* err) { vector ins, outs; { EvalString out; if (!lexer_.ReadPath(&out, err)) return false; if (out.empty()) return lexer_.Error("expected path", err); do { outs.push_back(out); out.Clear(); if (!lexer_.ReadPath(&out, err)) return false; } while (!out.empty()); } if (!ExpectToken(Lexer::COLON, err)) return false; string rule_name; if (!lexer_.ReadIdent(&rule_name)) return lexer_.Error("expected build command name", err); const Rule* rule = state_->LookupRule(rule_name); if (!rule) return lexer_.Error("unknown build rule '" + rule_name + "'", err); for (;;) { // XXX should we require one path here? EvalString in; if (!lexer_.ReadPath(&in, err)) return false; if (in.empty()) break; ins.push_back(in); } // Add all implicit deps, counting how many as we go. int implicit = 0; if (lexer_.PeekToken(Lexer::PIPE)) { for (;;) { EvalString in; if (!lexer_.ReadPath(&in, err)) return err; if (in.empty()) break; ins.push_back(in); ++implicit; } } // Add all order-only deps, counting how many as we go. int order_only = 0; if (lexer_.PeekToken(Lexer::PIPE2)) { for (;;) { EvalString in; if (!lexer_.ReadPath(&in, err)) return false; if (in.empty()) break; ins.push_back(in); ++order_only; } } if (!ExpectToken(Lexer::NEWLINE, err)) return false; // XXX scoped_ptr to handle error case. BindingEnv* env = new BindingEnv(env_); while (lexer_.PeekToken(Lexer::INDENT)) { string key; EvalString val; if (!ParseLet(&key, &val, err)) return false; env->AddBinding(key, val.Evaluate(env_)); } Edge* edge = state_->AddEdge(rule); edge->env_ = env; string pool_name = edge->GetBinding("pool"); if (!pool_name.empty()) { Pool* pool = state_->LookupPool(pool_name); if (pool == NULL) return lexer_.Error("unknown pool name", err); edge->pool_ = pool; } for (vector::iterator i = ins.begin(); i != ins.end(); ++i) { string path = i->Evaluate(env); string path_err; if (!CanonicalizePath(&path, &path_err)) return lexer_.Error(path_err, err); state_->AddIn(edge, path); } for (vector::iterator i = outs.begin(); i != outs.end(); ++i) { string path = i->Evaluate(env); string path_err; if (!CanonicalizePath(&path, &path_err)) return lexer_.Error(path_err, err); state_->AddOut(edge, path); } edge->implicit_deps_ = implicit; edge->order_only_deps_ = order_only; // Multiple outputs aren't (yet?) supported with depslog. string deps_type = edge->GetBinding("deps"); if (!deps_type.empty() && edge->outputs_.size() > 1) { return lexer_.Error("multiple outputs aren't (yet?) supported by depslog; " "bring this up on the mailing list if it affects you", err); } return true; } bool ManifestParser::ParseFileInclude(bool new_scope, string* err) { // XXX this should use ReadPath! EvalString eval; if (!lexer_.ReadPath(&eval, err)) return false; string path = eval.Evaluate(env_); string contents; string read_err; if (!file_reader_->ReadFile(path, &contents, &read_err)) return lexer_.Error("loading '" + path + "': " + read_err, err); ManifestParser subparser(state_, file_reader_); if (new_scope) { subparser.env_ = new BindingEnv(env_); } else { subparser.env_ = env_; } if (!subparser.Parse(path, contents, err)) return false; if (!ExpectToken(Lexer::NEWLINE, err)) return false; return true; } bool ManifestParser::ExpectToken(Lexer::Token expected, string* err) { Lexer::Token token = lexer_.ReadToken(); if (token != expected) { string message = string("expected ") + Lexer::TokenName(expected); message += string(", got ") + Lexer::TokenName(token); message += Lexer::TokenErrorHint(expected); return lexer_.Error(message, err); } return true; } ninja-1.3.4/src/manifest_parser.h000066400000000000000000000037351215343310300167430ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_MANIFEST_PARSER_H_ #define NINJA_MANIFEST_PARSER_H_ #include using namespace std; #include "lexer.h" struct BindingEnv; struct EvalString; struct State; /// Parses .ninja files. struct ManifestParser { struct FileReader { virtual ~FileReader() {} virtual bool ReadFile(const string& path, string* content, string* err) = 0; }; ManifestParser(State* state, FileReader* file_reader); /// Load and parse a file. bool Load(const string& filename, string* err); /// Parse a text string of input. Used by tests. bool ParseTest(const string& input, string* err) { return Parse("input", input, err); } private: /// Parse a file, given its contents as a string. bool Parse(const string& filename, const string& input, string* err); /// Parse various statement types. bool ParsePool(string* err); bool ParseRule(string* err); bool ParseLet(string* key, EvalString* val, string* err); bool ParseEdge(string* err); bool ParseDefault(string* err); /// Parse either a 'subninja' or 'include' line. bool ParseFileInclude(bool new_scope, string* err); /// If the next token is not \a expected, produce an error string /// saying "expectd foo, got bar". bool ExpectToken(Lexer::Token expected, string* err); State* state_; BindingEnv* env_; FileReader* file_reader_; Lexer lexer_; }; #endif // NINJA_MANIFEST_PARSER_H_ ninja-1.3.4/src/manifest_parser_test.cc000066400000000000000000000504651215343310300201420ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "manifest_parser.h" #include #include #include #include "graph.h" #include "state.h" struct ParserTest : public testing::Test, public ManifestParser::FileReader { void AssertParse(const char* input) { ManifestParser parser(&state, this); string err; ASSERT_TRUE(parser.ParseTest(input, &err)) << err; ASSERT_EQ("", err); } virtual bool ReadFile(const string& path, string* content, string* err) { files_read_.push_back(path); map::iterator i = files_.find(path); if (i == files_.end()) { *err = "No such file or directory"; // Match strerror() for ENOENT. return false; } *content = i->second; return true; } State state; map files_; vector files_read_; }; TEST_F(ParserTest, Empty) { ASSERT_NO_FATAL_FAILURE(AssertParse("")); } TEST_F(ParserTest, Rules) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n" " command = cat $in > $out\n" "\n" "rule date\n" " command = date > $out\n" "\n" "build result: cat in_1.cc in-2.O\n")); ASSERT_EQ(3u, state.rules_.size()); const Rule* rule = state.rules_.begin()->second; EXPECT_EQ("cat", rule->name()); EXPECT_EQ("[cat ][$in][ > ][$out]", rule->GetBinding("command")->Serialize()); } TEST_F(ParserTest, RuleAttributes) { // Check that all of the allowed rule attributes are parsed ok. ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n" " command = a\n" " depfile = a\n" " deps = a\n" " description = a\n" " generator = a\n" " restat = a\n" " rspfile = a\n" " rspfile_content = a\n" )); } TEST_F(ParserTest, IgnoreIndentedComments) { ASSERT_NO_FATAL_FAILURE(AssertParse( " #indented comment\n" "rule cat\n" " command = cat $in > $out\n" " #generator = 1\n" " restat = 1 # comment\n" " #comment\n" "build result: cat in_1.cc in-2.O\n" " #comment\n")); ASSERT_EQ(2u, state.rules_.size()); const Rule* rule = state.rules_.begin()->second; EXPECT_EQ("cat", rule->name()); Edge* edge = state.GetNode("result")->in_edge(); EXPECT_TRUE(edge->GetBindingBool("restat")); EXPECT_FALSE(edge->GetBindingBool("generator")); } TEST_F(ParserTest, IgnoreIndentedBlankLines) { // the indented blanks used to cause parse errors ASSERT_NO_FATAL_FAILURE(AssertParse( " \n" "rule cat\n" " command = cat $in > $out\n" " \n" "build result: cat in_1.cc in-2.O\n" " \n" "variable=1\n")); // the variable must be in the top level environment EXPECT_EQ("1", state.bindings_.LookupVariable("variable")); } TEST_F(ParserTest, ResponseFiles) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat_rsp\n" " command = cat $rspfile > $out\n" " rspfile = $rspfile\n" " rspfile_content = $in\n" "\n" "build out: cat_rsp in\n" " rspfile=out.rsp\n")); ASSERT_EQ(2u, state.rules_.size()); const Rule* rule = state.rules_.begin()->second; EXPECT_EQ("cat_rsp", rule->name()); EXPECT_EQ("[cat ][$rspfile][ > ][$out]", rule->GetBinding("command")->Serialize()); EXPECT_EQ("[$rspfile]", rule->GetBinding("rspfile")->Serialize()); EXPECT_EQ("[$in]", rule->GetBinding("rspfile_content")->Serialize()); } TEST_F(ParserTest, InNewline) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat_rsp\n" " command = cat $in_newline > $out\n" "\n" "build out: cat_rsp in in2\n" " rspfile=out.rsp\n")); ASSERT_EQ(2u, state.rules_.size()); const Rule* rule = state.rules_.begin()->second; EXPECT_EQ("cat_rsp", rule->name()); EXPECT_EQ("[cat ][$in_newline][ > ][$out]", rule->GetBinding("command")->Serialize()); Edge* edge = state.edges_[0]; EXPECT_EQ("cat in\nin2 > out", edge->EvaluateCommand()); } TEST_F(ParserTest, Variables) { ASSERT_NO_FATAL_FAILURE(AssertParse( "l = one-letter-test\n" "rule link\n" " command = ld $l $extra $with_under -o $out $in\n" "\n" "extra = -pthread\n" "with_under = -under\n" "build a: link b c\n" "nested1 = 1\n" "nested2 = $nested1/2\n" "build supernested: link x\n" " extra = $nested2/3\n")); ASSERT_EQ(2u, state.edges_.size()); Edge* edge = state.edges_[0]; EXPECT_EQ("ld one-letter-test -pthread -under -o a b c", edge->EvaluateCommand()); EXPECT_EQ("1/2", state.bindings_.LookupVariable("nested2")); edge = state.edges_[1]; EXPECT_EQ("ld one-letter-test 1/2/3 -under -o supernested x", edge->EvaluateCommand()); } TEST_F(ParserTest, VariableScope) { ASSERT_NO_FATAL_FAILURE(AssertParse( "foo = bar\n" "rule cmd\n" " command = cmd $foo $in $out\n" "\n" "build inner: cmd a\n" " foo = baz\n" "build outer: cmd b\n" "\n" // Extra newline after build line tickles a regression. )); ASSERT_EQ(2u, state.edges_.size()); EXPECT_EQ("cmd baz a inner", state.edges_[0]->EvaluateCommand()); EXPECT_EQ("cmd bar b outer", state.edges_[1]->EvaluateCommand()); } TEST_F(ParserTest, Continuation) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule link\n" " command = foo bar $\n" " baz\n" "\n" "build a: link c $\n" " d e f\n")); ASSERT_EQ(2u, state.rules_.size()); const Rule* rule = state.rules_.begin()->second; EXPECT_EQ("link", rule->name()); EXPECT_EQ("[foo bar baz]", rule->GetBinding("command")->Serialize()); } TEST_F(ParserTest, Backslash) { ASSERT_NO_FATAL_FAILURE(AssertParse( "foo = bar\\baz\n" "foo2 = bar\\ baz\n" )); EXPECT_EQ("bar\\baz", state.bindings_.LookupVariable("foo")); EXPECT_EQ("bar\\ baz", state.bindings_.LookupVariable("foo2")); } TEST_F(ParserTest, Comment) { ASSERT_NO_FATAL_FAILURE(AssertParse( "# this is a comment\n" "foo = not # a comment\n")); EXPECT_EQ("not # a comment", state.bindings_.LookupVariable("foo")); } TEST_F(ParserTest, Dollars) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule foo\n" " command = ${out}bar$$baz$$$\n" "blah\n" "x = $$dollar\n" "build $x: foo y\n" )); EXPECT_EQ("$dollar", state.bindings_.LookupVariable("x")); EXPECT_EQ("$dollarbar$baz$blah", state.edges_[0]->EvaluateCommand()); } TEST_F(ParserTest, EscapeSpaces) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule spaces\n" " command = something\n" "build foo$ bar: spaces $$one two$$$ three\n" )); EXPECT_TRUE(state.LookupNode("foo bar")); EXPECT_EQ(state.edges_[0]->outputs_[0]->path(), "foo bar"); EXPECT_EQ(state.edges_[0]->inputs_[0]->path(), "$one"); EXPECT_EQ(state.edges_[0]->inputs_[1]->path(), "two$ three"); EXPECT_EQ(state.edges_[0]->EvaluateCommand(), "something"); } TEST_F(ParserTest, CanonicalizeFile) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n" " command = cat $in > $out\n" "build out: cat in/1 in//2\n" "build in/1: cat\n" "build in/2: cat\n")); EXPECT_TRUE(state.LookupNode("in/1")); EXPECT_TRUE(state.LookupNode("in/2")); EXPECT_FALSE(state.LookupNode("in//1")); EXPECT_FALSE(state.LookupNode("in//2")); } TEST_F(ParserTest, PathVariables) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n" " command = cat $in > $out\n" "dir = out\n" "build $dir/exe: cat src\n")); EXPECT_FALSE(state.LookupNode("$dir/exe")); EXPECT_TRUE(state.LookupNode("out/exe")); } TEST_F(ParserTest, CanonicalizePaths) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n" " command = cat $in > $out\n" "build ./out.o: cat ./bar/baz/../foo.cc\n")); EXPECT_FALSE(state.LookupNode("./out.o")); EXPECT_TRUE(state.LookupNode("out.o")); EXPECT_FALSE(state.LookupNode("./bar/baz/../foo.cc")); EXPECT_TRUE(state.LookupNode("bar/foo.cc")); } TEST_F(ParserTest, ReservedWords) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule build\n" " command = rule run $out\n" "build subninja: build include default foo.cc\n" "default subninja\n")); } TEST_F(ParserTest, Errors) { { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("foobar", &err)); EXPECT_EQ("input:1: expected '=', got eof\n" "foobar\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("x 3", &err)); EXPECT_EQ("input:1: expected '=', got identifier\n" "x 3\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("x = 3", &err)); EXPECT_EQ("input:1: unexpected EOF\n" "x = 3\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("x = 3\ny 2", &err)); EXPECT_EQ("input:2: expected '=', got identifier\n" "y 2\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("x = $", &err)); EXPECT_EQ("input:1: bad $-escape (literal $ must be written as $$)\n" "x = $\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("x = $\n $[\n", &err)); EXPECT_EQ("input:2: bad $-escape (literal $ must be written as $$)\n" " $[\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("x = a$\n b$\n $\n", &err)); EXPECT_EQ("input:4: unexpected EOF\n" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("build x: y z\n", &err)); EXPECT_EQ("input:1: unknown build rule 'y'\n" "build x: y z\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("build x:: y z\n", &err)); EXPECT_EQ("input:1: expected build command name\n" "build x:: y z\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cat\n command = cat ok\n" "build x: cat $\n :\n", &err)); EXPECT_EQ("input:4: expected newline, got ':'\n" " :\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cat\n", &err)); EXPECT_EQ("input:2: expected 'command =' line\n", err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cat\n" " command = ${fafsd\n" "foo = bar\n", &err)); EXPECT_EQ("input:2: bad $-escape (literal $ must be written as $$)\n" " command = ${fafsd\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cat\n" " command = cat\n" "build $.: cat foo\n", &err)); EXPECT_EQ("input:3: bad $-escape (literal $ must be written as $$)\n" "build $.: cat foo\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cat\n" " command = cat\n" "build $: cat foo\n", &err)); EXPECT_EQ("input:3: expected ':', got newline ($ also escapes ':')\n" "build $: cat foo\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule %foo\n", &err)); EXPECT_EQ("input:1: expected rule name\n", err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cc\n" " command = foo\n" " othervar = bar\n", &err)); EXPECT_EQ("input:3: unexpected variable 'othervar'\n" " othervar = bar\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cc\n command = foo\n" "build $.: cc bar.cc\n", &err)); EXPECT_EQ("input:3: bad $-escape (literal $ must be written as $$)\n" "build $.: cc bar.cc\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cc\n command = foo\n" "build $: cc bar.cc\n", &err)); EXPECT_EQ("input:3: expected ':', got newline ($ also escapes ':')\n" "build $: cc bar.cc\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("default\n", &err)); EXPECT_EQ("input:1: expected target name\n" "default\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("default nonexistent\n", &err)); EXPECT_EQ("input:1: unknown target 'nonexistent'\n" "default nonexistent\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule r\n command = r\n" "build b: r\n" "default b:\n", &err)); EXPECT_EQ("input:4: expected newline, got ':'\n" "default b:\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("default $a\n", &err)); EXPECT_EQ("input:1: empty path\n" "default $a\n" " ^ near here" , err); } { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule r\n" " command = r\n" "build $a: r $c\n", &err)); // XXX the line number is wrong; we should evaluate paths in ParseEdge // as we see them, not after we've read them all! EXPECT_EQ("input:4: empty path\n", err); } { State state; ManifestParser parser(&state, NULL); string err; // the indented blank line must terminate the rule // this also verifies that "unexpected (token)" errors are correct EXPECT_FALSE(parser.ParseTest("rule r\n" " command = r\n" " \n" " generator = 1\n", &err)); EXPECT_EQ("input:4: unexpected indent\n", err); } } TEST_F(ParserTest, MissingInput) { State state; ManifestParser parser(&state, this); string err; EXPECT_FALSE(parser.Load("build.ninja", &err)); EXPECT_EQ("loading 'build.ninja': No such file or directory", err); } TEST_F(ParserTest, MultipleOutputs) { State state; ManifestParser parser(&state, NULL); string err; EXPECT_TRUE(parser.ParseTest("rule cc\n command = foo\n depfile = bar\n" "build a.o b.o: cc c.cc\n", &err)); EXPECT_EQ("", err); } TEST_F(ParserTest, MultipleOutputsWithDeps) { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("rule cc\n command = foo\n deps = gcc\n" "build a.o b.o: cc c.cc\n", &err)); EXPECT_EQ("input:5: multiple outputs aren't (yet?) supported by depslog; " "bring this up on the mailing list if it affects you\n", err); } TEST_F(ParserTest, SubNinja) { files_["test.ninja"] = "var = inner\n" "build $builddir/inner: varref\n"; ASSERT_NO_FATAL_FAILURE(AssertParse( "builddir = some_dir/\n" "rule varref\n" " command = varref $var\n" "var = outer\n" "build $builddir/outer: varref\n" "subninja test.ninja\n" "build $builddir/outer2: varref\n")); ASSERT_EQ(1u, files_read_.size()); EXPECT_EQ("test.ninja", files_read_[0]); EXPECT_TRUE(state.LookupNode("some_dir/outer")); // Verify our builddir setting is inherited. EXPECT_TRUE(state.LookupNode("some_dir/inner")); ASSERT_EQ(3u, state.edges_.size()); EXPECT_EQ("varref outer", state.edges_[0]->EvaluateCommand()); EXPECT_EQ("varref inner", state.edges_[1]->EvaluateCommand()); EXPECT_EQ("varref outer", state.edges_[2]->EvaluateCommand()); } TEST_F(ParserTest, MissingSubNinja) { ManifestParser parser(&state, this); string err; EXPECT_FALSE(parser.ParseTest("subninja foo.ninja\n", &err)); EXPECT_EQ("input:1: loading 'foo.ninja': No such file or directory\n" "subninja foo.ninja\n" " ^ near here" , err); } TEST_F(ParserTest, Include) { files_["include.ninja"] = "var = inner\n"; ASSERT_NO_FATAL_FAILURE(AssertParse( "var = outer\n" "include include.ninja\n")); ASSERT_EQ(1u, files_read_.size()); EXPECT_EQ("include.ninja", files_read_[0]); EXPECT_EQ("inner", state.bindings_.LookupVariable("var")); } TEST_F(ParserTest, Implicit) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n" " command = cat $in > $out\n" "build foo: cat bar | baz\n")); Edge* edge = state.LookupNode("foo")->in_edge(); ASSERT_TRUE(edge->is_implicit(1)); } TEST_F(ParserTest, OrderOnly) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n command = cat $in > $out\n" "build foo: cat bar || baz\n")); Edge* edge = state.LookupNode("foo")->in_edge(); ASSERT_TRUE(edge->is_order_only(1)); } TEST_F(ParserTest, DefaultDefault) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n command = cat $in > $out\n" "build a: cat foo\n" "build b: cat foo\n" "build c: cat foo\n" "build d: cat foo\n")); string err; EXPECT_EQ(4u, state.DefaultNodes(&err).size()); EXPECT_EQ("", err); } TEST_F(ParserTest, DefaultStatements) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule cat\n command = cat $in > $out\n" "build a: cat foo\n" "build b: cat foo\n" "build c: cat foo\n" "build d: cat foo\n" "third = c\n" "default a b\n" "default $third\n")); string err; vector nodes = state.DefaultNodes(&err); EXPECT_EQ("", err); ASSERT_EQ(3u, nodes.size()); EXPECT_EQ("a", nodes[0]->path()); EXPECT_EQ("b", nodes[1]->path()); EXPECT_EQ("c", nodes[2]->path()); } TEST_F(ParserTest, UTF8) { ASSERT_NO_FATAL_FAILURE(AssertParse( "rule utf8\n" " command = true\n" " description = compilaci\xC3\xB3\n")); } // We might want to eventually allow CRLF to be nice to Windows developers, // but for now just verify we error out with a nice message. TEST_F(ParserTest, CRLF) { State state; ManifestParser parser(&state, NULL); string err; EXPECT_FALSE(parser.ParseTest("# comment with crlf\r\n", &err)); EXPECT_EQ("input:1: lexing error\n", err); EXPECT_FALSE(parser.ParseTest("foo = foo\nbar = bar\r\n", &err)); EXPECT_EQ("input:2: carriage returns are not allowed, use newlines\n" "bar = bar\r\n" " ^ near here", err); } ninja-1.3.4/src/metrics.cc000066400000000000000000000062151215343310300153610ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "metrics.h" #include #include #include #ifndef _WIN32 #include #else #include #endif #include "util.h" Metrics* g_metrics = NULL; namespace { #ifndef _WIN32 /// Compute a platform-specific high-res timer value that fits into an int64. int64_t HighResTimer() { timeval tv; if (gettimeofday(&tv, NULL) < 0) Fatal("gettimeofday: %s", strerror(errno)); return (int64_t)tv.tv_sec * 1000*1000 + tv.tv_usec; } /// Convert a delta of HighResTimer() values to microseconds. int64_t TimerToMicros(int64_t dt) { // No conversion necessary. return dt; } #else int64_t LargeIntegerToInt64(const LARGE_INTEGER& i) { return ((int64_t)i.HighPart) << 32 | i.LowPart; } int64_t HighResTimer() { LARGE_INTEGER counter; if (!QueryPerformanceCounter(&counter)) Fatal("QueryPerformanceCounter: %s", GetLastErrorString().c_str()); return LargeIntegerToInt64(counter); } int64_t TimerToMicros(int64_t dt) { static int64_t ticks_per_sec = 0; if (!ticks_per_sec) { LARGE_INTEGER freq; if (!QueryPerformanceFrequency(&freq)) Fatal("QueryPerformanceFrequency: %s", GetLastErrorString().c_str()); ticks_per_sec = LargeIntegerToInt64(freq); } // dt is in ticks. We want microseconds. return (dt * 1000000) / ticks_per_sec; } #endif } // anonymous namespace ScopedMetric::ScopedMetric(Metric* metric) { metric_ = metric; if (!metric_) return; start_ = HighResTimer(); } ScopedMetric::~ScopedMetric() { if (!metric_) return; metric_->count++; int64_t dt = TimerToMicros(HighResTimer() - start_); metric_->sum += dt; } Metric* Metrics::NewMetric(const string& name) { Metric* metric = new Metric; metric->name = name; metric->count = 0; metric->sum = 0; metrics_.push_back(metric); return metric; } void Metrics::Report() { int width = 0; for (vector::iterator i = metrics_.begin(); i != metrics_.end(); ++i) { width = max((int)(*i)->name.size(), width); } printf("%-*s\t%-6s\t%-9s\t%s\n", width, "metric", "count", "avg (us)", "total (ms)"); for (vector::iterator i = metrics_.begin(); i != metrics_.end(); ++i) { Metric* metric = *i; double total = metric->sum / (double)1000; double avg = metric->sum / (double)metric->count; printf("%-*s\t%-6d\t%-8.1f\t%.1f\n", width, metric->name.c_str(), metric->count, avg, total); } } uint64_t Stopwatch::Now() const { return TimerToMicros(HighResTimer()); } int64_t GetTimeMillis() { return TimerToMicros(HighResTimer()) / 1000; } ninja-1.3.4/src/metrics.h000066400000000000000000000051651215343310300152260ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_METRICS_H_ #define NINJA_METRICS_H_ #include #include using namespace std; #include "util.h" // For int64_t. /// The Metrics module is used for the debug mode that dumps timing stats of /// various actions. To use, see METRIC_RECORD below. /// A single metrics we're tracking, like "depfile load time". struct Metric { string name; /// Number of times we've hit the code path. int count; /// Total time (in micros) we've spent on the code path. int64_t sum; }; /// A scoped object for recording a metric across the body of a function. /// Used by the METRIC_RECORD macro. struct ScopedMetric { explicit ScopedMetric(Metric* metric); ~ScopedMetric(); private: Metric* metric_; /// Timestamp when the measurement started. /// Value is platform-dependent. int64_t start_; }; /// The singleton that stores metrics and prints the report. struct Metrics { Metric* NewMetric(const string& name); /// Print a summary report to stdout. void Report(); private: vector metrics_; }; /// Get the current time as relative to some epoch. /// Epoch varies between platforms; only useful for measuring elapsed time. int64_t GetTimeMillis(); /// A simple stopwatch which returns the time /// in seconds since Restart() was called. struct Stopwatch { public: Stopwatch() : started_(0) {} /// Seconds since Restart() call. double Elapsed() const { return 1e-6 * static_cast(Now() - started_); } void Restart() { started_ = Now(); } private: uint64_t started_; uint64_t Now() const; }; /// The primary interface to metrics. Use METRIC_RECORD("foobar") at the top /// of a function to get timing stats recorded for each call of the function. #define METRIC_RECORD(name) \ static Metric* metrics_h_metric = \ g_metrics ? g_metrics->NewMetric(name) : NULL; \ ScopedMetric metrics_h_scoped(metrics_h_metric); extern Metrics* g_metrics; #endif // NINJA_METRICS_H_ ninja-1.3.4/src/minidump-win32.cc000066400000000000000000000055341215343310300165000ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_BOOTSTRAP #include #include #include "util.h" typedef BOOL (WINAPI *MiniDumpWriteDumpFunc) ( IN HANDLE, IN DWORD, IN HANDLE, IN MINIDUMP_TYPE, IN CONST PMINIDUMP_EXCEPTION_INFORMATION, OPTIONAL IN CONST PMINIDUMP_USER_STREAM_INFORMATION, OPTIONAL IN CONST PMINIDUMP_CALLBACK_INFORMATION OPTIONAL ); /// Creates a windows minidump in temp folder. void CreateWin32MiniDump(_EXCEPTION_POINTERS* pep) { char temp_path[MAX_PATH]; GetTempPath(sizeof(temp_path), temp_path); char temp_file[MAX_PATH]; sprintf(temp_file, "%s\\ninja_crash_dump_%d.dmp", temp_path, GetCurrentProcessId()); // Delete any previous minidump of the same name. DeleteFile(temp_file); // Load DbgHelp.dll dynamically, as library is not present on all // Windows versions. HMODULE dbghelp = LoadLibrary("dbghelp.dll"); if (dbghelp == NULL) { Error("failed to create minidump: LoadLibrary('dbghelp.dll'): %s", GetLastErrorString().c_str()); return; } MiniDumpWriteDumpFunc mini_dump_write_dump = (MiniDumpWriteDumpFunc)GetProcAddress(dbghelp, "MiniDumpWriteDump"); if (mini_dump_write_dump == NULL) { Error("failed to create minidump: GetProcAddress('MiniDumpWriteDump'): %s", GetLastErrorString().c_str()); return; } HANDLE hFile = CreateFileA(temp_file, GENERIC_READ | GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); if (hFile == NULL) { Error("failed to create minidump: CreateFileA(%s): %s", temp_file, GetLastErrorString().c_str()); return; } MINIDUMP_EXCEPTION_INFORMATION mdei; mdei.ThreadId = GetCurrentThreadId(); mdei.ExceptionPointers = pep; mdei.ClientPointers = FALSE; MINIDUMP_TYPE mdt = (MINIDUMP_TYPE) (MiniDumpWithDataSegs | MiniDumpWithHandleData); BOOL rv = mini_dump_write_dump(GetCurrentProcess(), GetCurrentProcessId(), hFile, mdt, (pep != 0) ? &mdei : 0, 0, 0); CloseHandle(hFile); if (!rv) { Error("MiniDumpWriteDump failed: %s", GetLastErrorString().c_str()); return; } Warning("minidump created: %s", temp_file); } #endif // NINJA_BOOTSTRAP ninja-1.3.4/src/msvc_helper-win32.cc000066400000000000000000000132161215343310300171610ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "msvc_helper.h" #include #include #include #include #include "includes_normalize.h" #include "util.h" namespace { /// Return true if \a input ends with \a needle. bool EndsWith(const string& input, const string& needle) { return (input.size() >= needle.size() && input.substr(input.size() - needle.size()) == needle); } string Replace(const string& input, const string& find, const string& replace) { string result = input; size_t start_pos = 0; while ((start_pos = result.find(find, start_pos)) != string::npos) { result.replace(start_pos, find.length(), replace); start_pos += replace.length(); } return result; } } // anonymous namespace string EscapeForDepfile(const string& path) { // Depfiles don't escape single \. return Replace(path, " ", "\\ "); } // static string CLParser::FilterShowIncludes(const string& line) { static const char kMagicPrefix[] = "Note: including file: "; const char* in = line.c_str(); const char* end = in + line.size(); if (end - in > (int)sizeof(kMagicPrefix) - 1 && memcmp(in, kMagicPrefix, sizeof(kMagicPrefix) - 1) == 0) { in += sizeof(kMagicPrefix) - 1; while (*in == ' ') ++in; return line.substr(in - line.c_str()); } return ""; } // static bool CLParser::IsSystemInclude(string path) { transform(path.begin(), path.end(), path.begin(), ::tolower); // TODO: this is a heuristic, perhaps there's a better way? return (path.find("program files") != string::npos || path.find("microsoft visual studio") != string::npos); } // static bool CLParser::FilterInputFilename(string line) { transform(line.begin(), line.end(), line.begin(), ::tolower); // TODO: other extensions, like .asm? return EndsWith(line, ".c") || EndsWith(line, ".cc") || EndsWith(line, ".cxx") || EndsWith(line, ".cpp"); } string CLParser::Parse(const string& output) { string filtered_output; // Loop over all lines in the output to process them. size_t start = 0; while (start < output.size()) { size_t end = output.find_first_of("\r\n", start); if (end == string::npos) end = output.size(); string line = output.substr(start, end - start); string include = FilterShowIncludes(line); if (!include.empty()) { include = IncludesNormalize::Normalize(include, NULL); if (!IsSystemInclude(include)) includes_.insert(include); } else if (FilterInputFilename(line)) { // Drop it. // TODO: if we support compiling multiple output files in a single // cl.exe invocation, we should stash the filename. } else { filtered_output.append(line); filtered_output.append("\n"); } if (end < output.size() && output[end] == '\r') ++end; if (end < output.size() && output[end] == '\n') ++end; start = end; } return filtered_output; } int CLWrapper::Run(const string& command, string* output) { SECURITY_ATTRIBUTES security_attributes = {}; security_attributes.nLength = sizeof(SECURITY_ATTRIBUTES); security_attributes.bInheritHandle = TRUE; // Must be inheritable so subprocesses can dup to children. HANDLE nul = CreateFile("NUL", GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, &security_attributes, OPEN_EXISTING, 0, NULL); if (nul == INVALID_HANDLE_VALUE) Fatal("couldn't open nul"); HANDLE stdout_read, stdout_write; if (!CreatePipe(&stdout_read, &stdout_write, &security_attributes, 0)) Win32Fatal("CreatePipe"); if (!SetHandleInformation(stdout_read, HANDLE_FLAG_INHERIT, 0)) Win32Fatal("SetHandleInformation"); PROCESS_INFORMATION process_info = {}; STARTUPINFO startup_info = {}; startup_info.cb = sizeof(STARTUPINFO); startup_info.hStdInput = nul; startup_info.hStdError = stdout_write; startup_info.hStdOutput = stdout_write; startup_info.dwFlags |= STARTF_USESTDHANDLES; if (!CreateProcessA(NULL, (char*)command.c_str(), NULL, NULL, /* inherit handles */ TRUE, 0, env_block_, NULL, &startup_info, &process_info)) { Win32Fatal("CreateProcess"); } if (!CloseHandle(nul) || !CloseHandle(stdout_write)) { Win32Fatal("CloseHandle"); } // Read all output of the subprocess. DWORD read_len = 1; while (read_len) { char buf[64 << 10]; read_len = 0; if (!::ReadFile(stdout_read, buf, sizeof(buf), &read_len, NULL) && GetLastError() != ERROR_BROKEN_PIPE) { Win32Fatal("ReadFile"); } output->append(buf, read_len); } // Wait for it to exit and grab its exit code. if (WaitForSingleObject(process_info.hProcess, INFINITE) == WAIT_FAILED) Win32Fatal("WaitForSingleObject"); DWORD exit_code = 0; if (!GetExitCodeProcess(process_info.hProcess, &exit_code)) Win32Fatal("GetExitCodeProcess"); if (!CloseHandle(stdout_read) || !CloseHandle(process_info.hProcess) || !CloseHandle(process_info.hThread)) { Win32Fatal("CloseHandle"); } return exit_code; } ninja-1.3.4/src/msvc_helper.h000066400000000000000000000042621215343310300160640ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include #include using namespace std; string EscapeForDepfile(const string& path); /// Visual Studio's cl.exe requires some massaging to work with Ninja; /// for example, it emits include information on stderr in a funny /// format when building with /showIncludes. This class parses this /// output. struct CLParser { /// Parse a line of cl.exe output and extract /showIncludes info. /// If a dependency is extracted, returns a nonempty string. /// Exposed for testing. static string FilterShowIncludes(const string& line); /// Return true if a mentioned include file is a system path. /// Filtering these out reduces dependency information considerably. static bool IsSystemInclude(string path); /// Parse a line of cl.exe output and return true if it looks like /// it's printing an input filename. This is a heuristic but it appears /// to be the best we can do. /// Exposed for testing. static bool FilterInputFilename(string line); /// Parse the full output of cl, returning the output (if any) that /// should printed. string Parse(const string& output); set includes_; }; /// Wraps a synchronous execution of a CL subprocess. struct CLWrapper { CLWrapper() : env_block_(NULL) {} /// Set the environment block (as suitable for CreateProcess) to be used /// by Run(). void SetEnvBlock(void* env_block) { env_block_ = env_block; } /// Start a process and gather its raw output. Returns its exit code. /// Crashes (calls Fatal()) on error. int Run(const string& command, string* output); void* env_block_; }; ninja-1.3.4/src/msvc_helper_main-win32.cc000066400000000000000000000067561215343310300202000ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "msvc_helper.h" #include #include #include #include #include "util.h" #include "getopt.h" namespace { void Usage() { printf( "usage: ninja -t msvc [options] -- cl.exe /showIncludes /otherArgs\n" "options:\n" " -e ENVFILE load environment block from ENVFILE as environment\n" " -o FILE write output dependency information to FILE.d\n" ); } void PushPathIntoEnvironment(const string& env_block) { const char* as_str = env_block.c_str(); while (as_str[0]) { if (_strnicmp(as_str, "path=", 5) == 0) { _putenv(as_str); return; } else { as_str = &as_str[strlen(as_str) + 1]; } } } void WriteDepFileOrDie(const char* object_path, const CLParser& parse) { string depfile_path = string(object_path) + ".d"; FILE* depfile = fopen(depfile_path.c_str(), "w"); if (!depfile) { unlink(object_path); Fatal("opening %s: %s", depfile_path.c_str(), GetLastErrorString().c_str()); } if (fprintf(depfile, "%s: ", object_path) < 0) { unlink(object_path); fclose(depfile); unlink(depfile_path.c_str()); Fatal("writing %s", depfile_path.c_str()); } const set& headers = parse.includes_; for (set::const_iterator i = headers.begin(); i != headers.end(); ++i) { if (fprintf(depfile, "%s\n", EscapeForDepfile(*i).c_str()) < 0) { unlink(object_path); fclose(depfile); unlink(depfile_path.c_str()); Fatal("writing %s", depfile_path.c_str()); } } fclose(depfile); } } // anonymous namespace int MSVCHelperMain(int argc, char** argv) { const char* output_filename = NULL; const char* envfile = NULL; const option kLongOptions[] = { { "help", no_argument, NULL, 'h' }, { NULL, 0, NULL, 0 } }; int opt; while ((opt = getopt_long(argc, argv, "e:o:h", kLongOptions, NULL)) != -1) { switch (opt) { case 'e': envfile = optarg; break; case 'o': output_filename = optarg; break; case 'h': default: Usage(); return 0; } } string env; if (envfile) { string err; if (ReadFile(envfile, &env, &err) != 0) Fatal("couldn't open %s: %s", envfile, err.c_str()); PushPathIntoEnvironment(env); } char* command = GetCommandLine(); command = strstr(command, " -- "); if (!command) { Fatal("expected command line to end with \" -- command args\""); } command += 4; CLWrapper cl; if (!env.empty()) cl.SetEnvBlock((void*)env.data()); string output; int exit_code = cl.Run(command, &output); if (output_filename) { CLParser parser; output = parser.Parse(output); WriteDepFileOrDie(output_filename, parser); } // CLWrapper's output already as \r\n line endings, make sure the C runtime // doesn't expand this to \r\r\n. _setmode(_fileno(stdout), _O_BINARY); printf("%s", output.c_str()); return exit_code; } ninja-1.3.4/src/msvc_helper_test.cc000066400000000000000000000070451215343310300172630ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "msvc_helper.h" #include #include "test.h" #include "util.h" TEST(CLParserTest, ShowIncludes) { ASSERT_EQ("", CLParser::FilterShowIncludes("")); ASSERT_EQ("", CLParser::FilterShowIncludes("Sample compiler output")); ASSERT_EQ("c:\\Some Files\\foobar.h", CLParser::FilterShowIncludes("Note: including file: " "c:\\Some Files\\foobar.h")); ASSERT_EQ("c:\\initspaces.h", CLParser::FilterShowIncludes("Note: including file: " "c:\\initspaces.h")); } TEST(CLParserTest, FilterInputFilename) { ASSERT_TRUE(CLParser::FilterInputFilename("foobar.cc")); ASSERT_TRUE(CLParser::FilterInputFilename("foo bar.cc")); ASSERT_TRUE(CLParser::FilterInputFilename("baz.c")); ASSERT_TRUE(CLParser::FilterInputFilename("FOOBAR.CC")); ASSERT_FALSE(CLParser::FilterInputFilename( "src\\cl_helper.cc(166) : fatal error C1075: end " "of file found ...")); } TEST(CLParserTest, ParseSimple) { CLParser parser; string output = parser.Parse( "foo\r\n" "Note: including file: foo.h\r\n" "bar\r\n"); ASSERT_EQ("foo\nbar\n", output); ASSERT_EQ(1u, parser.includes_.size()); ASSERT_EQ("foo.h", *parser.includes_.begin()); } TEST(CLParserTest, ParseFilenameFilter) { CLParser parser; string output = parser.Parse( "foo.cc\r\n" "cl: warning\r\n"); ASSERT_EQ("cl: warning\n", output); } TEST(CLParserTest, ParseSystemInclude) { CLParser parser; string output = parser.Parse( "Note: including file: c:\\Program Files\\foo.h\r\n" "Note: including file: d:\\Microsoft Visual Studio\\bar.h\r\n" "Note: including file: path.h\r\n"); // We should have dropped the first two includes because they look like // system headers. ASSERT_EQ("", output); ASSERT_EQ(1u, parser.includes_.size()); ASSERT_EQ("path.h", *parser.includes_.begin()); } TEST(CLParserTest, DuplicatedHeader) { CLParser parser; string output = parser.Parse( "Note: including file: foo.h\r\n" "Note: including file: bar.h\r\n" "Note: including file: foo.h\r\n"); // We should have dropped one copy of foo.h. ASSERT_EQ("", output); ASSERT_EQ(2u, parser.includes_.size()); } TEST(CLParserTest, DuplicatedHeaderPathConverted) { CLParser parser; string output = parser.Parse( "Note: including file: sub/foo.h\r\n" "Note: including file: bar.h\r\n" "Note: including file: sub\\foo.h\r\n"); // We should have dropped one copy of foo.h. ASSERT_EQ("", output); ASSERT_EQ(2u, parser.includes_.size()); } TEST(CLParserTest, SpacesInFilename) { ASSERT_EQ("sub\\some\\ sdk\\foo.h", EscapeForDepfile("sub\\some sdk\\foo.h")); } TEST(MSVCHelperTest, EnvBlock) { char env_block[] = "foo=bar\0"; CLWrapper cl; cl.SetEnvBlock(env_block); string output; cl.Run("cmd /c \"echo foo is %foo%", &output); ASSERT_EQ("foo is bar\r\n", output); } ninja-1.3.4/src/ninja.cc000066400000000000000000000706121215343310300150140ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include #include #include #include #ifdef _WIN32 #include "getopt.h" #include #include #else #include #include #endif #include "browse.h" #include "build.h" #include "build_log.h" #include "deps_log.h" #include "clean.h" #include "disk_interface.h" #include "explain.h" #include "graph.h" #include "graphviz.h" #include "manifest_parser.h" #include "metrics.h" #include "state.h" #include "util.h" #include "version.h" #ifdef _MSC_VER // Defined in msvc_helper_main-win32.cc. int MSVCHelperMain(int argc, char** argv); // Defined in minidump-win32.cc. void CreateWin32MiniDump(_EXCEPTION_POINTERS* pep); #endif namespace { struct Tool; /// Command-line options. struct Options { /// Build file to load. const char* input_file; /// Directory to change into before running. const char* working_dir; /// Tool to run rather than building. const Tool* tool; }; /// The Ninja main() loads up a series of data structures; various tools need /// to poke into these, so store them as fields on an object. struct NinjaMain { NinjaMain(const char* ninja_command, const BuildConfig& config) : ninja_command_(ninja_command), config_(config) {} /// Command line used to run Ninja. const char* ninja_command_; /// Build configuration set from flags (e.g. parallelism). const BuildConfig& config_; /// Loaded state (rules, nodes). State state_; /// Functions for accesssing the disk. RealDiskInterface disk_interface_; /// The build directory, used for storing the build log etc. string build_dir_; BuildLog build_log_; DepsLog deps_log_; /// The type of functions that are the entry points to tools (subcommands). typedef int (NinjaMain::*ToolFunc)(int, char**); /// Get the Node for a given command-line path, handling features like /// spell correction. Node* CollectTarget(const char* cpath, string* err); /// CollectTarget for all command-line arguments, filling in \a targets. bool CollectTargetsFromArgs(int argc, char* argv[], vector* targets, string* err); // The various subcommands, run via "-t XXX". int ToolGraph(int argc, char* argv[]); int ToolQuery(int argc, char* argv[]); int ToolBrowse(int argc, char* argv[]); int ToolMSVC(int argc, char* argv[]); int ToolTargets(int argc, char* argv[]); int ToolCommands(int argc, char* argv[]); int ToolClean(int argc, char* argv[]); int ToolCompilationDatabase(int argc, char* argv[]); int ToolUrtle(int argc, char** argv); /// Open the build log. /// @return false on error. bool OpenBuildLog(); /// Open the deps log: load it, then open for writing. /// @return false on error. bool OpenDepsLog(); /// Ensure the build directory exists, creating it if necessary. /// @return false on error. bool EnsureBuildDirExists(); /// Rebuild the manifest, if necessary. /// Fills in \a err on error. /// @return true if the manifest was rebuilt. bool RebuildManifest(const char* input_file, string* err); /// Build the targets listed on the command line. /// @return an exit code. int RunBuild(int argc, char** argv); /// Dump the output requested by '-d stats'. void DumpMetrics(); }; /// Subtools, accessible via "-t foo". struct Tool { /// Short name of the tool. const char* name; /// Description (shown in "-t list"). const char* desc; /// When to run the tool. enum { /// Run after parsing the command-line flags (as early as possible). RUN_AFTER_FLAGS, /// Run after loading build.ninja. RUN_AFTER_LOAD, /// Run after loading the build/deps logs. RUN_AFTER_LOGS, } when; /// Implementation of the tool. NinjaMain::ToolFunc func; }; /// Print usage information. void Usage(const BuildConfig& config) { fprintf(stderr, "usage: ninja [options] [targets...]\n" "\n" "if targets are unspecified, builds the 'default' target (see manual).\n" "\n" "options:\n" " --version print ninja version (\"%s\")\n" "\n" " -C DIR change to DIR before doing anything else\n" " -f FILE specify input build file [default=build.ninja]\n" "\n" " -j N run N jobs in parallel [default=%d, derived from CPUs available]\n" " -l N do not start new jobs if the load average is greater than N\n" #ifdef _WIN32 " (not yet implemented on Windows)\n" #endif " -k N keep going until N jobs fail [default=1]\n" " -n dry run (don't run commands but act like they succeeded)\n" " -v show all command lines while building\n" "\n" " -d MODE enable debugging (use -d list to list modes)\n" " -t TOOL run a subtool (use -t list to list subtools)\n" " terminates toplevel options; further flags are passed to the tool\n", kNinjaVersion, config.parallelism); } /// Choose a default value for the -j (parallelism) flag. int GuessParallelism() { switch (int processors = GetProcessorCount()) { case 0: case 1: return 2; case 2: return 3; default: return processors + 2; } } /// An implementation of ManifestParser::FileReader that actually reads /// the file. struct RealFileReader : public ManifestParser::FileReader { virtual bool ReadFile(const string& path, string* content, string* err) { return ::ReadFile(path, content, err) == 0; } }; /// Rebuild the build manifest, if necessary. /// Returns true if the manifest was rebuilt. bool NinjaMain::RebuildManifest(const char* input_file, string* err) { string path = input_file; if (!CanonicalizePath(&path, err)) return false; Node* node = state_.LookupNode(path); if (!node) return false; Builder builder(&state_, config_, &build_log_, &deps_log_, &disk_interface_); if (!builder.AddTarget(node, err)) return false; if (builder.AlreadyUpToDate()) return false; // Not an error, but we didn't rebuild. if (!builder.Build(err)) return false; // The manifest was only rebuilt if it is now dirty (it may have been cleaned // by a restat). return node->dirty(); } Node* NinjaMain::CollectTarget(const char* cpath, string* err) { string path = cpath; if (!CanonicalizePath(&path, err)) return NULL; // Special syntax: "foo.cc^" means "the first output of foo.cc". bool first_dependent = false; if (!path.empty() && path[path.size() - 1] == '^') { path.resize(path.size() - 1); first_dependent = true; } Node* node = state_.LookupNode(path); if (node) { if (first_dependent) { if (node->out_edges().empty()) { *err = "'" + path + "' has no out edge"; return NULL; } Edge* edge = node->out_edges()[0]; if (edge->outputs_.empty()) { edge->Dump(); Fatal("edge has no outputs"); } node = edge->outputs_[0]; } return node; } else { *err = "unknown target '" + path + "'"; if (path == "clean") { *err += ", did you mean 'ninja -t clean'?"; } else if (path == "help") { *err += ", did you mean 'ninja -h'?"; } else { Node* suggestion = state_.SpellcheckNode(path); if (suggestion) { *err += ", did you mean '" + suggestion->path() + "'?"; } } return NULL; } } bool NinjaMain::CollectTargetsFromArgs(int argc, char* argv[], vector* targets, string* err) { if (argc == 0) { *targets = state_.DefaultNodes(err); return err->empty(); } for (int i = 0; i < argc; ++i) { Node* node = CollectTarget(argv[i], err); if (node == NULL) return false; targets->push_back(node); } return true; } int NinjaMain::ToolGraph(int argc, char* argv[]) { vector nodes; string err; if (!CollectTargetsFromArgs(argc, argv, &nodes, &err)) { Error("%s", err.c_str()); return 1; } GraphViz graph; graph.Start(); for (vector::const_iterator n = nodes.begin(); n != nodes.end(); ++n) graph.AddTarget(*n); graph.Finish(); return 0; } int NinjaMain::ToolQuery(int argc, char* argv[]) { if (argc == 0) { Error("expected a target to query"); return 1; } for (int i = 0; i < argc; ++i) { string err; Node* node = CollectTarget(argv[i], &err); if (!node) { Error("%s", err.c_str()); return 1; } printf("%s:\n", node->path().c_str()); if (Edge* edge = node->in_edge()) { printf(" input: %s\n", edge->rule_->name().c_str()); for (int in = 0; in < (int)edge->inputs_.size(); in++) { const char* label = ""; if (edge->is_implicit(in)) label = "| "; else if (edge->is_order_only(in)) label = "|| "; printf(" %s%s\n", label, edge->inputs_[in]->path().c_str()); } } printf(" outputs:\n"); for (vector::const_iterator edge = node->out_edges().begin(); edge != node->out_edges().end(); ++edge) { for (vector::iterator out = (*edge)->outputs_.begin(); out != (*edge)->outputs_.end(); ++out) { printf(" %s\n", (*out)->path().c_str()); } } } return 0; } #if !defined(_WIN32) && !defined(NINJA_BOOTSTRAP) int NinjaMain::ToolBrowse(int argc, char* argv[]) { if (argc < 1) { Error("expected a target to browse"); return 1; } RunBrowsePython(&state_, ninja_command_, argv[0]); // If we get here, the browse failed. return 1; } #endif // _WIN32 #if defined(_MSC_VER) int NinjaMain::ToolMSVC(int argc, char* argv[]) { // Reset getopt: push one argument onto the front of argv, reset optind. argc++; argv--; optind = 0; return MSVCHelperMain(argc, argv); } #endif int ToolTargetsList(const vector& nodes, int depth, int indent) { for (vector::const_iterator n = nodes.begin(); n != nodes.end(); ++n) { for (int i = 0; i < indent; ++i) printf(" "); const char* target = (*n)->path().c_str(); if ((*n)->in_edge()) { printf("%s: %s\n", target, (*n)->in_edge()->rule_->name().c_str()); if (depth > 1 || depth <= 0) ToolTargetsList((*n)->in_edge()->inputs_, depth - 1, indent + 1); } else { printf("%s\n", target); } } return 0; } int ToolTargetsSourceList(State* state) { for (vector::iterator e = state->edges_.begin(); e != state->edges_.end(); ++e) { for (vector::iterator inps = (*e)->inputs_.begin(); inps != (*e)->inputs_.end(); ++inps) { if (!(*inps)->in_edge()) printf("%s\n", (*inps)->path().c_str()); } } return 0; } int ToolTargetsList(State* state, const string& rule_name) { set rules; // Gather the outputs. for (vector::iterator e = state->edges_.begin(); e != state->edges_.end(); ++e) { if ((*e)->rule_->name() == rule_name) { for (vector::iterator out_node = (*e)->outputs_.begin(); out_node != (*e)->outputs_.end(); ++out_node) { rules.insert((*out_node)->path()); } } } // Print them. for (set::const_iterator i = rules.begin(); i != rules.end(); ++i) { printf("%s\n", (*i).c_str()); } return 0; } int ToolTargetsList(State* state) { for (vector::iterator e = state->edges_.begin(); e != state->edges_.end(); ++e) { for (vector::iterator out_node = (*e)->outputs_.begin(); out_node != (*e)->outputs_.end(); ++out_node) { printf("%s: %s\n", (*out_node)->path().c_str(), (*e)->rule_->name().c_str()); } } return 0; } int NinjaMain::ToolTargets(int argc, char* argv[]) { int depth = 1; if (argc >= 1) { string mode = argv[0]; if (mode == "rule") { string rule; if (argc > 1) rule = argv[1]; if (rule.empty()) return ToolTargetsSourceList(&state_); else return ToolTargetsList(&state_, rule); } else if (mode == "depth") { if (argc > 1) depth = atoi(argv[1]); } else if (mode == "all") { return ToolTargetsList(&state_); } else { const char* suggestion = SpellcheckString(mode.c_str(), "rule", "depth", "all", NULL); if (suggestion) { Error("unknown target tool mode '%s', did you mean '%s'?", mode.c_str(), suggestion); } else { Error("unknown target tool mode '%s'", mode.c_str()); } return 1; } } string err; vector root_nodes = state_.RootNodes(&err); if (err.empty()) { return ToolTargetsList(root_nodes, depth, 0); } else { Error("%s", err.c_str()); return 1; } } void PrintCommands(Edge* edge, set* seen) { if (!edge) return; if (!seen->insert(edge).second) return; for (vector::iterator in = edge->inputs_.begin(); in != edge->inputs_.end(); ++in) PrintCommands((*in)->in_edge(), seen); if (!edge->is_phony()) puts(edge->EvaluateCommand().c_str()); } int NinjaMain::ToolCommands(int argc, char* argv[]) { vector nodes; string err; if (!CollectTargetsFromArgs(argc, argv, &nodes, &err)) { Error("%s", err.c_str()); return 1; } set seen; for (vector::iterator in = nodes.begin(); in != nodes.end(); ++in) PrintCommands((*in)->in_edge(), &seen); return 0; } int NinjaMain::ToolClean(int argc, char* argv[]) { // The clean tool uses getopt, and expects argv[0] to contain the name of // the tool, i.e. "clean". argc++; argv--; bool generator = false; bool clean_rules = false; optind = 1; int opt; while ((opt = getopt(argc, argv, const_cast("hgr"))) != -1) { switch (opt) { case 'g': generator = true; break; case 'r': clean_rules = true; break; case 'h': default: printf("usage: ninja -t clean [options] [targets]\n" "\n" "options:\n" " -g also clean files marked as ninja generator output\n" " -r interpret targets as a list of rules to clean instead\n" ); return 1; } } argv += optind; argc -= optind; if (clean_rules && argc == 0) { Error("expected a rule to clean"); return 1; } Cleaner cleaner(&state_, config_); if (argc >= 1) { if (clean_rules) return cleaner.CleanRules(argc, argv); else return cleaner.CleanTargets(argc, argv); } else { return cleaner.CleanAll(generator); } } void EncodeJSONString(const char *str) { while (*str) { if (*str == '"' || *str == '\\') putchar('\\'); putchar(*str); str++; } } int NinjaMain::ToolCompilationDatabase(int argc, char* argv[]) { bool first = true; vector cwd; do { cwd.resize(cwd.size() + 1024); errno = 0; } while (!getcwd(&cwd[0], cwd.size()) && errno == ERANGE); if (errno != 0 && errno != ERANGE) { Error("cannot determine working directory: %s", strerror(errno)); return 1; } putchar('['); for (vector::iterator e = state_.edges_.begin(); e != state_.edges_.end(); ++e) { for (int i = 0; i != argc; ++i) { if ((*e)->rule_->name() == argv[i]) { if (!first) putchar(','); printf("\n {\n \"directory\": \""); EncodeJSONString(&cwd[0]); printf("\",\n \"command\": \""); EncodeJSONString((*e)->EvaluateCommand().c_str()); printf("\",\n \"file\": \""); EncodeJSONString((*e)->inputs_[0]->path().c_str()); printf("\"\n }"); first = false; } } } puts("\n]"); return 0; } int NinjaMain::ToolUrtle(int argc, char** argv) { // RLE encoded. const char* urtle = " 13 ,3;2!2;\n8 ,;<11!;\n5 `'<10!(2`'2!\n11 ,6;, `\\. `\\9 .,c13$ec,.\n6 " ",2;11!>; `. ,;!2> .e8$2\".2 \"?7$e.\n <:<8!'` 2.3,.2` ,3!' ;,(?7\";2!2'<" "; `?6$PF ,;,\n2 `'4!8;<3;5! J2$b,`!>;2!:2!`,d?b`!>\n26 `'-;,(<9!> $F3 )3.:!.2 d\"" "2 ) !>\n30 7`2'<3!- \"=-='5 .2 `2-=\",!>\n25 .ze9$er2 .,cd16$bc.'\n22 .e" "14$,26$.\n21 z45$c .\n20 J50$c\n20 14$P\"`?34$b\n20 14$ dbc `2\"?22$?7$c" "\n20 ?18$c.6 4\"8?4\" c8$P\n9 .2,.8 \"20$c.3 ._14 J9$\n .2,2c9$bec,.2 `?" "21$c.3`4%,3%,3 c8$P\"\n22$c2 2\"?21$bc2,.2` .2,c7$P2\",cb\n23$b bc,.2\"2" "?14$2F2\"5?2\",J5$P\" ,zd3$\n24$ ?$3?%3 `2\"2?12$bcucd3$P3\"2 2=7$\n23$P" "\" ,3;<5!>2;,. `4\"6?2\"2 ,9;, `\"?2$\n"; int count = 0; for (const char* p = urtle; *p; p++) { if ('0' <= *p && *p <= '9') { count = count*10 + *p - '0'; } else { for (int i = 0; i < std::max(count, 1); ++i) printf("%c", *p); count = 0; } } return 0; } /// Find the function to execute for \a tool_name and return it via \a func. /// Returns a Tool, or NULL if Ninja should exit. const Tool* ChooseTool(const string& tool_name) { static const Tool kTools[] = { #if !defined(_WIN32) && !defined(NINJA_BOOTSTRAP) { "browse", "browse dependency graph in a web browser", Tool::RUN_AFTER_LOAD, &NinjaMain::ToolBrowse }, #endif #if defined(_MSC_VER) { "msvc", "build helper for MSVC cl.exe (EXPERIMENTAL)", Tool::RUN_AFTER_FLAGS, &NinjaMain::ToolMSVC }, #endif { "clean", "clean built files", Tool::RUN_AFTER_LOAD, &NinjaMain::ToolClean }, { "commands", "list all commands required to rebuild given targets", Tool::RUN_AFTER_LOAD, &NinjaMain::ToolCommands }, { "graph", "output graphviz dot file for targets", Tool::RUN_AFTER_LOAD, &NinjaMain::ToolGraph }, { "query", "show inputs/outputs for a path", Tool::RUN_AFTER_LOGS, &NinjaMain::ToolQuery }, { "targets", "list targets by their rule or depth in the DAG", Tool::RUN_AFTER_LOAD, &NinjaMain::ToolTargets }, { "compdb", "dump JSON compilation database to stdout", Tool::RUN_AFTER_LOAD, &NinjaMain::ToolCompilationDatabase }, { "urtle", NULL, Tool::RUN_AFTER_FLAGS, &NinjaMain::ToolUrtle }, { NULL, NULL, Tool::RUN_AFTER_FLAGS, NULL } }; if (tool_name == "list") { printf("ninja subtools:\n"); for (const Tool* tool = &kTools[0]; tool->name; ++tool) { if (tool->desc) printf("%10s %s\n", tool->name, tool->desc); } return NULL; } for (const Tool* tool = &kTools[0]; tool->name; ++tool) { if (tool->name == tool_name) return tool; } vector words; for (const Tool* tool = &kTools[0]; tool->name; ++tool) words.push_back(tool->name); const char* suggestion = SpellcheckStringV(tool_name, words); if (suggestion) { Fatal("unknown tool '%s', did you mean '%s'?", tool_name.c_str(), suggestion); } else { Fatal("unknown tool '%s'", tool_name.c_str()); } return NULL; // Not reached. } /// Enable a debugging mode. Returns false if Ninja should exit instead /// of continuing. bool DebugEnable(const string& name) { if (name == "list") { printf("debugging modes:\n" " stats print operation counts/timing info\n" " explain explain what caused a command to execute\n" "multiple modes can be enabled via -d FOO -d BAR\n"); return false; } else if (name == "stats") { g_metrics = new Metrics; return true; } else if (name == "explain") { g_explaining = true; return true; } else { const char* suggestion = SpellcheckString(name.c_str(), "stats", "explain", NULL); if (suggestion) { Error("unknown debug setting '%s', did you mean '%s'?", name.c_str(), suggestion); } else { Error("unknown debug setting '%s'", name.c_str()); } return false; } } bool NinjaMain::OpenBuildLog() { string log_path = ".ninja_log"; if (!build_dir_.empty()) log_path = build_dir_ + "/" + log_path; string err; if (!build_log_.Load(log_path, &err)) { Error("loading build log %s: %s", log_path.c_str(), err.c_str()); return false; } if (!err.empty()) { // Hack: Load() can return a warning via err by returning true. Warning("%s", err.c_str()); err.clear(); } if (!config_.dry_run) { if (!build_log_.OpenForWrite(log_path, &err)) { Error("opening build log: %s", err.c_str()); return false; } } return true; } /// Open the deps log: load it, then open for writing. /// @return false on error. bool NinjaMain::OpenDepsLog() { string path = ".ninja_deps"; if (!build_dir_.empty()) path = build_dir_ + "/" + path; string err; if (!deps_log_.Load(path, &state_, &err)) { Error("loading deps log %s: %s", path.c_str(), err.c_str()); return false; } if (!err.empty()) { // Hack: Load() can return a warning via err by returning true. Warning("%s", err.c_str()); err.clear(); } if (!config_.dry_run) { if (!deps_log_.OpenForWrite(path, &err)) { Error("opening deps log: %s", err.c_str()); return false; } } return true; } void NinjaMain::DumpMetrics() { g_metrics->Report(); printf("\n"); int count = (int)state_.paths_.size(); int buckets = (int)state_.paths_.bucket_count(); printf("path->node hash load %.2f (%d entries / %d buckets)\n", count / (double) buckets, count, buckets); } bool NinjaMain::EnsureBuildDirExists() { build_dir_ = state_.bindings_.LookupVariable("builddir"); if (!build_dir_.empty() && !config_.dry_run) { if (!disk_interface_.MakeDirs(build_dir_ + "/.") && errno != EEXIST) { Error("creating build directory %s: %s", build_dir_.c_str(), strerror(errno)); return false; } } return true; } int NinjaMain::RunBuild(int argc, char** argv) { string err; vector targets; if (!CollectTargetsFromArgs(argc, argv, &targets, &err)) { Error("%s", err.c_str()); return 1; } Builder builder(&state_, config_, &build_log_, &deps_log_, &disk_interface_); for (size_t i = 0; i < targets.size(); ++i) { if (!builder.AddTarget(targets[i], &err)) { if (!err.empty()) { Error("%s", err.c_str()); return 1; } else { // Added a target that is already up-to-date; not really // an error. } } } if (builder.AlreadyUpToDate()) { printf("ninja: no work to do.\n"); return 0; } if (!builder.Build(&err)) { printf("ninja: build stopped: %s.\n", err.c_str()); if (err.find("interrupted by user") != string::npos) { return 2; } return 1; } return 0; } #ifdef _MSC_VER /// This handler processes fatal crashes that you can't catch /// Test example: C++ exception in a stack-unwind-block /// Real-world example: ninja launched a compiler to process a tricky /// C++ input file. The compiler got itself into a state where it /// generated 3 GB of output and caused ninja to crash. void TerminateHandler() { CreateWin32MiniDump(NULL); Fatal("terminate handler called"); } /// On Windows, we want to prevent error dialogs in case of exceptions. /// This function handles the exception, and writes a minidump. int ExceptionFilter(unsigned int code, struct _EXCEPTION_POINTERS *ep) { Error("exception: 0x%X", code); // e.g. EXCEPTION_ACCESS_VIOLATION fflush(stderr); CreateWin32MiniDump(ep); return EXCEPTION_EXECUTE_HANDLER; } #endif // _MSC_VER /// Parse argv for command-line options. /// Returns an exit code, or -1 if Ninja should continue. int ReadFlags(int* argc, char*** argv, Options* options, BuildConfig* config) { config->parallelism = GuessParallelism(); enum { OPT_VERSION = 1 }; const option kLongOptions[] = { { "help", no_argument, NULL, 'h' }, { "version", no_argument, NULL, OPT_VERSION }, { NULL, 0, NULL, 0 } }; int opt; while (!options->tool && (opt = getopt_long(*argc, *argv, "d:f:j:k:l:nt:vC:h", kLongOptions, NULL)) != -1) { switch (opt) { case 'd': if (!DebugEnable(optarg)) return 1; break; case 'f': options->input_file = optarg; break; case 'j': { char* end; int value = strtol(optarg, &end, 10); if (*end != 0 || value <= 0) Fatal("invalid -j parameter"); config->parallelism = value; break; } case 'k': { char* end; int value = strtol(optarg, &end, 10); if (*end != 0) Fatal("-k parameter not numeric; did you mean -k 0?"); // We want to go until N jobs fail, which means we should allow // N failures and then stop. For N <= 0, INT_MAX is close enough // to infinite for most sane builds. config->failures_allowed = value > 0 ? value : INT_MAX; break; } case 'l': { char* end; double value = strtod(optarg, &end); if (end == optarg) Fatal("-l parameter not numeric: did you mean -l 0.0?"); config->max_load_average = value; break; } case 'n': config->dry_run = true; break; case 't': options->tool = ChooseTool(optarg); if (!options->tool) return 0; break; case 'v': config->verbosity = BuildConfig::VERBOSE; break; case 'C': options->working_dir = optarg; break; case OPT_VERSION: printf("%s\n", kNinjaVersion); return 0; case 'h': default: Usage(*config); return 1; } } *argv += optind; *argc -= optind; return -1; } int real_main(int argc, char** argv) { BuildConfig config; Options options = {}; options.input_file = "build.ninja"; setvbuf(stdout, NULL, _IOLBF, BUFSIZ); int exit_code = ReadFlags(&argc, &argv, &options, &config); if (exit_code >= 0) return exit_code; if (options.tool && options.tool->when == Tool::RUN_AFTER_FLAGS) { // None of the RUN_AFTER_FLAGS actually use a NinjaMain, but it's needed // by other tools. NinjaMain ninja(argv[0], config); return (ninja.*options.tool->func)(argc, argv); } if (options.working_dir) { // The formatting of this string, complete with funny quotes, is // so Emacs can properly identify that the cwd has changed for // subsequent commands. // Don't print this if a tool is being used, so that tool output // can be piped into a file without this string showing up. if (!options.tool) printf("ninja: Entering directory `%s'\n", options.working_dir); if (chdir(options.working_dir) < 0) { Fatal("chdir to '%s' - %s", options.working_dir, strerror(errno)); } } // The build can take up to 2 passes: one to rebuild the manifest, then // another to build the desired target. for (int cycle = 0; cycle < 2; ++cycle) { NinjaMain ninja(argv[0], config); RealFileReader file_reader; ManifestParser parser(&ninja.state_, &file_reader); string err; if (!parser.Load(options.input_file, &err)) { Error("%s", err.c_str()); return 1; } if (options.tool && options.tool->when == Tool::RUN_AFTER_LOAD) return (ninja.*options.tool->func)(argc, argv); if (!ninja.EnsureBuildDirExists()) return 1; if (!ninja.OpenBuildLog() || !ninja.OpenDepsLog()) return 1; if (options.tool && options.tool->when == Tool::RUN_AFTER_LOGS) return (ninja.*options.tool->func)(argc, argv); // The first time through, attempt to rebuild the manifest before // building anything else. if (cycle == 0) { if (ninja.RebuildManifest(options.input_file, &err)) { // Start the build over with the new manifest. continue; } else if (!err.empty()) { Error("rebuilding '%s': %s", options.input_file, err.c_str()); return 1; } } int result = ninja.RunBuild(argc, argv); if (g_metrics) ninja.DumpMetrics(); return result; } return 1; // Shouldn't be reached. } } // anonymous namespace int main(int argc, char** argv) { #if !defined(NINJA_BOOTSTRAP) && defined(_MSC_VER) // Set a handler to catch crashes not caught by the __try..__except // block (e.g. an exception in a stack-unwind-block). set_terminate(TerminateHandler); __try { // Running inside __try ... __except suppresses any Windows error // dialogs for errors such as bad_alloc. return real_main(argc, argv); } __except(ExceptionFilter(GetExceptionCode(), GetExceptionInformation())) { // Common error situations return exitCode=1. 2 was chosen to // indicate a more serious problem. return 2; } #else return real_main(argc, argv); #endif } ninja-1.3.4/src/ninja_test.cc000066400000000000000000000052601215343310300160500ustar00rootroot00000000000000// Copyright 2013 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include #include "gtest/gtest.h" #include "line_printer.h" string StringPrintf(const char* format, ...) { const int N = 1024; char buf[N]; va_list ap; va_start(ap, format); vsnprintf(buf, N, format, ap); va_end(ap); return buf; } /// A test result printer that's less wordy than gtest's default. struct LaconicPrinter : public testing::EmptyTestEventListener { LaconicPrinter() : tests_started_(0), test_count_(0), iteration_(0) {} virtual void OnTestProgramStart(const testing::UnitTest& unit_test) { test_count_ = unit_test.test_to_run_count(); } virtual void OnTestIterationStart(const testing::UnitTest& test_info, int iteration) { tests_started_ = 0; iteration_ = iteration; } virtual void OnTestStart(const testing::TestInfo& test_info) { ++tests_started_; printer_.Print( StringPrintf("[%d/%d%s] %s.%s", tests_started_, test_count_, iteration_ ? StringPrintf(" iter %d", iteration_).c_str() : "", test_info.test_case_name(), test_info.name()), LinePrinter::ELIDE); } virtual void OnTestPartResult( const testing::TestPartResult& test_part_result) { if (!test_part_result.failed()) return; printer_.PrintOnNewLine(StringPrintf( "*** Failure in %s:%d\n%s\n", test_part_result.file_name(), test_part_result.line_number(), test_part_result.summary())); } virtual void OnTestProgramEnd(const testing::UnitTest& unit_test) { printer_.PrintOnNewLine(unit_test.Passed() ? "passed\n" : "failed\n"); } private: LinePrinter printer_; int tests_started_; int test_count_; int iteration_; }; int main(int argc, char **argv) { testing::InitGoogleTest(&argc, argv); testing::TestEventListeners& listeners = testing::UnitTest::GetInstance()->listeners(); delete listeners.Release(listeners.default_result_printer()); listeners.Append(new LaconicPrinter); return RUN_ALL_TESTS(); } ninja-1.3.4/src/parser_perftest.cc000066400000000000000000000040501215343310300171160ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include #include "depfile_parser.h" #include "util.h" #include "metrics.h" int main(int argc, char* argv[]) { if (argc < 2) { printf("usage: %s \n", argv[0]); return 1; } vector times; for (int i = 1; i < argc; ++i) { const char* filename = argv[i]; for (int limit = 1 << 10; limit < (1<<20); limit *= 2) { int64_t start = GetTimeMillis(); for (int rep = 0; rep < limit; ++rep) { string buf; string err; if (ReadFile(filename, &buf, &err) < 0) { printf("%s: %s\n", filename, err.c_str()); return 1; } DepfileParser parser; if (!parser.Parse(&buf, &err)) { printf("%s: %s\n", filename, err.c_str()); return 1; } } int64_t end = GetTimeMillis(); if (end - start > 100) { int delta = (int)(end - start); float time = delta*1000 / (float)limit; printf("%s: %.1fus\n", filename, time); times.push_back(time); break; } } } if (!times.empty()) { float min = times[0]; float max = times[0]; float total = 0; for (size_t i = 0; i < times.size(); ++i) { total += times[i]; if (times[i] < min) min = times[i]; else if (times[i] > max) max = times[i]; } printf("min %.1fus max %.1fus avg %.1fus\n", min, max, total / times.size()); } return 0; } ninja-1.3.4/src/state.cc000066400000000000000000000133221215343310300150300ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "state.h" #include #include #include "edit_distance.h" #include "graph.h" #include "metrics.h" #include "util.h" void Pool::EdgeScheduled(const Edge& edge) { if (depth_ != 0) current_use_ += edge.weight(); } void Pool::EdgeFinished(const Edge& edge) { if (depth_ != 0) current_use_ -= edge.weight(); } void Pool::DelayEdge(Edge* edge) { assert(depth_ != 0); delayed_.insert(edge); } void Pool::RetrieveReadyEdges(set* ready_queue) { DelayedEdges::iterator it = delayed_.begin(); while (it != delayed_.end()) { Edge* edge = *it; if (current_use_ + edge->weight() > depth_) break; ready_queue->insert(edge); EdgeScheduled(*edge); ++it; } delayed_.erase(delayed_.begin(), it); } void Pool::Dump() const { printf("%s (%d/%d) ->\n", name_.c_str(), current_use_, depth_); for (DelayedEdges::const_iterator it = delayed_.begin(); it != delayed_.end(); ++it) { printf("\t"); (*it)->Dump(); } } bool Pool::WeightedEdgeCmp(const Edge* a, const Edge* b) { if (!a) return b; if (!b) return false; int weight_diff = a->weight() - b->weight(); return ((weight_diff < 0) || (weight_diff == 0 && a < b)); } Pool State::kDefaultPool("", 0); const Rule State::kPhonyRule("phony"); State::State() { AddRule(&kPhonyRule); AddPool(&kDefaultPool); } void State::AddRule(const Rule* rule) { assert(LookupRule(rule->name()) == NULL); rules_[rule->name()] = rule; } const Rule* State::LookupRule(const string& rule_name) { map::iterator i = rules_.find(rule_name); if (i == rules_.end()) return NULL; return i->second; } void State::AddPool(Pool* pool) { assert(LookupPool(pool->name()) == NULL); pools_[pool->name()] = pool; } Pool* State::LookupPool(const string& pool_name) { map::iterator i = pools_.find(pool_name); if (i == pools_.end()) return NULL; return i->second; } Edge* State::AddEdge(const Rule* rule) { Edge* edge = new Edge(); edge->rule_ = rule; edge->pool_ = &State::kDefaultPool; edge->env_ = &bindings_; edges_.push_back(edge); return edge; } Node* State::GetNode(StringPiece path) { Node* node = LookupNode(path); if (node) return node; node = new Node(path.AsString()); paths_[node->path()] = node; return node; } Node* State::LookupNode(StringPiece path) { METRIC_RECORD("lookup node"); Paths::iterator i = paths_.find(path); if (i != paths_.end()) return i->second; return NULL; } Node* State::SpellcheckNode(const string& path) { const bool kAllowReplacements = true; const int kMaxValidEditDistance = 3; int min_distance = kMaxValidEditDistance + 1; Node* result = NULL; for (Paths::iterator i = paths_.begin(); i != paths_.end(); ++i) { int distance = EditDistance( i->first, path, kAllowReplacements, kMaxValidEditDistance); if (distance < min_distance && i->second) { min_distance = distance; result = i->second; } } return result; } void State::AddIn(Edge* edge, StringPiece path) { Node* node = GetNode(path); edge->inputs_.push_back(node); node->AddOutEdge(edge); } void State::AddOut(Edge* edge, StringPiece path) { Node* node = GetNode(path); edge->outputs_.push_back(node); if (node->in_edge()) { Warning("multiple rules generate %s. " "builds involving this target will not be correct; " "continuing anyway", path.AsString().c_str()); } node->set_in_edge(edge); } bool State::AddDefault(StringPiece path, string* err) { Node* node = LookupNode(path); if (!node) { *err = "unknown target '" + path.AsString() + "'"; return false; } defaults_.push_back(node); return true; } vector State::RootNodes(string* err) { vector root_nodes; // Search for nodes with no output. for (vector::iterator e = edges_.begin(); e != edges_.end(); ++e) { for (vector::iterator out = (*e)->outputs_.begin(); out != (*e)->outputs_.end(); ++out) { if ((*out)->out_edges().empty()) root_nodes.push_back(*out); } } if (!edges_.empty() && root_nodes.empty()) *err = "could not determine root nodes of build graph"; assert(edges_.empty() || !root_nodes.empty()); return root_nodes; } vector State::DefaultNodes(string* err) { return defaults_.empty() ? RootNodes(err) : defaults_; } void State::Reset() { for (Paths::iterator i = paths_.begin(); i != paths_.end(); ++i) i->second->ResetState(); for (vector::iterator e = edges_.begin(); e != edges_.end(); ++e) (*e)->outputs_ready_ = false; } void State::Dump() { for (Paths::iterator i = paths_.begin(); i != paths_.end(); ++i) { Node* node = i->second; printf("%s %s [id:%d]\n", node->path().c_str(), node->status_known() ? (node->dirty() ? "dirty" : "clean") : "unknown", node->id()); } if (!pools_.empty()) { printf("resource_pools:\n"); for (map::const_iterator it = pools_.begin(); it != pools_.end(); ++it) { if (!it->second->name().empty()) { it->second->Dump(); } } } } ninja-1.3.4/src/state.h000066400000000000000000000101551215343310300146730ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_STATE_H_ #define NINJA_STATE_H_ #include #include #include #include using namespace std; #include "eval_env.h" #include "hash_map.h" struct Edge; struct Node; struct Rule; /// A pool for delayed edges. /// Pools are scoped to a State. Edges within a State will share Pools. A Pool /// will keep a count of the total 'weight' of the currently scheduled edges. If /// a Plan attempts to schedule an Edge which would cause the total weight to /// exceed the depth of the Pool, the Pool will enque the Edge instead of /// allowing the Plan to schedule it. The Pool will relinquish queued Edges when /// the total scheduled weight diminishes enough (i.e. when a scheduled edge /// completes). struct Pool { explicit Pool(const string& name, int depth) : name_(name), current_use_(0), depth_(depth), delayed_(&WeightedEdgeCmp) { } // A depth of 0 is infinite bool is_valid() const { return depth_ >= 0; } int depth() const { return depth_; } const string& name() const { return name_; } /// true if the Pool might delay this edge bool ShouldDelayEdge() const { return depth_ != 0; } /// informs this Pool that the given edge is committed to be run. /// Pool will count this edge as using resources from this pool. void EdgeScheduled(const Edge& edge); /// informs this Pool that the given edge is no longer runnable, and should /// relinquish its resources back to the pool void EdgeFinished(const Edge& edge); /// adds the given edge to this Pool to be delayed. void DelayEdge(Edge* edge); /// Pool will add zero or more edges to the ready_queue void RetrieveReadyEdges(set* ready_queue); /// Dump the Pool and its edges (useful for debugging). void Dump() const; private: string name_; /// |current_use_| is the total of the weights of the edges which are /// currently scheduled in the Plan (i.e. the edges in Plan::ready_). int current_use_; int depth_; static bool WeightedEdgeCmp(const Edge* a, const Edge* b); typedef set DelayedEdges; DelayedEdges delayed_; }; /// Global state (file status, loaded rules) for a single run. struct State { static Pool kDefaultPool; static const Rule kPhonyRule; State(); void AddRule(const Rule* rule); const Rule* LookupRule(const string& rule_name); void AddPool(Pool* pool); Pool* LookupPool(const string& pool_name); Edge* AddEdge(const Rule* rule); Node* GetNode(StringPiece path); Node* LookupNode(StringPiece path); Node* SpellcheckNode(const string& path); void AddIn(Edge* edge, StringPiece path); void AddOut(Edge* edge, StringPiece path); bool AddDefault(StringPiece path, string* error); /// Reset state. Keeps all nodes and edges, but restores them to the /// state where we haven't yet examined the disk for dirty state. void Reset(); /// Dump the nodes and Pools (useful for debugging). void Dump(); /// @return the root node(s) of the graph. (Root nodes have no output edges). /// @param error where to write the error message if somethings went wrong. vector RootNodes(string* error); vector DefaultNodes(string* error); /// Mapping of path -> Node. typedef ExternalStringHashMap::Type Paths; Paths paths_; /// All the rules used in the graph. map rules_; /// All the pools used in the graph. map pools_; /// All the edges of the graph. vector edges_; BindingEnv bindings_; vector defaults_; }; #endif // NINJA_STATE_H_ ninja-1.3.4/src/state_test.cc000066400000000000000000000024201215343310300160640ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include #include "graph.h" #include "state.h" namespace { TEST(State, Basic) { State state; EvalString command; command.AddText("cat "); command.AddSpecial("in"); command.AddText(" > "); command.AddSpecial("out"); Rule* rule = new Rule("cat"); rule->AddBinding("command", command); state.AddRule(rule); Edge* edge = state.AddEdge(rule); state.AddIn(edge, "in1"); state.AddIn(edge, "in2"); state.AddOut(edge, "out"); EXPECT_EQ("cat in1 in2 > out", edge->EvaluateCommand()); EXPECT_FALSE(state.GetNode("in1")->dirty()); EXPECT_FALSE(state.GetNode("in2")->dirty()); EXPECT_FALSE(state.GetNode("out")->dirty()); } } // namespace ninja-1.3.4/src/string_piece.h000066400000000000000000000032471215343310300162320ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_STRINGPIECE_H_ #define NINJA_STRINGPIECE_H_ #include using namespace std; #include /// StringPiece represents a slice of a string whose memory is managed /// externally. It is useful for reducing the number of std::strings /// we need to allocate. struct StringPiece { StringPiece() : str_(NULL), len_(0) {} /// The constructors intentionally allow for implicit conversions. StringPiece(const string& str) : str_(str.data()), len_(str.size()) {} StringPiece(const char* str) : str_(str), len_(strlen(str)) {} StringPiece(const char* str, size_t len) : str_(str), len_(len) {} bool operator==(const StringPiece& other) const { return len_ == other.len_ && memcmp(str_, other.str_, len_) == 0; } bool operator!=(const StringPiece& other) const { return !(*this == other); } /// Convert the slice into a full-fledged std::string, copying the /// data into a new string. string AsString() const { return len_ ? string(str_, len_) : string(); } const char* str_; size_t len_; }; #endif // NINJA_STRINGPIECE_H_ ninja-1.3.4/src/subprocess-posix.cc000066400000000000000000000151721215343310300172450ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "subprocess.h" #include #include #include #include #include #include #include #include #include "util.h" Subprocess::Subprocess() : fd_(-1), pid_(-1) { } Subprocess::~Subprocess() { if (fd_ >= 0) close(fd_); // Reap child if forgotten. if (pid_ != -1) Finish(); } bool Subprocess::Start(SubprocessSet* set, const string& command) { int output_pipe[2]; if (pipe(output_pipe) < 0) Fatal("pipe: %s", strerror(errno)); fd_ = output_pipe[0]; #if !defined(USE_PPOLL) // On Linux and OpenBSD, we use ppoll in DoWork(); elsewhere we use pselect // and so must avoid overly-large FDs. if (fd_ >= static_cast(FD_SETSIZE)) Fatal("pipe: %s", strerror(EMFILE)); #endif // !USE_PPOLL SetCloseOnExec(fd_); pid_ = fork(); if (pid_ < 0) Fatal("fork: %s", strerror(errno)); if (pid_ == 0) { close(output_pipe[0]); // Track which fd we use to report errors on. int error_pipe = output_pipe[1]; do { if (setpgid(0, 0) < 0) break; if (sigaction(SIGINT, &set->old_act_, 0) < 0) break; if (sigprocmask(SIG_SETMASK, &set->old_mask_, 0) < 0) break; // Open /dev/null over stdin. int devnull = open("/dev/null", O_RDONLY); if (devnull < 0) break; if (dup2(devnull, 0) < 0) break; close(devnull); if (dup2(output_pipe[1], 1) < 0 || dup2(output_pipe[1], 2) < 0) break; // Now can use stderr for errors. error_pipe = 2; close(output_pipe[1]); execl("/bin/sh", "/bin/sh", "-c", command.c_str(), (char *) NULL); } while (false); // If we get here, something went wrong; the execl should have // replaced us. char* err = strerror(errno); if (write(error_pipe, err, strlen(err)) < 0) { // If the write fails, there's nothing we can do. // But this block seems necessary to silence the warning. } _exit(1); } close(output_pipe[1]); return true; } void Subprocess::OnPipeReady() { char buf[4 << 10]; ssize_t len = read(fd_, buf, sizeof(buf)); if (len > 0) { buf_.append(buf, len); } else { if (len < 0) Fatal("read: %s", strerror(errno)); close(fd_); fd_ = -1; } } ExitStatus Subprocess::Finish() { assert(pid_ != -1); int status; if (waitpid(pid_, &status, 0) < 0) Fatal("waitpid(%d): %s", pid_, strerror(errno)); pid_ = -1; if (WIFEXITED(status)) { int exit = WEXITSTATUS(status); if (exit == 0) return ExitSuccess; } else if (WIFSIGNALED(status)) { if (WTERMSIG(status) == SIGINT) return ExitInterrupted; } return ExitFailure; } bool Subprocess::Done() const { return fd_ == -1; } const string& Subprocess::GetOutput() const { return buf_; } bool SubprocessSet::interrupted_; void SubprocessSet::SetInterruptedFlag(int signum) { (void) signum; interrupted_ = true; } SubprocessSet::SubprocessSet() { sigset_t set; sigemptyset(&set); sigaddset(&set, SIGINT); if (sigprocmask(SIG_BLOCK, &set, &old_mask_) < 0) Fatal("sigprocmask: %s", strerror(errno)); struct sigaction act; memset(&act, 0, sizeof(act)); act.sa_handler = SetInterruptedFlag; if (sigaction(SIGINT, &act, &old_act_) < 0) Fatal("sigaction: %s", strerror(errno)); } SubprocessSet::~SubprocessSet() { Clear(); if (sigaction(SIGINT, &old_act_, 0) < 0) Fatal("sigaction: %s", strerror(errno)); if (sigprocmask(SIG_SETMASK, &old_mask_, 0) < 0) Fatal("sigprocmask: %s", strerror(errno)); } Subprocess *SubprocessSet::Add(const string& command) { Subprocess *subprocess = new Subprocess; if (!subprocess->Start(this, command)) { delete subprocess; return 0; } running_.push_back(subprocess); return subprocess; } #ifdef USE_PPOLL bool SubprocessSet::DoWork() { vector fds; nfds_t nfds = 0; for (vector::iterator i = running_.begin(); i != running_.end(); ++i) { int fd = (*i)->fd_; if (fd < 0) continue; pollfd pfd = { fd, POLLIN | POLLPRI, 0 }; fds.push_back(pfd); ++nfds; } interrupted_ = false; int ret = ppoll(&fds.front(), nfds, NULL, &old_mask_); if (ret == -1) { if (errno != EINTR) { perror("ninja: ppoll"); return false; } return interrupted_; } nfds_t cur_nfd = 0; for (vector::iterator i = running_.begin(); i != running_.end(); ) { int fd = (*i)->fd_; if (fd < 0) continue; assert(fd == fds[cur_nfd].fd); if (fds[cur_nfd++].revents) { (*i)->OnPipeReady(); if ((*i)->Done()) { finished_.push(*i); i = running_.erase(i); continue; } } ++i; } return interrupted_; } #else // linux || __OpenBSD__ bool SubprocessSet::DoWork() { fd_set set; int nfds = 0; FD_ZERO(&set); for (vector::iterator i = running_.begin(); i != running_.end(); ++i) { int fd = (*i)->fd_; if (fd >= 0) { FD_SET(fd, &set); if (nfds < fd+1) nfds = fd+1; } } interrupted_ = false; int ret = pselect(nfds, &set, 0, 0, 0, &old_mask_); if (ret == -1) { if (errno != EINTR) { perror("ninja: pselect"); return false; } return interrupted_; } for (vector::iterator i = running_.begin(); i != running_.end(); ) { int fd = (*i)->fd_; if (fd >= 0 && FD_ISSET(fd, &set)) { (*i)->OnPipeReady(); if ((*i)->Done()) { finished_.push(*i); i = running_.erase(i); continue; } } ++i; } return interrupted_; } #endif // linux || __OpenBSD__ Subprocess* SubprocessSet::NextFinished() { if (finished_.empty()) return NULL; Subprocess* subproc = finished_.front(); finished_.pop(); return subproc; } void SubprocessSet::Clear() { for (vector::iterator i = running_.begin(); i != running_.end(); ++i) kill(-(*i)->pid_, SIGINT); for (vector::iterator i = running_.begin(); i != running_.end(); ++i) delete *i; running_.clear(); } ninja-1.3.4/src/subprocess-win32.cc000066400000000000000000000200071215343310300170360ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "subprocess.h" #include #include #include "util.h" Subprocess::Subprocess() : child_(NULL) , overlapped_(), is_reading_(false) { } Subprocess::~Subprocess() { if (pipe_) { if (!CloseHandle(pipe_)) Win32Fatal("CloseHandle"); } // Reap child if forgotten. if (child_) Finish(); } HANDLE Subprocess::SetupPipe(HANDLE ioport) { char pipe_name[100]; snprintf(pipe_name, sizeof(pipe_name), "\\\\.\\pipe\\ninja_pid%lu_sp%p", GetCurrentProcessId(), this); pipe_ = ::CreateNamedPipeA(pipe_name, PIPE_ACCESS_INBOUND | FILE_FLAG_OVERLAPPED, PIPE_TYPE_BYTE, PIPE_UNLIMITED_INSTANCES, 0, 0, INFINITE, NULL); if (pipe_ == INVALID_HANDLE_VALUE) Win32Fatal("CreateNamedPipe"); if (!CreateIoCompletionPort(pipe_, ioport, (ULONG_PTR)this, 0)) Win32Fatal("CreateIoCompletionPort"); memset(&overlapped_, 0, sizeof(overlapped_)); if (!ConnectNamedPipe(pipe_, &overlapped_) && GetLastError() != ERROR_IO_PENDING) { Win32Fatal("ConnectNamedPipe"); } // Get the write end of the pipe as a handle inheritable across processes. HANDLE output_write_handle = CreateFile(pipe_name, GENERIC_WRITE, 0, NULL, OPEN_EXISTING, 0, NULL); HANDLE output_write_child; if (!DuplicateHandle(GetCurrentProcess(), output_write_handle, GetCurrentProcess(), &output_write_child, 0, TRUE, DUPLICATE_SAME_ACCESS)) { Win32Fatal("DuplicateHandle"); } CloseHandle(output_write_handle); return output_write_child; } bool Subprocess::Start(SubprocessSet* set, const string& command) { HANDLE child_pipe = SetupPipe(set->ioport_); SECURITY_ATTRIBUTES security_attributes; memset(&security_attributes, 0, sizeof(SECURITY_ATTRIBUTES)); security_attributes.nLength = sizeof(SECURITY_ATTRIBUTES); security_attributes.bInheritHandle = TRUE; // Must be inheritable so subprocesses can dup to children. HANDLE nul = CreateFile("NUL", GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, &security_attributes, OPEN_EXISTING, 0, NULL); if (nul == INVALID_HANDLE_VALUE) Fatal("couldn't open nul"); STARTUPINFOA startup_info; memset(&startup_info, 0, sizeof(startup_info)); startup_info.cb = sizeof(STARTUPINFO); startup_info.dwFlags = STARTF_USESTDHANDLES; startup_info.hStdInput = nul; startup_info.hStdOutput = child_pipe; startup_info.hStdError = child_pipe; PROCESS_INFORMATION process_info; memset(&process_info, 0, sizeof(process_info)); // Do not prepend 'cmd /c' on Windows, this breaks command // lines greater than 8,191 chars. if (!CreateProcessA(NULL, (char*)command.c_str(), NULL, NULL, /* inherit handles */ TRUE, CREATE_NEW_PROCESS_GROUP, NULL, NULL, &startup_info, &process_info)) { DWORD error = GetLastError(); if (error == ERROR_FILE_NOT_FOUND) { // File (program) not found error is treated as a normal build // action failure. if (child_pipe) CloseHandle(child_pipe); CloseHandle(pipe_); CloseHandle(nul); pipe_ = NULL; // child_ is already NULL; buf_ = "CreateProcess failed: The system cannot find the file " "specified.\n"; return true; } else { Win32Fatal("CreateProcess"); // pass all other errors to Win32Fatal } } // Close pipe channel only used by the child. if (child_pipe) CloseHandle(child_pipe); CloseHandle(nul); CloseHandle(process_info.hThread); child_ = process_info.hProcess; return true; } void Subprocess::OnPipeReady() { DWORD bytes; if (!GetOverlappedResult(pipe_, &overlapped_, &bytes, TRUE)) { if (GetLastError() == ERROR_BROKEN_PIPE) { CloseHandle(pipe_); pipe_ = NULL; return; } Win32Fatal("GetOverlappedResult"); } if (is_reading_ && bytes) buf_.append(overlapped_buf_, bytes); memset(&overlapped_, 0, sizeof(overlapped_)); is_reading_ = true; if (!::ReadFile(pipe_, overlapped_buf_, sizeof(overlapped_buf_), &bytes, &overlapped_)) { if (GetLastError() == ERROR_BROKEN_PIPE) { CloseHandle(pipe_); pipe_ = NULL; return; } if (GetLastError() != ERROR_IO_PENDING) Win32Fatal("ReadFile"); } // Even if we read any bytes in the readfile call, we'll enter this // function again later and get them at that point. } ExitStatus Subprocess::Finish() { if (!child_) return ExitFailure; // TODO: add error handling for all of these. WaitForSingleObject(child_, INFINITE); DWORD exit_code = 0; GetExitCodeProcess(child_, &exit_code); CloseHandle(child_); child_ = NULL; return exit_code == 0 ? ExitSuccess : exit_code == CONTROL_C_EXIT ? ExitInterrupted : ExitFailure; } bool Subprocess::Done() const { return pipe_ == NULL; } const string& Subprocess::GetOutput() const { return buf_; } HANDLE SubprocessSet::ioport_; SubprocessSet::SubprocessSet() { ioport_ = ::CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 1); if (!ioport_) Win32Fatal("CreateIoCompletionPort"); if (!SetConsoleCtrlHandler(NotifyInterrupted, TRUE)) Win32Fatal("SetConsoleCtrlHandler"); } SubprocessSet::~SubprocessSet() { Clear(); SetConsoleCtrlHandler(NotifyInterrupted, FALSE); CloseHandle(ioport_); } BOOL WINAPI SubprocessSet::NotifyInterrupted(DWORD dwCtrlType) { if (dwCtrlType == CTRL_C_EVENT || dwCtrlType == CTRL_BREAK_EVENT) { if (!PostQueuedCompletionStatus(ioport_, 0, 0, NULL)) Win32Fatal("PostQueuedCompletionStatus"); return TRUE; } return FALSE; } Subprocess *SubprocessSet::Add(const string& command) { Subprocess *subprocess = new Subprocess; if (!subprocess->Start(this, command)) { delete subprocess; return 0; } if (subprocess->child_) running_.push_back(subprocess); else finished_.push(subprocess); return subprocess; } bool SubprocessSet::DoWork() { DWORD bytes_read; Subprocess* subproc; OVERLAPPED* overlapped; if (!GetQueuedCompletionStatus(ioport_, &bytes_read, (PULONG_PTR)&subproc, &overlapped, INFINITE)) { if (GetLastError() != ERROR_BROKEN_PIPE) Win32Fatal("GetQueuedCompletionStatus"); } if (!subproc) // A NULL subproc indicates that we were interrupted and is // delivered by NotifyInterrupted above. return true; subproc->OnPipeReady(); if (subproc->Done()) { vector::iterator end = std::remove(running_.begin(), running_.end(), subproc); if (running_.end() != end) { finished_.push(subproc); running_.resize(end - running_.begin()); } } return false; } Subprocess* SubprocessSet::NextFinished() { if (finished_.empty()) return NULL; Subprocess* subproc = finished_.front(); finished_.pop(); return subproc; } void SubprocessSet::Clear() { for (vector::iterator i = running_.begin(); i != running_.end(); ++i) { if ((*i)->child_) { if (!GenerateConsoleCtrlEvent(CTRL_BREAK_EVENT, GetProcessId((*i)->child_))) { Win32Fatal("GenerateConsoleCtrlEvent"); } } } for (vector::iterator i = running_.begin(); i != running_.end(); ++i) delete *i; running_.clear(); } ninja-1.3.4/src/subprocess.h000066400000000000000000000047171215343310300157520ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_SUBPROCESS_H_ #define NINJA_SUBPROCESS_H_ #include #include #include using namespace std; #ifdef _WIN32 #include #else #include #endif #include "exit_status.h" /// Subprocess wraps a single async subprocess. It is entirely /// passive: it expects the caller to notify it when its fds are ready /// for reading, as well as call Finish() to reap the child once done() /// is true. struct Subprocess { ~Subprocess(); /// Returns ExitSuccess on successful process exit, ExitInterrupted if /// the process was interrupted, ExitFailure if it otherwise failed. ExitStatus Finish(); bool Done() const; const string& GetOutput() const; private: Subprocess(); bool Start(struct SubprocessSet* set, const string& command); void OnPipeReady(); string buf_; #ifdef _WIN32 /// Set up pipe_ as the parent-side pipe of the subprocess; return the /// other end of the pipe, usable in the child process. HANDLE SetupPipe(HANDLE ioport); HANDLE child_; HANDLE pipe_; OVERLAPPED overlapped_; char overlapped_buf_[4 << 10]; bool is_reading_; #else int fd_; pid_t pid_; #endif friend struct SubprocessSet; }; /// SubprocessSet runs a ppoll/pselect() loop around a set of Subprocesses. /// DoWork() waits for any state change in subprocesses; finished_ /// is a queue of subprocesses as they finish. struct SubprocessSet { SubprocessSet(); ~SubprocessSet(); Subprocess* Add(const string& command); bool DoWork(); Subprocess* NextFinished(); void Clear(); vector running_; queue finished_; #ifdef _WIN32 static BOOL WINAPI NotifyInterrupted(DWORD dwCtrlType); static HANDLE ioport_; #else static void SetInterruptedFlag(int signum); static bool interrupted_; struct sigaction old_act_; sigset_t old_mask_; #endif }; #endif // NINJA_SUBPROCESS_H_ ninja-1.3.4/src/subprocess_test.cc000066400000000000000000000123441215343310300171420ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "subprocess.h" #include "test.h" #ifndef _WIN32 // SetWithLots need setrlimit. #include #include #include #endif namespace { #ifdef _WIN32 const char* kSimpleCommand = "cmd /c dir \\"; #else const char* kSimpleCommand = "ls /"; #endif struct SubprocessTest : public testing::Test { SubprocessSet subprocs_; }; } // anonymous namespace // Run a command that fails and emits to stderr. TEST_F(SubprocessTest, BadCommandStderr) { Subprocess* subproc = subprocs_.Add("cmd /c ninja_no_such_command"); ASSERT_NE((Subprocess *) 0, subproc); while (!subproc->Done()) { // Pretend we discovered that stderr was ready for writing. subprocs_.DoWork(); } EXPECT_EQ(ExitFailure, subproc->Finish()); EXPECT_NE("", subproc->GetOutput()); } // Run a command that does not exist TEST_F(SubprocessTest, NoSuchCommand) { Subprocess* subproc = subprocs_.Add("ninja_no_such_command"); ASSERT_NE((Subprocess *) 0, subproc); while (!subproc->Done()) { // Pretend we discovered that stderr was ready for writing. subprocs_.DoWork(); } EXPECT_EQ(ExitFailure, subproc->Finish()); EXPECT_NE("", subproc->GetOutput()); #ifdef _WIN32 ASSERT_EQ("CreateProcess failed: The system cannot find the file " "specified.\n", subproc->GetOutput()); #endif } #ifndef _WIN32 TEST_F(SubprocessTest, InterruptChild) { Subprocess* subproc = subprocs_.Add("kill -INT $$"); ASSERT_NE((Subprocess *) 0, subproc); while (!subproc->Done()) { subprocs_.DoWork(); } EXPECT_EQ(ExitInterrupted, subproc->Finish()); } TEST_F(SubprocessTest, InterruptParent) { Subprocess* subproc = subprocs_.Add("kill -INT $PPID ; sleep 1"); ASSERT_NE((Subprocess *) 0, subproc); while (!subproc->Done()) { bool interrupted = subprocs_.DoWork(); if (interrupted) return; } ADD_FAILURE() << "We should have been interrupted"; } #endif TEST_F(SubprocessTest, SetWithSingle) { Subprocess* subproc = subprocs_.Add(kSimpleCommand); ASSERT_NE((Subprocess *) 0, subproc); while (!subproc->Done()) { subprocs_.DoWork(); } ASSERT_EQ(ExitSuccess, subproc->Finish()); ASSERT_NE("", subproc->GetOutput()); ASSERT_EQ(1u, subprocs_.finished_.size()); } TEST_F(SubprocessTest, SetWithMulti) { Subprocess* processes[3]; const char* kCommands[3] = { kSimpleCommand, #ifdef _WIN32 "cmd /c echo hi", "cmd /c time /t", #else "whoami", "pwd", #endif }; for (int i = 0; i < 3; ++i) { processes[i] = subprocs_.Add(kCommands[i]); ASSERT_NE((Subprocess *) 0, processes[i]); } ASSERT_EQ(3u, subprocs_.running_.size()); for (int i = 0; i < 3; ++i) { ASSERT_FALSE(processes[i]->Done()); ASSERT_EQ("", processes[i]->GetOutput()); } while (!processes[0]->Done() || !processes[1]->Done() || !processes[2]->Done()) { ASSERT_GT(subprocs_.running_.size(), 0u); subprocs_.DoWork(); } ASSERT_EQ(0u, subprocs_.running_.size()); ASSERT_EQ(3u, subprocs_.finished_.size()); for (int i = 0; i < 3; ++i) { ASSERT_EQ(ExitSuccess, processes[i]->Finish()); ASSERT_NE("", processes[i]->GetOutput()); delete processes[i]; } } // OS X's process limit is less than 1025 by default // (|sysctl kern.maxprocperuid| is 709 on 10.7 and 10.8 and less prior to that). #if defined(linux) || defined(__OpenBSD__) TEST_F(SubprocessTest, SetWithLots) { // Arbitrary big number; needs to be over 1024 to confirm we're no longer // hostage to pselect. const size_t kNumProcs = 1025; // Make sure [ulimit -n] isn't going to stop us from working. rlimit rlim; ASSERT_EQ(0, getrlimit(RLIMIT_NOFILE, &rlim)); ASSERT_GT(rlim.rlim_cur, kNumProcs) << "Raise [ulimit -n] well above " << kNumProcs << " to make this test go"; vector procs; for (size_t i = 0; i < kNumProcs; ++i) { Subprocess* subproc = subprocs_.Add("/bin/echo"); ASSERT_NE((Subprocess *) 0, subproc); procs.push_back(subproc); } while (!subprocs_.running_.empty()) subprocs_.DoWork(); for (size_t i = 0; i < procs.size(); ++i) { ASSERT_EQ(ExitSuccess, procs[i]->Finish()); ASSERT_NE("", procs[i]->GetOutput()); } ASSERT_EQ(kNumProcs, subprocs_.finished_.size()); } #endif // linux || __OpenBSD__ // TODO: this test could work on Windows, just not sure how to simply // read stdin. #ifndef _WIN32 // Verify that a command that attempts to read stdin correctly thinks // that stdin is closed. TEST_F(SubprocessTest, ReadStdin) { Subprocess* subproc = subprocs_.Add("cat -"); while (!subproc->Done()) { subprocs_.DoWork(); } ASSERT_EQ(ExitSuccess, subproc->Finish()); ASSERT_EQ(1u, subprocs_.finished_.size()); } #endif // _WIN32 ninja-1.3.4/src/test.cc000066400000000000000000000110501215343310300146630ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "test.h" #include #include #include "build_log.h" #include "manifest_parser.h" #include "util.h" #ifdef _WIN32 #include #endif namespace { #ifdef _WIN32 #ifndef _mktemp_s /// mingw has no mktemp. Implement one with the same type as the one /// found in the Windows API. int _mktemp_s(char* templ) { char* ofs = strchr(templ, 'X'); sprintf(ofs, "%d", rand() % 1000000); return 0; } #endif /// Windows has no mkdtemp. Implement it in terms of _mktemp_s. char* mkdtemp(char* name_template) { int err = _mktemp_s(name_template); if (err < 0) { perror("_mktemp_s"); return NULL; } err = _mkdir(name_template); if (err < 0) { perror("mkdir"); return NULL; } return name_template; } #endif // _WIN32 string GetSystemTempDir() { #ifdef _WIN32 char buf[1024]; if (!GetTempPath(sizeof(buf), buf)) return ""; return buf; #else const char* tempdir = getenv("TMPDIR"); if (tempdir) return tempdir; return "/tmp"; #endif } } // anonymous namespace StateTestWithBuiltinRules::StateTestWithBuiltinRules() { AddCatRule(&state_); } void StateTestWithBuiltinRules::AddCatRule(State* state) { AssertParse(state, "rule cat\n" " command = cat $in > $out\n"); } Node* StateTestWithBuiltinRules::GetNode(const string& path) { return state_.GetNode(path); } void AssertParse(State* state, const char* input) { ManifestParser parser(state, NULL); string err; ASSERT_TRUE(parser.ParseTest(input, &err)) << err; ASSERT_EQ("", err); } void AssertHash(const char* expected, uint64_t actual) { ASSERT_EQ(BuildLog::LogEntry::HashCommand(expected), actual); } void VirtualFileSystem::Create(const string& path, const string& contents) { files_[path].mtime = now_; files_[path].contents = contents; files_created_.insert(path); } TimeStamp VirtualFileSystem::Stat(const string& path) { FileMap::iterator i = files_.find(path); if (i != files_.end()) return i->second.mtime; return 0; } bool VirtualFileSystem::WriteFile(const string& path, const string& contents) { Create(path, contents); return true; } bool VirtualFileSystem::MakeDir(const string& path) { directories_made_.push_back(path); return true; // success } string VirtualFileSystem::ReadFile(const string& path, string* err) { files_read_.push_back(path); FileMap::iterator i = files_.find(path); if (i != files_.end()) return i->second.contents; return ""; } int VirtualFileSystem::RemoveFile(const string& path) { if (find(directories_made_.begin(), directories_made_.end(), path) != directories_made_.end()) return -1; FileMap::iterator i = files_.find(path); if (i != files_.end()) { files_.erase(i); files_removed_.insert(path); return 0; } else { return 1; } } void ScopedTempDir::CreateAndEnter(const string& name) { // First change into the system temp dir and save it for cleanup. start_dir_ = GetSystemTempDir(); if (start_dir_.empty()) Fatal("couldn't get system temp dir"); if (chdir(start_dir_.c_str()) < 0) Fatal("chdir: %s", strerror(errno)); // Create a temporary subdirectory of that. char name_template[1024]; strcpy(name_template, name.c_str()); strcat(name_template, "-XXXXXX"); char* tempname = mkdtemp(name_template); if (!tempname) Fatal("mkdtemp: %s", strerror(errno)); temp_dir_name_ = tempname; // chdir into the new temporary directory. if (chdir(temp_dir_name_.c_str()) < 0) Fatal("chdir: %s", strerror(errno)); } void ScopedTempDir::Cleanup() { if (temp_dir_name_.empty()) return; // Something went wrong earlier. // Move out of the directory we're about to clobber. if (chdir(start_dir_.c_str()) < 0) Fatal("chdir: %s", strerror(errno)); #ifdef _WIN32 string command = "rmdir /s /q " + temp_dir_name_; #else string command = "rm -rf " + temp_dir_name_; #endif if (system(command.c_str()) < 0) Fatal("system: %s", strerror(errno)); temp_dir_name_.clear(); } ninja-1.3.4/src/test.h000066400000000000000000000054661215343310300145430ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_TEST_H_ #define NINJA_TEST_H_ #include #include "disk_interface.h" #include "state.h" #include "util.h" // Support utilites for tests. struct Node; /// A base test fixture that includes a State object with a /// builtin "cat" rule. struct StateTestWithBuiltinRules : public testing::Test { StateTestWithBuiltinRules(); /// Add a "cat" rule to \a state. Used by some tests; it's /// otherwise done by the ctor to state_. void AddCatRule(State* state); /// Short way to get a Node by its path from state_. Node* GetNode(const string& path); State state_; }; void AssertParse(State* state, const char* input); void AssertHash(const char* expected, uint64_t actual); /// An implementation of DiskInterface that uses an in-memory representation /// of disk state. It also logs file accesses and directory creations /// so it can be used by tests to verify disk access patterns. struct VirtualFileSystem : public DiskInterface { VirtualFileSystem() : now_(1) {} /// "Create" a file with contents. void Create(const string& path, const string& contents); /// Tick "time" forwards; subsequent file operations will be newer than /// previous ones. int Tick() { return ++now_; } // DiskInterface virtual TimeStamp Stat(const string& path); virtual bool WriteFile(const string& path, const string& contents); virtual bool MakeDir(const string& path); virtual string ReadFile(const string& path, string* err); virtual int RemoveFile(const string& path); /// An entry for a single in-memory file. struct Entry { int mtime; string contents; }; vector directories_made_; vector files_read_; typedef map FileMap; FileMap files_; set files_removed_; set files_created_; /// A simple fake timestamp for file operations. int now_; }; struct ScopedTempDir { /// Create a temporary directory and chdir into it. void CreateAndEnter(const string& name); /// Clean up the temporary directory. void Cleanup(); /// The temp directory containing our dir. string start_dir_; /// The subdirectory name for our dir, or empty if it hasn't been set up. string temp_dir_name_; }; #endif // NINJA_TEST_H_ ninja-1.3.4/src/timestamp.h000066400000000000000000000017241215343310300155600ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_TIMESTAMP_H_ #define NINJA_TIMESTAMP_H_ // When considering file modification times we only care to compare // them against one another -- we never convert them to an absolute // real time. On POSIX we use time_t (seconds since epoch) and on // Windows we use a different value. Both fit in an int. typedef int TimeStamp; #endif // NINJA_TIMESTAMP_H_ ninja-1.3.4/src/util.cc000066400000000000000000000220161215343310300146650ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "util.h" #ifdef _WIN32 #include #include #include #endif #include #include #include #include #include #include #include #include #ifndef _WIN32 #include #include #endif #include #if defined(__APPLE__) || defined(__FreeBSD__) #include #elif defined(__SVR4) && defined(__sun) #include #include #elif defined(linux) || defined(__GLIBC__) #include #endif #include "edit_distance.h" #include "metrics.h" void Fatal(const char* msg, ...) { va_list ap; fprintf(stderr, "ninja: fatal: "); va_start(ap, msg); vfprintf(stderr, msg, ap); va_end(ap); fprintf(stderr, "\n"); #ifdef _WIN32 // On Windows, some tools may inject extra threads. // exit() may block on locks held by those threads, so forcibly exit. fflush(stderr); fflush(stdout); ExitProcess(1); #else exit(1); #endif } void Warning(const char* msg, ...) { va_list ap; fprintf(stderr, "ninja: warning: "); va_start(ap, msg); vfprintf(stderr, msg, ap); va_end(ap); fprintf(stderr, "\n"); } void Error(const char* msg, ...) { va_list ap; fprintf(stderr, "ninja: error: "); va_start(ap, msg); vfprintf(stderr, msg, ap); va_end(ap); fprintf(stderr, "\n"); } bool CanonicalizePath(string* path, string* err) { METRIC_RECORD("canonicalize str"); size_t len = path->size(); char* str = 0; if (len > 0) str = &(*path)[0]; if (!CanonicalizePath(str, &len, err)) return false; path->resize(len); return true; } bool CanonicalizePath(char* path, size_t* len, string* err) { // WARNING: this function is performance-critical; please benchmark // any changes you make to it. METRIC_RECORD("canonicalize path"); if (*len == 0) { *err = "empty path"; return false; } const int kMaxPathComponents = 30; char* components[kMaxPathComponents]; int component_count = 0; char* start = path; char* dst = start; const char* src = start; const char* end = start + *len; if (*src == '/') { #ifdef _WIN32 // network path starts with // if (*len > 1 && *(src + 1) == '/') { src += 2; dst += 2; } else { ++src; ++dst; } #else ++src; ++dst; #endif } while (src < end) { if (*src == '.') { if (src + 1 == end || src[1] == '/') { // '.' component; eliminate. src += 2; continue; } else if (src[1] == '.' && (src + 2 == end || src[2] == '/')) { // '..' component. Back up if possible. if (component_count > 0) { dst = components[component_count - 1]; src += 3; --component_count; } else { *dst++ = *src++; *dst++ = *src++; *dst++ = *src++; } continue; } } if (*src == '/') { src++; continue; } if (component_count == kMaxPathComponents) Fatal("path has too many components : %s", path); components[component_count] = dst; ++component_count; while (*src != '/' && src != end) *dst++ = *src++; *dst++ = *src++; // Copy '/' or final \0 character as well. } if (dst == start) { *err = "path canonicalizes to the empty path"; return false; } *len = dst - start - 1; return true; } int ReadFile(const string& path, string* contents, string* err) { FILE* f = fopen(path.c_str(), "r"); if (!f) { err->assign(strerror(errno)); return -errno; } char buf[64 << 10]; size_t len; while ((len = fread(buf, 1, sizeof(buf), f)) > 0) { contents->append(buf, len); } if (ferror(f)) { err->assign(strerror(errno)); // XXX errno? contents->clear(); fclose(f); return -errno; } fclose(f); return 0; } void SetCloseOnExec(int fd) { #ifndef _WIN32 int flags = fcntl(fd, F_GETFD); if (flags < 0) { perror("fcntl(F_GETFD)"); } else { if (fcntl(fd, F_SETFD, flags | FD_CLOEXEC) < 0) perror("fcntl(F_SETFD)"); } #else HANDLE hd = (HANDLE) _get_osfhandle(fd); if (! SetHandleInformation(hd, HANDLE_FLAG_INHERIT, 0)) { fprintf(stderr, "SetHandleInformation(): %s", GetLastErrorString().c_str()); } #endif // ! _WIN32 } const char* SpellcheckStringV(const string& text, const vector& words) { const bool kAllowReplacements = true; const int kMaxValidEditDistance = 3; int min_distance = kMaxValidEditDistance + 1; const char* result = NULL; for (vector::const_iterator i = words.begin(); i != words.end(); ++i) { int distance = EditDistance(*i, text, kAllowReplacements, kMaxValidEditDistance); if (distance < min_distance) { min_distance = distance; result = *i; } } return result; } const char* SpellcheckString(const char* text, ...) { // Note: This takes a const char* instead of a string& because using // va_start() with a reference parameter is undefined behavior. va_list ap; va_start(ap, text); vector words; const char* word; while ((word = va_arg(ap, const char*))) words.push_back(word); va_end(ap); return SpellcheckStringV(text, words); } #ifdef _WIN32 string GetLastErrorString() { DWORD err = GetLastError(); char* msg_buf; FormatMessageA( FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, err, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), (char*)&msg_buf, 0, NULL); string msg = msg_buf; LocalFree(msg_buf); return msg; } void Win32Fatal(const char* function) { Fatal("%s: %s", function, GetLastErrorString().c_str()); } #endif static bool islatinalpha(int c) { // isalpha() is locale-dependent. return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z'); } string StripAnsiEscapeCodes(const string& in) { string stripped; stripped.reserve(in.size()); for (size_t i = 0; i < in.size(); ++i) { if (in[i] != '\33') { // Not an escape code. stripped.push_back(in[i]); continue; } // Only strip CSIs for now. if (i + 1 >= in.size()) break; if (in[i + 1] != '[') continue; // Not a CSI. i += 2; // Skip everything up to and including the next [a-zA-Z]. while (i < in.size() && !islatinalpha(in[i])) ++i; } return stripped; } #if defined(linux) || defined(__GLIBC__) int GetProcessorCount() { return get_nprocs(); } #elif defined(__APPLE__) || defined(__FreeBSD__) int GetProcessorCount() { int processors; size_t processors_size = sizeof(processors); int name[] = {CTL_HW, HW_NCPU}; if (sysctl(name, sizeof(name) / sizeof(int), &processors, &processors_size, NULL, 0) < 0) { return 0; } return processors; } #elif defined(_WIN32) int GetProcessorCount() { SYSTEM_INFO info; GetSystemInfo(&info); return info.dwNumberOfProcessors; } #else // This is what get_nprocs() should be doing in the Linux implementation // above, but in a more standard way. int GetProcessorCount() { return sysconf(_SC_NPROCESSORS_ONLN); } #endif #if defined(_WIN32) || defined(__CYGWIN__) double GetLoadAverage() { // TODO(nicolas.despres@gmail.com): Find a way to implement it on Windows. // Remember to also update Usage() when this is fixed. return -0.0f; } #else double GetLoadAverage() { double loadavg[3] = { 0.0f, 0.0f, 0.0f }; if (getloadavg(loadavg, 3) < 0) { // Maybe we should return an error here or the availability of // getloadavg(3) should be checked when ninja is configured. return -0.0f; } return loadavg[0]; } #endif // _WIN32 string ElideMiddle(const string& str, size_t width) { const int kMargin = 3; // Space for "...". string result = str; if (result.size() + kMargin > width) { size_t elide_size = (width - kMargin) / 2; result = result.substr(0, elide_size) + "..." + result.substr(result.size() - elide_size, elide_size); } return result; } bool Truncate(const string& path, size_t size, string* err) { #ifdef _WIN32 int fh = _sopen(path.c_str(), _O_RDWR | _O_CREAT, _SH_DENYNO, _S_IREAD | _S_IWRITE); int success = _chsize(fh, size); _close(fh); #else int success = truncate(path.c_str(), size); #endif // Both truncate() and _chsize() return 0 on success and set errno and return // -1 on failure. if (success < 0) { *err = strerror(errno); return false; } return true; } ninja-1.3.4/src/util.h000066400000000000000000000057361215343310300145410ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_UTIL_H_ #define NINJA_UTIL_H_ #ifdef _WIN32 #include "win32port.h" #else #include #endif #include #include using namespace std; #ifdef _MSC_VER #define NORETURN __declspec(noreturn) #else #define NORETURN __attribute__((noreturn)) #endif /// Log a fatal message and exit. NORETURN void Fatal(const char* msg, ...); /// Log a warning message. void Warning(const char* msg, ...); /// Log an error message. void Error(const char* msg, ...); /// Canonicalize a path like "foo/../bar.h" into just "bar.h". bool CanonicalizePath(string* path, string* err); bool CanonicalizePath(char* path, size_t* len, string* err); /// Read a file to a string (in text mode: with CRLF conversion /// on Windows). /// Returns -errno and fills in \a err on error. int ReadFile(const string& path, string* contents, string* err); /// Mark a file descriptor to not be inherited on exec()s. void SetCloseOnExec(int fd); /// Given a misspelled string and a list of correct spellings, returns /// the closest match or NULL if there is no close enough match. const char* SpellcheckStringV(const string& text, const vector& words); /// Like SpellcheckStringV, but takes a NULL-terminated list. const char* SpellcheckString(const char* text, ...); /// Removes all Ansi escape codes (http://www.termsys.demon.co.uk/vtansi.htm). string StripAnsiEscapeCodes(const string& in); /// @return the number of processors on the machine. Useful for an initial /// guess for how many jobs to run in parallel. @return 0 on error. int GetProcessorCount(); /// @return the load average of the machine. A negative value is returned /// on error. double GetLoadAverage(); /// Elide the given string @a str with '...' in the middle if the length /// exceeds @a width. string ElideMiddle(const string& str, size_t width); /// Truncates a file to the given size. bool Truncate(const string& path, size_t size, string* err); #ifdef _MSC_VER #define snprintf _snprintf #define fileno _fileno #define unlink _unlink #define chdir _chdir #define strtoull _strtoui64 #define getcwd _getcwd #define PATH_MAX _MAX_PATH #endif #ifdef _WIN32 /// Convert the value returned by GetLastError() into a string. string GetLastErrorString(); /// Calls Fatal() with a function name and GetLastErrorString. NORETURN void Win32Fatal(const char* function); #endif #endif // NINJA_UTIL_H_ ninja-1.3.4/src/util_test.cc000066400000000000000000000106321215343310300157250ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "util.h" #include "test.h" TEST(CanonicalizePath, PathSamples) { string path; string err; EXPECT_FALSE(CanonicalizePath(&path, &err)); EXPECT_EQ("empty path", err); path = "foo.h"; err = ""; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("foo.h", path); path = "./foo.h"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("foo.h", path); path = "./foo/./bar.h"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("foo/bar.h", path); path = "./x/foo/../bar.h"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("x/bar.h", path); path = "./x/foo/../../bar.h"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("bar.h", path); path = "foo//bar"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("foo/bar", path); path = "foo//.//..///bar"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("bar", path); path = "./x/../foo/../../bar.h"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("../bar.h", path); path = "foo/./."; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("foo", path); path = "foo/bar/.."; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("foo", path); path = "foo/.hidden_bar"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("foo/.hidden_bar", path); path = "/foo"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("/foo", path); path = "//foo"; EXPECT_TRUE(CanonicalizePath(&path, &err)); #ifdef _WIN32 EXPECT_EQ("//foo", path); #else EXPECT_EQ("/foo", path); #endif path = "/"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("", path); } TEST(CanonicalizePath, EmptyResult) { string path; string err; EXPECT_FALSE(CanonicalizePath(&path, &err)); EXPECT_EQ("empty path", err); path = "."; EXPECT_FALSE(CanonicalizePath(&path, &err)); EXPECT_EQ("path canonicalizes to the empty path", err); path = "./."; EXPECT_FALSE(CanonicalizePath(&path, &err)); EXPECT_EQ("path canonicalizes to the empty path", err); } TEST(CanonicalizePath, UpDir) { string path, err; path = "../../foo/bar.h"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("../../foo/bar.h", path); path = "test/../../foo/bar.h"; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("../foo/bar.h", path); } TEST(CanonicalizePath, AbsolutePath) { string path = "/usr/include/stdio.h"; string err; EXPECT_TRUE(CanonicalizePath(&path, &err)); EXPECT_EQ("/usr/include/stdio.h", path); } TEST(CanonicalizePath, NotNullTerminated) { string path; string err; size_t len; path = "foo/. bar/."; len = strlen("foo/."); // Canonicalize only the part before the space. EXPECT_TRUE(CanonicalizePath(&path[0], &len, &err)); EXPECT_EQ(strlen("foo"), len); EXPECT_EQ("foo/. bar/.", string(path)); path = "foo/../file bar/."; len = strlen("foo/../file"); EXPECT_TRUE(CanonicalizePath(&path[0], &len, &err)); EXPECT_EQ(strlen("file"), len); EXPECT_EQ("file ./file bar/.", string(path)); } TEST(StripAnsiEscapeCodes, EscapeAtEnd) { string stripped = StripAnsiEscapeCodes("foo\33"); EXPECT_EQ("foo", stripped); stripped = StripAnsiEscapeCodes("foo\33["); EXPECT_EQ("foo", stripped); } TEST(StripAnsiEscapeCodes, StripColors) { // An actual clang warning. string input = "\33[1maffixmgr.cxx:286:15: \33[0m\33[0;1;35mwarning: " "\33[0m\33[1musing the result... [-Wparentheses]\33[0m"; string stripped = StripAnsiEscapeCodes(input); EXPECT_EQ("affixmgr.cxx:286:15: warning: using the result... [-Wparentheses]", stripped); } TEST(ElideMiddle, NothingToElide) { string input = "Nothing to elide in this short string."; EXPECT_EQ(input, ElideMiddle(input, 80)); } TEST(ElideMiddle, ElideInTheMiddle) { string input = "01234567890123456789"; string elided = ElideMiddle(input, 10); EXPECT_EQ("012...789", elided); } ninja-1.3.4/src/version.cc000066400000000000000000000033041215343310300153740ustar00rootroot00000000000000// Copyright 2013 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #include "version.h" #include #include "util.h" const char* kNinjaVersion = "1.3.4"; void ParseVersion(const string& version, int* major, int* minor) { size_t end = version.find('.'); *major = atoi(version.substr(0, end).c_str()); *minor = 0; if (end != string::npos) { size_t start = end + 1; end = version.find('.', start); *minor = atoi(version.substr(start, end).c_str()); } } void CheckNinjaVersion(const string& version) { int bin_major, bin_minor; ParseVersion(kNinjaVersion, &bin_major, &bin_minor); int file_major, file_minor; ParseVersion(version, &file_major, &file_minor); if (bin_major > file_major) { Warning("ninja executable version (%s) greater than build file " "ninja_required_version (%s); versions may be incompatible.", kNinjaVersion, version.c_str()); return; } if ((bin_major == file_major && bin_minor < file_minor) || bin_major < file_major) { Fatal("ninja version (%s) incompatible with build file " "ninja_required_version version (%s).", kNinjaVersion, version.c_str()); } } ninja-1.3.4/src/version.h000066400000000000000000000021601215343310300152350ustar00rootroot00000000000000// Copyright 2013 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_VERSION_H_ #define NINJA_VERSION_H_ #include using namespace std; /// The version number of the current Ninja release. This will always /// be "git" on trunk. extern const char* kNinjaVersion; /// Parse the major/minor components of a version string. void ParseVersion(const string& version, int* major, int* minor); /// Check whether \a version is compatible with the current Ninja version, /// aborting if not. void CheckNinjaVersion(const string& required_version); #endif // NINJA_VERSION_H_ ninja-1.3.4/src/win32port.h000066400000000000000000000017161215343310300154250ustar00rootroot00000000000000// Copyright 2012 Google Inc. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. #ifndef NINJA_WIN32PORT_H_ #define NINJA_WIN32PORT_H_ typedef signed short int16_t; typedef unsigned short uint16_t; /// A 64-bit integer type typedef signed long long int64_t; typedef unsigned long long uint64_t; // printf format specifier for uint64_t, from C99. #ifndef PRIu64 #define PRIu64 "I64u" #define PRIx64 "I64x" #endif #endif // NINJA_WIN32PORT_H_