pax_global_header00006660000000000000000000000064136312231660014515gustar00rootroot0000000000000052 comment=25c570cdb50e11c19aac5f2398e845b570eca76b ctemplate-ctemplate-2.4/000077500000000000000000000000001363122316600153145ustar00rootroot00000000000000ctemplate-ctemplate-2.4/AUTHORS000066400000000000000000000000431363122316600163610ustar00rootroot00000000000000google-ctemplate@googlegroups.com ctemplate-ctemplate-2.4/COPYING000066400000000000000000000027071363122316600163550ustar00rootroot00000000000000Copyright (c) 2005, Google Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of Google Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ctemplate-ctemplate-2.4/ChangeLog000066400000000000000000000325441363122316600170760ustar00rootroot00000000000000Sun Mar 08 18:00:14 2020 Olaf van der Spek * ctemplate: version 2.4 release * Switch to Python 3 Wed Mar 19 20:02:51 2014 Olaf van der Spek * ctemplate: version 2.3 release * Fixed some more C++11 issues. * Removed deprecated Template::SetEscapedValueAndShowSection() * and Template::ReloadIfChanged(). Wed Apr 18 23:36:08 2012 Olaf van der Spek * ctemplate: version 2.2 release * Fixed several issues reported by G++ 4.7 Thu Mar 22 00:29:09 2012 Olaf van der Spek * ctemplate: version 2.1 release * Added operator[] to TemplateDictionary * Added begin(), end(), data() and size() to TemplateString * Refactored code Tue Jan 24 16:00:51 2012 Google Inc. * ctemplate: version 2.0 release * Changed the 'official' ctemplate email in setup.py/etc * Rewrote docs to refer to Ctemplate, not Google Template * Renamed google-ctemplate.sln to ctemplate.sln * Changed copyright text to reflect Google's relinquished ownership Thu Dec 22 14:03:27 2011 Google Inc. * ctemplate: version 1.1 release * Make reloads always prefer files earlier on the search patch (ssas) * PORTING: Add a default BSWAP32, needed for AIX (csilvers) * Die at configure-time when g++ isn't installed Fri Aug 26 13:10:46 2011 Google Inc. * ctemplate: version 1.0 release * No changes since 1.0rc2 Fri Jul 29 12:19:50 2011 Google Inc. * ctemplate: version 1.0rc2 release * BUGFIX: removed #includes of non-installed headers from installed ones * BUGFIX: Make compile-test test against an install Fri Jul 22 15:31:05 2011 Google Inc. * ctemplate: version 1.0rc1 release * BUGFIX: Dumpers were ignoring a tpl-string 'length' (csilvers) * BUGFIX: Add a needed $(top_srcdir) to Makefile.am * Replace 0x00-0x1F with space in xml_escape (alexvod) * Fix xml encoding to handle form-feed properly (ghurley) * DOCUMENTATION: Wer missing some docs in the tarball (csilvers) * BUILD: Update acx_pthread.m4 for nostdlib (Kacper) * BUGFIX: Make CreateSectionIterator check tpl-globals (williasr) * PORTING: Fix solaris build by getting rid of PATH_MAX (csilvers) Mon Jan 24 15:38:31 2011 Google Inc. * ctemplate: version 0.99 release * Accept . as part of tag/attr names when autoescaping (falmeida) * Optimize javascript template escaping (blount) * Allow inside :H=snippet modifiers (jdtang) * make_tpl_varnames can write multiple tpls to one file (jad) * Add a few escaping modifier equivalences (jad) * BUGFIX: Fix ReloadAllIfChanged() with path (panicker) * PORTING: Relace tr with more portable sed, in tests (csilvers) * Updated from autoconf 2.64 to autoconf 2.65 Fri Sep 24 11:38:27 2010 Google Inc. * ctemplate: version 0.98 release * Add new image URL modifier: :I=html, :I=css, etc (jshneier, dougy) * Allow lowercase words in pragma contexts (csilvers) * Enable full-word matching for tpl filenames (aneeshnaman) * Performance improvements for small_map (wonchun) * Avoid some compiler warnings (mac) * Fix a c++ conformance bug involving const (chandlerc) * BUGFIX: never reload string templates (panicker) * BUGFIX: Fix refcounting to avoid accessing freed memory (panicker) * PORTING: Avoid SIBGUS on sparc by aligning memory more (csilvers) * PORTING: Use isascii_is*() to avoid langtype issues (csilvers) * PORTING: Fix 'class not properly dll-epxorted' warnings (csilvers) * PORTING: Rename README.windows to README_windows.txt (csilvers) * Made NEWS file non-empty. * Added ctemplate.pc + ctemplate_nothreads.pc for pkg-config (csilvers) Mon Apr 19 15:55:23 2010 Google Inc. * ctemplate: version 0.97 release * Major API revamp: +TemplateCache, -Template (panicker, csilvers) * Major documentation revamp: howto -> guide + reference (csilvers) * Protect auto-generated #include files with header guard (dnovillo) * Allow ftp:// in ValidateUrl modifier (martone) * Speed up template modifiers (turnidge) * BUGFIX: Fix a dnagling pointer in template-modifiers (csilvers) * PORTING: cygwin/mingw now compile (due to autoconf macros) (csilvers) * PORTING: improve generate_fs_test on windows (csilvers) Mon Oct 19 11:42:57 2009 Google Inc. * ctemplate: version 0.96 release * Support multiple template root directories (williasr) * Remove TemplateContext from TemplateToString (jad) * Remove programmatic auto-escape code (jad) * Speedup: use linker to detect immutable TemplateStrings (csilvers) * Implement ShowTemplateGlobalSection (williasr) * Change SetIntValue to accept longs (omoikane) * Allow TemplateDictionary c'tor to take a TemplateString (csilvers) * Make Mutex safer to run during global destructors (csilvers) * BUILD: don't set TMPDIR when building; ar reads it (csilvers) * BUILD: fix compiling of C (as opposed to C++) test file (csilvers) * BUILD: remove bash-specific 'test x == y' from configure.ac (csilvers) * BUGFIX: use AllocAligned when allocating non-char objects (csilvers) * PORTING: remove obsolete MSVC Detect64BitPortabilityProblems (csilvers) * Updated from autoconf 2.61 to 2.62 Fri Jun 12 08:13:35 2009 Google Inc. * ctemplate: version 0.95 release * Change default namespace from google to ctemplate * Rename include directory from google to ctemplate * Remove deprecated code: template_from_string.{h,cc} * Remove deprecated code: modifiers in TemplateDictionary class * Remove last arg from StringToTemplate; use autoescape pragma instead * Remove template_modifiers namespace * Provide a script to help convert namespaces and #includes * TemplateDictionary constructor takes TemplateString, not string Thu May 7 11:27:28 2009 Google Inc. * ctemplate: version 0.94 release * Use arena for all memory allocations: 3-4% speedup (csilvers) * Add the ability to hook the annotation system (ryoji) * Expose Expand(ExpandEmitter*,...) to allow custom emitters (csilvers) * Add RemoveStringFromTemplateCache (csilvers) * Add new :url_escape_with_arg=css modifier for urls in CSS (jad) * Support tr1's unordered_map in preference to hash_map (csilvers) * Use Murmurhash for all string hashing, rather than hash<> (csilvers) * Better parsing of meta tags and dangling < for auto-escape (falmeida) * Add AddXssSafeModifier (jad) * Allow disabling auto-escape for 'trusted' vars (jad) * BUGFIX: resolve possible memory-leaks in CopyDictionary (csilvers) * BUGFIX: fix bug when reloading with AUTOESCAPE pragma (jad) * Updated autoconf version to 2.61 and libtool version to 1.5.26 Mon Feb 2 13:51:33 2009 Google Inc. * ctemplate: version 0.93 release * New AUTOESCAPE pragma: alternative to GetTemplateWithAutoescaping (jad) * Improve autoescape for CSS template and more (jad) * Replace RegisterStringAsTemplate with TemplateToString (csilvers) * Speed up template code by hashing and cacheing template strings (jcrim) * Add StaticTemplateString to make hashing/cacheing even faster * Autogenerate parser files from .config files (falmeida) * TESTING: new compile_test to test end-to-end compilation (csilvers) * Fix Mutex class to allow expanding template before main() (csilvers) Wed Nov 12 11:24:33 2008 Google Inc. * ctemplate: version 0.92 release * BUG FIX: affecting an included string-template with autoescaping (jad) * BUG FIX: long-name cmdline flag in diff_tpl_auto_escape (jad) * Do include-template indenting after applying modifiers (jad) * Add human-readable error messages to the html parser (falmeida) * Create an abstract base class for TemplateDictionary (williasr) * PORTING: Be more careful about Windows #defines and types (csilvers) * PORTING: Make unittest scripts more sh-compatible (falmeida) * Show missing includes in annotated output (ribrdb) * Improvements to javascript escaping (escape \f) (jad) * Improvements to json escaping (escape <>&) (jad) * Avoid raciness by preferring cached to new template-strings (csilvers) Tue Aug 19 16:20:27 2008 Google Inc. * ctemplate: version 0.91 release * NEW FEATURE: "separator" sections (csilvers) * NEW FEATURE: changing the markup separator via {{=XXX YYY=}} (csilvers) * NEW FEATURE: Add an Expand modifier to template dictionary (turnidge) * TemplateAsString class -> Template::RegisterStringAsTemplate (edwardcu) * TemplateModifier class -> PerExpandData class (csilvers) * Store more data in the arena, shrinking TemplateDictionary (csilvers) * Use TemplateString more widely internally; small speedup (adamk) * Lazily initialize hash-maps, saving time and space (csilvers) * Make template annotations more efficient (turnidge) * PORTABILITY: Fixed a testing script to run on solaris x86 (csilvers) * Fix namespaces to start with :: (csilvers) * Improve html parser in various ways (falmeida) Sun Apr 27 15:06:15 2008 Google Inc. * ctemplate: version 0.90 release * MAJOR NEW FEATURE: New "auto-escape" mode (jad, falmeida, mughda) * New tool, diff_tpl_auto_escape, to convert templates to auto-escaping * Fixed make install to install test datafiles (csilvers) * Add a new modifier: javascript_escape_with_arg=number (jad) * Javascript modifier now escapes all js newlines (msamuel) Thu Jan 24 16:09:43 2008 Google Inc. * ctemplate: version 0.9 release * Bugfix: now we honor "preserve newlines in javacript" (ktl) * Fix indentation of included templates (csilvers) * Deprecate the SetEscaped* methods in favor of modifiers (csilvers) * Revamp the way template modifiers are registered (csilvers) * Change the psuedo-name of main section from __MAIN__ to __{{MAIN}}__ * Fix types to quiet windows compiler (csilvers) * Allow infile and outfile for template-converter (ambrose) * Some doc fixes Thu Aug 16 21:42:55 2007 Google Inc. * ctemplate: version 0.8 release * Add the ability to dynamically add modifiers (ribrdb) * Support per-Expand() data for template modifiers (ribrdb) * New commandline flag -f for make_tpl_varnames_h (herbertc) * Windows: give debug dll and release dll different names (csilvers) * A few fixups of Windows includes, based on user reports (csilvers) Tue Jul 3 12:46:59 2007 Google Inc. * ctemplate: version 0.7 release * Bugfix: another lock hygiene fix, for recursive includes (csilvers) * Minor type-hygiene improvements: size_t for int, etc. (csilvers) * Porting improvements: tests pass on OS X, FreeBSD, Solaris (csilvers) * Windows port! -- VS solution provided for all unittests (csilvers) Thu Jun 21 14:02:32 2007 Google Inc. * ctemplate: version 0.6.1 release * Bugfix: data corruption bug with >2 template modifiers (jmacgill) * Bugfix: syntax error in template-namelist: configure-bug (csilvers) * Bugfix: improve lock hygiene to avoid potential deadlock (csilvers) Sat Jun 9 22:34:52 2007 Google Inc. * ctemplate: version 0.6 release * Use computed includes for hash_map/set: easier config (csilvers) * Added all used .m4 templates to the distribution (csilvers) * Beefed-up and revamped modifier code (csilvers) * New modifiers for url-escaping, attribute-cleansing, etc (ribrdb) * Annotations now include modifier information (csilvers) * Support embedded NULs in template names and values (csilvers) Mon May 14 17:27:10 2007 Google Inc. * ctemplate: version 0.5 release * Add new MakeCopy() method to copy template dictionaries (csilvers) * Add JSON-escaping (mikepurvis) * Internal change that should ease thread-handling a bit (csilvers) * Fix url_query_escape to avoid stack overflow (csilvers) Mon Jan 15 14:10:42 2007 Google Inc. * ctemplate: version 0.4 release * Improve html-escaping by adding single-quote (bdangelo) * Improve javascript-escaping by adding more characters too (smknappy) * Add url-escaping, for url query parameters (dcoker) * Add support for "pre" escaping, which preserves whitespace (dboswell) * Typo fixes in documentation (csilvers) * Expand() returns false if a template file failed to load (jmittleman) Mon Aug 21 17:44:32 2006 Google Inc. * ctemplate: version 0.3 release * New contrib/ directory entry: vi syntax highlighting (patlac) * New contrib/ directory entry: emacs syntax highlighting (tonyg) * Allow escape-modifiers to affect includes, not just vars (csilvers) separating logic from presentation. * Add JSON escape-functor (majewski) Wed Jun 14 14:56:04 2006 Google Inc. * ctemplate: version 0.2 release * API CHANGE: new typedefs (and types) for namelist functions (csilvers) * carry annotation status through to child templates/sections (ehamon) * Support registering templates after global construct time (njain) * Add pthread-rwlock support for darwin (csilvers) * SetTemplateGlobalValue(): new variable type with new scoping (ehamon) * Export a nothreads version of the ctemplate library (csilvers) * Got rid of scandir call, which should improve portability (csilvers) Mon Mar 13 22:20:46 2006 Google Inc. * ctemplate: initial release: The ctemplate package contains a library implementing a simple but powerful template language for C++. It emphasizes ctemplate-ctemplate-2.4/Makefile.am000077500000000000000000000620261363122316600173610ustar00rootroot00000000000000## Process this file with automake to produce Makefile.in # Make sure that when we re-make ./configure, we get the macros we need ACLOCAL_AMFLAGS = -I m4 # This is so we can #include AM_CPPFLAGS = -I$(top_srcdir)/src # These are good warnings to turn on by default if GCC AM_CXXFLAGS = -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare endif dist_doc_DATA = AUTHORS ChangeLog NEWS README # mingw does weird directory munging on /tmp, which causes some of our # tests to fail, so use a different tmpdir there. if MINGW TMPDIR = ./ dist_doc_DATA += README_windows.txt else TMPDIR = /tmp endif ctemplateincludedir = $(includedir)/ctemplate ## The .h files you want to install (that is, .h files that people ## who install this package can include in their own applications.) ## We have to include both the .h and .h.in forms. The latter we ## put in noinst_HEADERS. ## NOTE: If you add a file here, also add it to the end of configure.ac nodist_ctemplateinclude_HEADERS = \ src/ctemplate/template.h \ src/ctemplate/template_cache.h \ src/ctemplate/template_string.h \ src/ctemplate/template_enums.h \ src/ctemplate/template_pathops.h \ src/ctemplate/template_modifiers.h \ src/ctemplate/template_dictionary.h \ src/ctemplate/template_dictionary_interface.h \ src/ctemplate/template_annotator.h \ src/ctemplate/template_emitter.h \ src/ctemplate/template_namelist.h \ src/ctemplate/per_expand_data.h \ src/ctemplate/str_ref.h noinst_HEADERS = \ src/ctemplate/template.h.in \ src/ctemplate/template_cache.h.in \ src/ctemplate/template_string.h.in \ src/ctemplate/template_enums.h.in \ src/ctemplate/template_pathops.h.in \ src/ctemplate/template_modifiers.h.in \ src/ctemplate/template_dictionary.h.in \ src/ctemplate/template_dictionary_interface.h.in \ src/ctemplate/template_annotator.h.in \ src/ctemplate/template_emitter.h.in \ src/ctemplate/template_namelist.h.in \ src/ctemplate/per_expand_data.h.in \ src/ctemplate/str_ref.h.in ## This is for HTML and other documentation you want to install. ## Add your documentation files (in doc/) in addition to these ## top-level boilerplate files. Also add a TODO file if you have one. dist_html_DATA = doc/designstyle.css doc/index.html \ doc/howto.html doc/guide.html doc/reference.html \ doc/tips.html doc/example.html doc/auto_escape.html \ doc/xss_resources.html ## The libraries (.so's) you want to install lib_LTLIBRARIES = ## Libraries that we want to use only internally (for our unittest binaries) noinst_LTLIBRARIES = ## The binaries you want to install bin_PROGRAMS = bin_SCRIPTS = ## The location of the windows project file for each binary we make WINDOWS_PROJECTS = ctemplate.sln ## unittests you want to run when people type 'make check'. ## TESTS is for binary unittests, check_SCRIPTS for script-based unittests. ## TESTS_ENVIRONMENT sets environment variables for when you run unittest, ## but it only seems to take effect for *binary* unittests (argh!) TESTS = check_SCRIPTS = TESTS_ENVIRONMENT = TEMPLATE_ROOTDIR=$(top_srcdir) # Every time you add a unittest to check_SCRIPTS, add it here too noinst_SCRIPTS = # Add to this for tests that use data TESTDATA = ## vvvv RULES TO MAKE THE LIBRARIES, BINARIES, AND UNITTESTS # These files are auto-generated via generate_fsm.py. Since we don't want # to require python to build ctemplate, we include these in the dist src/htmlparser/htmlparser_fsm.h: src/htmlparser/generate_fsm.py src/htmlparser/fsm_config.py src/htmlparser/htmlparser_fsm.config $(top_srcdir)/src/htmlparser/generate_fsm.py $(top_srcdir)/src/htmlparser/htmlparser_fsm.config > $@ src/htmlparser/jsparser_fsm.h: src/htmlparser/generate_fsm.py src/htmlparser/fsm_config.py src/htmlparser/jsparser_fsm.config $(top_srcdir)/src/htmlparser/generate_fsm.py $(top_srcdir)/src/htmlparser/jsparser_fsm.config > $@ src/tests/statemachine_test_fsm.h: src/htmlparser/generate_fsm.py src/htmlparser/fsm_config.py src/tests/statemachine_test_fsm.config $(top_srcdir)/src/htmlparser/generate_fsm.py $(top_srcdir)/src/tests/statemachine_test_fsm.config > $@ # This is a required hack for auto-generated .h files: cf the automake info pages # NOTE: since we put these .h files in BUILT_SOURCES, we don't need to put # them in as deps for the binaries that use them. In fact, it's important # that we don't; instead we put them in EXTRA_DIST. This mean, in practice, # they'll go at the end of the distribution tarfile, which means they'll # have a later timestamp than the .config files that generate them, which # means users won't try to rebuild them. Ah, the joys of automake. BUILT_SOURCES = src/htmlparser/htmlparser_fsm.h \ src/htmlparser/jsparser_fsm.h \ src/tests/statemachine_test_fsm.h # These are the symbols (classes, mostly) we want to export from our library. # Note this regexp applies to the *mangled* name, which is why we have to # be careful where we want to assert [^A-Za-z]. (Particularly bad on Darwin.) # template_modifiers and BaseArena and UnsafeArena perhaps shouldn't be # exported, but they're needed -- at least -- for some unittests. CTEMPLATE_SYMBOLS = '(ctemplate|template_modifiers|BaseArena|UnsafeArena|[^A-Za-z]Template[^A-Za-z]|TemplateDictionary|TemplateNamelist|TemplateFromString|TemplateString|TemplateState|[^A-Za-z]Strip[^A-Za-z])' lib_LTLIBRARIES += libctemplate.la WINDOWS_PROJECTS += vsprojects/libctemplate/libctemplate.vcxproj libctemplate_la_SOURCES = $(nodist_ctemplateinclude_HEADERS) \ src/base/arena-inl.h \ src/base/arena.cc \ src/base/arena.h \ src/base/fileutil.h \ src/base/macros.h \ src/base/manual_constructor.h \ src/base/mutex.h \ src/base/small_map.h \ src/base/thread_annotations.h \ src/base/util.h \ src/indented_writer.h \ src/per_expand_data.cc \ src/template.cc \ src/template_annotator.cc \ src/template_cache.cc \ src/template_dictionary.cc \ src/template_modifiers.cc \ src/template_modifiers_internal.h \ src/template_namelist.cc \ src/template_pathops.cc \ src/template_string.cc \ src/htmlparser/htmlparser.cc \ src/htmlparser/htmlparser.h \ src/htmlparser/htmlparser_cpp.h \ src/htmlparser/jsparser.cc \ src/htmlparser/jsparser.h \ src/htmlparser/statemachine.cc \ src/htmlparser/statemachine.h libctemplate_la_CXXFLAGS = $(PTHREAD_CFLAGS) -DNDEBUG $(AM_CXXFLAGS) # -version-info gets passed to libtool libctemplate_la_LDFLAGS = $(PTHREAD_CFLAGS) \ -export-symbols-regex $(CTEMPLATE_SYMBOLS) \ -no-undefined \ -version-info @SO_VERSION@ libctemplate_la_LIBADD = $(PTHREAD_LIBS) # For MinGW, we need to bring in the windows port files if MINGW libctemplate_la_SOURCES += src/windows/port.h src/windows/port.cc endif MINGW # automake will make different .o files for this library, which is good, # because we use an extra compiler flag. lib_LTLIBRARIES += libctemplate_nothreads.la libctemplate_nothreads_la_SOURCES = $(libctemplate_la_SOURCES) libctemplate_nothreads_la_DEPENDENCIES = $(libctemplate_la_DEPENDENCIES) libctemplate_nothreads_la_CXXFLAGS = -DNDEBUG -DNO_THREADS $(AM_CXXFLAGS) libctemplate_nothreads_la_LDFLAGS = -export-symbols-regex $(CTEMPLATE_SYMBOLS) \ -no-undefined \ -version-info @SO_VERSION@ # For our tests, we want versions of these libraries that include asserts. noinst_LTLIBRARIES += libctemplate_debug.la libctemplate_debug_la_SOURCES = $(libctemplate_la_SOURCES) libctemplate_debug_la_DEPENDENCIES = $(libctemplate_la_DEPENDENCIES) libctemplate_debug_la_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) libctemplate_debug_la_LDFLAGS = $(libctemplate_la_LDFLAGS) libctemplate_debug_la_LIBADD = $(libctemplate_la_LIBADD) noinst_LTLIBRARIES += libctemplate_nothreads_debug.la libctemplate_nothreads_debug_la_SOURCES = $(libctemplate_nothreads_la_SOURCES) libctemplate_nothreads_debug_la_DEPENDENCIES = $(libctemplate_nothreads_la_DEPENDENCIES) libctemplate_nothreads_debug_la_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) libctemplate_nothreads_debug_la_LDFLAGS = $(libctemplate_nothreads_la_LDFLAGS) # We could also make a library that has the TemplateDictionaryPeer # class. This class is useful for testing (it provides introspection # on the TemplateDictionary hierarchy that's easier to use than the # default debugging method, TemplateDictionary::Dump()) but by the # same token also violates abstraction in ways that might be fragile. # We don't expose the library for now, until the demonstrated need # outweighs the costs. If you'd like to use this library, please send # mail to google-ctemplate@googlegroups.com! ##lib_LTLIBRARIES += libctemplate_testing.la noinst_LTLIBRARIES += libctemplate_testing.la libctemplate_testing_nothreads.la libctemplate_testing_la_SOURCES = $(nodist_ctemplateinclude_HEADERS) \ src/tests/template_test_util.h \ src/tests/template_test_util.cc libctemplate_testing_la_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) CTEMPLATE_TESTING_SYMBOLS = 'TemporaryRegisterTemplate|TemplateDictionaryPeer' libctemplate_testing_la_LDFLAGS = $(PTHREAD_CFLAGS) \ -export-symbols-regex $(CTEMPLATE_TESTING_SYMBOLS) \ -no-undefined \ -version-info @SO_VERSION@ libctemplate_testing_la_LIBADD = $(PTHREAD_LIBS) libctemplate_testing_nothreads_la_SOURCES = $(libctemplate_testing_la_SOURCES) libctemplate_testing_nothreads_la_CXXFLAGS = $(AM_CXXFLAGS) -DNO_THREADS libctemplate_testing_nothreads_la_LDFLAGS = -export-symbols-regex $(CTEMPLATE_TESTING_SYMBOLS) \ -no-undefined \ -version-info @SO_VERSION@ # This library depends on libctemplate, but it can use either libctemplate # or libctemplate_nothreads. Rather than have two versions of this # library, I just leave out the deps entirely: you must be sure to specify # one of the two whenever you link this library into your code. #libctemplate_testing_la_LIBADD = libctemplate.la # or libctemplate_nothreads.la # Helper apps bin_PROGRAMS += make_tpl_varnames_h WINDOWS_PROJECTS += vsprojects/make_tpl_varname_h/make_tpl_varname_h.vcxproj make_tpl_varnames_h_SOURCES = $(nodist_ctemplateinclude_HEADERS) \ src/make_tpl_varnames_h.cc make_tpl_varnames_h_LDADD = libctemplate_nothreads.la bin_PROGRAMS += diff_tpl_auto_escape WINDOWS_PROJECTS += vsprojects/diff_tpl_auto_escape/diff_tpl_auto_escape.vcxproj diff_tpl_auto_escape_SOURCES = $(nodist_ctemplateinclude_HEADERS) \ src/diff_tpl_auto_escape.cc diff_tpl_auto_escape_LDADD = libctemplate_nothreads.la bin_SCRIPTS += src/template-converter # For each of the tests, we test with and without threads TESTS += compile_test compile_nothreads_test WINDOWS_PROJECTS += vsprojects/compile_test/compile_test.vcxproj compile_test_SOURCES = $(nodist_ctemplateinclude_HEADERS) \ src/tests/compile_test.cc compile_test_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) compile_test_LDFLAGS = $(PTHREAD_CFLAGS) compile_test_LDADD = libctemplate_debug.la $(PTHREAD_LIBS) compile_nothreads_test_SOURCES = $(compile_test_SOURCES) compile_nothreads_test_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) compile_nothreads_test_LDADD = libctemplate_nothreads_debug.la # With compile_test, we are making sure that the code compiles without # needing config.h/etc. The best test is to make sure we can compile # with just the installed header files. The best way to do this is at # installcheck time. When this rule is run (via 'make distcheck'), # 'includedir' is the installed include-dir. installcheck-local: $(CXX) -I $(includedir) -c $(top_srcdir)/src/tests/compile_test.cc \ -o installcheck_compile_test.o TESTS += template_test_util_test template_test_util_nothreads_test WINDOWS_PROJECTS += vsprojects/template_test_util_test/template_test_util_test.vcxproj template_test_util_test_SOURCES = src/tests/config_for_unittests.h \ src/template_dictionary.cc \ src/tests/template_test_util_test.cc template_test_util_test_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) template_test_util_test_LDFLAGS = $(PTHREAD_CFLAGS) template_test_util_test_LDADD = libctemplate_testing.la libctemplate_debug.la \ $(PTHREAD_LIBS) template_test_util_nothreads_test_SOURCES = $(template_test_util_test_SOURCES) template_test_util_nothreads_test_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) template_test_util_nothreads_test_LDADD = libctemplate_testing_nothreads.la \ libctemplate_nothreads_debug.la TESTS += template_dictionary_unittest template_dictionary_nothreads_unittest WINDOWS_PROJECTS += vsprojects/template_dictionary_unittest/template_dictionary_unittest.vcxproj template_dictionary_unittest_SOURCES = src/tests/config_for_unittests.h \ src/base/arena.h \ src/tests/template_dictionary_unittest.cc template_dictionary_unittest_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) template_dictionary_unittest_LDFLAGS = $(PTHREAD_CFLAGS) template_dictionary_unittest_LDADD = libctemplate_testing.la libctemplate_debug.la \ $(PTHREAD_LIBS) template_dictionary_nothreads_unittest_SOURCES = $(template_dictionary_unittest_SOURCES) template_dictionary_nothreads_unittest_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) template_dictionary_nothreads_unittest_LDADD = libctemplate_testing_nothreads.la \ libctemplate_nothreads_debug.la TESTS += template_modifiers_unittest template_modifiers_nothreads_unittest WINDOWS_PROJECTS += vsprojects/template_modifiers_unittest/template_modifiers_unittest.vcxproj template_modifiers_unittest_SOURCES = src/tests/config_for_unittests.h \ src/tests/template_modifiers_unittest.cc template_modifiers_unittest_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) template_modifiers_unittest_LDFLAGS = $(PTHREAD_CFLAGS) template_modifiers_unittest_LDADD = libctemplate_testing.la libctemplate_debug.la \ $(PTHREAD_LIBS) template_modifiers_nothreads_unittest_SOURCES = $(template_modifiers_unittest_SOURCES) template_modifiers_nothreads_unittest_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) template_modifiers_nothreads_unittest_LDADD = libctemplate_testing_nothreads.la \ libctemplate_nothreads_debug.la TESTS += template_setglobals_unittest template_setglobals_nothreads_unittest WINDOWS_PROJECTS += vsprojects/template_setglobals_unittest/template_setglobals_unittest.vcxproj template_setglobals_unittest_SOURCES = src/tests/config_for_unittests.h \ src/tests/template_setglobals_unittest.cc template_setglobals_unittest_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) template_setglobals_unittest_LDFLAGS = $(PTHREAD_CFLAGS) template_setglobals_unittest_LDADD = libctemplate_testing.la libctemplate_debug.la \ $(PTHREAD_LIBS) template_setglobals_nothreads_unittest_SOURCES = $(template_setglobals_unittest_SOURCES) template_setglobals_nothreads_unittest_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) template_setglobals_nothreads_unittest_LDADD = libctemplate_testing_nothreads.la \ libctemplate_nothreads_debug.la TESTS += template_cache_test template_cache_nothreads_test WINDOWS_PROJECTS += vsprojects/template_cache_test/template_cache_test.vcxproj template_cache_test_SOURCES = src/tests/config_for_unittests.h \ src/template_modifiers_internal.h \ src/tests/template_cache_test.cc template_cache_test_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) template_cache_test_LDFLAGS = $(PTHREAD_CFLAGS) template_cache_test_LDADD = libctemplate_testing.la libctemplate_debug.la \ $(PTHREAD_LIBS) template_cache_nothreads_test_SOURCES = $(template_cache_test_SOURCES) template_cache_nothreads_test_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) template_cache_nothreads_test_LDADD = libctemplate_testing_nothreads.la \ libctemplate_nothreads_debug.la TESTS += template_unittest template_nothreads_unittest WINDOWS_PROJECTS += vsprojects/template_unittest/template_unittest.vcxproj WINDOWS_PROJECTS += vsprojects/template_unittest_static/template_unittest_static.vcxproj template_unittest_SOURCES = src/tests/config_for_unittests.h \ src/template_modifiers_internal.h \ src/tests/template_unittest.cc template_unittest_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) template_unittest_LDFLAGS = $(PTHREAD_CFLAGS) template_unittest_LDADD = libctemplate_testing.la libctemplate_debug.la \ $(PTHREAD_LIBS) template_nothreads_unittest_SOURCES = $(template_unittest_SOURCES) template_nothreads_unittest_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) template_nothreads_unittest_LDADD = libctemplate_testing_nothreads.la \ libctemplate_nothreads_debug.la TESTS += template_regtest template_nothreads_regtest WINDOWS_PROJECTS += vsprojects/template_regtest/template_regtest.vcxproj template_regtest_SOURCES = src/tests/template_regtest.cc template_regtest_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) template_regtest_LDFLAGS = $(PTHREAD_CFLAGS) template_regtest_LDADD = libctemplate_debug.la $(PTHREAD_LIBS) template_nothreads_regtest_SOURCES = $(template_regtest_SOURCES) template_nothreads_regtest_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) template_nothreads_regtest_LDADD = libctemplate_nothreads_debug.la TESTDATA += \ src/tests/template_unittest_test_footer.in \ src/tests/template_unittest_test_html.in \ src/tests/template_unittest_test_invalid1.in \ src/tests/template_unittest_test_invalid2.in \ src/tests/template_unittest_test_markerdelim.in \ src/tests/template_unittest_test_modifiers.in \ src/tests/template_unittest_test_nul.in \ src/tests/template_unittest_test_selective_css.in \ src/tests/template_unittest_test_selective_html.in \ src/tests/template_unittest_test_selective_js.in \ src/tests/template_unittest_test_simple.in \ src/tests/template_unittest_test_valid1.in \ src/tests/template_unittest_test_footer_dict01.out \ src/tests/template_unittest_test_footer_dict02.out \ src/tests/template_unittest_test_html_dict01.out \ src/tests/template_unittest_test_html_dict02.out \ src/tests/template_unittest_test_markerdelim_dict01.out \ src/tests/template_unittest_test_markerdelim_dict02.out \ src/tests/template_unittest_test_modifiers_dict01.out \ src/tests/template_unittest_test_nul_dict01.out \ src/tests/template_unittest_test_selective_css_dict01.out \ src/tests/template_unittest_test_selective_css_dict02.out \ src/tests/template_unittest_test_selective_html_dict01.out \ src/tests/template_unittest_test_selective_html_dict02.out \ src/tests/template_unittest_test_selective_js_dict01.out \ src/tests/template_unittest_test_selective_js_dict02.out \ src/tests/template_unittest_test_simple_dict01.out \ src/tests/template_unittest_test_simple_dict02.out \ src/tests/template_unittest_test_simple_dict03.out \ src/tests/template_unittest_test_valid1_dict01.out TESTS += htmlparser_test htmlparser_nothreads_test WINDOWS_PROJECTS += vsprojects/htmlparser_test/htmlparser_test.vcxproj htmlparser_test_SOURCES = src/tests/htmlparser_cpp_test.cc htmlparser_test_CXXFLAGS = $(PTHREAD_CFLAGS) $(AM_CXXFLAGS) htmlparser_test_LDFLAGS = $(PTHREAD_CFLAGS) htmlparser_test_LDADD = libctemplate_debug.la $(PTHREAD_LIBS) htmlparser_nothreads_test_SOURCES = $(htmlparser_test_SOURCES) htmlparser_nothreads_test_CXXFLAGS = -DNO_THREADS $(AM_CXXFLAGS) htmlparser_nothreads_test_LDADD = libctemplate_nothreads_debug.la TESTDATA += \ src/tests/htmlparser_testdata/cdata.html \ src/tests/htmlparser_testdata/comments.html \ src/tests/htmlparser_testdata/context.html \ src/tests/htmlparser_testdata/google.html \ src/tests/htmlparser_testdata/javascript_attribute.html \ src/tests/htmlparser_testdata/javascript_block.html \ src/tests/htmlparser_testdata/javascript_regexp.html \ src/tests/htmlparser_testdata/position.html \ src/tests/htmlparser_testdata/reset.html \ src/tests/htmlparser_testdata/simple.html \ src/tests/htmlparser_testdata/tags.html # We want to compile statemachine.cc as a c file as well as a c++ file statemachine.c: $(top_srcdir)/src/htmlparser/statemachine.cc cp $< $@ BUILT_SOURCES += statemachine.c CLEANFILES = statemachine.c TESTS += statemachine_test WINDOWS_PROJECTS += vsprojects/statemachine_test/statemachine_test.vcxproj statemachine_test_SOURCES = src/tests/statemachine_test.c \ statemachine.c statemachine_test_CFLAGS = -I$(top_srcdir)/src/htmlparser TESTS += generate_fsm_c_test WINDOWS_PROJECTS += vsprojects/generate_fsm_c_test/generate_fsm_c_test.vcxproj generate_fsm_c_test_SOURCES = src/tests/generate_fsm_c_test.c generate_fsm_c_test_DEPENDENCIES = src/tests/htmlparser_testdata/sample_fsm.c # This is something only maintainers need (since they're the only ones # who generate .h or .c files from .config files). check_SCRIPTS += generate_fsm_test_sh noinst_SCRIPTS += src/tests/generate_fsm_test.sh generate_fsm_test_sh: src/tests/generate_fsm_test.sh \ src/tests/htmlparser_testdata/sample_fsm.config \ src/tests/htmlparser_testdata/sample_fsm.c \ src/htmlparser/generate_fsm.py sh $(top_srcdir)/src/tests/generate_fsm_test.sh $(top_srcdir) TESTDATA += \ src/tests/htmlparser_testdata/sample_fsm.config \ src/tests/htmlparser_testdata/sample_fsm.c check_SCRIPTS += make_tpl_varnames_h_unittest_sh noinst_SCRIPTS += src/tests/make_tpl_varnames_h_unittest.sh make_tpl_varnames_h_unittest_sh: src/tests/make_tpl_varnames_h_unittest.sh \ make_tpl_varnames_h sh $(top_srcdir)/src/tests/make_tpl_varnames_h_unittest.sh \ $(top_builddir)/make_tpl_varnames_h $(TMPDIR)/$@_dir check_SCRIPTS += diff_tpl_auto_escape_unittest_sh noinst_SCRIPTS += src/tests/diff_tpl_auto_escape_unittest.sh diff_tpl_auto_escape_unittest_sh: src/tests/diff_tpl_auto_escape_unittest.sh \ diff_tpl_auto_escape sh $(top_srcdir)/src/tests/diff_tpl_auto_escape_unittest.sh \ $(top_builddir)/diff_tpl_auto_escape $(TMPDIR)/$@_dir ## ^^^^ END OF RULES TO MAKE THE LIBRARIES, BINARIES, AND UNITTESTS ## This should always include $(TESTS), but may also include other ## binaries that you compile but don't want automatically installed. check_PROGRAMS = $(TESTS) rpm: dist-gzip packages/rpm.sh packages/rpm/rpm.spec @cd packages && ./rpm.sh ${PACKAGE} ${VERSION} deb: dist-gzip packages/deb.sh packages/deb/* @cd packages && ./deb.sh ${PACKAGE} ${VERSION} # http://linux.die.net/man/1/pkg-config, http://pkg-config.freedesktop.org/wiki pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = lib${PACKAGE}.pc lib${PACKAGE}_nothreads.pc CLEANFILES += $(pkgconfig_DATA) # I get the description and URL lines from the rpm spec. I use sed to # try to rewrite exec_prefix, libdir, and includedir in terms of # prefix, if possible. lib${PACKAGE}.pc: Makefile packages/rpm/rpm.spec echo 'prefix=$(prefix)' > "$@".tmp echo 'exec_prefix='`echo '$(exec_prefix)' | sed 's@^$(prefix)@$${prefix}@'` >> "$@".tmp echo 'libdir='`echo '$(libdir)' | sed 's@^$(exec_prefix)@$${exec_prefix}@'` >> "$@".tmp echo 'includedir='`echo '$(includedir)' | sed 's@^$(prefix)@$${prefix}@'` >> "$@".tmp echo '' >> "$@".tmp echo 'Name: $(PACKAGE)' >> "$@".tmp echo 'Version: $(VERSION)' >> "$@".tmp -grep '^Summary:' $(top_srcdir)/packages/rpm/rpm.spec | sed s/^Summary:/Description:/ | head -n1 >> "$@".tmp -grep '^URL: ' $(top_srcdir)/packages/rpm/rpm.spec >> "$@".tmp echo 'Requires:' >> "$@".tmp echo 'Libs: -L$${libdir} -l$(PACKAGE)' >> "$@".tmp echo 'Libs.private: $(PTHREAD_CFLAGS) $(PTHREAD_LIBS)' >> "$@".tmp echo 'Cflags: -I$${includedir}' >> "$@".tmp mv -f "$@".tmp "$@" # The nothreads version is mostly the same lib${PACKAGE}_nothreads.pc: lib${PACKAGE}.pc grep -v Libs.private lib${PACKAGE}.pc | sed s/-l$(PACKAGE)/-l$(PACKAGE)_nothreads/ > "$@" # Windows wants write permission to .vcxproj files and maybe even sln files. dist-hook: test -e "$(distdir)/vsprojects" \ && chmod -R u+w $(distdir)/*.sln $(distdir)/vsprojects/ ## If you're using libtool, add 'libtool' here. Also add this rule: libtool: $(LIBTOOL_DEPS) $(SHELL) ./config.status --recheck # In addition to the normal stuff, we include all the windows-specific # code, and also the code used to generate the html-parser fsm files. # NOTE: BUILT_SOURCES should come after the .config files, so they # come later (and thus end up with a later timestamp) when we tar/untar. EXTRA_DIST = packages/rpm.sh packages/rpm/rpm.spec packages/deb.sh packages/deb \ $(SCRIPTS) $(TESTDATA) libtool autogen.sh \ src/htmlparser/generate_fsm.py src/htmlparser/fsm_config.py \ src/tests/statemachine_test_fsm.config \ src/htmlparser/htmlparser_fsm.config \ src/htmlparser/jsparser_fsm.config \ $(BUILT_SOURCES) \ src/windows/config.h src/windows/preprocess.sh \ src/windows/ctemplate $(WINDOWS_PROJECTS) \ src/solaris/libstdc++.la contrib ctemplate-ctemplate-2.4/NEWS000066400000000000000000000277331363122316600160270ustar00rootroot00000000000000== 08 March 2020 == Ctemplate 2.4 Switch to Python 3 == 19 March 2014 == Ctemplate 2.3 has been released. Fixed some more C++11 issues. Removed deprecated Template::SetEscapedValueAndShowSection() and Template::ReloadIfChanged(). == 18 April 2012 == Ctemplate 2.2 has been released. Several issues reported by G++ 4.7 have been fixed. == 22 March 2012 == 2.1 has been released. operator[] has been added to TemplateDictionary. == 24 January 2012 == I've just released ctemplate 2.0. It has no functional changes from ctemplate 1.2. The `google-ctemplate` project has been renamed to `ctemplate`. I (csilvers) am stepping down as maintainer, to be replaced by Olaf van der Spek. Welcome to the team, Olaf! I've been impressed by your contributions to the project discussions and code to date, and look forward to having you on the team. I bumped the major version number up to 2 to reflect the new community ownership of the project. All the [http://google-ctemplate.googlecode.com/svn/tags/ctemplate-2.0/ChangeLog changes] are related to the renaming. === 18 January 2011 === The `google-ctemplate` Google Code page has been renamed to `ctemplate`, in preparation for the project being renamed to `ctemplate`. In the coming weeks, I'll be stepping down as maintainer for the ctemplate project, and as part of that Google is relinquishing ownership of the project; it will now be entirely community run. The name change reflects that shift. === 22 December 2011 === I've just released ctemplate 1.1. The only functionality change is obscure: when a reload is done (via, say, `ReloadAllIfChanged`), and a new file has been created since the last reload that both a) has the same filename as a template file that had been loaded in the past, and b) is earlier on the template search-path than the previously loaded file, *and* c) the previously loaded file hasn't changed since the last reload, then at reload-time we now load the new file into the template, replacing the old file. Before we would only load the new file if the old file had changed on disk, and would otherwise leave the template alone. Even more minor changes are in the [http://google-ctemplate.googlecode.com/svn/tags/ctemplate-1.1/ChangeLog ChangeLog]. === 26 August 2011 === I've just released ctemplate 1.0! (I've decided 4 weeks is well within the definition of "the next week or so"...) A bit anti-climactic: there are no changes from ctemplate 1.0rc2. === 29 July 2011 === I've just released ctemplate 1.0rc2. This fixes a serious bug where I had added #includes in installed header files, that tried to include non-installed header files. This means it was impossible to use installed ctemplate; it only worked if you were using it from the tarball directory. I also fixed the unittest that was supposed to catch this, but didn't (it was also building in the tarball directory). If no further problems are found in the next week or so, I will release ctemplate 1.0. === 22 July 2011 === I've just released ctemplate 1.0rc1. If no problems are found in the next week or two, I will release ctemplate 1.0. ctemplate 1.0rc1 has relatively few changes from ctemplate 0.99. xml-escaping has been improved a bit. A couple of bugs have been fixed, including one where we were ignoring template-global sections (a relatively new feature) in some places. A full list of changes is available in the [http://google-ctemplate.googlecode.com/svn/tags/ctemplate-1.0rc1/ChangeLog ChangeLog]. I've also changed the internal tools used to integrate Google-supplied patches to ctemplate into the opensource release. These new tools should result in more frequent updates with better change descriptions. They will also result in future ChangeLog entries being much more verbose (for better or for worse). === 24 January 2011 === I've just released ctemplate 0.99. I expect this to be the last release before ctemplate 1.0. Code has settled down; the big change here is some efficiency improvements to javascript template escaping. There is also a bugfix for an obscure case where ReloadAllIfChanged() is used with multiple ctemplate search paths, where files are deleted in one part of the search path between reloads. Unless you need either of the above, there's no particular reason to upgrade. A full list of changes is available in the [http://google-ctemplate.googlecode.com/svn/tags/ctemplate-0.99/ChangeLog ChangeLog]. === 23 September 2010 === I've just released ctemplate 0.98. The changes are settling down as we approach ctemplate 1.0 -- a few new default modifiers, a few performance tweaks, a few portability improvements, but nothing disruptive. In my testing for this release, I noticed that the template regression test (but not other template tests) would segfault on gcc 4.1.1 when compiled with -O2. This seems pretty clearly to be a compiler bug; if you need to use gcc 4.1.1 to compile ctemplate, you may wish to build via `./configure CXXFLAGS="-O1 -g"` just to be safe. === 20 April 2010 === I've just released ctemplate 0.97. This change consists primarily of a significant change to the API: the addition of the `TemplateCache` class, combined with deprecation of the `Template` class. `TemplateCache` is a class that holds a collection of templates; this concept always existed in ctemplate, but was not previously exposed. Many static methods of the `Template` class, such as `ReloadAllIfChanged()`, have become methods on `TemplateCache` instead (the `Template` methods have been retained for backwards compatibility.) Other methods, such as `Expand()`, have become free functions. In fact, the entire `Template` class has been deprecated. The deprecation of `Template` calls for changes in all clients of the template code -- you can see in the example at the top of this page how the code has changed from `Template* tpl = ctemplate::Template::GetTemplate("example.tpl", ctemplate::DO_NOT_STRIP); tpl->Expand(&output, &dict);` to `ctemplate::ExpandTemplate("example.tpl", ctemplate::DO_NOT_STRIP, &dict, &output);`. These changes will make the code simpler and more thread-safe. Old code should continue to work -- the `Template` class remains -- but new code should use the new API, and old code should transition as convenient. One old API method is intrinsically thread-unsafe, and should be prioritized to change: `tpl->ReloadIfChanged` should change to `ctemplate::Template::ReloadAllIfChanged()`. Note this is a semantic change: all templates are now reloaded, rather than just one. However, since templates are reloaded lazily, and only if they've changed on disk, I'm hopeful it will always be a reasonable change to make. To go along with these changes, the documentation has been almost entirely revamped and made more accessible. Obscure ctemplate features have been excised from the user's guide and moved into a separate reference document. The new API is fully documented, including new flexibility around reloading templates, made available by the introduction of `TemplateCache`. There are some more minor changes as well, such as the addition of #include guards in the auto-generated .tpl.h files, to make it safe to multiply-include them. I've also been continuing the portability work: ctemplate should now work under Cygwin and MinGW. A full list of changes is available in the [http://google-ctemplate.googlecode.com/svn/tags/ctemplate-0.97/ChangeLog ChangeLog]. I know I've said this before, but I don't expect major API changes before the 1.0 release. The most significant changes I expect to see are the potential removal of some of the 'forwarding' methods in the (deprecated) `Template` class. === 12 June 2009 === I've just released ctemplate 0.95. This is entirely an API cleanup release. Actually, relatively little of the API proper has changed: `StringToTemplate` no longer takes an autoescape-context arg (instead you specify this as part of the template-string, using the `AUTOESCAPE` pragma). A few obsolete constructs have gone away, such as the `TemplateFromString` class and `TemplateDictionary::html_escape` and friends (just use the top-level `html_escape`). See the [http://google-ctemplate.googlecode.com/svn/tags/ctemplate-0.95/ChangeLog ChangeLog] for a full list of these changes. The biggest change is a renaming: the default namespace is now `ctemplate` rather than `google`, and the include directory is `ctemplate` rather than `google`. Other namespaces, such as `template_modifiers`, have gone away. All these changes will require you to modify your old code to get it working with ctemplate 0.95. I've written a [http://google-ctemplate.googlecode.com/svn/trunk/contrib/convert_to_95.pl script] to help you do that. Please open an [http://code.google.com/p/google-ctemplate/issues/list issue] if you see a problem with the script. I've tested it, but not as widely as I'd like. Also note the script will not be perfect for more complex constructs, which you will have to clean up by hand. I hope (expect) the API is now stable, and we won't see any more such changes before ctemplate 1.0. I tried to isolate them all in this release; except for the API changes, this release should behave identically to ctemplate 0.94. === 7 May 2009 === I've just released ctemplate 0.94. A few new features have been added, such as the ability to expand a template into your own custom `ExpandEmitter` instance, and the ability to hook the annotation system (typically used for debugging). You can now remove strings from the template cache in addition to adding them. Also, there continues to be a trickle of new modifiers, in this case a modifier for URL's in a CSS context. However, the most invasive changes were made for speed reasons. The biggest is that (almost) all `TemplateDictionary` allocations are now done on the arena -- this includes allocations by the STL classes inside the dictionary. This allows us to free all the memory at once, rather than item by item, and has yielded a 3-4% speed improvement in our tests. I've combined this with a `small_map` class that stores items in a vector instead of a hash-map until we get to 3 or 4 items; this gives another speed increase in the (common) case a template has only a few sections or includes. I also changed the hashing code to use [http://murmurhash.googlepages.com/ MurmurHash] everywhere, rather than the string hash function built into the STL library. This should be faster. All these changes should not be outwardly visible, but they do use more advanced features of C++ than ctemplate has to date. This may result in some problems compiling, or conceivably when running. If you see any, please file an [http://code.google.com/p/google-ctemplate/issues/list issue report]. You can see a full list of changes on the [http://google-ctemplate.googlecode.com/svn/tags/ctemplate-0.94/ChangeLog ChangeLog]. === 20 August 2008 === ctemplate 0.91 introduces the beginning of some API changes, as I look to clean up the API in preparation for ctemplate 1.0. After 1.0, the API will remain backwards compatible, but until that time, the API may change. Please take a look at the [http://google-ctemplate.googlecode.com/svn/trunk/ChangeLog ChangeLog] to see if any of these changes affect you. One change is the introduction of a new `PerExpandData` class, which holds some state that was formerly in the `TemplateDictionary` class. I'm still not sure if this class is a good idea, if it should be separate from `TemplateDictionary` or a member, or what functionality should move there (for instance, should `SetTemplateGlobal` move there, since template-global variables are really, in some sense, per-expand variables?) If you have any feedback, ideally based on your own experience using the current API, feel free to post it at `google-ctemplate@googlegroups.com`. ctemplate also has several new features, including the addition of "separator" sections, and the ability to change the markup character (from `{{`). See the [http://google-ctemplate.googlecode.com/svn/trunk/ChangeLog ChangeLog] for a complete list, and the [http://google-ctemplate.googlecode.com/svn/trunk/doc/howto.html howto documentation] for more details on these new features. ctemplate-ctemplate-2.4/README000066400000000000000000000027401363122316600161770ustar00rootroot00000000000000See the documentation in the doc/ directory for information about how to use the ctemplate library. COMPILING --------- To compile test applications with these classes, run ./configure followed by make. To install these header files on your system, run 'make install'. (On Windows, the instructions are different; see README.windows.) See INSTALL for more details. This code should work on any modern C++ system. It has been tested on Linux (Ubuntu, Fedora, RedHat), Solaris 10 x86, FreeBSD 6.0, OS X 10.3 and 10.4, and Windows under both VC++7 and VC++8. There are a few Windows-specific details; see README.windows for more information. CTEMPLATE AND THREADS --------------------- The ctemplate library has thread support, so it works properly in a threaded environment. For this to work, if you link libraries with -lctemplate you may find you also need to add -pthread (or, on some systems, -pthreads, and on others, -lpthread) to get the library to compile. If you leave out the -pthread, you'll see errors like this: symbol lookup error: /usr/local/lib/libctemplate.so.0: undefined symbol: pthread_rwlock_init If your code isn't multi-threaded, you can instead use the ctemplate_nothread library: -lctemplate_nothreads To summarize, there are two ways to link in ctemlpate in non-threaded applications. For instance: 1) gcc -o my_app my_app.o -lctemplate -pthread 2) gcc -o my_app my_app.o -lctemplate_nothreads If your application uses threads, you should use form (1). ctemplate-ctemplate-2.4/README.md000066400000000000000000000046111363122316600165750ustar00rootroot00000000000000Welcome to the C++ CTemplate system! ==================================== This library provides an easy to use and lightning fast text templating system to use with C++ programs. It was originally called Google Templates, due to its origin as the template system used for Google search result pages. Now it has a more general name matching its community-owned nature. Documentation ------------- Refer to the [Project Documentation]() to learn how to use the CTemplate system. There is also a [HOWTO]() and [Examples](). Compiling --------- To compile test applications with these classes, run `./autogen.sh && ./configure` followed by `make` on unixoid platforms like Linux or MacOSX. To install these header files on your system, run `make install`. See INSTALL for more details. This code should work on any modern C++ system. It has been tested on Linux (Ubuntu, Fedora, RedHat), Solaris 10 x86, FreeBSD 6.0, OS X 10.3 and 10.4, and Windows under both VC++7 and VC++8. There are a few Windows-specific details; see README.windows for more information. CTemplate and Threads --------------------- The ctemplate library has thread support, so it works properly in a threaded environment. For this to work, if you link libraries with `-lctemplate` you may also need to add `-pthread` (or, on some systems, `-pthreads`, and on others, `-lpthread`) to get the library to compile. If you leave out the `-pthread`, you'll see errors like this: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ symbol lookup error: /usr/local/lib/libctemplate.so.0: undefined symbol: pthread_rwlock_init ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If your code isn't multi-threaded, you can instead use the ctemplate\_nothread library: `-lctemplate_nothreads` To summarize, there are two ways to link in ctemlpate in non-threaded applications. For instance: 1. Thread safe build:`gcc -o my_app my_app.o -lctemplate -pthread` 2. Unthreaded code:`gcc -o my_app my_app.o -lctemplate_nothreads` If your application uses threads, you should use form (1). ctemplate-ctemplate-2.4/README_windows.txt000066400000000000000000000040471363122316600205710ustar00rootroot00000000000000This project has been ported to Windows. A working solution file exists in this directory: ctemplate.sln You can load this solution file into either VC++ 7.1 (Visual Studio 2003) or VC++ 8.0 (Visual Studio 2005) -- in the latter case, it will automatically convert the files to the latest format for you. When you build the solution, it will create libctemplate.dll, the main library for this project, plus a number of unittests, which you can run by hand (or, more easily, under the Visual Studio debugger) to make sure everything is working properly on your system. The binaries will end up in a directory called "debug" or "release" in the top-level directory (next to the .sln file). I don't know very much about how to install DLLs on Windows, so you'll have to figure out that part for yourself. If you choose to just re-use the existing .sln, make sure you set the IncludeDir's appropriately! Look at the properties for libctemplate.dll. If you wish to link to ctemplate statically instead of using the .dll, you can; see the example project template_unittest_static. For this to work, you'll need to add "/D CTEMPLATE_DLL_DECL=" to the compile line of every ctemplate .cc file. Note that these systems are set to build in Debug mode by default. You may want to change them to Release mode. Currently, Template::StringToTemplate returns a Template object that you, the caller, must free. We've heard reports that Windows can have trouble allocating memory in a .dll that is meant to be freed in the application. Thus, we suggest you not use StringToTemplate from Windows. Instead, you can use Template::StringToTemplateCache() followed by Template::GetTemplate(). I have little experience with Windows programming, so there may be better ways to set this up than I've done! If you run across any problems, please post to the google-ctemplate Google Group, or report them on the ctemplate Google Code site: http://groups.google.com/group/google-ctemplate http://code.google.com/p/ctemplate/issues/list -- craig ctemplate-ctemplate-2.4/autogen.sh000077500000000000000000000011361363122316600173160ustar00rootroot00000000000000#!/bin/sh # Before using, you should figure out all the .m4 macros that your # configure.m4 script needs and make sure they exist in the m4/ # directory. # # These are the files that this script might edit: # aclocal.m4 configure Makefile.in src/config.h.in \ # depcomp config.guess config.sub install-sh missing mkinstalldirs \ # # Here's a command you can run to see what files aclocal will import: # aclocal -I ../autoconf --output=- | sed -n 's/^m4_include..\([^]]*\).*/\1/p' set -ex rm -rf autom4te.cache autoreconf --force --install --warnings all,no-obsolete rm -rf autom4te.cache exit 0 ctemplate-ctemplate-2.4/configure.ac000066400000000000000000000112351363122316600176040ustar00rootroot00000000000000## Process this file with autoconf to produce configure. ## In general, the safest way to proceed is to run ./autogen.sh # make sure we're interpreted by some minimal autoconf AC_PREREQ([2.69]) AC_INIT([ctemplate],[2.3],[google-ctemplate@googlegroups.com]) AM_SILENT_RULES([yes]) # Update this value for every release! (A:B:C will map to foo.so.(A-C).C.B) # http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html SO_VERSION=3:0:0 # The argument here is just something that should be in the current directory # (for sanity checking) AC_CONFIG_SRCDIR(README) AC_CONFIG_MACRO_DIR([m4]) AC_CANONICAL_HOST AM_INIT_AUTOMAKE([subdir-objects dist-zip]) AC_CONFIG_HEADERS([src/config.h]) # Checks for programs. AC_PROG_CXX AC_PROG_CC AM_PROG_AR AM_CONDITIONAL(GCC, test "$GCC" = yes) # let the Makefile know if we're gcc # MinGW uses autoconf, but also needs the windows shim routines # (since it doesn't have its own support for, say, pthreads). # This requires us to #include a special header file, and also to # link in some windows versions of .o's instead of the unix versions. AH_BOTTOM([ #if defined( __MINGW32__) || defined(__MINGW64__) #include "windows/port.h" #endif ]) AM_CONDITIONAL(MINGW, expr $host : '.*-mingw' >/dev/null 2>&1) LT_INIT([disable-fast-install]) AC_SUBST(LIBTOOL_DEPS) AC_SUBST(SO_VERSION) AX_C___ATTRIBUTE__ # Defines PRIuS AC_COMPILER_CHARACTERISTICS # Here are some examples of how to check for the existence of a fn or file AC_CHECK_FUNCS([getopt_long getopt]) AC_CHECK_HEADERS([getopt.h]) AC_CHECK_HEADERS([utime.h]) # used by unittests to mock file-times AC_HEADER_DIRENT # for template_unittest.cc, template_regtest.cc # We need to do byte-swapping efficiently to hash efficiently in # template_string.cc. Different architectures do this differently. AC_CHECK_HEADERS(byteswap.h) # Linux (GNU in general) AC_CHECK_HEADERS(libkern/OSByteOrder.h) # OS X AC_CHECK_HEADERS(sys/byteorder.h) # Solaris 10 AC_CHECK_HEADERS(endian.h) # Linux AC_CHECK_HEADERS(machine/endian.h) # OS X AC_CHECK_HEADERS(sys/endian.h) # FreeBSD AC_CHECK_HEADERS(sys/isa_defs.h) # Solaris 10 # A lot of the code in this directory depends on pthreads AX_PTHREAD # We'd like to use read/write locks in several places in the code. # See if our pthreads support extends to that. Note: for linux, it # does as long as you define _XOPEN_SOURCE appropriately. AC_RWLOCK # For mingw/cygwin, figure out if the mutex code needs to use # 'volatile' in some places. They differ from MSVC, and the API is # unclear, so it's best just to check. AC_INTERLOCKED_EXCHANGE_NONVOLATILE # Figures out where hash_map and hash_set live, and what namespace they use AC_CXX_STL_HASH AC_SUBST(ac_cv_cxx_hash_namespace) AC_SUBST(ac_cv_cxx_hash_map) AC_SUBST(ac_cv_cxx_hash_set) if test "$ac_cv___attribute__" = "yes"; then AC_SUBST(ac_google_attribute, 1) else AC_SUBST(ac_google_attribute, 0) fi # On some systems (eg gnu/linux), the linker defines _start and # data_start to indicate the extent of the .text section. We can use # this to know strings are immutable. In the code, we make the # variables weak, just in case, but for this check, we only want to # say "yes" if the linker supports the vars, *and* the compiler supports # attribute-weak. AC_LINK_IFELSE([ AC_LANG_PROGRAM([], [[ extern char _start[]; extern char data_start[]; extern char dummy[] __attribute__((weak)); return (_start && data_start && dummy); ]]) ], [ ac_have_attribute_weak=1 ], [ ac_have_attribute_weak=0 ]) AC_SUBST(ac_have_attribute_weak) # In unix (that is, using this script), this dll-export stuff will always # be the empty string. In windows, we'll replace it with windows-specific # text. ac_windows_dllexport_defines= ac_windows_dllexport= AC_SUBST(ac_windows_dllexport_defines) AC_SUBST(ac_windows_dllexport) # Write generated configuration file, and also .h files AC_CONFIG_FILES([Makefile \ src/ctemplate/template_string.h \ src/ctemplate/template_enums.h \ src/ctemplate/template.h \ src/ctemplate/template_cache.h \ src/ctemplate/template_modifiers.h \ src/ctemplate/template_emitter.h \ src/ctemplate/template_annotator.h \ src/ctemplate/template_dictionary.h \ src/ctemplate/template_pathops.h \ src/ctemplate/template_namelist.h \ src/ctemplate/find_ptr.h \ src/ctemplate/per_expand_data.h \ src/ctemplate/str_ref.h \ src/ctemplate/template_dictionary_interface.h \ ]) AC_OUTPUT ctemplate-ctemplate-2.4/contrib/000077500000000000000000000000001363122316600167545ustar00rootroot00000000000000ctemplate-ctemplate-2.4/contrib/README.contrib000066400000000000000000000013001363122316600212650ustar00rootroot00000000000000All files under this contrib directory are UNSUPPORTED; use at your own risk. They were provided by users of ctemplate and were not tested by the authors of ctemplate. These files are not owned by Google. Please contact the authors of the contributions for help about these, not the ctemplate authors. Thanks!, and enjoy. highlighting.vim by Patrick Lacasse How to set up syntax highlighting (colorization) for ctemplate .tpl files, using vim. A shar file; see top-of-file for how to extract. tpl-mode.el by Tony Gentilcore Elisp file for syntax highlighting (colorization) for ctemplate .tpl files, using emacs. See top-of-file for how to use. ctemplate-ctemplate-2.4/contrib/convert_to_95.pl000077500000000000000000000132231363122316600220140ustar00rootroot00000000000000#!/usr/bin/perl -pli.bak # Copyright (c) 2006, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # --- # Author: Craig Silverstein # This is a perl script for editing your source files that were # written with ctemplate <0.95, to use the new #include paths, # namespace names, and function locations in ctemplate 0.95. # # This script will modify your files in place. It is safe to # call it multiple times. To call it, do something like: # find -print0 | xargs -0 ./convert_to_95.pl # # Issues: # TemplateFromString logic is tricky. Best to check up by hand. # Can't always tell when 1st arg to TemplateFromString is empty string. # If it might be for your app, change the logic to use StringToTemplate. # Fails if TemplateFromString or StringToTemplate span multiple lines. # Change all the #include lines s@google/template.h@ctemplate/template.h@g; s@google/per_expand_data.h@ctemplate/per_expand_data.h@g; s@google/template.h@ctemplate/template.h@g; s@google/template_annotator.h@ctemplate/template_annotator.h@g; s@google/template_dictionary.h@ctemplate/template_dictionary.h@g; s@google/template_dictionary_interface.h@ctemplate/template_dictionary_interface.h@g; s@google/template_emitter.h@ctemplate/template_emitter.h@g; s@google/template_enums.h@ctemplate/template_enums.h@g; s@google/template_from_string.h@ctemplate/template_from_string.h@g; s@google/template_modifiers.h@ctemplate/template_modifiers.h@g; s@google/template_namelist.h@ctemplate/template_namelist.h@g; s@google/template_pathops.h@ctemplate/template_pathops.h@g; s@google/template_string.h@ctemplate/template_string.h@g; # Change all the namespace uses s@namespace\s+google@namespace ctemplate@g; # brought into google namespace s@\bgoogle::@ctemplate::@g; s@namespace\s+template_modifiers\s*{@@ && ($in_tm = 1); $in_tm && s@}@@ && ($in_tm = 0); s@\btemplate_modifiers::@@g; # Change TemplateFromString::GetTemplate to # Template::StringToTemplateCache(cache_key, template_text); # return Template::GetTemplate(cache_key, strip); # unless cache_key is the empty string, in which case it should be # return Template::StringToTemplate(template_text, strip, TC_MANUAL); # We have to be tricky, to do this in one command. # Better would be to just do Template::StringToTemplate(), but then we # have to free the memory when we're done, which is too hard to do here. s@TemplateFromString::GetTemplate\s*\(""\s*,([^,]+),([^,]+)\)@Template::StringToTemplate($1, $2, ctemplate::TC_MANUAL)@; s@TemplateFromString::GetTemplate\s*\(([^,]+),([^,]+),([^,]+)\)@Template::GetTemplate((ctemplate::Template::StringToTemplateCache($1, $2), $1), $3)@; if (m@TemplateFromString::GetTemplate\s*\([^)]*$@) { warn("$ARGV:$.: TemplateFromString spans mutiple lines. You will need " . "to clean it up by hand. See comments in $0."); } # Get rid of the 4th arg to Template::TemplateToString. If it's # TC_MANUAL, that's fine. If it's not, we need to tell the user to # fix this up manually by adding an {{%PRAGMA AUTOESCAPE}} to the # beginning of their string. if (s@(\bStringToTemplate\s*\([^)]*),\s*TC_([A-Z_]*)@$1@g) { if ($2 ne "MANUAL") { # TC_MANUAL is the new default warn("WARNING: $ARGV:$.: You will need to manually fix up the " . "StringToTemplate change to add an AUTOESCAPE {{\%pragma}} to " . "the beginning of the input string, to set the context to TC_$2\n"); } } if (m@\bStringToTemplate\s*\([^)]*$@) { warn("$ARGV:$.: StringToTemplate spans mutiple lines. You will need " . "to clean it up by hand: get rid of the TC_foo arg. If it's not " . "TC_MANUAL, you will need to add an AUTOESCAPE pragma. " . "See comments in $0."); } # Change all the TemplateDictionary escapes. Now they are globals in # the ctemplate namespace. s@TemplateDictionary::html_escape@html_escape@g; s@TemplateDictionary::pre_escape@pre_escape@g; s@TemplateDictionary::xml_escape@xml_escape@g; s@TemplateDictionary::javascript_escape@javascript_escape@g; s@TemplateDictionary::url_query_escape@url_query_escape@g; s@TemplateDictionary::json_escape@json_escape@g; # This cleans up anything messy we left behind. s@\bctemplate::ctemplate::@ctemplate::@g; # get rid of redundant specifier close ARGV if eof; # to reset $. for each file ctemplate-ctemplate-2.4/contrib/highlighting.vim000066400000000000000000000155051363122316600221440ustar00rootroot00000000000000#!/bin/sh # This is a shell archive (produced by GNU sharutils 4.2.1). # To extract the files from this archive, save it to some FILE, remove # everything before the `!/bin/sh' line above, then type `sh FILE'. # # Made on 2006-08-08 13:38 PDT by . # Commandline: shar -T .vim # # Existing files will *not* be overwritten unless `-c' is specified. # # This shar contains: # length mode name # ------ ---------- ------------------------------------------ # 1477 -rw-r--r-- .vim/syntax/tpl.vim # 56 -rw-r--r-- .vim/ftdetect/tpl.vim # echo >/dev/null <<_NOTES_EOF From: Patrick Lacasse Subject: vim color for google-ctemplate howto To: google-ctemplate@googlegroups.com Date: Fri, 4 Aug 2006 11:38:39 -0400 Hi group, I'm now using google-ctemplate. My text editor is vim. Here is a little gift for other people like me. Howto have google-ctemplate colored by vim : In your home directory, run 'sh $0'. Now restart vim. This will autodetects file with tpl extension and colors them. You can change the color by changing the HiLink lines ( try changing String by Identifier ) . I'm not sure exactly about what are the legal marker names. Feel free to change the regexes. I only tryed this with vim 6.4 , I just cut and past the things about version checking. For more information about syntax higlithning in vim, try :help syntax Amusez-vous bien, Patrick Lacasse patlac@borabora.crchul.ulaval.ca _NOTES_EOF save_IFS="${IFS}" IFS="${IFS}:" gettext_dir=FAILED locale_dir=FAILED first_param="$1" for dir in $PATH do if test "$gettext_dir" = FAILED && test -f $dir/gettext \ && ($dir/gettext --version >/dev/null 2>&1) then set `$dir/gettext --version 2>&1` if test "$3" = GNU then gettext_dir=$dir fi fi if test "$locale_dir" = FAILED && test -f $dir/shar \ && ($dir/shar --print-text-domain-dir >/dev/null 2>&1) then locale_dir=`$dir/shar --print-text-domain-dir` fi done IFS="$save_IFS" if test "$locale_dir" = FAILED || test "$gettext_dir" = FAILED then echo=echo else TEXTDOMAINDIR=$locale_dir export TEXTDOMAINDIR TEXTDOMAIN=sharutils export TEXTDOMAIN echo="$gettext_dir/gettext -s" fi if touch -am -t 200112312359.59 $$.touch >/dev/null 2>&1 && test ! -f 200112312359.59 -a -f $$.touch; then shar_touch='touch -am -t $1$2$3$4$5$6.$7 "$8"' elif touch -am 123123592001.59 $$.touch >/dev/null 2>&1 && test ! -f 123123592001.59 -a ! -f 123123592001.5 -a -f $$.touch; then shar_touch='touch -am $3$4$5$6$1$2.$7 "$8"' elif touch -am 1231235901 $$.touch >/dev/null 2>&1 && test ! -f 1231235901 -a -f $$.touch; then shar_touch='touch -am $3$4$5$6$2 "$8"' else shar_touch=: echo $echo 'WARNING: not restoring timestamps. Consider getting and' $echo "installing GNU \`touch', distributed in GNU File Utilities..." echo fi rm -f 200112312359.59 123123592001.59 123123592001.5 1231235901 $$.touch # if mkdir _sh09814; then $echo 'x -' 'creating lock directory' else $echo 'failed to create lock directory' exit 1 fi # ============= .vim/syntax/tpl.vim ============== if test ! -d '.vim'; then $echo 'x -' 'creating directory' '.vim' mkdir '.vim' fi if test ! -d '.vim/syntax'; then $echo 'x -' 'creating directory' '.vim/syntax' mkdir '.vim/syntax' fi if test -f '.vim/syntax/tpl.vim' && test "$first_param" != -c; then $echo 'x -' SKIPPING '.vim/syntax/tpl.vim' '(file already exists)' else $echo 'x -' extracting '.vim/syntax/tpl.vim' '(text)' sed 's/^X//' << 'SHAR_EOF' > '.vim/syntax/tpl.vim' && " Vim syntax file " Language: google-ctemplate " Maintainer: Patrick Lacasse " Last Change: 2006 Août 03 " " For information about google-ctemplate see " http://goog-ctemplate.sourceforge.net/ " " This vim syntax file works on vim 5.6, 5.7, 5.8 and 6.x. " It implements Bram Moolenaar's April 25, 2001 recommendations to make " the syntax file maximally portable across different versions of vim. X " For version 5.x: Clear all syntax items " For version 6.x: Quit when a syntax file was already loaded if version < 600 X syntax clear elseif exists("b:current_syntax") X finish endif X syntax match tplMarkerError "{{}\?\([^}]\+}\?\)*}}" syntax match tplSectionMarker "{{[#/][A-Za-z_]\+}}" syntax match tplInclude "{{>[A-Za-z_]\+}}" syntax match tplComment "{{![A-Za-z_]\+}}" syntax match tplVariableMarker "{{[_A-Za-z]\+}}" X " Define the default highlighting. " For version 5.7 and earlier: only when not done already " For version 5.8 and later: only when an item doesn't have highlighting yet if version >= 508 || !exists("did_tpl_syn_inits") X if version < 508 X let did_tpl_syn_inits = 1 X command -nargs=+ HiLink hi link X else X command -nargs=+ HiLink hi def link X endif X X HiLink tplSectionMarker Repeat X HiLink tplInclude Include X HiLink tplComment Comment X HiLink tplVariableMarker String X HiLink tplMarkerError Error X X delcommand HiLink endif X let b:current_syntax = "tpl" SHAR_EOF (set 20 06 08 08 13 34 11 '.vim/syntax/tpl.vim'; eval "$shar_touch") && chmod 0644 '.vim/syntax/tpl.vim' || $echo 'restore of' '.vim/syntax/tpl.vim' 'failed' if ( md5sum --help 2>&1 | grep 'sage: md5sum \[' ) >/dev/null 2>&1 \ && ( md5sum --version 2>&1 | grep -v 'textutils 1.12' ) >/dev/null; then md5sum -c << SHAR_EOF >/dev/null 2>&1 \ || $echo '.vim/syntax/tpl.vim:' 'MD5 check failed' 536faef79eff0597e642c5db04c1f79d .vim/syntax/tpl.vim SHAR_EOF else shar_count="`LC_ALL= LC_CTYPE= LANG= wc -c < '.vim/syntax/tpl.vim'`" test 1477 -eq "$shar_count" || $echo '.vim/syntax/tpl.vim:' 'original size' '1477,' 'current size' "$shar_count!" fi fi # ============= .vim/ftdetect/tpl.vim ============== if test ! -d '.vim/ftdetect'; then $echo 'x -' 'creating directory' '.vim/ftdetect' mkdir '.vim/ftdetect' fi if test -f '.vim/ftdetect/tpl.vim' && test "$first_param" != -c; then $echo 'x -' SKIPPING '.vim/ftdetect/tpl.vim' '(file already exists)' else $echo 'x -' extracting '.vim/ftdetect/tpl.vim' '(text)' sed 's/^X//' << 'SHAR_EOF' > '.vim/ftdetect/tpl.vim' && au BufRead,BufNewFile *.tpl set filetype=tpl SHAR_EOF (set 20 06 08 08 13 34 20 '.vim/ftdetect/tpl.vim'; eval "$shar_touch") && chmod 0644 '.vim/ftdetect/tpl.vim' || $echo 'restore of' '.vim/ftdetect/tpl.vim' 'failed' if ( md5sum --help 2>&1 | grep 'sage: md5sum \[' ) >/dev/null 2>&1 \ && ( md5sum --version 2>&1 | grep -v 'textutils 1.12' ) >/dev/null; then md5sum -c << SHAR_EOF >/dev/null 2>&1 \ || $echo '.vim/ftdetect/tpl.vim:' 'MD5 check failed' 774fd4a092b77400ef6e74a7256ff8ef .vim/ftdetect/tpl.vim SHAR_EOF else shar_count="`LC_ALL= LC_CTYPE= LANG= wc -c < '.vim/ftdetect/tpl.vim'`" test 56 -eq "$shar_count" || $echo '.vim/ftdetect/tpl.vim:' 'original size' '56,' 'current size' "$shar_count!" fi fi rm -fr _sh09814 exit 0 ctemplate-ctemplate-2.4/contrib/tpl-mode.el000066400000000000000000000200561363122316600210220ustar00rootroot00000000000000;;; tpl-mode.el -- a major mode for editing Google CTemplate files. ;;; By Tony Gentilcore, July 2006 ;;; ;;; TO USE: ;;; 1) Copy this file somewhere you in emacs load-path. To see what ;;; your load-path is, run inside emacs: C-h v load-path ;;; 2) Add the following two lines to your .emacs file: ;;; (setq auto-mode-alist (cons '("\\.tpl$" . tpl-mode) auto-mode-alist)) ;;; (autoload 'tpl-mode "tpl-mode" "Major mode for editing CTemplate files." t) ;;; 3) Optionally (but recommended), add this third line as well: ;;; (add-hook 'tpl-mode-hook '(lambda () (font-lock-mode 1))) ;;; --- ;;; ;;; While the CTemplate language can be used for any types of text, ;;; this mode is intended for using CTemplate to write HTML. ;;; ;;; The indentation still has minor bugs due to the fact that ;;; templates do not require valid HTML. ;;; ;;; It would be nice to be able to highlight attributes of HTML tags, ;;; however this is difficult due to the presence of CTemplate symbols ;;; embedded within attributes. (eval-when-compile (require 'font-lock)) (defgroup tpl-mode nil "Major mode for editing CTemplate files" :group 'languages) (defvar tpl-mode-version "1.0" "Version of `tpl-mode.el'.") (defvar tpl-mode-abbrev-table nil "Abbrev table for use in tpl-mode buffers.") (define-abbrev-table 'tpl-mode-abbrev-table ()) (defcustom tpl-mode-hook nil "*Hook that runs upon entering tpl-mode." :type 'hook ) (defvar tpl-mode-map nil "Keymap for tpl-mode major mode") (if tpl-mode-map nil (setq tpl-mode-map (make-sparse-keymap)) ) (define-key tpl-mode-map "\t" 'tpl-indent-command) (define-key tpl-mode-map "\C-m" 'newline-and-indent) (defvar tpl-mode-syntax-table nil "Syntax table in use in tpl-mode buffers.") ;; Syntax table. (if tpl-mode-syntax-table nil (setq tpl-mode-syntax-table (make-syntax-table text-mode-syntax-table)) (modify-syntax-entry ?< "(> " tpl-mode-syntax-table) (modify-syntax-entry ?> ")< " tpl-mode-syntax-table) (modify-syntax-entry ?\" ". " tpl-mode-syntax-table) (modify-syntax-entry ?\\ ". " tpl-mode-syntax-table) (modify-syntax-entry ?' "w " tpl-mode-syntax-table) ) (defvar tpl-basic-offset 2 "The basic indentation offset.") ;; Constant regular expressions to identify template elements. (defconst tpl-mode-tpl-token "[a-zA-Z][a-zA-Z0-9_:=\-]*?") (defconst tpl-mode-section (concat "\\({{[#/]" tpl-mode-tpl-token "}}\\)")) (defconst tpl-mode-open-section (concat "\\({{#" tpl-mode-tpl-token "}}\\)")) (defconst tpl-mode-close-section (concat "{{/\\(" tpl-mode-tpl-token "\\)}}")) (defconst tpl-mode-comment "\\({{![^}]+?}}\\)") (defconst tpl-mode-include (concat "\\({{>" tpl-mode-tpl-token "}}\\)")) (defconst tpl-mode-variable (concat "\\({{" tpl-mode-tpl-token "}}\\)")) (defconst tpl-mode-builtins (concat "\\({{\\<" (regexp-opt '("BI_NEWLINE" "BI_SPACE") t) "\\>}}\\)")) (defconst tpl-mode-close-section-at-start (concat "^[ \t]*?" tpl-mode-close-section)) ;; Constant regular expressions to identify html tags. ;; Taken from HTML 4.01 / XHTML 1.0 Reference found at: ;; http://www.w3schools.com/tags/default.asp. (defconst tpl-mode-html-constant "\\(&#?[a-z0-9]\\{2,5\\};\\)") (defconst tpl-mode-pair-tag (concat "\\<" (regexp-opt '("a" "abbr" "acronym" "address" "applet" "area" "b" "bdo" "big" "blockquote" "body" "button" "caption" "center" "cite" "code" "col" "colgroup" "dd" "del" "dfn" "dif" "div" "dl" "dt" "em" "fieldset" "font" "form" "frame" "frameset" "h1" "h2" "h3" "h4" "h5" "h6" "head" "html" "i" "iframe" "ins" "kbd" "label" "legend" "li" "link" "map" "menu" "noframes" "noscript" "object" "ol" "optgroup" "option" "p" "pre" "q" "s" "samp" "script" "select" "small" "span" "strike" "strong" "style" "sub" "sup" "table" "tbody" "td" "textarea" "tfoot" "th" "thead" "title" "tr" "tt" "u" "ul" "var") t) "\\>")) (defconst tpl-mode-standalone-tag (concat "\\<" (regexp-opt '("base" "br" "hr" "img" "input" "meta" "param") t) "\\>")) (defconst tpl-mode-open-tag (concat "<\\(" tpl-mode-pair-tag "\\)")) (defconst tpl-mode-close-tag (concat "")) (defconst tpl-mode-close-tag-at-start (concat "^[ \t]*?" tpl-mode-close-tag)) (defconst tpl-mode-blank-line "^[ \t]*?$") (defconst tpl-mode-dangling-open (concat "\\(" tpl-mode-open-section "\\)\\|\\(" tpl-mode-open-tag "\\)[^/]*$")) (defun tpl-indent-command () "Command for indenting text. Just calls tpl-indent." (interactive) (tpl-indent)) ;; Function to control indenting. (defun tpl-indent () "Indent current line" ;; Set the point to beginning of line. (beginning-of-line) ;; If we are at the beginning of the file, indent to 0. (if (bobp) (indent-line-to 0) (let ((tag-stack 1) (close-tag "") (cur-indent 0) (old-pnt (point-marker)) (close-at-start) (open-token) (dangling-open)) (progn ;; Determine if this is a template line or an html line. (if (looking-at "^[ \t]*?{{") (setq close-at-start tpl-mode-close-section-at-start open-token "{{#") (setq close-at-start tpl-mode-close-tag-at-start open-token "<") ) ;; If there is a closing tag at the start of the line, search back ;; for its opener and indent to that level. (if (looking-at close-at-start) (progn (save-excursion (setq close-tag (match-string 1)) ;; Keep searching for a match for the close tag until ;; the tag-stack is 0. (while (and (not (bobp)) (> tag-stack 0) (re-search-backward (concat open-token "\\(/?\\)" close-tag) nil t)) (if (string-equal (match-string 1) "/") ;; We found another close tag, so increment tag-stack. (setq tag-stack (+ tag-stack 1)) ;; We found an open tag, so decrement tag-stack. (setq tag-stack (- tag-stack 1)) ) (setq cur-indent (current-indentation)) ) ) (if (> tag-stack 0) (save-excursion (forward-line -1) (setq cur-indent (current-indentation)) ) ) ) ;; This was not a closing tag, so we check if the previous line ;; was an opening tag. (save-excursion ;; Keep moving back until we find a line that is not blank (while (progn (forward-line -1) (and (not (bobp)) (looking-at tpl-mode-blank-line)) ) ) (setq cur-indent (current-indentation)) (if (re-search-forward tpl-mode-dangling-open old-pnt t) (setq cur-indent (+ cur-indent tpl-basic-offset)) ) ) ) ;; Finally, we execute the actual indentation. (if (> cur-indent 0) (indent-line-to cur-indent) (indent-line-to 0) ) ) ) ) ) ;; controls highlighting (defconst tpl-mode-font-lock-keywords (list (list tpl-mode-section '(1 font-lock-keyword-face)) (list tpl-mode-comment '(1 font-lock-comment-face)) (list tpl-mode-include '(1 font-lock-builtin-face)) (list tpl-mode-builtins '(1 font-lock-variable-name-face)) (list tpl-mode-variable '(1 font-lock-reference-face)) (list (concat " Guide to using Auto Escape

Guide to using Auto Escape

Introduction

Auto Escape is an optional mode of execution in the Template System developed to provide a better defense against cross-site scripting (XSS) in web applications. In this mode, the Template System assumes responsibility for applying the proper escaping modifiers for each variable in your template (and templates it may include). As such, the template developer no longer needs to manually apply escaping modifiers to each variable, a process which is repetitive and error-prone particularly in larger and more complex applications.

In order for the Template System to properly determine the escaping modifiers to apply to variables, it activates a built-in HTML-aware parser during template initialization. The parser scans the template to determines the context in which each variable is being emitted. The scanning is performed at initialization-time only hence does not contribute to processing costs at template expansion time.

Refer to Security Considerations for additional security information on escaping directives and their use according to the context in which content is being expanded.

Current Status

Auto Escape currently supports well HTML and Javascript templates. It also provides very basic support for CSS, JSON and XML templates where it applies the corresponding escaping modifier but without utilizing a parser. We may in the future provide parsers specific to these contexts and modifiers for them that are finer-grained.

Overview

This section provides more background on Auto Escape. Refer to How to Auto Escape for information on how to leverage this mode in your application.

Using template modifiers

In the simplest case, where you only want to use template-modifiers to achieve html-safety (that is, you only use html_escape, url_query_escape, and so forth), you can stop specifying template-modifiers in your template at all. The Auto Escape mode will generate the modifiers on its own and perform the correct escaping.

If you need other types of modifiers, including your own custom modifiers, you can still specify them in your template and they will continue to work just the same. The Auto Escape mode will simply apply the appropriate escaping modifiers after your own.

Other reasons for manually specifying variable modifiers include:

  1. To indicate a variable is safe. You may add the :none modifier as the last modifier for a given variable, in which case the Auto Escape mode will leave it untouched and not apply escaping directives to it. Use only when you are sure the variable is trusted to be safe from XSS.
  2. To select a different escaping modifier from a list of equivalent modifiers. Certain modifiers are equivalent from an escaping point of view in that they would all ensure given content is safe when escaped in a specific context. The Auto Escape mode choses the most common escaping modifier for that use but allows you to indicate a different choice as shown below:
    TemplateContextPrimary Modifier Accepted alternatives
    TC_HTML :html_escape
    • :pre_escape
    • :html_escape_with_arg=snippet
    • :html_escape_with_arg=attribute
    • :html_escape_with_arg=url
    • :url_query_escape
    • :url_escape_with_arg=html
    • :img_src_url_escape_with_arg=html
    TC_HTML and TC_CSS :cleanse_css
    • :url_escape_with_arg=css
    • :img_src_url_escape_with_arg=css
    TC_XML :xml_escape
    • :html_escape
    • :html_escape_with_arg=attribute
    TC_JS :javascript_escape
    • :url_escape_with_arg=javascript
    • :img_src_url_escape_with_arg=javascript
    TC_JSON :javascript_escape
    • :json_escape
  3. To add additional escaping modifiers. The Template System will never remove escaping directives you explicitly specify.

Escaping Contexts

The table belows indicates which modifier Auto-Escape selects to escape a given variable, based on the context of the variable being inserted and, possibly, on more specific information such as whether the variable is inside quotation marks. The table only applies for the well-supported TemplateContext values (currently TC_HTML and TC_JS).

In a few cases, we do not have a modifier that is both safe against XSS and respecting of the semantics of the variable therefore we fail the template initialization. An error is logged indicating the cause of the failure.

ContextHTML Quoted? ExamplesAction Performed
Regular HTML Body and HTML commentsAny 
<p>Hello {{USER}}</p>
Escape USER using :html_escape
In URL attribute: Starts at pos 0Yes 
<a href="{{URL}}">;
 Escape URL using :url_escape_with_arg=html
In URL attribute: OtherYes 
<a href="/foo?q={{QUERY}}">
 Escape QUERY using :html_escape
In URL attribute: Starting at pos 0No 
<form action={{URL}}>
 Fail template initialization
In URL attribute: OtherNo 
<a href=/foo?q={{QUERY}}>My Link</a>
 Escape QUERY using :url_query_escape
In STYLE attributeYes 
<div style="color:{{COLOR}};">
 Escape COLOR using :cleanse_css
In STYLE attributeNo 
<div style=color:{{COLOR}};>
 Fail template initialization
In Javascript attribute: String literalYes 
<a href="url" onclick="doFoo('{{ARG}}');">
 Escape ARG using :javascript_escape
In Javascript attribute: Non-string literalYes 
<a href="url" onclick="doFoo({{ARG}});">
 Escape ARG using :javascript_escape_with_arg=number
In Javascript attribute: AnyNo 
<a href="url" onclick=doFoo('{{ARG}}');>
 Fail template initialization.
In all other attributesYes 
<b class="{{CLASS}}">
 Escape CLASS using :html_escape
In all other attributesNo 
<table border={{BORDER}}>
 Escape BORDER using :html_escape_with_arg=attribute
In Javascript code: In a string literalAny 
<script>var a = '{{VALUE}}';</script>
 Escape VALUE using :javascript_escape
In Javascript code: Non-string literalAny 
<script>var a = {{VALUE}};</script>
 Escape VALUE using :javascript_escape_with_arg=number
In a <style> tagAny 
<style>font-size={{FONTSIZE}};</style>
 Escape FONTSIZE using :cleanse_css

Comments:

  • For values of URL-accepting attributes, we apply a different modifier if the variable is at the beginning of the attribute value or not because in the former case we consider the variable to represent a complete URL and hence also check that its scheme is safe (HTTP or HTTPS).
  • We recommend that you enclose attribute values in quotes in particular:
    1. Under some conditions outlined in the table above, when variables are present in attribute values that are not enclosed in quotes, we may fail template initialization because we do not have escaping modifiers that are safe to use.
    2. Also the determination of which escaping modifier to apply for unquoted attribute values specifically is likely to change in the future.

The Auto Escape mode is designed to be simple to use and to produce correct correct escaping with minimal input. It does however come with restrictions governing the use of this template system.

Limitations

  • Templates may only be included in regular HTML text. Including a template within HTML comments or inside an HTML tag is currently not supported. In the example below, HEADER and FOOTER are properly included but TAG and COMMENT are not.

              <html><head>{{>HEADER}}</head><body>
          <{{>TAG}}>
          <p>
          <!-- {{>COMMENT}} -->
          <p>{{>FOOTER}}
          </body></html>

    If the template system detects that you are including a template in another HTML context, it will log a warning.

  • Included templates may not cross HTML boundaries. They cannot start with one TemplateContext and end in another. For example, the template below crosses boundaries -- it starts in HTML and ends in Javascript -- and is therefore not a valid included template for auto-escaping.

            <html>
          <head>
          </head>
          <body>
            Hello USER.
              <script>
                 var a = 'not a nice template';

    To rectify this violation, continue the template by completing the javascript code and terminating it with a </script> tag.

  • Error reporting. We currently report errors by logging a warning or an error during initial template initialization and parsing if we suspect something is incorrect. Please check for them, as they indicate an error that may cause auto-escaping to work improperly.

  • Beware of double escaping. When you use the Auto Escape mode, you surrender all variable escaping to the template system. If you also perform your own escaping in your source code, you may face situations where content was escaped multiple times.

    In particular, do not use any of the legacy methods below or others that perform variable escaping.

    • SetEscapedValue
    • SetEscapedFormatttedValue
    • SetEscapedValueAndShowSection

  • Supported HTML contexts. HTML (TC_HTML) and Javascript (TC_JS) templates are well supported. Other template contexts have only basic support. For these contexts, variables are escaped as follows:

    TemplateContextEscaping Applied
    TC_JSON :javascript_escape
    TC_XML :xml_escape
    TC_CSS :cleanse_css

  • New restrictions on the use of BI_SPACE and BI_NEWLINE. These system variables may not be re-assigned to a semantically different value via dict->SetValue or related methods, and they may not have modifiers attached to them in templates. For instance: it's ok to redefine BI_NEWLINE from \n to \r\n, since html treats them identically. But changing it to hello, world! is not allowed.

A new filename convention

The Auto Escape feature codifies simple conventions of template filenames:

  1. CSS templates optionally include, in their filename, the keywords style or stylesheet or css.
  2. Javascript templates optionally include, in their filename, the keywords js or javascript.

If we find one of these keywords in the filename but it does not match the template type given, we log a warning.

Jad Boutros
ctemplate-ctemplate-2.4/doc/designstyle.css000066400000000000000000000037101363122316600211260ustar00rootroot00000000000000body { background-color: #ffffff; color: black; margin-right: 1in; margin-left: 1in; } h1, h2, h3, h4, h5, h6 { color: #3366ff; font-family: sans-serif; } @media print { /* Darker version for printing */ h1, h2, h3, h4, h5, h6 { color: #000080; font-family: helvetica, sans-serif; } } h1 { text-align: center; font-size: 18pt; } h2 { margin-left: -0.5in; } h3 { margin-left: -0.25in; } h4 { margin-left: -0.125in; } hr { margin-left: -1in; } /* Definition lists: definition term bold */ dt { font-weight: bold; } address { text-align: right; } /* Use the tag for bits of code and for variables and objects. */ code,pre,samp,var { color: #006000; } /* Use the tag for file and directory paths and names. */ file { color: #905050; font-family: monospace; } /* Use the tag for stuff the user should type. */ kbd { color: #600000; } div.note p { float: right; width: 3in; margin-right: 0%; padding: 1px; border: 2px solid #6060a0; background-color: #fffff0; } UL.nobullets { list-style-type: none; list-style-image: none; margin-left: -1em; } /* pretty printing styles. See prettify.js */ .str { color: #080; } .kwd { color: #008; } .com { color: #800; } .typ { color: #606; } .lit { color: #066; } .pun { color: #660; } .pln { color: #000; } .tag { color: #008; } .atn { color: #606; } .atv { color: #080; } pre.prettyprint { padding: 2px; border: 1px solid #888; } .embsrc { background: #eee; } @media print { .str { color: #060; } .kwd { color: #006; font-weight: bold; } .com { color: #600; font-style: italic; } .typ { color: #404; font-weight: bold; } .lit { color: #044; } .pun { color: #440; } .pln { color: #000; } .tag { color: #006; font-weight: bold; } .atn { color: #404; } .atv { color: #060; } } /* Table Column Headers */ .hdr { color: #006; font-weight: bold; background-color: #dddddd; } .hdr2 { color: #006; background-color: #eeeeee; }ctemplate-ctemplate-2.4/doc/example.html000066400000000000000000000275601363122316600204140ustar00rootroot00000000000000 Template Examples

Template Examples

Simple Example

One reason this example is so simple is that it doesn't even require a separate template file, but instead uses StringToTemplateCache(). It also doesn't use sections or template-includes.


int main() {
  static const char template_text[] =
    "ERROR: {{FUNCTION}}({{ARGS}}) returned {{ERROR_CODE}}: {{ERROR_MESSAGE}}\n";
  ctemplate::StringToTemplateCache("mytpl", template_text, ctemplate::DO_NOT_STRIP);
  FILE* fp = fopen(argv[1], "r");
  if (fp == NULL) {
    int err_no = errno;   // squirrel this away
    ctemplate::TemplateDictionary dict("error_msg: fopen()");
    dict.SetValue("FUNCTION", "fopen");
    dict.SetValue("ARGS", argv[1]);
    dict.SetIntValue("ERROR_CODE", err_no);
    dict.SetValue("ERROR_MESSAGE", strerror(err_no));

    string error_text;
    ctemplate::ExpandTemplate("mytpl", ctemplate::DO_NOT_STRIP, &dict, &error_text);
    puts(error_text.c_str());
  }
}
Note: If this template was intended to run in a web application, you can leverage the functionality provided by the auto-escape mode. Simply add the AUTOESCAPE pragma directive at the top of the template text and your variables will be automatically escaped for the context you specify. For example, if your template is returned in an HTML context, change the template_text declaration as follows: 
  static const char template_text[] =
    "{{%AUTOESCAPE context=\"HTML\"}}"
    "ERROR: {{FUNCTION}}({{ARGS}}) returned {{ERROR_CODE}}: {{ERROR_MESSAGE}}\n";

This example is only slightly more complicated: we only print the ": <error message>" part when the error message isn't the empty string.


int main() {
  static const char template_text[] =
     "ERROR: {{FUNCTION}}({{ARGS}}) returned {{ERROR_CODE}}"
     "{{#MSG_SECTION}}: {{ERROR_MESSAGE}}{{/MSG_SECTION}}\n";
  ctemplate::StringToTemplateCache("mytpl", template_text, ctemplate::DO_NOT_STRIP);
  FILE* fp = fopen(argv[1], "r");
  if (fp == NULL) {
    int err_no = errno;   // squirrel this away
    ctemplate::TemplateDictionary dict("file_error_message");
    dict.SetValue("FUNCTION", "fopen");
    dict.SetValue("ARGS", argv[1]);
    dict.SetIntValue("ERROR_CODE", err_no);
    if (err_no > 0)
      dict.SetValueAndShowSection("ERROR_MESSAGE", strerror(err_no),
                                  "MSG_SECTION");

    string error_text;
    ctemplate::ExpandTemplate("mytpl", ctemplate::DO_NOT_STRIP, &dict, &error_text);
    puts(error_text.c_str());
  }
  delete tpl;
}

This maybe-show-text functionality is one way the template machinery is more powerful than just using printf. Another nice property of templates is you can reuse the same variable multiple times in your template string. You can also define the variable values in any order.

Search Results Page

Here is an example template that could be used to format a Google search results page:


{{>HEADER}}
<body bgcolor=white>

{{>PAGE_HEADING}}{{!The following div must be on the same line}}<div>

{{!The ONE_RESULT section displays a single search item}}
{{#ONE_RESULT}}
    {{! Note: there are two SUBITEM_SECTIONs. They both show or hide together}}
    {{#SUBITEM_SECTION}}<blockquote>{{/SUBITEM_SECTION}}
    {{! LEAD_LINE is received HTML-escaped from the backend.}}
    <p><a href="{{JUMP_TO_URL:html_escape}}"  target=nw>{{LEAD_LINE}}</a><font size=-1>

    {{! SNIPPET1, SNIPPET2 are HTML-escaped in the snippet generator.}}
    {{#SNIPPET1_SECTION}}
        <br>{{SNIPPET1}}
    {{/SNIPPET1_SECTION}}

    {{#SNIPPET2_SECTION}}
        <br>{{SNIPPET2}}
    {{/SNIPPET2_SECTION}}

    {{#DESCRIPTION_SECTION}}
        {{! DESC is received HTML-escaped from the backend.}}
        <br><span class=f>Description:</span> {{DESC}}
    {{/DESCRIPTION_SECTION}}

    {{#CATEGORY_SECTION}}
        <br><span class=f>Category:</span> <a href="{{CAT_URL:html_escape}}" class=f>
        {{CATEGORY:html_escape}}</a>
    {{/CATEGORY_SECTION}}

    {{#LASTLINE_SECTION}}
        <br><font color="{{ALT_TEXT_COLOR:h}}">{{URL:h}}
        {{#KS_SECTION}}} - {{KSIZE:h}}{{/KS_SECTION}}}
        {{#CACHE_SECTION}}} - <a href="{{CACHE_URL:h}}" class=f>Cached</A>
        {{/CACHE_SECTION}}}
        {{#SIM_SECTION}}} - <a href="{{SIM_PAGES_URL:h}}" class=f>Similar pages</A>
        {{/SIM_SECTION}}}

        {{#STOCK_SECTION}}
             -  <a href="{{STOCK_URL:h}}" class=f>Stock quotes: {{STOCK_SYMBOL:h}}</a>
        {{/STOCK_SECTION}}
        </font>
    {{/LASTLINE_SECTION}}           

    {{#MORE_SECTION}}
        <br>[ <a href="{{MORE_URL:h}}" class=f>More results from {{MORE_LABEL:h}}</a> ]
    {{/MORE_SECTION}}

    </font><br>
    {{! Note: there are two SUBITEM_SECTIONs. They both show or hide together}}
    {{#SUBITEM_SECTION}}</blockquote>{{/SUBITEM_SECTION}}
{{/ONE_RESULT}}
</div> {{! this /div closes the div at the top of this file}}
{{>PAGE_FOOTING}}

Here is a sample procedure that could populate a dictionary for expanding that template. The "one procedure" entry point is fill_search_results_dictionary. The SetTemplateValues function is a separate entry point for initializing each top-level template with some standard values.

#include "template.h"

RegisterTemplateFilename(SEARCH_RESULTS_FN, "search_results.tpl");
#include "search_results.tpl.varnames.h"  // defines ksr_HEADER, etc.

using ctemplate::TemplateDictionary;
using ctemplate::ExpandTemplate;
using ctemplate::STRIP_WHITESPACE;

// IsEmpty
//    A simple utility function
static bool IsEmpty(const string &str) {
  return str.empty();
}

// SetTemplateValues
//   Use the TemplateDictionary object to set template-wide values that
//   may be used in the top-level template and all its sub-sections
//   and included templates. The template-wide values are all
//   colors from the Palette object
void SetTemplateValues(TemplateDictionary *dictionary, const Palette* colors) {
  // better would be to use ksr_LINK_COLOR, etc, assuming those are
  // defined in search_results.tpl.varnames.h.  But using literal
  // text, as here, is legal as well.
  dictionary->SetValue("LINK_COLOR", colors->link_color);
  dictionary->SetValue("BAR_TEXT_COLOR", colors->bar_text_color);
  dictionary->SetValue("TEXT_COLOR", colors->text_color);
  dictionary->SetValue("FAINT_COLOR", colors->faint_color);
  dictionary->SetValue("IMPORTANT_COLOR", colors->important_color);
  dictionary->SetValue("BAR_COLOR", colors->bar_color);
  dictionary->SetValue("ALT_TEXT_COLOR", colors->alt_text_color);
  dictionary->SetValue("ALINK_COLOR", colors->alink_color);
  dictionary->SetValue("VLINK_COLOR", colors->vlink_color);
}

// fill_search_results_dictionary
//   Iterates through all the QueryResults contained in the Query object.
//   For each one, it sets corresponding template dictionary values
//   (or hides sections containing their variables, if appropriate) in
//   a sub-dictionary and then adds that dictionary to the parent
void fill_search_results_dictionary(TemplateDictionary *dictionary,
                                    const Query *query) {
  dictionary->SetFilename(SEARCH_RESULTS_FN);

  // These two functions are defined elsewhere
  fill_header_dictionary(dictionary->AddIncludeDictionary(ksr_HEADER));
  fill_page_heading_dictionary(dictionary->AddIncludeDictionary(ksr_PAGE_HEADING),
                               query);

  ResultsList *results = query->GetResults();
  int resCount = 0;

  for (ResultsList::const_iterator iter = results->begin();
       iter != results->end();
       ++iter) {
    QueryResult *qr = (*iter);

    // Create a new sub-dictionary named "Result Dict <n>" for this entry

    ++resCount;

    TemplateDictionary *result_dictionary =
      dictionary->AddSectionDictionary(ksr_ONE_RESULT);

    result_dictionary->SetValue(ksr_JUMP_TO_URL, qr->GetUrl());

    if (qr->IsSubItem()) {
      result_dictionary->ShowSection(ksr_SUBITEM_SECTION);
    }

    result_dictionary->SetValue(ksr_LEAD_LINE, qr->GetLeadLine());

    result_dictionary->SetValueAndShowSection(ksr_SNIPPET1, qr->GetSnippet1(),
                                              ksr_SNIPPET1_SECTION);
    
    result_dictionary->SetValueAndShowSection(ksr_SNIPPET2, qr->GetSnippet2(),
                                              ksr_SNIPPET2_SECTION);
    
    result_dictionary->SetValueAndShowSection(ksr_DESC, qr->GetDescription(),
                                              ksr_DESCRIPTION_SECTION);

    result_dictionary->SetValueAndShowSection(ksr_CAT_URL, qr->GetCategoryUrl(),
                                              ksr_CATEGORY_SECTION);

    result_dictionary->SetValueAndShowSection("CATEGORY", qr->GetCategoryName(),
                                              "CATEGORY_SECTION");


    if (IsEmpty(qr->GetDisplayUrl()) &&
        IsEmpty(qr->GetPageSize()) &&
        IsEmpty(qr->GetCachedUrl()) &&
        IsEmpty(qr->GetSimilarPagesUrl()) &&
        (IsEmpty(qr->GetStockUrl()) ||
         IsEmpty(qr->GetStockSymbol())) ) {
      // there is nothing on the last line, so hide it altogether
    } else {
      result_dictionary->ShowSection("LASTLINE_SECTION");

      result_dictionary->SetValue(ksr_URL, qr->GetDisplayUrl());

      result_dictionary->SetValueAndShowSection(ksr_KSIZE, qr->GetPageSize(),
                                                ksr_KS_SECTION);

      result_dictionary->SetValueAndShowSection(ksr_CACHE_URL, qr->GetCachedUrl(),
                                                ksr_CACHE_SECTION);

      result_dictionary->SetValueAndShowSection(ksr_SIM_PAGES_URL,
                                                qr->GetSimilarPagesUrl(),
                                                ksr_SIM_SECTION);

      result_dictionary->SetValueAndShowSection(ksr_STOCK_URL, qr->GetStockUrl(),
                                                ksr_STOCK_SECTION);

      result_dictionary->SetValueAndShowSection(ksr_STOCK_SYMBOL,
                                                qr->GetStockSymbol(),
                                                ksr_STOCK_SECTION);
    }

    result_dictionary->SetValueAndShowSection(ksr_MORE_URL, qr->GetMoreUrl(),
                                              ksr_MORE_SECTION);

    result_dictionary->SetValueAndShowSection(ksr_MORE_LABEL, qr->GetMoreLabel(),
                                              ksr_MORE_SECTION);

  }    

  fill_page_footing_dictionary(dictionary->AddIncludeDictionary(ksr_PAGE_FOOTING),
                               query);
}        

void output_page(const Query* query) {
  TemplateDictionary dict("search-results dict");
  string output;
  fill_search_results_dictionary(&dict, query);
  ctemplate::ExpandTemplate(SEARCH_RESULTS_FN, STRIP_WHITESPACE, &dict, &output);
  // output now holds the expanded template
}
Craig Silverstein
ctemplate-ctemplate-2.4/doc/guide.html000066400000000000000000001271441363122316600200550ustar00rootroot00000000000000 How To Use the Ctemplate (formerly Google Template) System

How To Use the Ctemplate (formerly Google Template) System

(as of )

Motivation

A template system can be used to separate output formatting specifications, which govern the appearance and location of output text and data elements, from the executable logic which prepares the data and makes decisions about what appears in the output.

Template systems lie along a continuum of power versus separation. "Powerful" constructs like variable assignment or conditional statements make it easy to modify the look of an application within the template system exclusively, without having to modify any of the underlying "application logic". They do so, however, at the cost of separation, turning the templates themselves into part of the application logic.

This template system leans strongly towards preserving the separation of logic and presentation. It is intentionally constrained in the features it supports and, as a result, applications tend to require quite a bit of code to instantiate a template. This may not be to everybody's tastes. However, while this design limits the power of the template language, it does not limit the power or flexibility of the template system. This system supports arbitrarily complex text formatting. Many Google applications, including the "main" Google web search, use this system for formatting output.

Finally, this system is designed with an eye towards efficiency. Template instantiation is very quick, with an eye towards minimizing both memory use and memory fragmentation.

Overview

There are two main parts to the Ctemplate System:

  • Templates
  • Data dictionaries

The templates are text files that contain the format specification for the formatted output, i.e, the template language. The data dictionaries contain the mappings from the template elements (markers) embedded in the templates to the data that they will format. Here's a simple template:

   <html><head><title>{{TITLE}}</title>{{META_TAGS}}</head>
   <body>{{BODY}}</body></html>

Here's a dictionary that one could use to instantiate the template:

   {"TITLE": "Template example",
    "BODY": "This is a simple template example.\nIt's boring",
    "DATE": "11/20/2005"}

If we instantiated the template with this dictionary (a process we call "expanding"), here's the output we would get:

   <html><head><title>Template example</title></head>
   <body>This is a simple template example.
It's boring</body></html>

{{TITLE}} and {{BODY}} are template elements, also called markers. In the dictionary, TITLE, BODY, and DATE are dictionary names, and the values associated with each one, such as 11/20/2005, are dictionary values.

A few points are clear even from this simple example:

  1. Dictionary keys and values are strings; the Ctemplate system is not typed.
  2. Dictionary values come already formatted. It was up to the application code to decide how to format the value for DATE, and to insert the date into the dictionary already formatted.
  3. Not all dictionary values must be used by a template. DATE is entirely ignored.
  4. Not all template elements may exist in the dictionary. In this example, {{META_TAGS}} is not found in the dictionary. This is perfectly legal; missing dictionary names evaluate to the empty string.

Templates

The template language has four major types of markers (the full list of marker types is described in the reference guide):

  1. Variable markers, which are replaced by text based on dictionary values. All markers in the above example are variable markers. Variable markers look like this: {{FOO}}
  2. Start section and end section markers, which delimit sections which may appear zero, one, or N times in the output. The number of times a section appears is determined by the data dictionaries, as explained below. Each time a section is expanded, it uses a different dictionary, so that the output values may be different from one iteration of a section expansion to another. Note that the specification of how sections expand is entirely dependent on the dictionary, as set up by the application; there is no way to specify a repeat count in the template language itself. Section markers look like this: {{#FOO}}...{{/FOO}}
  3. Template-include markers, which designate other templates to be expanded and inserted at the location where the marker appears. These are treated much like sections -- one may think of them as sections whose content is specified in a different file instead of inline -- and just like sections, can be expanded zero, one or N times in the output, each with a different dictionary and even a different include-file. Template-include markers look like this: {{>FOO}}
  4. Comment markers, which may annotate the template structure but drop completely out of the expanded output. Comment markers look like this: {{! comment lives here -- cool, no?}}

These marker types each have their own namespace. For readability, however, it is best to not overuse a single name.

Anything found in a template of the form {{...}} is interpreted as a template marker. All other text is considered formatting text and is output verbatim at template expansion time. Formatting text may consist of HTML tags, XML tags, linefeeds and other spacing characters, constant text, etc.

Data Dictionaries

A data dictionary is a map from keys to values. The keys are always strings, each string representing either a variable, a section, or a template-include file. (Comments are not stored in the data dictionary!) These values correspond to the name of the associated template marker: a section {{#FOO}} in the template text is matched to the key "FOO" in the dictionary, if it exists. Note the case must match as well.

The value associated with a key differs according to key type. The value associated with a variable is simple: it's the value for that variable. Both keys and values can be any 8-bit character-string, and may include internal NULs (\0).

The value associated with a section is more complicated, and somewhat recursive: it's a list of data dictionaries. Come template-expansion time, the section is expanded once for each dictionary in the list, so if there are two dictionaries in the list, then the section text will occur in the output twice. The first time, all variables/etc. in the section will be evaluated taking into account the first dictionary. The second time, all variables/etc. will be evaluated taking into account the second dictionary. (See below for a definition of "taking into account.")

A template-include is a special type of section, so the associated value is the same: a list of dictionaries. Template-includes also have one other, mandatory associated piece of information: the filename of the template to include.

The application program is responsible for building this data dictionary, including all nesting. It then applies this dictionary to a single template to produce formatted output.

Expanding a Template

A program using Ctemplate typically reads in templates at load time. During the course of program execution, the program will repeatedly perform the following two steps: first, instantiate a data dictionary, and second, apply the dictionary to the template to produce output.

The template system applies a dictionary to a template by finding all template markers in the template, and replacing them with the appropriate dictionary values. It matches template markers to dictionary keys in the obvious way. For instance, a template marker {{FOO}} matches the dictionary key FOO. The marker {{#BAR}} matches the dictionary key BAR, as does the marker {{/BAR}}. The marker {{>BAZ}} matches the dictionary key BAZ. (And of course, the marker {{! comment}} doesn't match any dictionary key at all.)

If no dictionary key is found for a given template marker, then the template marker is ignored: if a variable, it expands to the empty string; if a section or include-template, the section or include-template is expanded zero times.

All names are case sensitive. Names -- that is, variable keys and, as a result, template markers -- must be made of (7-bit ascii) alphanumeric characters and the underscore. The comment marker, which does not map to dictionary keys, may contain any characters whatsoever except }, the close-curly brace. It's a syntax error for any template marker to violate this rule.

Outside of the template markers, templates may contain any text whatsoever, including (single) curly braces and NUL characters.

Auto Escape Mode

The Auto Escape mode helps protect against cross-site scripting (XSS) attacks in web-applications by automatically escaping variables in your template. The Guide to using Auto Escape has an overview of Auto Escape as well as discussion of its limitations.

Auto Escape is enabled on a template-by-template basis. Simply add the AUTOESCAPE pragma to the desired template. That template will then be automatically escaped, independently of the templates it may include or it may be included from. The AUTOESCAPE pragma must be placed at the top of the template. It takes a 'context' argument saying what context the template is used in: html, javascript, css, xml, etc. (There's also a state=IN_TAG argument that is used when the template is just a snippet of html intended for use in a tag.) See the reference guide for a full description of autoescape arguments.

The table below shows four small sample templates along with their corresponding AUTOESCAPE pragma. The two most common contexts are HTML and JAVASCRIPT. We also show a sample template for the CSS context as well as a sample template for the IN_TAG state of HTML (although it is expected to be rarely used).

HTML JAVASCRIPT CSS HTML IN_TAG (uncommon) 
    {{%AUTOESCAPE context="HTML"}}

    <body>
      <p>Hello {{USER}}</p>
      <p><a href="{{URL}}">Your Account</a></p>
    </body>
    
    {{%AUTOESCAPE context="JAVASCRIPT"}}

    function showMessage(user, msg) {
      alert("Hello: " + user + " Message: " + msg);
    }

    var user = '{{USER}}';
    var msg = '{{MSG}}';
    showMessage(user, msg);
    
    {{%AUTOESCAPE context="CSS"}}

    P.abstract {
      text-align:{{EDGE}};
      font-size:{{FONT_SIZE_PC}};
    }
    .italic {font-style:{{ITALIC}}}
    
    {{%AUTOESCAPE context="HTML" state="IN_TAG"}}

    class="{{CLASS}}" id="{{ID}}"

Auto-escaping works by automatically applying modifiers to every variable in the template. You can manually apply modifiers as well, and even define your own. See the reference guide for a full discussion of modifiers and how to use them.

Details on Dictionary Lookup

The dictionary structure is a tree: there's a 'main' dictionary, and then a list of sub-dictionaries for each section or include-template. Even with all this complexity, the lookup rules are mostly straightforward: when looking up a marker -- be it a variable, section, or include-template marker -- the system looks in the currently applicable dictionary. If it's found there, great. If not, and the parent dictionary is not an include-template, it continues the look in the parent dictionary, and possibly the grandparent, etc. That is, lookup has static scoping: you look in your dictionary and any parent dictionary that is associated with the same template-file. As soon as continuing the lookup would require you to jump to a new template-file (which is what include-template would do), we stop the lookup.

For instance, for a template that says {{#RESULTS}}{{RESULTNUM}}. {{>ONE_RESULT}}{{/RESULTS}}, "ONE_RESULT" is looked for in the "RESULTS" dictionary, and if not found there, is looked for in the main, top-level dictionary. Likewise, the variable "RESULTNUM" is looked for first in the "RESULTS" dictionary, then in the main dictionary if necessary. However, "ONE_RESULT" will not do equivalent cascading lookups. In fact, it will have no parent dictionaries at all, because it's a different template file and thus in a different scope.

Because of these scoping rules, it's perfectly reasonable to set all variables that are needed in a given template file, in the top-level dictionary for that template. In fact, the ShowSection() function is provided to support just this idiom. To avoid confusion in such a usage mode, it's strongly encouraged that you give unique names to all sections and include-templates in a single template file. (It's no problem, given the template scoping rules, for a single section or include-template name to be repeated across different template files.)

There's a single special case: the global variable dictionary. Every dictionary inherits its initial set of values from the global dictionary. Clients can set variables in the global dictionary just like they can in normal template dictionaries they create.

The system initializes the global dictionary with a few useful values for your convenience. All system variables are prefixed with BI, to emphasize they are "built in" variables.

  • BI_SPACE, which has the value <space>. It is used to force a space at the beginning or end of a line in the template, where it would normally be suppressed. (See below.)
  • BI_NEWLINE, which has the value <newline> It is used to force a newline at the end of a line, where it would normally be suppressed. (See below.)

As is usual for inheritance, if a user explicitly assigns a value to these variable-names in its own dictionary, this overrides the inherited value. So, dict->SetValue("BI_SPACE", "&nbsp;") causes BI_SPACE to have the value &nbsp;, rather than <space>, when expanding dict.

Note that only variables can be inherited from the global dictionary, not section dictionaries or include-file dictionaries.

A couple of small implementation notes: global inheritance is "last chance", so if a section's parent dictionary redefined BI_SPACE, say, the section dictionary inherits the parent-dict value, not the global-dict value. Second, variable inheritance happens at expand time, not at dictionary-create time. So if you create a section dictionary, and then afterwards set a variable in its parent dictionary (or in the global dictionary), the section will inherit that variable value, if it doesn't define the value itself.

Writing Application Code To Use Templates

Most application code concerns filling a template dictionary, but there is also code for expanding templates given a dictionary. A final category of code lets you inspect and control the template system.

Creating A Template Dictionary

The class TemplateDictionary is used for all template dictionary operations. new ctemplate::TemplateDictionary(name) is used to create a new top-level dictionary. dict->AddSectionDictionary(name) and dict->AddIncludeDictionary(name) are used to create sub-dictionaries for sections or include-files. After creating a dictionary, the application should call one or more functions for each marker in the template. As an example, consider the following template: 

<html><body> {{! This page has no head section.}}
{{#CHANGE_USER}}
<A HREF="/login">Click here</A> if you are not {{USERNAME}}<br>
{{/CHANGE_USER}}

Last five searches:<ol>
{{#PREV_SEARCHES}
<li> {{PREV_SEARCH}}
{{/PREV_SEARCHES}}
</ol>

{{>RESULT_TEMPLATE}}

{{FOOTER}}
</body></html>

To instantiate the template, the user should call a function to set up FOOTER, and a function to say what to do for the sections CHANGE_USER and PREV_SEARCHES, and for the include-template RESULT_TEMPLATE. Quite likely, the application will also want to create a sub-dictionary for CHANGE_USER, and in that sub-dictionary call a function to set up USERNAME. There will also be sub-dictionaries for PREV_SEARCHES, each of which will need to set PREV_SEARCH. Only when this is all set up will the application be able to apply the dictionary to the template to get output.

The appropriate function to call for a given template marker depends on its type.

Variables

For variables, the only interesting action is to set the variable's value. For most variables, the right method to call is dict->SetValue(name, value). (The name and value can be specified as strings in a variety of ways: C++ strings, char *'s, or char *'s plus length.)

There are two other ways to set a variable's value as well, each with a different scoping rule. You can call ctemplate::TemplateDictionary::SetGlobalValue(name, value) -- no TemplateDictionary instance needed here -- to set a variable that can be used by all templates in an application. This is quite rare.

You can also call dict->SetTemplateGlobalValue(name, value). This sets a variable that is seen by all child dictionaries of this dictionary: sub-sections you create via AddSectionDictionary, and included templates you create via AddIncludeDictionary (both described below). This differs from SetValue(), because SetValue() values are never inherited across template-includes. Almost always, SetValue is what you want; SetTemplateGlobalValue is intended for variables that are "global" to a particular template tree not all template trees, such as a color scheme to use, a language code, etc.

To make it easier to use SetValue(), there are a few helper routines to help setting values of a few special forms.

  • SetIntValue(name, int): takes an int as the value.
  • SetFormattedValue(name, fmt, ...): the fmt and ... work just like in printf: SetFormattedValue("HOMEPAGE", "http://%s/", hostname).

Example:

   ctemplate::TemplateDictionary* dict = new ctemplate::TemplateDictionary("var example");
   dict->SetValue("FOOTER", "Aren't these great results?");

Sections

Sections are used in two ways in templates. One is to expand some text multiple times. This is how PREV_SEARCHES is used in the example above. In this case we'll have one small sub-dictionary for each of the five previous searches the user did. To do this, call AddSectionDictionary(section_name) to create the sub-dictionary. It returns a TemplateDictionary* that you can use to fill the sub-dictionary.

The other use of sections is to conditionally show or hide a block of text at template-expand time. This is how CHANGE_USER is used in the example template: if the user is logged in, we show the section with the user's username, otherwise we choose not to show the section.

This second case is a special case of the first, and the "standard" way to show a section is to expand it exactly one time, by calling AddSectionDictionary() once, and then setting USERNAME in the sub-dictionary.

However, the hide/show idiom is so common there are a few convenience methods to make it simpler. The first takes advantage of the fact sections inherit variables from their parent: you set USERNAME in the parent dictionary, rather than a section sub-dictionary, and then call ShowSection(), which adds a single, empty dictionary for that section. This causes the section to be shown once, and to inherit all its variable values from its parent.

A second convenience method is written for the particular case we have with USERNAME: if the user's username is non-empty, we wish to show the section with USERNAME set to the username, otherwise we wish to hide the section and show neither USERNAME nor the text around it. The method SetValueAndShowSection(name, value, section_name) does exactly that: if value is non-empty, add a single dictionary to section_name and call section_dict->AddValue(name, value).

Example:

   ctemplate::TemplateDictionary* dict = new ctemplate::TemplateDictionary("section example");
   const char* username = GetUsername();   // returns "" for no user
   if (username[0] != '\0') {
      ctemplate::TemplateDictionary* sub_dict = dict->AddSectionDictionary("CHANGE_USER");
      sub_dict->SetValue("USERNAME", username);
   } else {
      // don't need to do anything; we want a hidden section, which is the default
   }

   // Instead of the above 'if' statement, we could have done this:
   if (username[0] != '\0') {
      dict->ShowSection("CHANGE_USER");       // adds a single, empty dictionary
      dict->SetValue("USERNAME", username);   // take advantage of inheritance
   } else {
      // don't need to do anything; we want a hidden section, which is the default
   }

   // Or we could have done this:
   dict->SetValueAndShowSection("USERNAME", username, "CHANGE_USER");

   // Moving on...
   GetPrevSearches(prev_searches, &num_prev_searches);
   if (num_prev_searches > 0) {
      for (int i = 0; i < num_prev_searches; ++i) {
         TemplateDictionary* sub_dict = dict->AddSectionDictionary("PREV_SEARCHES");
         sub_dict->SetValue("PREV_SEARCH", prev_searches[i]);
      }
   }

Template-includes

Template-include markers are much like section markers, so AddIncludeDictionary(name) acts, not surprisingly, exactly like AddSectionDictionary(name). However, since variable inheritance doesn't work across include boundaries, there is no template-include equivalent to ShowSection() or SetValueAndShowSection().

One difference between template-includes and sections is that for a sub-dictionary that you create via AddIncludeDictionary(), you must call subdict->SetFilename() to indicate the name of the template to include. If you do not set this, the sub-dictionary will be ignored. The filename may be absolute, or relative, in which case it's relative to some entry in the template search path.

Example:

   using ctemplate::TemplateDictionary;
   TemplateDictionary* dict = new TemplateDictionary("include example");
   GetResults(results, &num_results);
   for (int i = 0; i < num_results; ++i) {
      TemplateDictionary* sub_dict = dict->AddIncludeDictionary("RESULT_TEMPLATE");
      sub_dict->SetFilename("results.tpl");
      FillResultsTemplate(sub_dict, results[i]);
   }

In practice, it's much more likely that FillResultsTemplate() will be the one to call SetFilename(). Note that it's not an error to call SetFilename() on a dictionary even if the dictionary is not being used for a template-include; in that case, the function is a no-op, but is perhaps still useful as self-documenting code.

Another property of template-includes is that they set the indentation level for the included template, that is, every line in the included template is indented by the same amount as the template-includes line itself. For instance, if you have a template PRINT_STUFF like this:

print "Hello!"
print "You are the 10th caller!"
print "Congratulations!"

and you include it in the template:

   if ShouldPrintStuff():
     {{>PRINT_STUFF}}
   else:
     pass

then when it is expanded, all three print lines will be indented, not just the first one:

   if ShouldPrintStuff():
     print "Hello!"
     print "You are the 10th caller!"
     print "Congratulations!"
   else:
     pass

Note that this behavior is immaterial when using STRIP_WHITESPACE, since in that case all leading whitespace is stripped.

Expanding a Template

Once you have a template dictionary, it's simplicity itself to expand the template with those dictionary values, putting the output in a string:

   ctemplate::TemplateDictionary dict("debug-name");
   FillDictionary(&dict, ...);
   string output;
   bool error_free = ctemplate::ExpandTemplate(<filename>, ctemplate::STRIP_WHITESPACE, &dict, &output);
   // output now holds the expanded template.
   // ExpandTemplate returns false if the system cannot load-and-parse
   // <filename> or any of the template files referenced by the
   // TemplateDictionary.

The first argument to ExpandTemplate is the filename holding the template to expand (though there are ways to use non-file-based templates as well). The second argument is the "strip" mode, which specifies how to treat whitespace in the template. It can take the following values:

  • ctemplate::DO_NOT_STRIP: do nothing. This expands the template file verbatim.
  • ctemplate::STRIP_BLANK_LINES: remove all blank lines. This ignores any blank lines found in the template file when parsing it. When the template is html, this reduces the size of the output text without requiring a sacrifice of readability for the input file.
  • ctemplate::STRIP_WHITESPACE: remove not only blank lines when parsing, but also whitespace at the beginning and end of each line. It also removes any linefeed (possibly following whitespace) that follows a closing }} of any kind of template marker except a template variable. (This means a linefeed may be removed anywhere by simply placing a comment marker as the last element on the line.) When the template is html, this reduces the size of the output html without changing the way it renders (except in a few special cases.) When using this flag, the built-in template variables BI_NEWLINE and BI_SPACE can be useful to force a space or newline in a particular situation.

The expanded template is written to the string output. If output was not empty before calling Expand(), the expanded template is appended to the end of output.

There is also a "power user" version of ExpandTemplate(), called ExpandWithData(), that allows you to pass in per-expand data. Another "power user" version allows you to expand the template into a custom output container, rather than a string. See the reference guide for more information about these advanced methods.

Getting a Template From a String Rather Than a File

The first argument to ExpandTemplate is named "filename", suggesting that the template has to be read from an on-disk file. But in reality, the "filename" argument is just a key into an internal cache. (By default, the template system looks on disk to satisfy a cache miss, hence the "filename" to describe this variable.) If you have a template specified in a string rather than a file, you can manually insert it into the cache via ctemplate::StringToTemplateCache().

StringToTemplateCache() parses the string you pass in as if it were a template file, and inserts it into the global cache with the key and strip-mode that you provide. You can then use this key and strip-mode as the first two arguments to ExpandTemplate. You can also use the key as the argument to ctemplate::TemplateDictionary::SetFilename().

Prefer file-based to string-based templates where possible. Updating a file-based template requires merely a data push, rather than pushing the new executable, and it also makes it easier for non-programmers to modify the template. One reason to use string-based templates is if you are in an environment where having data files could be dangerous—for instance, you work on a disk that is usually full, or need the template to work even in the face of disk I/O errors.

This package comes with a script, template-converter, that takes a template file as input and emits a C++ code snippet (an .h file) that defines a string with those template contents. This makes it easy to start by using a normal, file-based template, and then switch to StringToTemplateCache() later if you so desire.

Copying a Template Dictionary

You can use the MakeCopy() method on a template dictionary to make a "deep" copy of the template. This can be useful for situations like the following: you want to fill a template several times, each time with 90% of the values the same, but the last 10% different. Computing the values is slow. Here's how you can use MakeCopy() to do it:

  1. fill dict with 90%
  2. newdict1 = dict->MakeCopy();
  3. fill newdict1 with last 10%
  4. newdict2 = dict->MakeCopy();
  5. fill newdict2 with last 10%
  6. etc.

The Template Cache

When templates are loaded from disk, they are stored in an internal template cache, which is used by ExpandTemplate() and (most obviously) StringToTemplateCache(). You can define your own template cache with the TemplateCache class.

This is an advanced technique used when you want to support having several versions of a template file in memory at the same time (for instance, a webserver might want to keep an old version of a template file around until all old requests using that template are done being serviced). It also supports advanced operations like removing a template from the cache, and reloading templates from disk if they have changed.

See the reference manual for more details about the template cache.

Template and Threads

All expansion functions and static TemplateDictionary methods are threadsafe: you can safely call ctemplate::TemplateDictionary::SetGlobalValue() without needing to worry about locking.

Non-static TemplateDictionary methods are not thread-safe. It is not safe for two threads to assign values to the same template-dictionary without doing their own locking. Note that this is expected to be quite rare: usually only one thread will care about a given template-dictionary.

Testing Templates

Templates have the advantage that they separate the presentation of your data from the application logic that generates the data. Naturally, you want to do the same for testing: you would like to test the application's logic in a way that is robust to changes in the presentation.

The easiest way to test this logic is with unit tests that exercise the functions in your code that fill template dictionaries. The class ctemplate::TemplateDictionaryPeer in template_test_util.h is designed to help you do exactly that.

Here's a sample test using TemplateDictionaryPeer:

   void MyTestMethod() {
     // Create and populate the dictionary as your app normally would.
     // In this case, we create a dict with data like this:
     // { color:blue,
     //   shape_section: [
     //     { size:big, num_sides:7 },
     //     { size:big, num_sides:7 },
     //     { size:big, num_sides:7 }
     //   ]
     // }
     MyObject obj;
     ctemplate::TemplateDictionary dict;
     obj.FillDictionary(&dict);
     // Create a TemplateDictionaryPeer to gain access to the dict contents.
     ctemplate::TemplateDictionaryPeer peer(&dict);
     // Expect color:blue at the top level of this dictionary.
     EXPECT_STREQ("blue", peer.GetSectionValue("color"));
     // Fetch sub-dictionaries from the dict.
     vector<const ctemplate::TemplateDictionary*> shape_dicts;
     peer.GetSectionDictionaries("shape_section", &shape_dicts);
     EXPECT_EQ(3, shape_dicts.size());
     for (int i = 0; i < 3; ++i) {
       // Create another peer for each sub-dict, and assert that each sub-dict
       // contains the expected data.
       ctemplate::TemplateDictionaryPeer shape_peer(dicts[i]);
       EXPECT_STREQ("big", shape_peer.GetSectionValue("size"));
       EXPECT_STREQ("7", shape_peer.GetSectionValue("num_sides"));
     }
   }

Note that by using TemplateDictionaryPeer, you can unit test the code for filling a TemplateDictionary independent of the consuming Template.

The above tests your dictionary filling functions, but doesn't touch the template expansion. Naturally, you may also want to test the template itself. In this case, you will want to avoid using your program's dictionary filling functions, and will instead provide a custom dictionary in your test. There are a variety of properties you might want to test about the template, but here are a few sample tests:

   void TestTemplateHTMLEscapesColor() {
     // Populate a dictionary that lets us exercise the condition we're testing.
     // In this case, that the 'color' tag will be HTML escaped.
     ctemplate::TemplateDictionary dict("t1");
     dict.SetValue("color", "<blue>");
     // Expand the template with this dictionary.
     string result;
     ctemplate::ExpandTemplate(FLAGS_test_srcdir + "templates/my_template.html",
                               ctemplate::STRIP_WHITESPACE, &dict, &result);
     // Assert that the expansion has the appropriate text in it.
     EXPECT_THAT(result, HasSubstr("&lt;blue&gt;"));
   }

   void TestColorAppearsBeforeShape() {
     // This time, the condition under test is that the "color" element of
     // the dictionary appears before the "shape" element.
     // Create an appropriate dictionary.
     ctemplate::TemplateDictionary dict("t2"); // Note: use sufficiently unique
     dict.SetValue("color", "white_asdf");     // strings that they won't occur
     dict.SetValue("shape", "square_asdf");    // "bare" in the template.

     string result;
     ctemplate::ExpandTemplate(FLAGS_test_srcdir + "templates/my_template.html",
                               ctemplate::STRIP_WHITESPACE, &dict, &result);
     // Assert that color comes before shape.
     EXPECT_THAT(result, ContainsRegex("white_asdf.*square_asdf"));
   }

Security Considerations

Like all web applications, programs that use the Ctemplate System to create HTML documents can be vulnerable to Cross-Site-Scripting (XSS) attacks unless data inserted into a template is appropriately sanitized and/or escaped. Which specific form of escaping or sanitization is required depends on the context in which the template variable appears within a HTML document (such as, regular "inner text", within a <script> tag, or within an onClick handler).

If you are concerned with XSS, your are strongly encouraged to leverage the Auto Escape mode developed specifically to better defend your application against XSS. The Auto Escape mode follows the guidelines outlined below. Do note however that regardless of whether you use Auto Escape or not, escaping alone while generally required, is often not enough! You also may need to sanitize or validate the input, as for instance with URL attributes. For further information, refer to additional resources on Cross-Site-Scripting issues.

The remainder of this section provides a brief summary of techniques to prevent XSS vulnerabilities due to template variables in various HTML contexts.
  1. Regular text (outside of tags and other special situations).

    Use the auto-escape pragma to html-escape the variable.

  2. HTML tag attributes.

    In addition to the auto-escape pragma, ensure that the attribute is enclosed in double quotes in the template.

  3. URL attributes (eg., href/src).

    In addition to the auto-escape pragma, ensure that the attribute is enclosed in double quotes in the template.

  4. Beware of inserting variables containing data from untrusted sources into the context of a style tag or attribute.

    Certain CSS style-sheet constructs can result in the invocation of javascript. To prevent XSS, the variable must be carefully validated and sanitized.

  5. Populating javascript variables.

    In addition to the auto-escape pragma, ensure that the literal is enclosed in quotes in the template:

         <script>
       var msg_text  = '{{MESSAGE}}';
     </script>

    Literals of non-string types cannot be quoted and escaped. Instead, ensure that the variable's value is set such that it is guaranteed that the resulting string corresponds to a javascript literal of the expected type. For example, use

           dict->SetValueInt("NUM_ITEMS", num_items);

    to populate an integer javascript variable in the template fragment

         <script>
       var num_items = {{NUM_ITEMS}};
     </script>
  6. Populating javascript variables within event handlers such as onClick.

    Tag attributes whose values are evaluated as a javascript expression (such as on{Click,Load,etc} handlers) generally require HTML-Escape in addition to Javascript-Escape, since the attribute's value is HTML-unescaped by the browser before it is passed to the javascript interpreter.

    However, the javascript-escaping provided by auto-escape makes a subsequent HTML-Escape unnecessary. As such you can apply the same rules for variables within event handlers as you would for javascript variables in string literals:

           <button ...
                    onclick='GotoUrl("{{TARGET_URL}}");'>
  7. Consider potential non-template sources of XSS.

    There are a number of scenarios in which XSS can arise that are unrelated to the insertion of values into HTML templates, including,

    • injection into HTTP headers such as Location,
    • incorrect browser-side guess of the content-encoding of a HTML document without explicitly specified charset,
    • incorrect browser-side guess of a non-HTML document's content-type that overrides the document's specified Content-Type,
    • browser-side handling of documents served for download-to-disk (Content-Disposition: attachment).

    Please consult additional documentation on Cross-Site-Scripting for more detailed discussion of such issues.

Working Effectively with Templates

Registering Template Strings

Both dictionary keys and template filenames are strings. Instead of using raw strings, we encourage you to use a bit of machinery to help protect against various types of errors.

For dictionary keys, you can use the make_tpl_varnames_h tool to create static string variables to use instead of a string constant. This will protect against typos, as the make_tpl_varnames_h documentation describes. It also yields slightly more efficient code.

For template filenames that a program uses -- including sub-templates -- we suggest the following idiom:

   #include "example.tpl.varnames.h"   // defines 1 string per dictionary key
   RegisterTemplateFilename(EXAMPLE_FN, "example.tpl");   // defines template
   ...
   include_dict->SetFilename(EXAMPLE_FN);
   ...
   ctemplate::ExpandTemplate(EXAMPLE_FN, ctemplate::DO_NOT_STRIP, ...);
   ...

By registering the filename, you can query the template system to detect syntax errors, reload-status, and so forth.

Managing Templates

There are methods to change the global state of the template system, to examine templates, and to examine template dictionaries. See the reference guide for more information.

Craig Silverstein
ctemplate-ctemplate-2.4/doc/howto.html000066400000000000000000000014411363122316600201070ustar00rootroot00000000000000 How To Use the Ctemplate (formerly Google Template) System

This document has split into two parts: a user's guide and a reference guide. Use the links below to see these, and other documents about the template system.

Craig Silverstein
ctemplate-ctemplate-2.4/doc/index.html000066400000000000000000000064151363122316600200640ustar00rootroot00000000000000 Ctemplate (formerly Google Template) System

Ctemplate System

Status: Current   (as of 25 April 2008)

Welcome to the C++ CTemplate system! (This project was originally called Google Templates, due to its origin as the template system used for Google search result pages, but now has a more general name matching its community-owned nature.)

As a quick start, here's a small but complete program that uses this template library. For more details see, the links below.

Template file example.tpl

   Hello {{NAME}},
   You have just won ${{VALUE}}!
   {{#IN_CA}}Well, ${{TAXED_VALUE}}, after taxes.{{/IN_CA}}

C++ program example.cc

   #include <stdlib.h>
   #include <string>
   #include <iostream>
   #include <ctemplate/template.h>
   int main(int argc, char** argv) {
      ctemplate::TemplateDictionary dict("example");
      dict.SetValue("NAME", "John Smith");
      int winnings = rand() % 100000;
      dict.SetIntValue("VALUE", winnings);
      dict.SetFormattedValue("TAXED_VALUE", "%.2f", winnings * 0.83);
      // For now, assume everyone lives in CA.
      // (Try running the program with a 0 here instead!)
      if (1) {
        dict.ShowSection("IN_CA");
      }
      std::string output;
      ctemplate::ExpandTemplate("example.tpl", ctemplate::DO_NOT_STRIP, &dict, &output);
      std::cout << output;
      return 0;
   }

Compiling and linking (using gcc)

   gcc -o example example.cc -lctemplate_nothreads

I can use the "nothreads" library because example.cc doesn't use threads. If example.cc were threaded, I would do something like this instead:

   gcc -o example example.cc -lctemplate -pthread

See the README for more details about the two different ctemplate libraries.

In-depth Documentation

  1. Howto: Introduction to the Ctemplate system, and a tutorial for using it.
  2. Auto Escape: Guide to using the optional Auto Escape mode to protect your web application better against XSS.
  3. Tips: Advice, tips, and recommendations for best practices with templates, to make them easier to write and maintain, and to avoid common template mistakes.
  4. Examples: Some example templates and application code that uses them. These are taken from actual Google applications.
Craig Silverstein
Last modified: Mon Feb 22 10:59:03 PST 2010
ctemplate-ctemplate-2.4/doc/reference.html000066400000000000000000002133721363122316600207150ustar00rootroot00000000000000 Ctemplate System Reference Guide

Ctemplate System Reference Guide

(as of )

Overview

The main class used by the template system is TemplateDictionary, which is used to expand a template file. It is used by the main functions for expanding a template, found in template.h.

TemplateCache is used to hold a collection of Template objects. TemplateNamelist provides various introspection routines on collections of Template objects.

TemplateModifier and PerExpandData are used to modify the values of a TemplateDictionary at expand time. TemplateAnnotator does too, but is intended for debugging purposes. TemplateDictionaryPeer is used for testing template code.

ExpandEmitter provides the ability to emit an expanded template to an arbitrary output store.

TemplateString is a string-like class that is built to be very efficient when used with the template system. For instance, tools are available to hash constant TemplateString objects at compile-time, speeding up their use at run-time.

The rest of this document describes these classes and functions in more detail, as well as build tools and other mechanisms for handling templates.

(Note: the code snippets below all assume the default configuration option is set, which puts template code in namespace ctemplate.)

Expanding Templates

The Template Language

Templates are strings that contain special formatting code called template markers. In the Ctemplate System, template strings are usually read from a file.

Anything found in a template of the form {{...}} is interpreted as a template marker. All other text is considered formatting text and is output verbatim at template expansion time. Outside of the template markers, templates may contain any text whatsoever, including (single) curly braces and NUL characters.

The template language has six types of markers:

  1. Variable markers, which are replaced by text based on dictionary values. Variable markers look like this: {{FOO}}
  2. Start section and end section markers, which delimit sections which may appear zero, one, or N times in the output. Section markers look like this: {{#FOO}}...{{/FOO}}
  3. Template-include markers, which designate other templates to be expanded and inserted at the location where the marker appears. These are treated much like sections -- one may think of them as sections whose content is specified in a different file instead of inline -- and just like sections, can be expanded zero, one or N times in the output, each with a different dictionary. Template-include markers look like this: {{>FOO}}
  4. Comment markers, which may annotate the template structure but drop completely out of the expanded output. Comment markers look like this: {{! comment lives here -- cool, no?}}
  5. Set-delimiter markers, which change the marker delimiters from {{ and }} to custom strings. (The only requirement is that these strings not contain whitespace or the equals sign.) This is useful for languages like TeX, where double-braces may occur in the text and are awkward to use for markup. Set-delimiter markers look like this: {{=< >=}} <! Now markers are delimited by braces > <=| |=> |! And now markers are delimited by bars! |
  6. Pragma markers, which invoke additional built-in template features when processing the template. Pragma markers look like this: {{%PRAGMA [name="value"...]}}. Currently, AUTOESCAPE is the only pragma defined.

These marker types each have their own namespace. For readability, however, it is best to not overuse a single name.

Modifiers

A variable and include-template can have one or more modifiers attached to them, like so:

   {{MYVAR:mod1:mod2:mod3=value:mod4=value with spaces:mod5}}

A modifier is a filter that's applied at template-expand time, that munges the value of the variable before it's output. See the TemplateModifier class for more details.

When expanding a variable (or template-include) with a modifier, the modifiers are applied in order, left to right. For a template-include, first the entire sub-template is expanded, as a single string, and then the modifiers are applied to that string.

In general, using explicit modifiers does not turn off auto-escaping on a variable. The explicit modifier :none does suppress auto-escaping.

Special Markers

Some marker names may have a special meaning in the template system. Right now, there's one such name.

Separator Sections

If you have a section named FOO, you can define inside of it a section named FOO_separator, and the template system will automatically expand that section every time FOO is expanded, except for the last. Thus, the contents of FOO_separator can be used to separate repeated values of a section.

Here's an example:

   Here are the meeting attendees:
   {{#ATTENDEES}}
      {{NAME}}
      {{#ATTENDEES_separator}}, {{/ATTENDEES_separator}}
   {{/ATTENDEES}}
   .

Here is a more convoluted example, to show the date:

   {{#DATE}}{{DATE_COMPONENT}}{{#DATE_separator}}{{DATE_SEP}}{{/DATE_separator}}{{/DATE}}

You'd set up a template dictionary to repeat DATE three times, with DATE_COMPONENT set to the month, day, and year (or day, month, and year, depending on your locale...), and DATE_SEP set to / or - or whatever date-separator is called for.

SEP_separator is always evaluated with the current dictionary. Thus, in the date example, if you wanted a different separator each time, you could do so by setting DATE_SEP to a different value for each repetition of DATE.

While SEP_separator is automatically expanded by the template system, it is otherwise a perfectly normal section. You can even instantiate it yourself by calling AddSectionDictionary("SEP_separator"). In that case, the section will be expanded both via the automatic expansion as a separator, and as a normal section via the section dictionary you added. This is more confusing than helpful, and you should probably never do it.

There can be at most one "separator" sub-section per section. If there are more, only the last is automatically expanded.

Specifying a template

In the template system -- in functions like ExpandTemplate() -- a template is specified by a pair: filename + strip-mode. The filename specifies the name of the template file on disk (but see below for string templates). The strip mode specifies how this file should be parsed as it's read from disk, in particular, how whitespace should be handled. It can take the following values:

  • ctemplate::DO_NOT_STRIP: do nothing. This expands the template file verbatim.
  • ctemplate::STRIP_BLANK_LINES: remove all blank lines. This ignores any blank lines found in the template file when parsing it. When the template is html, this reduces the size of the output text without requiring a sacrifice of readability for the input file.
  • ctemplate::STRIP_WHITESPACE: remove not only blank lines when parsing, but also whitespace at the beginning and end of each line. It also removes any linefeed (possibly following whitespace) that follows a closing }} of any kind of template marker except a template variable. (This means a linefeed may be removed anywhere by simply placing a comment marker as the last element on the line.) When the template is html, this reduces the size of the output html without changing the way it renders (except in a few special cases.) When using this flag, the built-in template variables BI_NEWLINE and BI_SPACE can be useful to force a space or newline in a particular situation.

The filename is where the template file lives on disk. It can be either an absolute filename, or relative to a directory in the template search path.

In addition to reading a template from disk, it is possible to read a template from a string, using StringToTemplateCache(). The first argument of StringToTemplateCache() is a "key" to use to refer to this string-based template. This key takes the place of the filename, for any routine that asks for a filename + strip.

ExpandTemplate()

This is the main workhorse function of the template system. It takes the filename of a template, a template dictionary, and a string to emit to, and emits an "expanded" version of the template using the given template dictionary to fill in the template markers.

If the template specified by the given filename is already found in an internal cache, the cached version of the template is used, otherwise the template is loaded from disk, parsed, and stored in the cache before expansion is performed. As always, the "filename" can also be a key to a string-based template, inserted directly into the cache via StringToTemplateCache().

There is an overloaded version of Expand() that takes an ExpandEmitter rather than a string, as the source to expand the template into.

This function returns true if the template was successfully expanded into the output parameter. It returns false if expansion failed or was only partially successful. It might fail because the template file cannot be found on disk, or because the template has syntax errors and cannot be parsed, or because the template sub-includes another template, and that sub-template has errors. To minimize the risk of errors at expand-time, you can call LoadTemplate() (below) first to load and parse the template into the cache. This will catch all errors except for errors involving sub-templates.

In the case of partial failures -- typically, failures resulting in an error in a sub-inclued template -- there may be partial data emitted to the output before the error was detected. If ExpandTemplate() returns false, you should be careful to check for and remove this partial data, if desired.

ExpandWithData()

ExpandWithData() is like ExpandTemplate(), with the addition that it allows you to pass in per-expand data. It's called like this:

   ctemplate::TemplateDictionary dict(...);
   ctemplate::PerExpandData per_expand_data;
   string output;
   ctemplate::ExpandWithData(filename, strip_mode, &output, &dict,
                             &per_expand_data);

Per-expand data is applied to all templates that are seen while expanding: not only the template you called Expand() on, but also sub-templates that are brought in via template-includes ({{>INCLUDE}}). See the description of the PerExpandData class for more details about how expansion can be modified by per-expand data.

The return value has the same meaning as for ExpandTemplate() In fact, if you pass in NULL as the per_expand_data argument, this function is exactly equivalent to ExpandTemplate().

LoadTemplate()

This function takes a filename and a strip-mode, and loads the file into the default template cache. Future calls to ExpandTemplate() or ExpandWithData() will get the parsed template from the cache, without needing to go to disk.

This function returns true if the template was successfully read from disk, parsed, and inserted into the cache. It will also return true if the template is already in the cache (even if the file has changed on disk since the template was inserted into the cache). It will return false if the file cannot be found on disk, or cannot be successfully parsed. (Note that LoadTemplate() cannot detect errors in sub-included templates, since the identity of sub-included templates is specified in a TemplateDictionary, not in the template itself.)

StringToTemplateCache()

In addition to reading a template file from disk, it is also possible to read a template file from a string, using StringToTemplateCache(). It takes the content as a string. It also takes in a key and a strip mode. The given key and strip mode can be used as the filename/strip pair in calls to ExpandTemplate() and similar functions. The key can also be used as the "filename" in calls to ctemplate::TemplateDictionary::SetFilename(), allowing this template to be included inside other templates.

StringToTemplateCache() returns true if the string is successfully inserted into the cache. It returns false otherwise, probably because there is already a string or filename in the cache with the same key.

default_template_cache() and mutable_default_template_cache()

All the above routines -- ExpandTemplate(), LoadTemplate(), and the like -- read and write parsed templates to the default template cache. This is just a static instance of the TemplateCache class. It can be accessed via these two functions: default_template_cache() and, if you need a non-const version of the cache, mutable_default_template_cache(). These can be useful if you need the advanced features of template caches, such as ReloadAllIfChanged() or ClearCache().

The TemplateDictionary Class

The class TemplateDictionary is used for all template dictionary operations. In general, an application will need to call a TemplateDictionary method for every marker in the associated template (the major exception: markers that evaluate to the empty string can be ignored).

The appropriate function to call for a given template marker depends on its type.

Data Dictionaries

A data dictionary is a map from keys to values. The keys are always strings, corresponding to the name of the associated template marker, so a section {{#FOO}} in the template text is matched to the key FOO in the dictionary, if it exists. Note the case must match as well.

The value associated with a key differs according to key type. The value associated with a variable is simple: it's the value for that variable. Both keys and values can be any 8-bit character-string, and may include internal NULs (\0).

The value associated with a section is a list of data dictionaries. At template-expansion time, the section is expanded once for each dictionary in the list. The first time, all variables/etc. in the section will be evaluated taking into account the first dictionary. The second time, all variables/etc. will be evaluated taking into account the second dictionary. (See below for a definition of "taking into account.")

The value associated with a template-include is also a list of data dictionaries. Each data dictionary in this list must also have one other, mandatory associated piece of associated information: the filename of the template to include. At expand-time, that filename is passed as-is to ctemplate::ExpandTemplate() (or, more exactly, the equivalent of ExpandTemplate() in the same TemplateCache that the parent template comes from).

Details on Dictionary Lookup

The dictionary structure is a tree: there's a 'main' dictionary, and then a list of sub-dictionaries for each section or include-template. When looking up a marker -- be it a variable, section, or include-template marker -- the system looks in the currently applicable dictionary. If it's found there, that value is used. If not, and the parent dictionary is not an include-template, it continues the look in the parent dictionary, and possibly the grandparent, etc. That is, lookup has static scoping: you look in your dictionary and any parent dictionary that is associated with the same template-file. As soon as continuing the lookup would require you to jump to a new template-file (which is what include-template would do), we stop the lookup.

If lookup fails in all dictionaries, the template system does a final lookup in the global variable dictionary.

The system initializes the global dictionary with a few useful values for your convenience (the user can add more). All system variables are prefixed with BI, to emphasize they are "built in" variables.

  • BI_SPACE, which has the value <space>. It is used to force a space at the beginning or end of a line in the template, where it would normally be suppressed. (See below.)
  • BI_NEWLINE, which has the value <newline> It is used to force a newline at the end of a line, where it would normally be suppressed. (See below.)

Variable inheritance happens at expand time, not at dictionary-create time. So if you create a section dictionary, and then afterwards set a variable in its parent dictionary (or in the global dictionary), the section will inherit that variable value, if it doesn't define the value itself.

SetValue() and variants

SetValue() is used to set the value for a variable marker in a dictionary. It takes a string as input -- nominally a TemplateString, though a C++ string or C char* will be auto-converted -- for the variable name and its value. The name is the same string as is used for the variable's template-marker inside the template this dictionary will be expanded with: if the template reads Hello {{NAME}} then the first argument to SetValue() must be NAME. Case matters.

SetIntValue() takes an integer as the value, rather than a string. On all platforms, the integer may be up to 64 bits.

SetFormattedValue() is a convenience routine. SetFormattedValue(key, arg1, arg2, ...) is logically equivalent to

   char buffer[A_BIG_ENOUGH_NUMBER];
   sprintf(buffer, arg1, arg2, ...);
   SetValue(key, buffer);

without having to worry about the size of buffer, or about stack overflow.

SetValueWithoutCopy() is provided to give an extra bit of efficiency when needed. SetValue() will copy the key and value into an internal data structure used by the TemplateDictionary; this avoids dangling pointers if the arguments to SetValue() are temporaries, or otherwise have shorter lifetime than the TemplateDictionary. But if you know that both the key and value strings have lifetime longer than the TemplateDictionary -- perhaps because they are global (static duration) char*'s -- you can call SetValueWIthoutCopy() to avoid the copy. This yields a small time savings at the cost of significant code fragility, so should only be used when absolutely necessary.

SetGlobalValue() and SetTemplateGlobalValue()

SetGlobalValue() is like SetValue() but works on the global dictionary. Since the global dictionary is shared across all template dictionaries, this is a static method on TemplateDictionary. It is thread-safe. It is also relatively slow.

SetTemplateGlobalValue() is like SetValue(), but the values are preserved across template-includes.

This example shows all three forms:

A.tpl:
   {{NAME}} has won {{>PRIZE}}.  It is worth {{AMOUNT}}.
B.tpl:
   {{AMOUNT}} dollars!  And it's all yours, {{NAME}}
C.tpl:
   To: {{NAME}}.  Amount: {{AMOUNT}}.
code:
   ctemplate::TemplateDictionary dict("set_value_demo");
   ctemplate::TemplateDictionary* subdict = dict.AddIncludeDictionary("PRIZE");
   subdict->SetFilename("B.tpl");
   dict->SetValue("NAME", "Jane McJane");
   dict->SetTemplateGlobalValue("AMOUNT", "One Million");
   ctemplate::TemplateDictionary::SetGlobalValue("NAME", "John Doe");
   ctemplate::TemplateDictionary dict_c("set_value_demo, part 2");

   ctemplate::ExpandTemplate("A.tpl", ..., &dict);
   ctemplate::ExpandTemplate("C.tpl", ..., &dict_c);

The first expand yields this:

   Jane McJane has won One Million dollars!  And it's all yours, John Doe.  It is worth One Million.

The second expand yields this:

   To: John Doe.  Amount: .

NAME was set via SetValue(), so its value was not propagated across the include-dict boundary; instead, the subdict (B.tpl) got its value for NAME from the global dictionary. AMOUNT, on the other hand, was set via SetTemplateGlobalValue(), so its value was propagated across the boundary, and the subdict saw it. However, the totally unrelated template, C.tpl, did not see either value, and only sees the values in the global dictionary. (Of course, had we filled dict_c with some values, C.tpl would have had access to those.)

AddSectionDictionary(), ShowSection(), and SetValueAndShowSection()

AddSectionDictionary(section_name) returns a sub-dictionary associated with the given section_name (for instance, dict.AddSectionDictionary("MYSECT") for a template like {{#MYSECT}}...{{/MYSECT}}). If called multiple times, it will return a new dictionary each time. During Expand(), the section will be expanded once for each time AddSectionDictionary() was called; that is, the text inside the section delimiters will be repeated once for each AddSectionDictionary() call, and within a single repetition, the dictionary returned by AddSectionDictionary() will be used to populate the text inside the section delimiters. (With the current dictionary as that dictionary's parent, for inheritance purposes.)

This suggests that if AddSectionDictionary() is never called on a section, the text inside the section will be omitted entirely by Expand().

ShowSection() is a convenience method used to show the text inside the section exactly once. It is equivalent to calling AddSectionDictionary() once, and ignoring the returned sub-dictionary. All variables in the section will depend on dictionary inheritence to get their values.

SetValueAndShowSection() is another convenience method,, used to show a section if and only if a related variable is set to a non-empty value. SetValueAndShowSection(name, value, section_name) is equivalent to this: if value is empty do nothing, otherwise add a single dictionary to section_name and call section_dict->AddValue(name, value).

Example:

   ctemplate::TemplateDictionary* dict = new ctemplate::TemplateDictionary("section example");
   const char* username = GetUsername();   // returns "" for no user
   if (username[0] != '\0') {
      ctemplate::TemplateDictionary* sub_dict = dict->AddSectionDictionary("CHANGE_USER");
      sub_dict->SetValue("USERNAME", username);
   } else {
      // don't need to do anything; we want a hidden section, which is the default
   }

   // Instead of the above 'if' statement, we could have done this:
   if (username[0] != '\0') {
      dict->ShowSection("CHANGE_USER");       // adds a single, empty dictionary
      dict->SetValue("USERNAME", username);   // take advantage of inheritance
   } else {
      // don't need to do anything; we want a hidden section, which is the default
   }

   // Or we could have done this:
   dict->SetValueAndShowSection("USERNAME", username, "CHANGE_USER");

   // Moving on...
   GetPrevSearches(prev_searches, &num_prev_searches);
   if (num_prev_searches > 0) {
      for (int i = 0; i < num_prev_searches; ++i) {
         TemplateDictionary* sub_dict = dict->AddSectionDictionary("PREV_SEARCHES");
         sub_dict->SetValue("PREV_SEARCH", prev_searches[i]);
      }
   }

AddIncludeDictionary() and SetFilename()

AddIncludeDictionary(section_name) returns a sub-dictionary associated with the given include_name (for instance, dict.AddIncludeDictionary("MYTPL") for a template like {{>MYTPL}}). If called multiple times, it will return a new dictionary each time. It is the responsibility of the caller to then call SetFilename() on the sub-dictionary returned.

During Expand(), each dictionary returned by AddIncludeDictionary will be examined in turn. For each dictionary for which SetFilename() is called, that filename will be read as a template (via LoadTemplate(), with the same "strip" value as the current template), and expanded using the dictionary. (Note that the dictionary will not inherit SetValue() values from the parent dictionary, though it will inherit SetTemplateGlobalValue() values.) This expanded template will then be emitted to the output.

Note that if AddIncludeDictionary() is never called, the template-include will be a no-op during Expand(). Likewise, if it is called but SetFilename() is never called on the resulting sub-dictionary, the template-include will be a no-op. On the other hand, if it is called multiple times, multiple templates -- possibly all from the same file, possibly not -- will be inserted at the point of the template-include.

If a user has called StringToTemplateCache(key, ...), then the user can call SetFilename(key) to include the contents of the string as the sub-template.

Dump() and DumpToString()

These routines dump the contents of a dictionary and its sub-dictionaries. Dump() dumps to stdout, DumpToString() to a string. They are intended for debugging.

The TemplateCache Class

This class holds a collection of templates. It can be used to give you a coherent view of the template system: you load all the templates you are going to use into the cache, and then reload them from disk in one atomic operation. You can have multiple TemplateCache objects in your executable, perhaps one for each service you provide, or perhaps one per thread (so as to avoid thread contention when loading templates).

One intended use of the template cache is to make it safe to reload template files while serving user requests from a webserver. The idea is that every user request, as it's created, is associated with the current template cache. When you wish to reload, you Clone() the current cache, and then reload inside the cloned copy. New requests will get the cloned cache, while old requests will continue to render using the old cache. TemplateCache is written to make this use-case efficient: templates are shared between caches when appropriate, with reference-counting to keep memory management easy.

The default template cache

The template system creates a default template cache, which is available via the functions default_template_cache() and mutable_default_template_cache().

For simple uses of the template system, using the default cache, via ExpandTemplate() and the like, may be perfectly adequate. If you want more sophisticated features, such as the ability to have different versions of the same template file active at one time, or to change the template root-directory, you will have to use an explicit TemplateCache.

LoadTemplate() and StringToTemplateCache()

These two routines are how you explicitly insert data into the cache. LoadTemplate() reads a template from disk, while StringToTemplateCache() takes the data from a user-specified string. These work exactly the same as the global LoadTemplate() and StringToTemplateCache(), except they insert into the given TemplateCache, rather than the global cache.

LoadTemplate() is not strictly necessary: if the cache cannot find a template it needs at Expand*() time, it will automatically try to fetch it from disk. It is intended mostly for use with Freeze(), which disables this auto-fetch behavior.

Both of these routines take a strip mode specifying how the template system should treat whitespace while parsing.

If the template-cache is frozen, LoadTemplate() will only return true if the template is already stored in the cache, and will return false in every other case. For a frozen cache, StringToTemplateCache() will always return false.

ExpandWithData() and ExpandFrozen()

These routines takes a string that represents either a template filename (for disk-based templates), or a key used in StringToTemplateCache() (for string-based templates), a strip mode saying how to parse the template's whitespace, and per-expand data (which can be NULL). Overloads are provided to output the expanded template to either a string or to an arbitrary ExpandEmitter.

Expand uses the filename + strip pair to fetch the template from the cache, if it's there. If not, ExpandWithData() will fetch the template from disk and insert it into the cache. ExpandFrozen(), on the other hand, will just fail to expand, and return false. This is the only difference in behavior between the two routines. To reinforce this behavior, ExpandFrozen() will return false immediately if Freeze() has not been called on the cache.

While expanding, the template system may come across template sub-includes ({{>SUB_TEMPLATE}}). It will attempt to fetch these templates from the cache as well; failing that, it will try to load the template from disk (for ExpandWithData()) or else fail the expand and return false (for ExpandFrozen()).

Because ExpandFrozen() never updates the cache, it is a const method, unlike ExpandWithData().

Note that TemplateCache does not provide the convenience ExpandTemplate() routine, as the analogue of the global ExpandTemplate(). If you are not interested in the per-expand data, just call ExpandWithData() with the per-expand data set to NULL.

ReloadAllIfChanged()

For every file-based template in the cache (this method ignores string-based templates), ReloadAllIfChanged() checks the filesystem to see if the file has changed since it was loaded into the cache. If so, the cache loads a new version of the file into the cache. Note that at this point both the old and new versions of the file exist in the cache! Only the new version is accessible via new calls to Expand(), but any expansions currently in flight during the ReloadAllIfChanged() call will continue to use the old version.

NOTE: ReloadAllIfChanged() never modifies existing items in the cache in any way. It only loads new entries into the cache.

ReloadAllIfChanged() comes in two flavors, controlled by the passed-in enum. In "immediate" mode, it synchronously iterates through the cache and reloads all files that need it. In "lazy" mode, it waits to reload a given template file until the next time it's actually used (in a call to Expand*()). "Immediate" mode is safer in the case where templates depend on each other and the files change a lot, since all files are reloaded at about the same time. "Lazy" mode avoids the latency spike of "immediate" mode, and is preferable in the (common) case that files on disk change only rarely.

SetTemplateRootDirectory(), AddAlternateTemplateRootDirectory(), and template_root_directory()

By default, filenames passed in to Expand*() are taken to be relative to the current working directory. These functions change that behavior. SetTemplateRootDirectory() resets the search path to be the single path passed in to this function. Every time AddAlternateTemplateRootDirectory() is called, it adds another directory to the end of the search path. The template system will follow the search path, in order, when looking for a filename.

Note that the template search path is only meaningful when the filename passed to Expand*() (or specified for a sub-include) is a relative filename. If the filename is an absolute filename, starting with /, the search path is ignored.

template_root_directory() returns the first entry in the search path. There is currently no way to access other entries in the search path.

FindTemplateFilename()

FindTemplateFilename() does the work of finding a template file in the filesystem, using the current template search path. It takes a relative pathname as input, and returns an absolute pathname as output, indicating where the template file lives on the filesystem. If the template file is not found, this method returns the empty string.

Freeze()

Freeze() marks the template cache as immutable. After this method is called, the cache can no longer be modified by loading new templates or reloading existing templates. During expansion only cached included templates will be used; they won't be loaded on-demand.

Before calling Freeze(), you should make sure your cache has all the templates it might need, using LoadTemplate() and StringToTemplateCache(). Otherwise, ExpandWithData() and ExpandFrozen() may fail.

Once the cache is frozen, calls to SetTemplateRootDirectory(), AddAlternateTemplateRootDirectory(), Delete(), and ReloadAllIfChanged() will fail.

After the cache is frozen, the TemplateCache object is effectively const.

Delete()

This method deletes an entry from the cache. If the entry is in the cache multiple times (each with a different "strip" mode), this method deletes all of them.

ClearCache()

This method deletes all entries from the cache. It can be called even on a frozen cache.

Note: this method is not typically necessary unless you are testing for memory leaks. It is intended to be called just before exiting the program, and after all template expansions are completed. Using it in that way may prevent unnecessary reporting by the leak checker.

Clone()

Clone() makes a shallow copy of the given TemplateCache object, and returns the copy. The new copy and the old share the same template objects; since it's a shallow copy, they actually share pointers to the exact same object. (Since the template cache never changes a template object once it's loaded into the cache, sharing the pointer isn't a risk.)

The intended use of Clone(), as described above, is to allow fine control over "epochal" changes in templates. One TemplateCache can hold all versions of the template files before the big update; after the update is done, you can clone the old cache and then call ReloadAllIfChanged() on the clone. This is a low-cost operation, since the two copies share most resources. Smart pointers take care of most memory management issues.

The caller is responsible for calling delete on the returned TemplateCache object.

The TemplateNamelist Class

This class provides information about registered templates. Or more precisely, all templates corresponding to registered filenames.

All methods in this class are static.

RegisterTemplate() and RegisterTemplateFilename()

TemplateNamelist::RegisterTemplate() takes a filename and adds it to the static namelist used by TemplateNamelist. All other methods of TemplateNamelist work only on registered filenames.

It's fairly rare to call this method directly. Instead, the more common use is to use the macro RegisterTemplateFilename somewhere in the global scope, like so:

RegisterTemplateFilename(EXAMPLE_FN, "example.tpl");

The reason to prefer the macro is it defines a global variable that you can use instead of the hard-coded template name. This helps catch typos. The RegisterTemplateFilename() example is functionally equivalent to:

#define EXAMPLE_FN "example.tpl"

It can be used like this:

   ctemplate::ExpandTemplate(EXAMPLE_FN, ctemplate::DO_NOT_STRIP, ...);

GetMissingList()

TemplateNamelist::GetMissingList() returns a list of all registered templates (or rather, all filenames) where the file could not be found on disk.

AllDoExist()

Returns true if and only if the missing-list is empty.

GetBadSyntax()

Returns a list of all registered templates where the template contains a syntax error, and thus cannot be used.

IsAllSyntaxOkay()

True if and only if the bad-syntax list is empty.

GetLastmodTime()

Returns the latest last-modified time for any registered template-file.

The TemplateModifier Class and Associated Functions

A modifier is a filter that's applied at template-expand time, that munges the value of the variable before it's output. For instance, <html><body>{{NAME:html_escape}}</body></html> asks the template system to apply the built-in html_escape modifier when expanding {{NAME}}. If you set NAME in your dictionary to be Jim & Bob, what will actually be emitted in the template is Jim &amp; Bob.

You can chain modifiers together. This template first html-escapes NAME, and then javascript-escapes that result:

   <html><body>{{NAME:html_escape:javascript_escape}}</body></html>

Modifiers typically have a long, descriptive name and also a one-letter abbreviation. So this example is equivalent to the previous one:

   <html><body>{{NAME:h:j}}</body></html>

Some modifiers take an argument, specified after an equals sign:

   <html><body>{{NAME:html_escape_with_arg=pre}}</body></html>

Built-in Modifiers

Here are the modifiers that are built in to the template system. They are all defined in template_modifiers.cc:

long nameshort namedescription
:cleanse_css:c Removes characters not safe for a CSS value. Safe characters are alphanumeric, space, underscore, period, coma, exclamation mark, pound, percent, and dash.
:html_escape:h html-escapes the variable before output (eg & -> &amp;)
:html_escape_with_arg:H special purpose html escaping. See :H Arguments below for details.
:img_src_url_escape_with_arg:I special purpose image url escaping. See :I and :U Arguments below for details.
:javascript_escape:j javascript-escapes the variable before output (eg " -> \x27 and & -> \x26)
:javascript_escape_with_arg:J special purpose javascript escaping. See :J Arguments below for details.
:json_escape:o json-escapes a variable before output as a string in json; HTML characters are escaped using Unicode escape sequences (e.g & -> \u0026) to comply with RFC 4627.
:none leaves the variable as is (used to disable auto-escaping)
:pre_escape:p pre-escapes the variable before output (same as html_escape but whitespace is preserved; useful for <pre>...</pre>)
:url_escape_with_arg:U special purpose url escaping. See :I and :U Arguments below for details.
:url_query_escape:u performs URL escaping on the variable before output. space is turned into +, and everything other than [0-9a-zA-Z.,_:*/~!()-], is transformed into %-style escapes. Use this when you are building URLs with variables as parameters: 
<a href="http://google.com/search?q={{QUERY:u}}">{{QUERY:h}}</a>
:xml_escape xml-escapes the variable before output (the five characters <>&"' become &lt&gt;&amp;&quot;&#39;) suitable for content returned in raw XML. It is not intended for escaping content within CDATA blocks.

The *_with_arg modifiers require an argument to specify the type of escaping to use. The following sections list the supported arguments for each of these modifiers.

:H Arguments

Here are the values that are supported by the html_escape_with_arg modifier:

valuedescription
=snippet like html_escape, but allows HTML entities and some tags to pass through unchanged. The allowed tags are <br>, <wbr>, <b>, and </b>.
=pre same as pre_escape
=url same as :U=html below. For backwards compatibility.
=attribute replaces characters not safe for an use in an unquoted attribute with underscore. Safe characters are alphanumeric, underscore, dash, period, and colon.

:I and :U Arguments

Here are the values that are supported by the img_src_url_escape_with_arg and url_escape_with_arg modifiers:

valuedescription
=html Ensures that a variable contains a safe URL. Safe means that it is either a http or https URL, or else it has no protocol specified.
  • If the URL is safe, the modifier HTML-escapes the URL.
  • Otherwise, the modifier replaces the unsafe URL with one of the following values:
    • /images/cleardot.gif (the :I modifier)
    • # (the :U modifier)
Do not use :U for image URLs. Use :I instead. # is not a safe replacement for an image URL. <img src=#> can cause each browser to request the entire page again.
=javascript Same as =html, but using javascript escaping instead of html escaping.
=css Same as =html but using CSS escaping instead of html escaping so that the variable can be safely inserted within a CSS @import statement or a CSS property. The characters in [\r\n()'"<>*\] are transformed into %-style escapes.
=query (Supported for :U only) Same as url_query_escape.

:J Arguments

Here are the values that are supported by the javascript_escape_with_arg modifier:

valuedescription
=number Ensures that the variable is a valid number or boolean javascript literal. This includes booleans true and false, decimal numbers (e.g. 4.10 or -5.01e+10) as well as hex numbers (e.g. 0x5FF). This modifier is intended to ensure the variable not enclosed in quotes cannot contain javascript code that may execute. It does not guarantee that the variable is syntactically well-formed. If the variable is safe, it is returned as-is, otherwise it is replaced with null. In the future we may add more logic to support objects and arrays.

Custom Modifiers

In addition to the built-in modifiers, you can write your own modifier. Custom modifiers must have a name starting with "x-", and the name can contain alphanumeric characters plus dashes and underscores. Custom modifiers can also accept values with any character except for : and }. For example this template could be a valid use of a custom modifier:

{{VAR:x-my_modifier:value1,value2,value3 has spaces,etc}}

See <template_modifiers.h> for details on how to write a modifier and how to register it. In short, you write a modifier by subclassing ctemplate::TemplateModifier and overriding the Modify method, and you register it by calling ctemplate::AddModifier() Here is an example of the code for a custom modifier:

   class StarEscape : public ctemplate::TemplateModifier {
     void Modify(const char* in, size_t inlen,
                 const ctemplate::PerExpandData* per_expand_data,
                 ctemplate::ExpandEmitter* outbuf, const string& arg) const {
       outbuf->Emit(string("*") + string(in, inlen) + string("*"));
     }
   };

Subclassing TemplateModifier

The minimum work to create a custom modifier is to subclass TemplateModifier and override the Modify() method. Modify() takes as input the value of the text to be modified, as well as the per expand data associated with this Expand() call. The method may use this input, plus any other data it may have, to generate output, which it should write into the given ExpandEmitter.

The subclass can also override MightModify(). This is useful for modifiers that are typically a no-op (in which case the modifier is just doing busy-work, copying its input to the output ExpandEmitter). If MightModify() returns false, the template system will avoid calling Modify() at all on that variable, avoiding the busy-work copy.

AddModifier()

AddModifier() is used to register a custom modifier, so the modifier can be used in templates. The name of the modifier must start with x-.

AddXssSafeModifier()

AddXssSafeModifier() is used to register a custom modifier that can work well with the auto-escape system. It is used when the modifier produces output that is "safe" from cross-site scripting attacks in all contexts in which it might be used. For instance, a modifier that only emits numbers is xss-safe if it's only used in html or javascript contexts.

Auto-escaping and Manual Escaping

If the auto-escape pragma is used in a document, then all variables will be auto-escaped, even if explicit modifiers are used on the variable. The rules are a little complicated:

  • If the explicit modifier is a built-in modifier, and that modifier is "compatible" with the modifier that auto-escape would perform ("compatible" means it safely escapes in at least as many contexts as the auto-escape modifier), then only the explicit modifier is applied, and no further auto-escaping is done.
  • If the explicit modifier is a custom modifier registered using AddXssSafeModifier(), no further auto-escaping is done.
  • If the explicit modifier is the :none modifier, no further auto-escaping is done. This is the mechanism for manually turning off auto-escaping.
  • In all other situations, auto-escaping will be performed after the explicit modifiers have all run.

The PerExpandData Class

ExpandWithData() and TemplateModifier::Modify() both take a PerExpandData object. Per-expand data is applied to all templates that are seen while expanding: not only the template you called ExpandWithData() on, but also sub-templates that are brought in via template-includes ({{>INCLUDE}}).

There are several types of per-expand data you can set, by calling the appropriate method on a PerExpandData object.

SetAnnotateOutput()

This is a debugging function. When expanding this template, it adds marker-strings to the output to indicate what template-substitutions the system made. This takes a string argument which can be used to shorten the filenames printed in the annotations: if the filename contains the string you give, everything before that string is elided from the filename before printing. (It's safe to just always pass in the empty string.)

SetAnnotator()

This overrides the default text-based annotation used by SetAnnotateOutput(). If this is set and SetAnnotateOutput() is called, the per-expand data will use this TemplateAnnotator instance to do the annotation, rather than the default instance.

InsertForModifiers() and LookupForModifiers()

InsertForModifiers() stores an arbitrary key/value pair in the PerExpandData structure. This is used with template modifiers: the PerExpandData object passed to ExpandWithData() is made available to every template modifier that is called during expand time (including any custom modifiers). The intended use of this functionality is to allow a modifier to work one way when expanding a template with dictionary A, and another way when expanding a template with dictionary B. For instance, a modifier might encrypt part of a webpage using a user's secret-key, which is of course different for every expansion of the webpage.

LookupForModifiers() can be used by a template-modifier to read the key/value pair inserted by InsertForModifiers(). LookupForModifiersAsString() is the same, but returns the value as a char* rather than a void*, for convenience.

SetTemplateExpansionModifier()

This is an advanced feature for those who need a custom hook into template expansion. It will not be used by most programmers. It takes a template-modifier as its argument, but this template modifier is treated specially: instead of applying when an appropriate magic string appears in the text of a template, it applies every time a template is expanded during a call to ExpandWithData(). Note this means that it's called not only after the top-level template, that ExpandWithData() is called on, is expanded, but also on every sub-template that is expanded due to a template-include.

This unusual functionality has a few unusual properties. Since the template expansion modifier is not called in normal fashion, the normal arg argument to the template modifier does not make sense. Instead, the arg is set to the path of the template which is being expanded. Also, template expansion modifiers can be expensive, since they are applied pretty indiscriminately, so it can be worth implementing the MightModifiy() predicate for the passed-in TemplateModifier to avoid unnecessary work.

The TemplateAnnotator Class

The TemplateAnnotator class is used to pass in to PerExpandData::SetAnnotator(), to control how expand-time annotation is done. This is meant to be used as a debugging routine.

This class is an abstract base class; SetAnnotator() takes a subclass that implements the various methods of the base class. These methods control the action taken when various template markers are seen during template expansion.

template_annotator.h defines the abstract base class, and also a concrete subclass that is used by default for annotation if SetAnnotator() is not called.

The TemplateDictionaryPeer Class

By design, it is not possible to view the contents of a TemplateDictionary; we want to make sure the interface between the logic layer of the application and the presentation layer of the template goes in only one direction. While generally this keeps programs as clean as possible, it presents problems in testing code, which may want to verify that a given TemplateDictionary has been filled properly. The TemplateDictionaryPeer addresses this need. It lives in template_test_util.h.

Some of the methods of TemplateDictionaryPeer are useful for internal tests only. Below are some of the methods that are most useful for user-level tests.

STS_INIT_FOR_TEST

This macro allows use of STS_INIT for testing, even when the input pointer is not guaranteed to be allocated for the entire length of the test (it must, of course, be allocated for the entire lifetime of the template being tested). Since normally STS_INIT requires inputs to have static duration, this allows for more flexibility in tests, at the cost of worse memory management and decreased code safety (in that it's possible to accidentally violate the lifetime requirements).

GetSectionValue()

This returns the value for the named variable. It uses normal template scoping rules to resolve the name.

IsHiddenSection() and IsHiddenTemplate()

Checks if a section or sub-template is "hidden" (that is, won't be displayed at all).

GetSectionDictionaries() and GetIncludeDictionaries()

Returns all the sub-dictionaries for a given section or template-include, in a vector.

Other Testing Functions

template_test_util.h has other useful routines for testing, such as ExpandIs(), which tests whether a template + dictionary pair expands into an expected string. See the header file for details.

The ExpandEmitter Class

There are two overloads of ExpandTemplate(): the first emits the expanded template to a C++ string, and the second emits to an ExpandEmitter: an abstract base class that serves as a data source. It supports just one overloaded method, Emit(), that can take a char, a char*, or a C++ string as input.

Using this class requires subclassing ExpandEmitter and providing the various definitions for Emit(). For instance, a subclass might provide definitions of Emit() that send the input bytes out to a network socket.

In addition to defining the abstract base class, template_emitter.h provides a sample concrete subclass implementation, for emitting to a string.

The TemplateString and StaticTemplateString Classes

TemplateString is a string-like implementation. Ctemplate uses TemplateString almost exclusively, internally. Many of the public API methods, such as TemplateDictionary::SetValue(), also take TemplateString as input. TemplateString has implicit constructors for both C++ strings and char*'s, so every method that takes a TemplateString will also work if given a C++ string or a char*. If you have a char* and know its length, you can save a bit of work by explicitly constructing a TemplateString with a char* + length, for instance:

   dict->SetValue(ctemplate::TemplateString("MYVAR", 5), value);

Some compile-time tools work with TemplateString to offload computation from runtime to compile-time. This is possible because the Ctemplate code often stores a hash of a string rather than a string directly. For static, immutable strings, TemplateString can store a pre-computed hash value. This functionality is used by make_tpl_varnames_h. Thus, using this tool to create constants to use for SetValue() keys provides not only protection against typos, but a speed improvement as well.

For immutable strings in your code, you can create efficient compile-time template-string objects of your own -- in this case of type StaticTemplateString -- by using the STS_INIT macro, like so:

static const StaticTemplateString kSectName =
    STS_INIT(kSectName, "test_SetAddSectionDictionary");

The string variable's name, kSectName is repeated twice. The variable's value is specified inside the macro. Note that this macro should be used at the top level of a file, not inside functions (even when the variables are made static), and you should define the StaticTemplateString exactly as above: static const StaticTemplateString. Otherwise, the undefined constructor/destructor order of C++ may result in surprising behavior, wherein the StaticTemplateString is not initialized when it ought to be.

The Template Class

In older version of ctemplate, the Template class, which holds parsed templates, was a major part of the template workflow: the common template use-case would be:

   Template* tpl = Template::GetTemplate(filename, strip_mode);
   TemplateDictionary dict(name);
   tpl->Expand(&dict, &outstring);

In current use, this model is deprecated in favor of the single ExpandTemplate() call; support for Template methods may be removed entirely in future versions of ctemplate. However, you may still find older code using this old formulation.

Auto-Escaping

The Guide to using Auto Escape has an overview of Auto Escape as well as discussion of its limitations. To use it, put the following text at the top of the template:

  {{%AUTOESCAPE context="CONTEXT" [state="STATE"]}}

Description of the arguments:

  • The context attribute is one of HTML, JAVASCRIPT, CSS, JSON or XML. It must correspond to the context in which the browser will interpret this template. Warning: Setting the wrong context will result in wrong escaping applied to all variables in the given template. In particular, if the template starts with a <script> tag, its context is HTML and not JAVASCRIPT. Auto-Escape will recognize the <script> tag and hence properly Javascript-escape the variables within it. Similarly, if the template starts with a <style> tag, its context is HTML and not CSS.
  • The state attribute is commonly omitted. It accepts the value IN_TAG in the HTML context to indicate that the template only contains (one or more) HTML attribute name and value pairs that are part of an HTML tag formed in a parent template.

This will auto-escape every variable in the template. To turn off auto-escaping for a particular variable, you can apply the none modifier, like so: {{MYVAR:none}}. Here is an example of an autoescaped document:

  {{%AUTOESCAPE context="HTML"}}

  <body>
    <p>Hello {{USER}}</p>
    <p><a href="{{URL}}">Your Account</a></p>
    <p>Your identifier: {{ID:none}}{{! This is dangerous! }}</p>
  </body>

Development Tools

This package includes several tools to make it easier to use write and use templates.

make_tpl_varnames_h: Template Syntax Checker and Header File Generator

make_tpl_varnames_h is a "lint" style syntax checker and header file generator. It takes the names of template files as command line arguments and loads each file into a Template object by retrieving the file via the Template factory method. The loading of the file does pure syntax checking and reports such errors as mis-matched section start/end markers, mis-matched open/close double-curly braces, such as "{{VAR}", or invalid characters in template variables/names/comments.

If the template passes the syntax check, by default the utility then creates a header file for use in the executable code that fills the dictionary for the template. If the developer includes this header file, then constants in the header file may be referenced in the dictionary building function, rather than hard-coding strings as variable and section names. By using these constants, the compiler can notify the developer of spelling errors and mismatched names. Here's an example of how this is used, and how it helps prevent errors:

   const char * const kosr_RESULT_NUMBER = "RESULT_NUMBER";  // script output
   dict.SetValue("RESSULT_NUMBER", "4");    // typo is silently missed
   dict.SetValue(kosr_RESSULT_NUMBER, "4");   // compiler catches typo

Each constant is named as follows:

  • The initial letter 'k', indicating a defined constant.
  • One or more prefix letters which are derived from the template file name. These prefix letters consist of the first letter of the file name, followed by the first letter following each underscore in the name, with the exception of the letter 'p' when it is followed by the letters "ost", as is a recommended convention for template versioning. For example, the prefix letters for the file one_search_result_post20020815.tpl are osr.
  • An underscore.
  • The variable or section name itself, same casing.

As an example, the section name "RESULT_NUMBER" in the file one_search_result_post20020815.tpl would be given the constant name kosr_RESULT_NUMBER and would appear in the header file as const char * const kosr_RESULT_NUMBER = "RESULT_NUMBER"; -- as in the example above. (The variable is actually a StaticTemplateString, not a char*, but the basic idea holds.)

An alternate output directory may be specified by the command line flag --header_dir.

The name of the generated header file is the same as the name of the template file with an extension added to the name. By default, that extension is .varnames.h. In the above example, the header file containing the constant declarations would be named one_search_result_post20020815.tpl.varnames.h. An alternate extension may be provided via the command line flag --outputfile_suffix.

Important command line flags:

  • --noheader -- Indicates that a header file should not be generated; only syntax checking should be done.
  • --header_dir -- sets the directory where the header is written. Default: "./"
  • --template_dir -- sets the template root directory. Default: ./ which is the correct specification when it is run from the directory where the templates are located. This is only used if the input template filenames are specified as relative paths rather than absolute paths.
  • --outputfile_suffix -- the extension added to the name of the template file to create the name of the generated header file. Default: .varnames.h.

For a full list of command line flags, run make_tpl_varnames_h --help.

template-converter: convert a template to a C++ string

StringToTemplateCache() lets you load a template from a string instead of a file. Applications may prefer this option to reduce the dependencies of the executable, or use it in environments where data files are not practical. In such cases, template-converter can be used as a template "compiler", letting the developer write a template file as a data file in the normal way, and then "compiling" it to a C++ string to be included in the executable.

Usage is template-converter <template filename>. C++ code is output is to stdout; it can be stored in a .h file or included directly into a C++ file. Perl must be installed to use this script.

Craig Silverstein
ctemplate-ctemplate-2.4/doc/tips.html000066400000000000000000000474761363122316600177500ustar00rootroot00000000000000 Tips and Guidelines for Using the Ctemplate System

Tips and Guidelines for Using the Ctemplate System

(as of 10 March 2010)

The basic rules of the template system are enough to use it, but over time, we at Google have developed some tips, guidelines, and best practices that make it easier to use templates effectively, and to avoid common template errors.

Program Design Considerations

Template naming and versioning

Early in Google's use of templates, we noticed a problem: if a binary that uses a template and its corresponding template were both modified, particularly if the change were such that the old binary could not work with the new template or the new binary could not work with the old template, then somehow they both had to be deployed at the same instant to not present errors to our users. This was hard to do. The solution was to adopt a template naming and versioning convention. The procedure to use it follows:

  • Each template name ends with _postYYYYMMDD.tpl, where YYYMMDD is the date of this version's initial creation.
  • Before making (non-backward-compatible) modifications to a template, copy the template to a new name, incorporating a later date than the original one being copied.
  • Edit the new file, and push it to the production server.
  • Finally, update the code to refer to the new template-name (ideally, using the RegisterTemplateFilename idiom), and push the new executable to the production server.

When this convention is followed, the new template file does not overwrite the old one when it is deployed, because it is a new file with a new name. The old template file is still there to be used as long as the old binary is still in production and the new template file just sits there being ignored. Then when the new binary finally gets deployed, it immediately starts using the new template file, because it is coded (in RegisterTemplateFilename) to do so. After that, it is the old template file that continues to sit there ignored.

The make_tpl_varnames_h utility knows about the "_postYYYYMMDD" naming convention, so it is important that you use that convention exactly if you use the make_tpl_varnames_h.

Processing Phases

Typically a program using the Ctemplate System will perform the following phases, usually in this order:

  1. Retrieve and prepare the data used to fill a dictionary.
  2. Build the data dictionary, including all its sub-dictionaries, that will supply the values to the designated template object, its sections, and its included templates.
  3. Retrieve the top-level template object required to format the data. (This may or may not involve reading and parsing a template file, depending on whether the requested file has already been read and parsed by the running program or whether that file has been marked "reload if changed" and was in fact changed.)
  4. Expand the template object into an output buffer using the completed data dictionary.
  5. Output the buffer.
  6. Clean up: Destroy the top-level data dictionary whenever it is no longer needed.
  7. Optionally, clear the cache at the end of program execution.

One template / One procedure call

Most of the code of the program will be in Phases 1 and 2. Clearly, Phase 1 is outside the scope of the template system. But in designing the code for Phase 2 (building the data dictionary), it is wise to have the structure of the program reflect the structure of the templates being used. Specifically, there should be a single procedure call to build the dictionary for a single template. That procedure call should take parameters that include all the data required to populate the data dictionary for that template and all the templates it includes. Following this "one template/one procedure call" guideline further, for each included template, another procedure should be called to populate the (or each) data dictionary for that included template. This maintains the "one template/one procedure call" principle in a nested fashion that reflects the nesting of the templates.

This is not to imply that the "one procedure call" for a template should not be modularized into sub-procedures for readability and maintainability, or that it should not call other auxilliary procedures for such things as formatting the data and converting it to the appropriate strings, etc. But it does mean that there should be one entry point for building the dictionary tree for one template and that entry point should show the data dependencies of that template through its parameter list. This code for populating the data dictionary should NOT be intermingled with data gathering code that should have been done in Phase 1.

(Inside Google, the convention has been used to name the dictionary building procedure using the pattern fill_..._dictionary where the dots are related to the name of the template the data is being prepared for. For instance, the data for the template named one_search_result.tpl might be placed in a dictionary via a function named fill_one_search_result_dictionary.)

Tips, Idioms, and Conventions

  1. Choose template names to create unique constant prefixes.

    Template names should contain at least two words to avoid constant prefix clashes (e.g. kxy_ instead of kx_ ) The name of a new template should be checked against the existing names before proceeding. If your new template name produces a prefix that conflicts with an already existing template, you should change the name of your new template, even though it may be the only perfect name you can come up with. You'll have to use a less than perfect name in that case. (See "Template Syntax Checker and Header File Generator" below for more explanation about constant prefixes.)

  2. Use SetFormattedValue discriminately.

    This method should never be used to sneak HTML into the executable as in

         dictionary->SetFormattedValue(kxy_VAR,
                                   "<b>%s</b>",
                                   some_const_char_string);

    In that case, the <b> and </b> should be moved into the template.

  3. Never have a section encompass an entire template.

    If the first line of a template is a start section marker and the last line is its matching end section marker, then those markers are unnecessary in almost all cases. They are usually put there to allow the entire template to be hidden or iterated, but since it encompasses the entire file, the section may be hidden by not expanding the file (or by hiding the template-include section that includes the file) and it may be iterated by iterating the template-include marker of the including template. (The only exception might be if the entire page is to be iterated, but this seems a bit of a stretch.)

  4. An included template is just a section whose contents are located in a separate file. You may iterate over it just like you do sections.

    For example, if your template has the following template-include marker:

         {{>MY_INCLUDED_TEMPLATE}}

    you may call

         ctemplate::TemplateDictionary *child_dict =
        dictionary->AddIncludeDictionary(kxy_MY_INCLUDED_TEMPLATE);

    to iterate that section. (Note: Make sure you call child_dict->SetFilename()! If your included template is not showing in the output, this is the first thing you should check.)

  5. The recommended idiom to fill an include-template dictionary is like this: 
            fill_include_template_dictionary(dict->AddIncludeDictionary(name), ...);

    But what do you do if you decide, in fill_include_template_dictionary, that you don't want to display anything for this include-template after all? It seems like it's too late: you've already created the sub-dictionary. The solution is simple: just be sure that fill_include_template_dictionary() doesn't call SetFilename() in that case.

  6. Never have a section which only contains another section.

    For example, don't do this:

         {{#OUTER_SECTION}}
        {{#INNER_SECTION}}
        section contents here
        {{/INNER_SECTION}}
     {{/OUTER_SECTION}}

    or this equivalent template code (see the previous item):

         {{#OUTER_SECTION}}
        {{>INCLUDED_SECTION}}
     {{/OUTER_SECTION}}

    This is usually done because the developer thinks the outer section must be used to hide the section when the inner section, intended for iteration, has no iterations. In both cases, you should only have one section (either INNER_SECTION or INCLUDED_SECTION in the examples) and iterate that section either 0 times or more than 0 times. It's the wonder of the dual use of sections, i.e. that they may be conditional or iterative or, in this case, both.

    A related suggestion: Do not have a section whose entire contents is one variable marker with nothing else, unless you need to iterate over that section with multiple values of that variable. You don't need the surrounding section just to hide the marker. A variable marker that is not set, does not produce output. By convention, we set such variables to the empty string. But in neither case do you need to hide it by hiding a surrounding section that contains nothing else.

  7. Use this hide/show idiom for if-else blocks.

    Since sections are hidden by default, you can use represent if-else logic in your code via ShowSection. For example:

         if ( my_test ) {
        dict->ShowSection(kxyz_TRUE_BLOCK);
        [ more code to fill the values for that section]
     } else {
        dict->ShowSection(kxyz_FALSE_BLOCK);
        [ more code to fill the values for that section]
     }
  8. Write... vs. Fill...Dictionary methods - Observe the proper division of labor, don't mix them.

    The output (or write) function should create the top level template dictionary, call one or more fill-dictionary routines with it, then get the template and expand it. It should not call dictionary modifying methods, like ShowSection and SetValue. By keeping these separated into their own fill-dictionary routine, the code is more modular and lends itself to template re-use. If you maintain the proper division of labor, the template you are filling and outputting may be filled and included in a larger template by someone else.

  9. Use AddSectionDictionary only when you want to iterate over a section or, secondarily, if you need to avoid name conflicts.

    Sometimes developers get the idea that every section requires its own child dictionary created by an AddSectionDictionary call. Because of variable inheritence, this isn't usually so. The intended purpose of AddSectionDictionary is to enable iteration over a section. Secondarily, if the section contains generic names that may conflict with the same name in other parts of the template, it may be safer to call AddSectionDictionary to create a separate namespace. In any case, do not assume you must call AddSectionDictionary just because you are working within a section. The main dictionary can be used for all levels of conditional sections as long as you avoid name conflicts by keeping the marker names unique.

  10. Do not place RegisterTemplateFilename statements in header (.h) files.

    RegisterTemplateFilename is a macro that instantiates a TemplateNamelist object. If you place it in a header file, a different object will get created each time it is included in another .cc file.

    The RegisterTemplateFilename statement and its associated #include of the varnames.h file should occur only in the .cc file that implements the fill-dictionary routine for that template. You should never have more than one RegisterTemplateFilename for a single template and you should try hard not to copy the #include file to other files as well. The template versioning makes this more important because a developer may not know that the template name with included version number needs to be updated in more than one file when versioning occurs. [Also see above for more information about what routine uses the filename declared by the RegisterTemplateFilename statement.]

  11. Never reference more than one template in a fill...dictionary method.

    Each template should have its own fill-dictionary routine. That routine should only reference marker names defined in that template. If this convention is followed, then all the prefixes in a fill-dictionary routine will be the same. [Note that an implication of this convention is that if the template includes another template, via a template-include marker, then containing template's fill-dictionary routine should call the included template's fill-dictionary routine (being careful to observe the convention described above). But then, this is merely a restatement of "One template / One procedure call".]

  12. Have fill...dictionary call SetFilename even if the dictionary is never used for a template-include.

    SetFilename() is required when a dictionary is created via AddIncludeDictionary(). However, it's safe to set all the time. By setting it always, you make the code work properly if this dictionary ever changes to be template-included after all. Even if not, by saying what template file the dictionary is intended to go with, you are self-documenting your code.

  13. Do not call c_str() on strings to pass them to TemplateDictionary methods.

    Note that all the TemplateDictionary methods are defined to take TemplateString objects. These are created automatically from both strings and char*'s (and can be created manually if you have a char* and a length). So if you have a string, it's safe and efficient to just pass it in directly; you do not need to extract the const char * from your string object to pass it to these methods. For some reason, this is a common error of novice template coders.

    The one exception to this rule is when using the method SetFormattedValue. When calling that method, you must call c_str() on strings that are to be inserted into the format string, just as you would when providing data for any other printf format string.

  14. Do not use SetGlobalValue when you could use SetValue or SetTemplateGlobalValue.

    SetGlobalValue should be used quite rarely, for constants that really are consistent across all your templates. It's slower to look up a value in the global dictionary than it is in the template-specific dictionary.

  15. Do not use StringToTemplateCache unless you have a specific need for its non-file-based attributes.

    ctemplate::StringToTemplateCache was created for use in highly constrained cases where file I/O may be impaired or undesirable, for instance to produce a server error message where there may be disk problems or to produce formatted output where there are processes that do not have a facility for updating data files dynamically. It is not recommended for ordinary use since it cannot be updated dynamically via a data-push; changes always require a binary push.

  16. Use auto-escaping to prevent Cross-Site-Scripting security vulnerabilities.

    Use {{%AUTOESCAPE}} consistently. Use the :none modifier to override autoesacping only in those (usually rare) cases where there is a specific reason the template variable should not be escaped, for example:

    • The template variable contains HTML markup that should be interpreted by the browser. In this case you must be very careful to ensure that the variable can in no case contain "harmful" HTML. Also, keep in mind the above recommendation on the use of SetFormattedValue and consider moving the HTML markup into the template.
    • The variable is known to be already escaped at the point it is inserted into the template (for example, the value might be kept in escaped form in a storage backend). Here, escaping again via a variable-modifier would result in "double escaping". You must ensure that the variable comes escaped in the appropriate context (that is, if you're inserting the variable into an html document, it's html-escaped and not java-escaped).
  17. Do not leave an extra space when using {{BI_SPACE}}

    The built-in template variable BI_SPACE is itself replaced by a single space. It is used where you need to make sure a space is preserved at the end of a line. It is a common mistake to leave an extra space before this marker, which results in not one, but two, spaces created in the document.

    Incorrect:

    <table border=0 {{BI_SPACE}}
       align=center>

    Correct:

    <table border=0{{BI_SPACE}}
       align=center>
Craig Silverstein
ctemplate-ctemplate-2.4/doc/xss_resources.html000066400000000000000000000045071363122316600216640ustar00rootroot00000000000000 Cross-Site Scripting Resources

Cross-Site Scripting Resources

Status: Current   (as of 17 August 2006)

Cross-Site Scripting (commonly abbreviated as XSS) is a security issue that arises when an attacker can cause client-side script (such as JavaScript) of his or her choosing to execute within another user's browser in the context of a given web-site or web-application. This may allow the attacker to steal that user's session cookies for the web-application in question, or otherwise manipulate that user's session context.

XSS vulnerabilities most often arise if a web-application renders data that originated from an untrusted source (such as a query parameter) in a HTML document without carefully validating or escaping that data.

The following online resources provide further information on XSS vulnerabilities and how to avoid them:

ctemplate-ctemplate-2.4/m4/000077500000000000000000000000001363122316600156345ustar00rootroot00000000000000ctemplate-ctemplate-2.4/m4/ac_have_attribute.m4000066400000000000000000000007641363122316600215560ustar00rootroot00000000000000AC_DEFUN([AX_C___ATTRIBUTE__], [ AC_MSG_CHECKING(for __attribute__) AC_CACHE_VAL(ac_cv___attribute__, [ AC_TRY_COMPILE( [#include static void foo(void) __attribute__ ((unused)); void foo(void) { exit(1); }], [], ac_cv___attribute__=yes, ac_cv___attribute__=no )]) if test "$ac_cv___attribute__" = "yes"; then AC_DEFINE(HAVE___ATTRIBUTE__, 1, [define if your compiler has __attribute__]) fi AC_MSG_RESULT($ac_cv___attribute__) ]) ctemplate-ctemplate-2.4/m4/ac_interlocked_exchange.m4000066400000000000000000000032731363122316600227130ustar00rootroot00000000000000# Check whether InterlockedExchange() takes a LONG or a volatile LONG # as its first argument. InterlockedCompareExchange is a windows # function; obviously, this macro is useful only for cygwin and mingw, # and other systems that compile against the windows API. # # Apparently the interface for this function is a bit inconsistent. # Windows likes volatile LONG, but mingw and cygwin don't, at least # for the versions I'm using. But rather than try to guess who # supports what, let's just check at configure time. (Note: this # is an error in C++ but only a warning in C, so we test in the former.) # # This function returns 'yes' if the type does not need volatile, # and defines the symbol INTERLOCKED_EXCHANGE_NONVOLATILE. (This # is the expected case for mingw and cygwin). It returns 'no', # and defines no symbol, otherwise. (This is the expected case for # MSVC.) The return value was sset this way so that we don't need # to define any symbols on windows, which doesn't run configure. AC_DEFUN([AC_INTERLOCKED_EXCHANGE_NONVOLATILE], [ AC_MSG_CHECKING(whether first argument to InterlockedExchange omits volatile) AC_CACHE_VAL(ac_cv_interlocked_exchange_nonvolatile, [AC_LANG_SAVE AC_LANG_CPLUSPLUS AC_TRY_COMPILE([#include ], [volatile LONG once; InterlockedExchange(&once, 1);], ac_cv_interlocked_exchange_nonvolatile="no", ac_cv_interlocked_exchange_nonvolatile="yes") AC_LANG_RESTORE ]) if test "$ac_cv_interlocked_exchange_nonvolatile" = "yes"; then AC_DEFINE(INTERLOCKED_EXCHANGE_NONVOLATILE, 1, [define if first argument to InterlockedExchange is just LONG]) fi AC_MSG_RESULT($ac_cv_interlocked_exchange_nonvolatile) ]) ctemplate-ctemplate-2.4/m4/ac_rwlock.m4000066400000000000000000000031101363122316600200350ustar00rootroot00000000000000# TODO(csilvers): it would be better to actually try to link against # -pthreads, to make sure it defines these methods, but that may be # too hard, since pthread support is really tricky. # Check for support for pthread_rwlock_init() etc. # These aren't posix, but are widely supported. To get them on linux, # you need to define _XOPEN_SOURCE first, so this check assumes your # application does that. # # Note: OS X (as of 6/1/06) seems to support pthread_rwlock, but # doesn't define PTHREAD_RWLOCK_INITIALIZER. Therefore, we don't test # that particularly macro. It's probably best if you don't use that # macro in your code either. # # Note: Cygwin (as of 12/1/08) has a bug in pthread_rwlock, where # if you try to acquire a read-lock twice, you get EDEADLCK, where # it should succeed. It would be nice to test for that, but we # can't do runtime checks here. So we just manually fail for CYGWIN. # TODO(csilvers): do better. AC_DEFUN([AC_RWLOCK], [AC_CACHE_CHECK(support for pthread_rwlock_* functions, ac_cv_rwlock, [AC_LANG_SAVE AC_LANG_C AC_TRY_COMPILE([#define _XOPEN_SOURCE 500 #include #ifdef __CYGWIN32__ # error Cygwin has a bug in pthread_rwlock; disabling #endif], [pthread_rwlock_t l; pthread_rwlock_init(&l, NULL); pthread_rwlock_rdlock(&l); return 0;], ac_cv_rwlock=yes, ac_cv_rwlock=no) AC_LANG_RESTORE ]) if test "$ac_cv_rwlock" = yes; then AC_DEFINE(HAVE_RWLOCK,1,[define if the compiler implements pthread_rwlock_*]) fi ]) ctemplate-ctemplate-2.4/m4/ax_pthread.m4000066400000000000000000000505201363122316600202170ustar00rootroot00000000000000# =========================================================================== # http://www.gnu.org/software/autoconf-archive/ax_pthread.html # =========================================================================== # # SYNOPSIS # # AX_PTHREAD([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]]) # # DESCRIPTION # # This macro figures out how to build C programs using POSIX threads. It # sets the PTHREAD_LIBS output variable to the threads library and linker # flags, and the PTHREAD_CFLAGS output variable to any special C compiler # flags that are needed. (The user can also force certain compiler # flags/libs to be tested by setting these environment variables.) # # Also sets PTHREAD_CC to any special C compiler that is needed for # multi-threaded programs (defaults to the value of CC otherwise). (This # is necessary on AIX to use the special cc_r compiler alias.) # # NOTE: You are assumed to not only compile your program with these flags, # but also to link with them as well. For example, you might link with # $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS # # If you are only building threaded programs, you may wish to use these # variables in your default LIBS, CFLAGS, and CC: # # LIBS="$PTHREAD_LIBS $LIBS" # CFLAGS="$CFLAGS $PTHREAD_CFLAGS" # CC="$PTHREAD_CC" # # In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant # has a nonstandard name, this macro defines PTHREAD_CREATE_JOINABLE to # that name (e.g. PTHREAD_CREATE_UNDETACHED on AIX). # # Also HAVE_PTHREAD_PRIO_INHERIT is defined if pthread is found and the # PTHREAD_PRIO_INHERIT symbol is defined when compiling with # PTHREAD_CFLAGS. # # ACTION-IF-FOUND is a list of shell commands to run if a threads library # is found, and ACTION-IF-NOT-FOUND is a list of commands to run it if it # is not found. If ACTION-IF-FOUND is not specified, the default action # will define HAVE_PTHREAD. # # Please let the authors know if this macro fails on any platform, or if # you have any other suggestions or comments. This macro was based on work # by SGJ on autoconf scripts for FFTW (http://www.fftw.org/) (with help # from M. Frigo), as well as ac_pthread and hb_pthread macros posted by # Alejandro Forero Cuervo to the autoconf macro repository. We are also # grateful for the helpful feedback of numerous users. # # Updated for Autoconf 2.68 by Daniel Richard G. # # LICENSE # # Copyright (c) 2008 Steven G. Johnson # Copyright (c) 2011 Daniel Richard G. # # This program is free software: you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation, either version 3 of the License, or (at your # option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. # # You should have received a copy of the GNU General Public License along # with this program. If not, see . # # As a special exception, the respective Autoconf Macro's copyright owner # gives unlimited permission to copy, distribute and modify the configure # scripts that are the output of Autoconf when processing the Macro. You # need not follow the terms of the GNU General Public License when using # or distributing such scripts, even though portions of the text of the # Macro appear in them. The GNU General Public License (GPL) does govern # all other use of the material that constitutes the Autoconf Macro. # # This special exception to the GPL applies to versions of the Autoconf # Macro released by the Autoconf Archive. When you make and distribute a # modified version of the Autoconf Macro, you may extend this special # exception to the GPL to apply to your modified version as well. #serial 23 AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD]) AC_DEFUN([AX_PTHREAD], [ AC_REQUIRE([AC_CANONICAL_HOST]) AC_REQUIRE([AC_PROG_CC]) AC_REQUIRE([AC_PROG_SED]) AC_LANG_PUSH([C]) ax_pthread_ok=no # We used to check for pthread.h first, but this fails if pthread.h # requires special compiler flags (e.g. on Tru64 or Sequent). # It gets checked for in the link test anyway. # First of all, check if the user has set any of the PTHREAD_LIBS, # etcetera environment variables, and if threads linking works using # them: if test "x$PTHREAD_CFLAGS$PTHREAD_LIBS" != "x"; then ax_pthread_save_CC="$CC" ax_pthread_save_CFLAGS="$CFLAGS" ax_pthread_save_LIBS="$LIBS" AS_IF([test "x$PTHREAD_CC" != "x"], [CC="$PTHREAD_CC"]) CFLAGS="$CFLAGS $PTHREAD_CFLAGS" LIBS="$PTHREAD_LIBS $LIBS" AC_MSG_CHECKING([for pthread_join using $CC $PTHREAD_CFLAGS $PTHREAD_LIBS]) AC_LINK_IFELSE([AC_LANG_CALL([], [pthread_join])], [ax_pthread_ok=yes]) AC_MSG_RESULT([$ax_pthread_ok]) if test "x$ax_pthread_ok" = "xno"; then PTHREAD_LIBS="" PTHREAD_CFLAGS="" fi CC="$ax_pthread_save_CC" CFLAGS="$ax_pthread_save_CFLAGS" LIBS="$ax_pthread_save_LIBS" fi # We must check for the threads library under a number of different # names; the ordering is very important because some systems # (e.g. DEC) have both -lpthread and -lpthreads, where one of the # libraries is broken (non-POSIX). # Create a list of thread flags to try. Items starting with a "-" are # C compiler flags, and other items are library names, except for "none" # which indicates that we try without any flags at all, and "pthread-config" # which is a program returning the flags for the Pth emulation library. ax_pthread_flags="pthreads none -Kthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config" # The ordering *is* (sometimes) important. Some notes on the # individual items follow: # pthreads: AIX (must check this before -lpthread) # none: in case threads are in libc; should be tried before -Kthread and # other compiler flags to prevent continual compiler warnings # -Kthread: Sequent (threads in libc, but -Kthread needed for pthread.h) # -pthread: Linux/gcc (kernel threads), BSD/gcc (userland threads), Tru64 # (Note: HP C rejects this with "bad form for `-t' option") # -pthreads: Solaris/gcc (Note: HP C also rejects) # -mt: Sun Workshop C (may only link SunOS threads [-lthread], but it # doesn't hurt to check since this sometimes defines pthreads and # -D_REENTRANT too), HP C (must be checked before -lpthread, which # is present but should not be used directly; and before -mthreads, # because the compiler interprets this as "-mt" + "-hreads") # -mthreads: Mingw32/gcc, Lynx/gcc # pthread: Linux, etcetera # --thread-safe: KAI C++ # pthread-config: use pthread-config program (for GNU Pth library) case $host_os in freebsd*) # -kthread: FreeBSD kernel threads (preferred to -pthread since SMP-able) # lthread: LinuxThreads port on FreeBSD (also preferred to -pthread) ax_pthread_flags="-kthread lthread $ax_pthread_flags" ;; hpux*) # From the cc(1) man page: "[-mt] Sets various -D flags to enable # multi-threading and also sets -lpthread." ax_pthread_flags="-mt -pthread pthread $ax_pthread_flags" ;; openedition*) # IBM z/OS requires a feature-test macro to be defined in order to # enable POSIX threads at all, so give the user a hint if this is # not set. (We don't define these ourselves, as they can affect # other portions of the system API in unpredictable ways.) AC_EGREP_CPP([AX_PTHREAD_ZOS_MISSING], [ # if !defined(_OPEN_THREADS) && !defined(_UNIX03_THREADS) AX_PTHREAD_ZOS_MISSING # endif ], [AC_MSG_WARN([IBM z/OS requires -D_OPEN_THREADS or -D_UNIX03_THREADS to enable pthreads support.])]) ;; solaris*) # On Solaris (at least, for some versions), libc contains stubbed # (non-functional) versions of the pthreads routines, so link-based # tests will erroneously succeed. (N.B.: The stubs are missing # pthread_cleanup_push, or rather a function called by this macro, # so we could check for that, but who knows whether they'll stub # that too in a future libc.) So we'll check first for the # standard Solaris way of linking pthreads (-mt -lpthread). ax_pthread_flags="-mt,pthread pthread $ax_pthread_flags" ;; esac # GCC generally uses -pthread, or -pthreads on some platforms (e.g. SPARC) AS_IF([test "x$GCC" = "xyes"], [ax_pthread_flags="-pthread -pthreads $ax_pthread_flags"]) # The presence of a feature test macro requesting re-entrant function # definitions is, on some systems, a strong hint that pthreads support is # correctly enabled case $host_os in darwin* | hpux* | linux* | osf* | solaris*) ax_pthread_check_macro="_REENTRANT" ;; aix*) ax_pthread_check_macro="_THREAD_SAFE" ;; *) ax_pthread_check_macro="--" ;; esac AS_IF([test "x$ax_pthread_check_macro" = "x--"], [ax_pthread_check_cond=0], [ax_pthread_check_cond="!defined($ax_pthread_check_macro)"]) # Are we compiling with Clang? AC_CACHE_CHECK([whether $CC is Clang], [ax_cv_PTHREAD_CLANG], [ax_cv_PTHREAD_CLANG=no # Note that Autoconf sets GCC=yes for Clang as well as GCC if test "x$GCC" = "xyes"; then AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG], [/* Note: Clang 2.7 lacks __clang_[a-z]+__ */ # if defined(__clang__) && defined(__llvm__) AX_PTHREAD_CC_IS_CLANG # endif ], [ax_cv_PTHREAD_CLANG=yes]) fi ]) ax_pthread_clang="$ax_cv_PTHREAD_CLANG" ax_pthread_clang_warning=no # Clang needs special handling, because older versions handle the -pthread # option in a rather... idiosyncratic way if test "x$ax_pthread_clang" = "xyes"; then # Clang takes -pthread; it has never supported any other flag # (Note 1: This will need to be revisited if a system that Clang # supports has POSIX threads in a separate library. This tends not # to be the way of modern systems, but it's conceivable.) # (Note 2: On some systems, notably Darwin, -pthread is not needed # to get POSIX threads support; the API is always present and # active. We could reasonably leave PTHREAD_CFLAGS empty. But # -pthread does define _REENTRANT, and while the Darwin headers # ignore this macro, third-party headers might not.) PTHREAD_CFLAGS="-pthread" PTHREAD_LIBS= ax_pthread_ok=yes # However, older versions of Clang make a point of warning the user # that, in an invocation where only linking and no compilation is # taking place, the -pthread option has no effect ("argument unused # during compilation"). They expect -pthread to be passed in only # when source code is being compiled. # # Problem is, this is at odds with the way Automake and most other # C build frameworks function, which is that the same flags used in # compilation (CFLAGS) are also used in linking. Many systems # supported by AX_PTHREAD require exactly this for POSIX threads # support, and in fact it is often not straightforward to specify a # flag that is used only in the compilation phase and not in # linking. Such a scenario is extremely rare in practice. # # Even though use of the -pthread flag in linking would only print # a warning, this can be a nuisance for well-run software projects # that build with -Werror. So if the active version of Clang has # this misfeature, we search for an option to squash it. AC_CACHE_CHECK([whether Clang needs flag to prevent "argument unused" warning when linking with -pthread], [ax_cv_PTHREAD_CLANG_NO_WARN_FLAG], [ax_cv_PTHREAD_CLANG_NO_WARN_FLAG=unknown # Create an alternate version of $ac_link that compiles and # links in two steps (.c -> .o, .o -> exe) instead of one # (.c -> exe), because the warning occurs only in the second # step ax_pthread_save_ac_link="$ac_link" ax_pthread_sed='s/conftest\.\$ac_ext/conftest.$ac_objext/g' ax_pthread_link_step=`$as_echo "$ac_link" | sed "$ax_pthread_sed"` ax_pthread_2step_ac_link="($ac_compile) && (echo ==== >&5) && ($ax_pthread_link_step)" ax_pthread_save_CFLAGS="$CFLAGS" for ax_pthread_try in '' -Qunused-arguments -Wno-unused-command-line-argument unknown; do AS_IF([test "x$ax_pthread_try" = "xunknown"], [break]) CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS" ac_link="$ax_pthread_save_ac_link" AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])], [ac_link="$ax_pthread_2step_ac_link" AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])], [break]) ]) done ac_link="$ax_pthread_save_ac_link" CFLAGS="$ax_pthread_save_CFLAGS" AS_IF([test "x$ax_pthread_try" = "x"], [ax_pthread_try=no]) ax_cv_PTHREAD_CLANG_NO_WARN_FLAG="$ax_pthread_try" ]) case "$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG" in no | unknown) ;; *) PTHREAD_CFLAGS="$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG $PTHREAD_CFLAGS" ;; esac fi # $ax_pthread_clang = yes if test "x$ax_pthread_ok" = "xno"; then for ax_pthread_try_flag in $ax_pthread_flags; do case $ax_pthread_try_flag in none) AC_MSG_CHECKING([whether pthreads work without any flags]) ;; -mt,pthread) AC_MSG_CHECKING([whether pthreads work with -mt -lpthread]) PTHREAD_CFLAGS="-mt" PTHREAD_LIBS="-lpthread" ;; -*) AC_MSG_CHECKING([whether pthreads work with $ax_pthread_try_flag]) PTHREAD_CFLAGS="$ax_pthread_try_flag" ;; pthread-config) AC_CHECK_PROG([ax_pthread_config], [pthread-config], [yes], [no]) AS_IF([test "x$ax_pthread_config" = "xno"], [continue]) PTHREAD_CFLAGS="`pthread-config --cflags`" PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`" ;; *) AC_MSG_CHECKING([for the pthreads library -l$ax_pthread_try_flag]) PTHREAD_LIBS="-l$ax_pthread_try_flag" ;; esac ax_pthread_save_CFLAGS="$CFLAGS" ax_pthread_save_LIBS="$LIBS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" LIBS="$PTHREAD_LIBS $LIBS" # Check for various functions. We must include pthread.h, # since some functions may be macros. (On the Sequent, we # need a special flag -Kthread to make this header compile.) # We check for pthread_join because it is in -lpthread on IRIX # while pthread_create is in libc. We check for pthread_attr_init # due to DEC craziness with -lpthreads. We check for # pthread_cleanup_push because it is one of the few pthread # functions on Solaris that doesn't have a non-functional libc stub. # We try pthread_create on general principles. AC_LINK_IFELSE([AC_LANG_PROGRAM([#include # if $ax_pthread_check_cond # error "$ax_pthread_check_macro must be defined" # endif static void routine(void *a) { a = 0; } static void *start_routine(void *a) { return a; }], [pthread_t th; pthread_attr_t attr; pthread_create(&th, 0, start_routine, 0); pthread_join(th, 0); pthread_attr_init(&attr); pthread_cleanup_push(routine, 0); pthread_cleanup_pop(0) /* ; */])], [ax_pthread_ok=yes], []) CFLAGS="$ax_pthread_save_CFLAGS" LIBS="$ax_pthread_save_LIBS" AC_MSG_RESULT([$ax_pthread_ok]) AS_IF([test "x$ax_pthread_ok" = "xyes"], [break]) PTHREAD_LIBS="" PTHREAD_CFLAGS="" done fi # Various other checks: if test "x$ax_pthread_ok" = "xyes"; then ax_pthread_save_CFLAGS="$CFLAGS" ax_pthread_save_LIBS="$LIBS" CFLAGS="$CFLAGS $PTHREAD_CFLAGS" LIBS="$PTHREAD_LIBS $LIBS" # Detect AIX lossage: JOINABLE attribute is called UNDETACHED. AC_CACHE_CHECK([for joinable pthread attribute], [ax_cv_PTHREAD_JOINABLE_ATTR], [ax_cv_PTHREAD_JOINABLE_ATTR=unknown for ax_pthread_attr in PTHREAD_CREATE_JOINABLE PTHREAD_CREATE_UNDETACHED; do AC_LINK_IFELSE([AC_LANG_PROGRAM([#include ], [int attr = $ax_pthread_attr; return attr /* ; */])], [ax_cv_PTHREAD_JOINABLE_ATTR=$ax_pthread_attr; break], []) done ]) AS_IF([test "x$ax_cv_PTHREAD_JOINABLE_ATTR" != "xunknown" && \ test "x$ax_cv_PTHREAD_JOINABLE_ATTR" != "xPTHREAD_CREATE_JOINABLE" && \ test "x$ax_pthread_joinable_attr_defined" != "xyes"], [AC_DEFINE_UNQUOTED([PTHREAD_CREATE_JOINABLE], [$ax_cv_PTHREAD_JOINABLE_ATTR], [Define to necessary symbol if this constant uses a non-standard name on your system.]) ax_pthread_joinable_attr_defined=yes ]) AC_CACHE_CHECK([whether more special flags are required for pthreads], [ax_cv_PTHREAD_SPECIAL_FLAGS], [ax_cv_PTHREAD_SPECIAL_FLAGS=no case $host_os in solaris*) ax_cv_PTHREAD_SPECIAL_FLAGS="-D_POSIX_PTHREAD_SEMANTICS" ;; esac ]) AS_IF([test "x$ax_cv_PTHREAD_SPECIAL_FLAGS" != "xno" && \ test "x$ax_pthread_special_flags_added" != "xyes"], [PTHREAD_CFLAGS="$ax_cv_PTHREAD_SPECIAL_FLAGS $PTHREAD_CFLAGS" ax_pthread_special_flags_added=yes]) AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT], [ax_cv_PTHREAD_PRIO_INHERIT], [AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]], [[int i = PTHREAD_PRIO_INHERIT;]])], [ax_cv_PTHREAD_PRIO_INHERIT=yes], [ax_cv_PTHREAD_PRIO_INHERIT=no]) ]) AS_IF([test "x$ax_cv_PTHREAD_PRIO_INHERIT" = "xyes" && \ test "x$ax_pthread_prio_inherit_defined" != "xyes"], [AC_DEFINE([HAVE_PTHREAD_PRIO_INHERIT], [1], [Have PTHREAD_PRIO_INHERIT.]) ax_pthread_prio_inherit_defined=yes ]) CFLAGS="$ax_pthread_save_CFLAGS" LIBS="$ax_pthread_save_LIBS" # More AIX lossage: compile with *_r variant if test "x$GCC" != "xyes"; then case $host_os in aix*) AS_CASE(["x/$CC"], [x*/c89|x*/c89_128|x*/c99|x*/c99_128|x*/cc|x*/cc128|x*/xlc|x*/xlc_v6|x*/xlc128|x*/xlc128_v6], [#handle absolute path differently from PATH based program lookup AS_CASE(["x$CC"], [x/*], [AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])], [AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])])]) ;; esac fi fi test -n "$PTHREAD_CC" || PTHREAD_CC="$CC" AC_SUBST([PTHREAD_LIBS]) AC_SUBST([PTHREAD_CFLAGS]) AC_SUBST([PTHREAD_CC]) # Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND: if test "x$ax_pthread_ok" = "xyes"; then ifelse([$1],,[AC_DEFINE([HAVE_PTHREAD],[1],[Define if you have POSIX threads libraries and header files.])],[$1]) : else ax_pthread_ok=no $2 fi AC_LANG_POP ])dnl AX_PTHREAD ctemplate-ctemplate-2.4/m4/compiler_characteristics.m4000066400000000000000000000022211363122316600231400ustar00rootroot00000000000000# Check compiler characteristics (e.g. type sizes, PRIxx macros, ...) # If types $1 and $2 are compatible, perform action $3 AC_DEFUN([AC_TYPES_COMPATIBLE], [AC_TRY_COMPILE([#include ], [$1 v1 = 0; $2 v2 = 0; return (&v1 - &v2)], $3)]) define(AC_PRIUS_COMMENT, [printf format code for printing a size_t and ssize_t]) AC_DEFUN([AC_COMPILER_CHARACTERISTICS], [AC_CACHE_CHECK(AC_PRIUS_COMMENT, ac_cv_formatting_prius_prefix, [AC_TYPES_COMPATIBLE(unsigned int, size_t, ac_cv_formatting_prius_prefix=; ac_cv_prius_defined=1) AC_TYPES_COMPATIBLE(unsigned long, size_t, ac_cv_formatting_prius_prefix=l; ac_cv_prius_defined=1) AC_TYPES_COMPATIBLE(unsigned long long, size_t, ac_cv_formatting_prius_prefix=ll; ac_cv_prius_defined=1 )]) if test -z "$ac_cv_prius_defined"; then ac_cv_formatting_prius_prefix=z; fi AC_DEFINE_UNQUOTED(PRIuS, "${ac_cv_formatting_prius_prefix}u", AC_PRIUS_COMMENT) AC_DEFINE_UNQUOTED(PRIxS, "${ac_cv_formatting_prius_prefix}x", AC_PRIUS_COMMENT) AC_DEFINE_UNQUOTED(PRIdS, "${ac_cv_formatting_prius_prefix}d", AC_PRIUS_COMMENT) ]) ctemplate-ctemplate-2.4/m4/stl_hash.m4000066400000000000000000000061571363122316600177140ustar00rootroot00000000000000# We check two things: where the include file is for # unordered_map/hash_map (we prefer the first form), and what # namespace unordered/hash_map lives in within that include file. We # include AC_TRY_COMPILE for all the combinations we've seen in the # wild. We define HASH_MAP_H to the location of the header file, and # HASH_NAMESPACE to the namespace the class (unordered_map or # hash_map) is in. We define HAVE_UNORDERED_MAP if the class we found # is named unordered_map, or leave it undefined if not. # This also checks if unordered map exists. AC_DEFUN([AC_CXX_STL_HASH], [ AC_MSG_CHECKING(the location of hash_map) AC_LANG_SAVE AC_LANG_CPLUSPLUS ac_cv_cxx_hash_map="" # First try unordered_map, but not on gcc's before 4.2 -- I've # seen unexplainable unordered_map bugs with -O2 on older gcc's. AC_TRY_COMPILE([#if defined(__GNUC__) && (__GNUC__ < 4 || (__GNUC__ == 4 && __GNUC_MINOR__ < 2)) # error GCC too old for unordered_map #endif ], [/* no program body necessary */], [stl_hash_old_gcc=no], [stl_hash_old_gcc=yes]) for location in unordered_map tr1/unordered_map; do for namespace in std std::tr1; do if test -z "$ac_cv_cxx_hash_map" -a "$stl_hash_old_gcc" != yes; then # Some older gcc's have a buggy tr1, so test a bit of code. AC_TRY_COMPILE([#include <$location>], [const ${namespace}::unordered_map t; return t.find(5) == t.end();], [ac_cv_cxx_hash_map="<$location>"; ac_cv_cxx_hash_namespace="$namespace"; ac_cv_cxx_have_unordered_map="yes";]) fi done done # Now try hash_map for location in ext/hash_map hash_map; do for namespace in __gnu_cxx "" std stdext; do if test -z "$ac_cv_cxx_hash_map"; then AC_TRY_COMPILE([#include <$location>], [${namespace}::hash_map t], [ac_cv_cxx_hash_map="<$location>"; ac_cv_cxx_hash_namespace="$namespace"; ac_cv_cxx_have_unordered_map="no";]) fi done done ac_cv_cxx_hash_set=`echo "$ac_cv_cxx_hash_map" | sed s/map/set/`; if test -n "$ac_cv_cxx_hash_map"; then AC_DEFINE(HAVE_HASH_MAP, 1, [define if the compiler has hash_map]) AC_DEFINE(HAVE_HASH_SET, 1, [define if the compiler has hash_set]) AC_DEFINE_UNQUOTED(HASH_MAP_H,$ac_cv_cxx_hash_map, [the location of or ]) AC_DEFINE_UNQUOTED(HASH_SET_H,$ac_cv_cxx_hash_set, [the location of or ]) AC_DEFINE_UNQUOTED(HASH_NAMESPACE,$ac_cv_cxx_hash_namespace, [the namespace of hash_map/hash_set]) if test "$ac_cv_cxx_have_unordered_map" = yes; then AC_DEFINE(HAVE_UNORDERED_MAP,1, [define if the compiler supports unordered_{map,set}]) fi AC_MSG_RESULT([$ac_cv_cxx_hash_map]) else AC_MSG_RESULT() AC_MSG_WARN([could not find an STL hash_map]) fi ]) ctemplate-ctemplate-2.4/packages/000077500000000000000000000000001363122316600170725ustar00rootroot00000000000000ctemplate-ctemplate-2.4/packages/deb.sh000077500000000000000000000050001363122316600201560ustar00rootroot00000000000000#!/bin/bash -e # This takes one commandline argument, the name of the package. If no # name is given, then we'll end up just using the name associated with # an arbitrary .tar.gz file in the rootdir. That's fine: there's probably # only one. # # Run this from the 'packages' directory, just under rootdir ## Set LIB to lib if exporting a library, empty-string else LIB= #LIB=lib PACKAGE="$1" VERSION="$2" # We can only build Debian packages, if the Debian build tools are installed if [ \! -x /usr/bin/debuild ]; then echo "Cannot find /usr/bin/debuild. Not building Debian packages." 1>&2 exit 0 fi # Double-check we're in the packages directory, just under rootdir if [ \! -r ../Makefile -a \! -r ../INSTALL ]; then echo "Must run $0 in the 'packages' directory, under the root directory." 1>&2 echo "Also, you must run \"make dist\" before running this script." 1>&2 exit 0 fi # Find the top directory for this package topdir="${PWD%/*}" # Find the tar archive built by "make dist" archive="${PACKAGE}-${VERSION}" archive_with_underscore="${PACKAGE}_${VERSION}" if [ -z "${archive}" ]; then echo "Cannot find ../$PACKAGE*.tar.gz. Run \"make dist\" first." 1>&2 exit 0 fi # Create a pristine directory for building the Debian package files trap 'rm -rf '`pwd`/tmp'; exit $?' EXIT SIGHUP SIGINT SIGTERM rm -rf tmp mkdir -p tmp cd tmp # Debian has very specific requirements about the naming of build # directories, and tar archives. It also wants to write all generated # packages to the parent of the source directory. We accommodate these # requirements by building directly from the tar file. ln -s "${topdir}/${archive}.tar.gz" "${LIB}${archive}.orig.tar.gz" # Some version of debuilder want foo.orig.tar.gz with _ between versions. ln -s "${topdir}/${archive}.tar.gz" "${LIB}${archive_with_underscore}.orig.tar.gz" tar zfx "${LIB}${archive}.orig.tar.gz" [ -n "${LIB}" ] && mv "${archive}" "${LIB}${archive}" cd "${LIB}${archive}" # This is one of those 'specific requirements': where the deb control files live cp -a "packages/deb" "debian" # Now, we can call Debian's standard build tool debuild -uc -us cd ../.. # get back to the original top-level dir # We'll put the result in a subdirectory that's named after the OS version # we've made this .deb file for. destdir="debian-$(cat /etc/debian_version 2>/dev/null || echo UNKNOWN)" rm -rf "$destdir" mkdir -p "$destdir" mv $(find tmp -mindepth 1 -maxdepth 1 -type f) "$destdir" echo echo "The Debian package files are located in $PWD/$destdir" ctemplate-ctemplate-2.4/packages/deb/000077500000000000000000000000001363122316600176245ustar00rootroot00000000000000ctemplate-ctemplate-2.4/packages/deb/README000066400000000000000000000004571363122316600205120ustar00rootroot00000000000000The list of files here isn't complete. For a step-by-step guide on how to set this package up correctly, check out http://www.debian.org/doc/maint-guide/ Most of the files that are in this directory are boilerplate. However, you may need to change the list of binary-arch dependencies in 'rules'. ctemplate-ctemplate-2.4/packages/deb/changelog000066400000000000000000000075171363122316600215100ustar00rootroot00000000000000ctemplate (2.1-1) unstable; urgency=low * New upstream release. -- Olaf van der Spek Thu, 22 Mar 2012 00:29:09 +0100 ctemplate (2.0-1) unstable; urgency=low * New upstream release. -- Google Inc. and others Tue, 24 Jan 2012 16:00:51 -0800 ctemplate (1.1-1) unstable; urgency=low * New upstream release. -- Google Inc. Thu, 22 Dec 2011 14:03:27 -0800 ctemplate (1.0-1) unstable; urgency=low * New upstream release. -- Google Inc. Fri, 26 Aug 2011 13:10:46 -0700 ctemplate (1.0rc2-1) unstable; urgency=low * New upstream release. -- Google Inc. Fri, 29 Jul 2011 12:19:50 -0700 ctemplate (1.0rc1-1) unstable; urgency=low * New upstream release. -- Google Inc. Fri, 22 Jul 2011 15:31:05 -0700 ctemplate (0.99-1) unstable; urgency=low * New upstream release. -- Google Inc. Mon, 24 Jan 2011 15:38:31 -0800 ctemplate (0.98-1) unstable; urgency=low * New upstream release. -- Google Inc. Fri, 24 Sep 2010 11:38:27 -0700 ctemplate (0.97-1) unstable; urgency=low * New upstream release. -- Google Inc. Mon, 19 Apr 2010 15:55:23 -0700 ctemplate (0.96-1) unstable; urgency=low * New upstream release. -- Google Inc. Mon, 19 Oct 2009 11:42:57 -0700 ctemplate (0.95-2) unstable; urgency=low * Fix the include directory in the spec files: google -> ctemplate -- Google Inc. Fri, 12 Jun 2009 08:13:35 -0700 ctemplate (0.95-1) unstable; urgency=low * New upstream release. -- Google Inc. Fri, 12 Jun 2009 08:13:35 -0700 ctemplate (0.94-1) unstable; urgency=low * New upstream release. -- Google Inc. Thu, 07 May 2009 11:27:28 -0700 ctemplate (0.93-1) unstable; urgency=low * New upstream release. -- Google Inc. Mon, 02 Feb 2009 13:51:33 -0800 ctemplate (0.92-1) unstable; urgency=low * New upstream release. -- Google Inc. Wed, 12 Nov 2008 11:24:33 -0800 ctemplate (0.91-1) unstable; urgency=low * New upstream release. -- Google Inc. Tue, 19 Aug 2008 16:20:27 -0700 ctemplate (0.90-1) unstable; urgency=low * New upstream release. -- Google Inc. Sun, 27 Apr 2008 15:06:15 -0700 ctemplate (0.9-1) unstable; urgency=low * New upstream release. -- Google Inc. Thu, 24 Jan 2008 16:09:43 -0800 ctemplate (0.8-1) unstable; urgency=low * New upstream release. -- Google Inc. Thu, 16 Aug 2007 21:42:55 -0700 ctemplate (0.7-1) unstable; urgency=low * New upstream release. -- Google Inc. Tue, 03 Jul 2007 12:46:59 -0700 ctemplate (0.6.1-1) unstable; urgency=low * New upstream release. -- Google Inc. Thu, 21 Jun 2007 14:02:32 -0700 ctemplate (0.6-1) unstable; urgency=low * New upstream release. -- Google Inc. Sat, 09 Jun 2007 22:34:52 -0700 ctemplate (0.5-1) unstable; urgency=low * New upstream release. -- Google Inc. Mon, 14 May 2007 17:27:10 -0700 ctemplate (0.4-1) unstable; urgency=low * New upstream release. -- Google Inc. Mon, 15 Jan 2007 14:10:42 -0800 ctemplate (0.3-1) unstable; urgency=low * New upstream release. -- Google Inc. Mon, 21 Aug 2006 17:44:32 -0700 ctemplate (0.2-1) unstable; urgency=low * New upstream release. -- Google Inc. Wed, 14 Jun 2006 14:56:04 -0700 ctemplate (0.1-1) unstable; urgency=low * Initial release. -- Google Inc. Mon, 13 Mar 2006 22:17:14 -0800 ctemplate-ctemplate-2.4/packages/deb/compat000066400000000000000000000000021363122316600210220ustar00rootroot000000000000004 ctemplate-ctemplate-2.4/packages/deb/control000066400000000000000000000021761363122316600212350ustar00rootroot00000000000000Source: ctemplate Section: libdevel Priority: optional Maintainer: Google Inc. and others Build-Depends: debhelper (>= 4.0.0) Standards-Version: 3.6.1 Package: libctemplate-dev Section: libdevel Architecture: any Depends: libctemplate0 (= ${Source-Version}) Description: This package contains a library implementing a simple but powerful template language for C++. It emphasizes separating logic from presentation: it is impossible to embed application logic in this template language. The devel package contains static and debug libraries and header files for developing applications that use the ctemplate package. Package: libctemplate0 Section: libs Architecture: any Description: This package contains a library implementing a simple but powerful template language for C++. It emphasizes separating logic from presentation: it is impossible to embed application logic in this template language. This limits the power of the template language without limiting the power of the template *system*. Indeed, Google's "main" web search uses this system exclusively for formatting output. ctemplate-ctemplate-2.4/packages/deb/copyright000066400000000000000000000032771363122316600215700ustar00rootroot00000000000000This package was debianized by Craig Silverstein on Tue, 24 Jan 2012 16:00:51 -0800. It was downloaded from http://code.google.com/p/ctemplate/downloads/list Upstream Author: google-ctemplate@googlegroups.com Copyright (c) 2006, Google Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of Google Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ctemplate-ctemplate-2.4/packages/deb/docs000066400000000000000000000003321363122316600204750ustar00rootroot00000000000000AUTHORS COPYING ChangeLog INSTALL NEWS README doc/designstyle.css doc/index.html doc/howto.html doc/tips.html doc/example.html doc/xss_resources.html contrib/README.contrib contrib/highlighting.vim contrib/tpl-mode.el ctemplate-ctemplate-2.4/packages/deb/libctemplate-dev.dirs000066400000000000000000000001041363122316600237230ustar00rootroot00000000000000usr/lib usr/lib/pkgconfig usr/include usr/include/ctemplate usr/bin ctemplate-ctemplate-2.4/packages/deb/libctemplate-dev.install000066400000000000000000000004141363122316600244340ustar00rootroot00000000000000usr/include/ctemplate/* usr/lib/lib*.so usr/lib/lib*.a usr/lib/lib*.la usr/bin/* usr/lib/pkgconfig/* debian/tmp/usr/include/ctemplate/* debian/tmp/usr/lib/lib*.so debian/tmp/usr/lib/lib*.a debian/tmp/usr/lib/lib*.la debian/tmp/usr/bin/* debian/tmp/usr/lib/pkgconfig/* ctemplate-ctemplate-2.4/packages/deb/libctemplate0.dirs000066400000000000000000000000101363122316600232230ustar00rootroot00000000000000usr/lib ctemplate-ctemplate-2.4/packages/deb/libctemplate0.install000066400000000000000000000000571363122316600237430ustar00rootroot00000000000000usr/lib/lib*.so.* debian/tmp/usr/lib/lib*.so.* ctemplate-ctemplate-2.4/packages/deb/rules000077500000000000000000000055421363122316600207120ustar00rootroot00000000000000#!/usr/bin/make -f # -*- makefile -*- # Sample debian/rules that uses debhelper. # This file was originally written by Joey Hess and Craig Small. # As a special exception, when this file is copied by dh-make into a # dh-make output file, you may use that output file without restriction. # This special exception was added by Craig Small in version 0.37 of dh-make. # Uncomment this to turn on verbose mode. #export DH_VERBOSE=1 # These are used for cross-compiling and for saving the configure script # from having to guess our platform (since we know it already) DEB_HOST_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE) DEB_BUILD_GNU_TYPE ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE) CFLAGS = -Wall -g ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS))) CFLAGS += -O0 else CFLAGS += -O2 endif ifeq (,$(findstring nostrip,$(DEB_BUILD_OPTIONS))) INSTALL_PROGRAM += -s endif # shared library versions, option 1 #version=2.0.5 #major=2 # option 2, assuming the library is created as src/.libs/libfoo.so.2.0.5 or so version=`ls src/.libs/lib*.so.* | \ awk '{if (match($$0,/[0-9]+\.[0-9]+\.[0-9]+$$/)) print substr($$0,RSTART)}'` major=`ls src/.libs/lib*.so.* | \ awk '{if (match($$0,/\.so\.[0-9]+$$/)) print substr($$0,RSTART+4)}'` config.status: configure dh_testdir # Add here commands to configure the package. CFLAGS="$(CFLAGS)" ./configure --host=$(DEB_HOST_GNU_TYPE) --build=$(DEB_BUILD_GNU_TYPE) --prefix=/usr --mandir=\$${prefix}/share/man --infodir=\$${prefix}/share/info build: build-stamp build-stamp: config.status dh_testdir # Add here commands to compile the package. $(MAKE) touch build-stamp clean: dh_testdir dh_testroot rm -f build-stamp # Add here commands to clean up after the build process. -$(MAKE) distclean ifneq "$(wildcard /usr/share/misc/config.sub)" "" cp -f /usr/share/misc/config.sub config.sub endif ifneq "$(wildcard /usr/share/misc/config.guess)" "" cp -f /usr/share/misc/config.guess config.guess endif dh_clean install: build dh_testdir dh_testroot dh_clean -k dh_installdirs # Add here commands to install the package into debian/tmp $(MAKE) install DESTDIR=$(CURDIR)/debian/tmp # Build architecture-independent files here. binary-indep: build install # We have nothing to do by default. # Build architecture-dependent files here. binary-arch: build install dh_testdir dh_testroot dh_installchangelogs ChangeLog dh_installdocs dh_installexamples dh_install --sourcedir=debian/tmp # dh_installmenu # dh_installdebconf # dh_installlogrotate # dh_installemacsen # dh_installpam # dh_installmime # dh_installinit # dh_installcron # dh_installinfo dh_installman dh_link dh_strip dh_compress dh_fixperms # dh_perl # dh_python dh_makeshlibs dh_installdeb dh_shlibdeps dh_gencontrol dh_md5sums dh_builddeb binary: binary-indep binary-arch .PHONY: build clean binary-indep binary-arch binary install ctemplate-ctemplate-2.4/packages/rpm.sh000077500000000000000000000047571363122316600202440ustar00rootroot00000000000000#!/bin/sh -e # Run this from the 'packages' directory, just under rootdir # We can only build rpm packages, if the rpm build tools are installed if [ \! -x /usr/bin/rpmbuild ] then echo "Cannot find /usr/bin/rpmbuild. Not building an rpm." 1>&2 exit 0 fi # Check the commandline flags PACKAGE="$1" VERSION="$2" fullname="${PACKAGE}-${VERSION}" archive=../$fullname.tar.gz if [ -z "$1" -o -z "$2" ] then echo "Usage: $0 " 1>&2 exit 0 fi # Double-check we're in the packages directory, just under rootdir if [ \! -r ../Makefile -a \! -r ../INSTALL ] then echo "Must run $0 in the 'packages' directory, under the root directory." 1>&2 echo "Also, you must run \"make dist\" before running this script." 1>&2 exit 0 fi if [ \! -r "$archive" ] then echo "Cannot find $archive. Run \"make dist\" first." 1>&2 exit 0 fi # Create the directory where the input lives, and where the output should live RPM_SOURCE_DIR="/tmp/rpmsource-$fullname" RPM_BUILD_DIR="/tmp/rpmbuild-$fullname" trap 'rm -rf $RPM_SOURCE_DIR $RPM_BUILD_DIR; exit $?' EXIT SIGHUP SIGINT SIGTERM rm -rf "$RPM_SOURCE_DIR" "$RPM_BUILD_DIR" mkdir "$RPM_SOURCE_DIR" mkdir "$RPM_BUILD_DIR" cp "$archive" "$RPM_SOURCE_DIR" # rpmbuild -- as far as I can tell -- asks the OS what CPU it has. # This may differ from what kind of binaries gcc produces. dpkg # does a better job of this, so if we can run 'dpkg --print-architecture' # to get the build CPU, we use that in preference of the rpmbuild # default. target=`dpkg --print-architecture 2>/dev/null || echo ""` if [ -n "$target" ] then target=" --target $target" fi rpmbuild -bb rpm/rpm.spec $target \ --define "NAME $PACKAGE" \ --define "VERSION $VERSION" \ --define "_sourcedir $RPM_SOURCE_DIR" \ --define "_builddir $RPM_BUILD_DIR" \ --define "_rpmdir $RPM_SOURCE_DIR" # We put the output in a directory based on what system we've built for destdir=rpm-unknown if [ -r /etc/issue ] then grep "Red Hat.*release 7" /etc/issue >/dev/null 2>&1 && destdir=rh7 grep "Red Hat.*release 8" /etc/issue >/dev/null 2>&1 && destdir=rh8 grep "Red Hat.*release 9" /etc/issue >/dev/null 2>&1 && destdir=rh9 if grep Fedora /etc/issue >/dev/null; then destdir=fc`grep Fedora /etc/issue | cut -d' ' -f 4`; fi fi rm -rf "$destdir" mkdir -p "$destdir" # We want to get not only the main package but devel etc, hence the middle * mv "$RPM_SOURCE_DIR"/*/"${PACKAGE}"-*"${VERSION}"*.rpm "$destdir" echo echo "The rpm package file(s) are located in $PWD/$destdir" ctemplate-ctemplate-2.4/packages/rpm/000077500000000000000000000000001363122316600176705ustar00rootroot00000000000000ctemplate-ctemplate-2.4/packages/rpm/rpm.spec000066400000000000000000000051171363122316600213460ustar00rootroot00000000000000%define RELEASE 1 %define rel %{?CUSTOM_RELEASE} %{!?CUSTOM_RELEASE:%RELEASE} %define prefix /usr Name: %NAME Summary: Simple but powerful template language for C++ Version: %VERSION Release: %rel Group: Development/Libraries URL: http://code.google.com/p/ctemplate License: BSD Vendor: Google Inc. and others Packager: Google Inc. and others Source: http://%{NAME}.googlecode.com/files/%{NAME}-%{VERSION}.tar.gz Distribution: Redhat 7 and above. Buildroot: %{_tmppath}/%{name}-root Prefix: %prefix %description The %name package contains a library implementing a simple but powerful template language for C++. It emphasizes separating logic from presentation: it is impossible to embed application logic in this template language. This limits the power of the template language without limiting the power of the template *system*. Indeed, Google's "main" web search uses this system exclusively for formatting output. %package devel Summary: Simple but powerful template language for C++ Group: Development/Libraries Requires: %{NAME} = %{VERSION} %description devel The %name-devel package contains static and debug libraries and header files for developing applications that use the %name package. %changelog * Wed Apr 22 2009 - Change build rule to use %configure instead of ./configure - Change install to use DESTDIR instead of prefix for make install - Use wildcards for doc/ and lib/ directories - Use {_libdir}/{_includedir}/etc instead of {prefix}/lib, etc * Mon Mar 13 2006 - First draft %prep %setup %build # I can't use '% configure', because it defines -m32 which breaks on # my development environment for some reason. But I do take # as much from % configure (in /usr/lib/rpm/macros) as I can. ./configure --prefix=%{_prefix} --exec-prefix=%{_exec_prefix} --bindir=%{_bindir} --sbindir=%{_sbindir} --sysconfdir=%{_sysconfdir} --datadir=%{_datadir} --includedir=%{_includedir} --libdir=%{_libdir} --libexecdir=%{_libexecdir} --localstatedir=%{_localstatedir} --sharedstatedir=%{_sharedstatedir} --mandir=%{_mandir} --infodir=%{_infodir} make %install rm -rf $RPM_BUILD_ROOT make DESTDIR=$RPM_BUILD_ROOT install %clean rm -rf $RPM_BUILD_ROOT %files %defattr(-,root,root) %docdir %{prefix}/share/doc/%{NAME}-%{VERSION} %{prefix}/share/doc/%{NAME}-%{VERSION}/* %{_libdir}/*.so.* %files devel %defattr(-,root,root) %{_includedir}/ctemplate %{_libdir}/*.a %{_libdir}/*.la %{_libdir}/*.so %{_bindir}/make_tpl_varnames_h %{_bindir}/template-converter %{_bindir}/diff_tpl_auto_escape %{_libdir}/pkgconfig/*.pc ctemplate-ctemplate-2.4/src/000077500000000000000000000000001363122316600161035ustar00rootroot00000000000000ctemplate-ctemplate-2.4/src/base/000077500000000000000000000000001363122316600170155ustar00rootroot00000000000000ctemplate-ctemplate-2.4/src/base/arena-inl.h000077500000000000000000000324301363122316600210410ustar00rootroot00000000000000// Copyright (c) 2000, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // Reorganized by Craig Silverstein // // In this file we define the arena template code. This includes the // ArenaAllocator, which is meant only to be used with STL, and also // the Gladiator (which needs to know how to new and delete various // types of objects). // // If you're only using the MALLOC-LIKE functionality of the arena, // you don't need to include this file at all! You do need to include // it (in your own .cc file) if you want to use the STRING, STL, or // NEW aspects of the arena. See arena.h for details on these types. // // ArenaAllocator is an STL allocator, but because it relies on unequal // instances, it may not work with all standards-conforming STL // implementations. But it works with SGI STL so we're happy. // // Here's an example of how the ArenaAllocator would be used. // Say we have a vector of ints that we want to have use the arena // for memory allocation. Here's one way to do it: // UnsafeArena* arena = new UnsafeArena(1000); // or SafeArena(), or 10000 // vector > v(arena); // // Note that every STL type always allows the allocator (in this case, // the arena, which is automatically promoted to an allocator) as the last // arg to the constructor. So if you would normally do // vector<...> v(foo, bar), // with the arena you can do // vector<...> v(foo, bar, arena); #ifndef BASE_ARENA_INL_H_ #define BASE_ARENA_INL_H_ #include #include "base/arena.h" #include #include #include #include namespace ctemplate { // T is the type we want to allocate, and C is the type of the arena. // ArenaAllocator has the thread-safety characteristics of C. template class ArenaAllocator { public: typedef T value_type; typedef size_t size_type; typedef ptrdiff_t difference_type; typedef T* pointer; typedef const T* const_pointer; typedef T& reference; typedef const T& const_reference; pointer address(reference r) const { return &r; } const_pointer address(const_reference r) const { return &r; } size_type max_size() const { return size_t(-1) / sizeof(T); } // DO NOT USE! The default constructor is for gcc3 compatibility only. ArenaAllocator() : arena_(0) { } // This is not an explicit constructor! So you can pass in an arena* // to functions needing an ArenaAllocator (like the astring constructor) // and everything will work ok. ArenaAllocator(C* arena) : arena_(arena) { } // NOLINT ~ArenaAllocator() { } pointer allocate(size_type n, std::allocator::const_pointer /*hint*/ = 0) { assert(arena_ && "No arena to allocate from!"); return reinterpret_cast(arena_->AllocAligned(n * sizeof(T), kAlignment)); } void deallocate(pointer p, size_type n) { arena_->Free(p, n * sizeof(T)); } void construct(pointer p, const T & val) { new(reinterpret_cast(p)) T(val); } void construct(pointer p) { new(reinterpret_cast(p)) T(); } void destroy(pointer p) { p->~T(); } C* arena(void) const { return arena_; } template struct rebind { typedef ArenaAllocator other; }; template ArenaAllocator(const ArenaAllocator& other) : arena_(other.arena()) { } template bool operator==(const ArenaAllocator& other) const { return arena_ == other.arena(); } template bool operator!=(const ArenaAllocator& other) const { return arena_ != other.arena(); } protected: static const int kAlignment; C* arena_; }; template const int ArenaAllocator::kAlignment = (1 == sizeof(T) ? 1 : BaseArena::kDefaultAlignment); // 'new' must be in the global namespace. } using ctemplate::UnsafeArena; // Operators for allocation on the arena // Syntax: new (AllocateInArena, arena) MyClass; // new (AllocateInArena, arena) MyClass[num]; // Useful for classes you can't descend from Gladiator, such as POD, // STL containers, etc. enum AllocateInArenaType { AllocateInArena }; inline void* operator new(size_t size, AllocateInArenaType /* unused */, UnsafeArena *arena) { return arena->Alloc(size); } inline void* operator new[](size_t size, AllocateInArenaType /* unused */, UnsafeArena *arena) { return arena->Alloc(size); } namespace ctemplate { // Ordinarily in C++, one allocates all instances of a class from an // arena. If that's what you want to do, you don't need Gladiator. // (However you may find ArenaOnlyGladiator useful.) // // However, for utility classes that are used by multiple clients, the // everything-in-one-arena model may not work. Some clients may wish // not to use an arena at all. Or perhaps a composite structure // (tree) will contain multiple objects (nodes) and some of those // objects will be created by a factory, using an arena, while other // objects will be created on-the-fly by an unsuspecting user who // doesn't know anything about the arena. // // To support that, have the arena-allocated class inherit from // Gladiator. The ordinary operator new will continue to allocate // from the heap. To allocate from an arena, do // Myclass * m = new (AllocateInArena, a) Myclass (args, to, constructor); // where a is either an arena or an allocator. Now you can call // delete on all the objects, whether they are allocated from an arena // or on the heap. Heap memory will be released, while arena memory will // not be. // // If a client knows that no objects were allocated on the heap, it // need not delete any objects (but it may if it wishes). The only // objects that must be deleted are those that were actually allocated // from the heap. // // NOTE: an exception to the google C++ style guide rule for "No multiple // implementation inheritance" is granted for this class: you can treat this // class as an "Interface" class, and use it in a multiple inheritence context, // even though it implements operator new/delete. class Gladiator { public: Gladiator() { } virtual ~Gladiator() { } // We do not override the array allocators, so array allocation and // deallocation will always be from the heap. Typically, arrays are // larger, and thus the costs of arena allocation are higher and the // benefits smaller. Since arrays are typically allocated and deallocated // very differently from scalars, this may not interfere too much with // the arena concept. If it does pose a problem, flesh out the // ArrayGladiator class below. void* operator new(size_t size) { void* ret = ::operator new(1 + size); static_cast(ret)[size] = 1; // mark as heap-allocated return ret; } // the ignored parameter keeps us from stepping on placement new template void* operator new(size_t size, const int ignored, T* allocator) { if (allocator) { void* ret = allocator->AllocAligned(1 + size, BaseArena::kDefaultAlignment); static_cast(ret)[size] = 0; // mark as arena-allocated return ret; } else { return operator new(size); // this is the function above } } void operator delete(void* memory, size_t size) { if (static_cast(memory)[size]) { assert (1 == static_cast(memory)[size]); ::operator delete(memory); } else { // We never call the allocator's Free method. If we need to do // that someday, we can store a pointer to the arena instead of // the Boolean marker flag. } } template void operator delete(void* memory, size_t size, const int ign, T* allocator) { // This "placement delete" can only be called if the constructor // throws an exception. if (allocator) { allocator->Free(memory, 1 + size); } else { ::operator delete(memory); } } }; // This avoids the space overhead of Gladiator if you just want to // override new and delete. It helps avoid some of the more common // problems that can occur when overriding new and delete. class ArenaOnlyGladiator { public: ArenaOnlyGladiator() { } // No virtual destructor is needed because we ignore the size // parameter in all the delete functions. // virtual ~ArenaOnlyGladiator() { } // can't just return NULL here -- compiler gives a warning. :-| void* operator new(size_t /*size*/) { assert(0); return reinterpret_cast(1); } void* operator new[](size_t /*size*/) { assert(0); return reinterpret_cast(1); } // the ignored parameter keeps us from stepping on placement new template void* operator new(size_t size, const int ignored, T* allocator) { assert(allocator); return allocator->AllocAligned(size, BaseArena::kDefaultAlignment); } template void* operator new[](size_t size, const int ignored, T* allocator) { assert(allocator); return allocator->AllocAligned (size, BaseArena::kDefaultAlignment); } void operator delete(void* /*memory*/, size_t /*size*/) { } template void operator delete(void* memory, size_t size, const int ign, T* allocator) { } void operator delete [](void* /*memory*/) { } template void operator delete(void* memory, const int ign, T* allocator) { } }; #if 0 // ********** for example purposes only; 100% untested. // Note that this implementation incurs an overhead of kHeaderSize for // every array that is allocated. *Before* the space is returned to the // user, we store the address of the Arena that owns the space, and // the length of th space itself. class ArrayGladiator : public Gladiator { public: void * operator new[] (size_t size) { const int sizeplus = size + kHeaderSize; void * const ret = ::operator new(sizeplus); *static_cast(ret) = NULL; // mark as heap-allocated *static_cast(ret + sizeof(Arena *)) = sizeplus; return ret + kHeaderSize; } // the ignored parameter keeps us from stepping on placement new template void * operator new[] (size_t size, const int ignored, T * allocator) { if (allocator) { const int sizeplus = size + kHeaderSize; void * const ret = allocator->AllocAligned(sizeplus, BaseArena::kDefaultAlignment); *static_cast(ret) = allocator->arena(); *static_cast(ret + sizeof(Arena *)) = sizeplus; return ret + kHeaderSize; } else { return operator new[](size); // this is the function above } } void operator delete [] (void * memory) { memory -= kHeaderSize; Arena * const arena = *static_cast(memory); size_t sizeplus = *static_cast(memory + sizeof(arena)); if (arena) { arena->SlowFree(memory, sizeplus); } else { ::operator delete (memory); } } template void * operator delete (void * memory, const int ign, T * allocator) { // This "placement delete" can only be called if the constructor // throws an exception. memory -= kHeaderSize; size_t sizeplus = *static_cast(memory + sizeof(Arena *)); if (allocator) { allocator->Free(memory, 1 + size); } else { operator delete (memory); } } protected: static const int kMinSize = sizeof size_t + sizeof(Arena *); static const int kHeaderSize = kMinSize > BaseArena::kDefaultAlignment ? 2 * BaseArena::kDefaultAlignment : BaseArena::kDefaultAlignment; }; #endif // ********** example } #endif // BASE_ARENA_INL_H_ ctemplate-ctemplate-2.4/src/base/arena.cc000066400000000000000000000511221363122316600204130ustar00rootroot00000000000000// Copyright (c) 2000, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // Reorganized by Craig Silverstein // "Handles" by Ilan Horn // // This approach to arenas overcomes many of the limitations described // in the "Specialized allocators" section of // http://www.pdos.lcs.mit.edu/~dm/c++-new.html // // A somewhat similar approach to Gladiator, but for heap-detection, was // suggested by Ron van der Wal and Scott Meyers at // http://www.aristeia.com/BookErrata/M27Comments_frames.html #include #include "base/arena.h" #include "base/arena-inl.h" #include #include #ifdef HAVE_UNISTD_H # include #endif #include #include // one place uintptr_t might be #ifdef HAVE_INTTYPES_H # include #endif // another place uintptr_t might be #ifdef HAVE_UNISTD_H # include #endif // last place uintptr_t might be #include "base/mutex.h" #include "base/util.h" // for DCHECK_* using std::min; using std::vector; // TODO(csilvers): add in a portable implementation of aligned_malloc static void* aligned_malloc(size_t size, size_t alignment) { LOG(FATAL) << "page_aligned_ not currently supported\n"; } // The value here doesn't matter until page_aligned_ is supported. static const int kPageSize = 8192; // should be getpagesize() namespace ctemplate { // We used to only keep track of how much space has been allocated in // debug mode. Now we track this for optimized builds, as well. If you // want to play with the old scheme to see if this helps performance, // change this ARENASET() macro to a NOP. However, NOTE: some // applications of arenas depend on this space information (exported // via bytes_allocated()). #define ARENASET(x) (x) // ---------------------------------------------------------------------- // BaseArena::BaseArena() // BaseArena::~BaseArena() // Destroying the arena automatically calls Reset() // ---------------------------------------------------------------------- BaseArena::BaseArena(char* first, const size_t block_size, bool align_to_page) : remaining_(0), first_block_we_own_(first ? 1 : 0), block_size_(block_size), freestart_(NULL), // set for real in Reset() last_alloc_(NULL), blocks_alloced_(1), overflow_blocks_(NULL), page_aligned_(align_to_page), handle_alignment_(1), handle_alignment_bits_(0), block_size_bits_(0) { assert(block_size > kDefaultAlignment); while ((static_cast(1) << block_size_bits_) < block_size_) { ++block_size_bits_; } if (page_aligned_) { // kPageSize must be power of 2, so make sure of this. CHECK(kPageSize > 0 && 0 == (kPageSize & (kPageSize - 1))) << "kPageSize[ " << kPageSize << "] is not " << "correctly initialized: not a power of 2."; } if (first) { CHECK(!page_aligned_ || (reinterpret_cast(first) & (kPageSize - 1)) == 0); first_blocks_[0].mem = first; } else { if (page_aligned_) { // Make sure the blocksize is page multiple, as we need to end on a page // boundary. CHECK_EQ(block_size & (kPageSize - 1), 0) << "block_size is not a" << "multiple of kPageSize"; first_blocks_[0].mem = reinterpret_cast(aligned_malloc(block_size_, kPageSize)); PCHECK(NULL != first_blocks_[0].mem); } else { first_blocks_[0].mem = reinterpret_cast(malloc(block_size_)); } } first_blocks_[0].size = block_size_; Reset(); } BaseArena::~BaseArena() { FreeBlocks(); assert(overflow_blocks_ == NULL); // FreeBlocks() should do that // The first X blocks stay allocated always by default. Delete them now. for ( int i = first_block_we_own_; i < blocks_alloced_; ++i ) free(first_blocks_[i].mem); } // ---------------------------------------------------------------------- // BaseArena::block_count() // Only reason this is in .cc file is because it involves STL. // ---------------------------------------------------------------------- int BaseArena::block_count() const { return (blocks_alloced_ + (overflow_blocks_ ? static_cast(overflow_blocks_->size()) : 0)); } // ---------------------------------------------------------------------- // BaseArena::Reset() // Clears all the memory an arena is using. // ---------------------------------------------------------------------- void BaseArena::Reset() { FreeBlocks(); freestart_ = first_blocks_[0].mem; remaining_ = first_blocks_[0].size; last_alloc_ = NULL; ARENASET(status_.bytes_allocated_ = block_size_); // We do not know for sure whether or not the first block is aligned, // so we fix that right now. const int overage = reinterpret_cast(freestart_) & (kDefaultAlignment-1); if (overage > 0) { const int waste = kDefaultAlignment - overage; freestart_ += waste; remaining_ -= waste; } freestart_when_empty_ = freestart_; assert(!(reinterpret_cast(freestart_)&(kDefaultAlignment-1))); } // ---------------------------------------------------------------------- // BaseArena::MakeNewBlock() // Our sbrk() equivalent. We always make blocks of the same size // (though GetMemory() can also make a new block for really big // data. // ---------------------------------------------------------------------- void BaseArena::MakeNewBlock() { AllocatedBlock *block = AllocNewBlock(block_size_); freestart_ = block->mem; remaining_ = block->size; } // ------------------------------------------------------------- // BaseArena::AllocNewBlock() // Adds and returns an AllocatedBlock. // The returned AllocatedBlock* is valid until the next call // to AllocNewBlock or Reset. (i.e. anything that might // affect overflow_blocks_). // ------------------------------------------------------------- BaseArena::AllocatedBlock* BaseArena::AllocNewBlock(const size_t block_size) { AllocatedBlock *block; // Find the next block. if ( blocks_alloced_ < ARRAYSIZE(first_blocks_) ) { // Use one of the pre-allocated blocks block = &first_blocks_[blocks_alloced_++]; } else { // oops, out of space, move to the vector if (overflow_blocks_ == NULL) overflow_blocks_ = new vector; // Adds another block to the vector. overflow_blocks_->resize(overflow_blocks_->size()+1); // block points to the last block of the vector. block = &overflow_blocks_->back(); } if (page_aligned_) { // We need the size to be multiple of kPageSize to mprotect it later. size_t num_pages = ((block_size - 1) / kPageSize) + 1; size_t new_block_size = num_pages * kPageSize; block->mem = reinterpret_cast(aligned_malloc(new_block_size, kPageSize)); PCHECK(NULL != block->mem); block->size = new_block_size; } else { block->mem = reinterpret_cast(malloc(block_size)); block->size = block_size; } ARENASET(status_.bytes_allocated_ += block_size); return block; } // ---------------------------------------------------------------------- // BaseArena::IndexToBlock() // Index encoding is as follows: // For blocks in the first_blocks_ array, we use index of the block in // the array. // For blocks in the overflow_blocks_ vector, we use the index of the // block in iverflow_blocks_, plus the size of the first_blocks_ array. // ---------------------------------------------------------------------- const BaseArena::AllocatedBlock *BaseArena::IndexToBlock(int index) const { if (index < ARRAYSIZE(first_blocks_)) { return &first_blocks_[index]; } CHECK(overflow_blocks_ != NULL); int index_in_overflow_blocks = index - ARRAYSIZE(first_blocks_); CHECK_GE(index_in_overflow_blocks, 0); CHECK_LT(static_cast(index_in_overflow_blocks), overflow_blocks_->size()); return &(*overflow_blocks_)[index_in_overflow_blocks]; } // ---------------------------------------------------------------------- // BaseArena::GetMemoryFallback() // We take memory out of our pool, aligned on the byte boundary // requested. If we don't have space in our current pool, we // allocate a new block (wasting the remaining space in the // current block) and give you that. If your memory needs are // too big for a single block, we make a special your-memory-only // allocation -- this is equivalent to not using the arena at all. // ---------------------------------------------------------------------- void* BaseArena::GetMemoryFallback(const size_t size, const int align_as_int) { if (0 == size) { return NULL; // stl/stl_alloc.h says this is okay } // This makes the type-checker happy. const size_t align = static_cast(align_as_int); assert(align_as_int > 0 && 0 == (align & (align - 1))); // must be power of 2 // If the object is more than a quarter of the block size, allocate // it separately to avoid wasting too much space in leftover bytes if (block_size_ == 0 || size > block_size_/4) { // then it gets its own block in the arena assert(align <= kDefaultAlignment); // because that's what new gives us // This block stays separate from the rest of the world; in particular // we don't update last_alloc_ so you can't reclaim space on this block. return AllocNewBlock(size)->mem; } const size_t overage = (reinterpret_cast(freestart_) & (align-1)); if (overage) { const size_t waste = align - overage; freestart_ += waste; if (waste < remaining_) { remaining_ -= waste; } else { remaining_ = 0; } } if (size > remaining_) { MakeNewBlock(); } remaining_ -= size; last_alloc_ = freestart_; freestart_ += size; assert(0 == (reinterpret_cast(last_alloc_) & (align-1))); return reinterpret_cast(last_alloc_); } // ---------------------------------------------------------------------- // BaseArena::ReturnMemoryFallback() // BaseArena::FreeBlocks() // Unlike GetMemory(), which does actual work, ReturnMemory() is a // no-op: we don't "free" memory until Reset() is called. We do // update some stats, though. Note we do no checking that the // pointer you pass in was actually allocated by us, or that it // was allocated for the size you say, so be careful here! // FreeBlocks() does the work for Reset(), actually freeing all // memory allocated in one fell swoop. // ---------------------------------------------------------------------- void BaseArena::FreeBlocks() { for ( int i = 1; i < blocks_alloced_; ++i ) { // keep first block alloced free(first_blocks_[i].mem); first_blocks_[i].mem = NULL; first_blocks_[i].size = 0; } blocks_alloced_ = 1; if (overflow_blocks_ != NULL) { vector::iterator it; for (it = overflow_blocks_->begin(); it != overflow_blocks_->end(); ++it) { free(it->mem); } delete overflow_blocks_; // These should be used very rarely overflow_blocks_ = NULL; } } // ---------------------------------------------------------------------- // BaseArena::AdjustLastAlloc() // If you realize you didn't want your last alloc to be for // the size you asked, after all, you can fix it by calling // this. We'll grow or shrink the last-alloc region if we // can (we can always shrink, but we might not be able to // grow if you want to grow too big. // RETURNS true if we successfully modified the last-alloc // region, false if the pointer you passed in wasn't actually // the last alloc or if you tried to grow bigger than we could. // ---------------------------------------------------------------------- bool BaseArena::AdjustLastAlloc(void *last_alloc, const size_t newsize) { // It's only legal to call this on the last thing you alloced. if (last_alloc == NULL || last_alloc != last_alloc_) return false; // last_alloc_ should never point into a "big" block, w/ size >= block_size_ assert(freestart_ >= last_alloc_ && freestart_ <= last_alloc_ + block_size_); assert(remaining_ >= 0); // should be: it's a size_t! if (newsize > (freestart_ - last_alloc_) + remaining_) return false; // not enough room, even after we get back last_alloc_ space const char* old_freestart = freestart_; // where last alloc used to end freestart_ = last_alloc_ + newsize; // where last alloc ends now remaining_ -= (freestart_ - old_freestart); // how much new space we've taken return true; } // ---------------------------------------------------------------------- // BaseArena::GetMemoryWithHandle() // First, memory is allocated using GetMemory, using handle_alignment_. // Since using different alignments for different handles would make // the handles incompatible (e.g., we could end up with the same handle // value referencing two different allocations, the alignment is not passed // as an argument to GetMemoryWithHandle, and handle_alignment_ is used // automatically for all GetMemoryWithHandle calls. // Then we go about building a handle to reference the allocated memory. // The block index used for the allocation, along with the offset inside // the block, are encoded into the handle as follows: // (block_index*block_size)+offset // offset is simply the difference between the pointer returned by // GetMemory and the starting pointer of the block. // The above value is then divided by the alignment. As we know that // both offset and the block_size are divisable by the alignment (this is // enforced by set_handle_alignment() for block_size, and by GetMemory() // for the offset), this does not lose any information, but allows to cram // more into the limited space in handle. // If the result does not fit into an unsigned 32-bit integer, we // have run out of space that the handle can represent, and return // an invalid handle. Note that the returned pointer is still usable, // but this allocation cannot be referenced by a handle. // ---------------------------------------------------------------------- void* BaseArena::GetMemoryWithHandle( const size_t size, BaseArena::Handle* handle) { CHECK(handle != NULL); // For efficiency, handles are always allocated aligned to a power of 2. void* p = GetMemory(size, (1 << handle_alignment_bits_)); // Find the index of the block the memory was allocated from. In most // cases, this will be the last block, so the following loop will // iterate exactly once. int block_index; const AllocatedBlock* block = NULL; for (block_index = block_count() - 1; block_index >= 0; --block_index) { block = IndexToBlock(block_index); if ((p >= block->mem) && (p < (block->mem + block->size))) { break; } } CHECK_GE(block_index, 0) << "Failed to find block that was allocated from"; CHECK(block != NULL) << "Failed to find block that was allocated from"; const uint64_t offset = reinterpret_cast(p) - block->mem; DCHECK_LT(offset, block_size_); DCHECK((offset & ((1 << handle_alignment_bits_) - 1)) == 0); DCHECK((block_size_ & ((1 << handle_alignment_bits_) - 1)) == 0); uint64_t handle_value = ((static_cast(block_index) << block_size_bits_) + offset) >> handle_alignment_bits_; if (handle_value >= static_cast(0xFFFFFFFF)) { // We ran out of space to be able to return a handle, so return an invalid // handle. handle_value = Handle::kInvalidValue; } handle->handle_ = static_cast(handle_value); return p; } // ---------------------------------------------------------------------- // BaseArena::set_handle_alignment() // Set the alignment to be used when Handles are requested. This can only // be set for an arena that is empty - it cannot be changed on the fly. // The alignment must be a power of 2 that the block size is divisable by. // The default alignment is 1. // Trying to set an alignment that does not meet the above constraints will // cause a CHECK-failure. // ---------------------------------------------------------------------- void BaseArena::set_handle_alignment(int align) { CHECK(align > 0 && 0 == (align & (align - 1))); // must be power of 2 CHECK(static_cast(align) < block_size_); CHECK((block_size_ % align) == 0); CHECK(is_empty()); handle_alignment_ = align; handle_alignment_bits_ = 0; while ((1 << handle_alignment_bits_) < handle_alignment_) { ++handle_alignment_bits_; } } // ---------------------------------------------------------------------- // BaseArena::HandleToPointer() // First, the handle value needs to gain back the alignment factor that // was divided out of it by GetMemoryWithHandle. Once this is done, it // becomes trivial to extract the block index and offset in the block out // of it, and calculate the pointer. // ---------------------------------------------------------------------- void* BaseArena::HandleToPointer(const Handle& h) const { CHECK(h.valid()); uint64_t handle = static_cast(h.handle_) << handle_alignment_bits_; int block_index = static_cast(handle >> block_size_bits_); size_t block_offset = static_cast(handle & ((1 << block_size_bits_) - 1)); const AllocatedBlock* block = IndexToBlock(block_index); CHECK(block != NULL); return reinterpret_cast(block->mem + block_offset); } // ---------------------------------------------------------------------- // UnsafeArena::Realloc() // SafeArena::Realloc() // If you decide you want to grow -- or shrink -- a memory region, // we'll do it for you here. Typically this will involve copying // the existing memory to somewhere else on the arena that has // more space reserved. But if you're reallocing the last-allocated // block, we may be able to accomodate you just by updating a // pointer. In any case, we return a pointer to the new memory // location, which may be the same as the pointer you passed in. // Here's an example of how you might use Realloc(): // // compr_buf = arena->Alloc(uncompr_size); // get too-much space // int compr_size; // zlib.Compress(uncompr_buf, uncompr_size, compr_buf, &compr_size); // compr_buf = arena->Realloc(compr_buf, uncompr_size, compr_size); // ---------------------------------------------------------------------- char* UnsafeArena::Realloc(char* s, size_t oldsize, size_t newsize) { assert(oldsize >= 0 && newsize >= 0); if ( AdjustLastAlloc(s, newsize) ) // in case s was last alloc return s; if ( newsize <= oldsize ) { return s; // no need to do anything; we're ain't reclaiming any memory! } char * newstr = Alloc(newsize); memcpy(newstr, s, min(oldsize, newsize)); return newstr; } char* SafeArena::Realloc(char* s, size_t oldsize, size_t newsize) { assert(oldsize >= 0 && newsize >= 0); { MutexLock lock(&mutex_); if ( AdjustLastAlloc(s, newsize) ) // in case s was last alloc return s; } if ( newsize <= oldsize ) { return s; // no need to do anything; we're ain't reclaiming any memory! } char * newstr = Alloc(newsize); memcpy(newstr, s, min(oldsize, newsize)); return newstr; } } ctemplate-ctemplate-2.4/src/base/arena.h000066400000000000000000000731611363122316600202640ustar00rootroot00000000000000// Copyright (c) 2000, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // Reorganized by Craig Silverstein // "Handles" by Ilan Horn // // Sometimes it is necessary to allocate a large number of small // objects. Doing this the usual way (malloc, new) is slow, // especially for multithreaded programs. A BaseArena provides a // mark/release method of memory management: it asks for a large chunk // from the operating system and doles it out bit by bit as required. // Then you free all the memory at once by calling BaseArena::Reset(). // // Use SafeArena for multi-threaded programs where multiple threads // could access the same arena at once. Use UnsafeArena otherwise. // Usually you'll want UnsafeArena. // // There are four ways to use the arena. arena.h and arena.cc are // sufficient for the MALLOC and STRINGS uses. For NEW and STL you'll // also need to include arena-inl.h in the appropriate .cc file. // However, we do *declare* (but not define) the template types here. // // LIKE MALLOC: --Uses UnsafeArena (or SafeArena)-- // This is the simplest way. Just create an arena, and whenever you // need a block of memory to put something in, call BaseArena::Alloc(). eg // s = arena.Alloc(100); // snprintf(s, 100, "%s:%d", host, port); // arena.Shrink(strlen(s)+1); // optional; see below for use // // You'll probably use the convenience routines more often: // s = arena.Strdup(host); // a copy of host lives in the arena // s = arena.Strndup(host, 100); // we guarantee to NUL-terminate! // s = arena.Memdup(protobuf, sizeof(protobuf); // // If you go the Alloc() route, you'll probably allocate too-much-space. // You can reclaim the extra space by calling Shrink() before the next // Alloc() (or Strdup(), or whatever), with the #bytes you actually used. // If you use this method, memory management is easy: just call Alloc() // and friends a lot, and call Reset() when you're done with the data. // // FOR STRINGS: --Uses UnsafeArena (or SafeArena)-- // This is a special case of STL (below), but is simpler. Use an // astring, which acts like a string but allocates from the passed-in // arena: // astring s(arena); // or "sastring" to use a SafeArena // s.assign(host); // astring s2(host, hostlen, arena); // // WITH NEW: --Uses BaseArena, Gladiator (or ArenaOnlyGladiator)-- // Use this to allocate a C++ class object (or any other object you // have to get via new/delete rather than malloc/free). // There are several things you have to do in this case: // 1) Your class (the one you new) must inherit from Gladiator. // 2) To actually allocate this class on the arena, use // myclass = new (AllocateInArena, arena) MyClass(constructor, args) // // Note that MyClass doesn't need to have the arena passed in. // But if it, in turn, wants to call "new" on some of its member // variables, and you want those member vars to be on the arena // too, you better pass in an arena so it can call new(0,arena). // // If you can guarantee that everyone who ever calls new on // MyClass uses the new(0,arena) form (ie nobody ever just says // new), you can have MyClass subclass from ArenaOnlyGladiator // rather than from Gladiator. ArenaOnlyGladiator is a bit more // efficient (faster and smaller), but is otherwise identical. // // If you allocate myclass using new(0,arena), and MyClass only // does memory management in the destructor, it's not necessary // to even call "delete myclass;", you can just call arena.Reset(); // If the destructor does something else (closes a file, logs // a message, whatever), you'll have to call destructor and Reset() // both: "delete myclass; arena.Reset();" // // Note that you can not allocate an array of classes this way: // noway = new (AllocateInArena, arena) MyClass[5]; // not supported! // It's not difficult to program, we just haven't done it. Arrays // are typically big and so there's little point to arena-izing them. // // WITH NEW: --Uses UnsafeArena-- // There are cases where you can't inherit the class from Gladiator, // or inheriting would be too expensive. Examples of this include // plain-old-data (allocated using new) and third-party classes (such // as STL containers). arena-inl.h provides a global operator new // that can be used as follows: // // #include "base/arena-inl.h" // // UnsafeArena arena(1000); // Foo* foo = new (AllocateInArena, &arena) Foo; // Foo* foo_array = new (AllocateInArena, &arena) Foo[10]; // // IN STL: --Uses BaseArena, ArenaAllocator-- // All STL containers (vector, hash_map, etc) take an allocator. // You can use the arena as an allocator. Then whenever the vector // (or whatever) wants to allocate memory, it will take it from the // arena. To use, you just indicate in the type that you want to use the // arena, and then actually give a pointer to the arena as the last // constructor arg: // vector > v(&arena); // v.push_back(3); // // WARNING: Careless use of STL within an arena-allocated object can // result in memory leaks if you rely on arena.Reset() to free // memory and do not call the object destructor. This is actually // a subclass of a more general hazard: If an arena-allocated // object creates (and owns) objects that are not also // arena-allocated, then the creating object must have a // destructor that deletes them, or they will not be deleted. // However, since the outer object is arena allocated, it's easy to // forget to call delete on it, and needing to do so may seem to // negate much of the benefit of arena allocation. A specific // example is use of vector in an arena-allocated object, // since type string is not atomic and is always allocated by the // default runtime allocator. The arena definition provided here // allows for much flexibility, but you ought to carefully consider // before defining arena-allocated objects which in turn create // non-arena allocated objects. // // WITH HANDLES: // The various arena classes can supply compact handles to data kept // in the arena. These handles consume only 4 bytes each, and are thus // more efficient than pointers - this may be interesting in cases // where a very large number of references to memory in the arena need // to be kept. // Note that handles are limited in the amount of data that can be reference // in the arena, typically to 4GB*the number given to set_handle_alignment() // (which defaults to 1). The number of allocations that can have handles // is, of course, smaller than 4G (that's what's representable by 32 bits). // It does depend on their sizes, however. In a worst-case scenario each // allocation consumes a page of its own, and we will run out of handles // after approximately (4G/block_size)*handle_alignment allocations. // When we run out of handles or allocate data over the amount of memory // that handles can reference, an invalid handle will be returned (but // the requested memory will still be allocated in the arena). // Handles memory use is most efficient when the arena block size is a power // of two. When this is not the case, we can run out of handles when at // most half of the addressable space (as described above) is not in use. // At worst handles can reference at least 2GB*handle_alignment. // Example use: // UnsafeArena arena(16384); // arena.set_handle_alignment(4); // // Assume you want to keep the string s in the arena. // Handle h = arena.MemdupWithHandle(s.c_str(), s.length()); // // Later, to get the memory from the handle, use: // void* p = arena.HandleToPointer(h); // // Note that there's no way to retrieve the size from the handle. // // It probably makes sense to encode the size into the buffer saved, // // unless the size is known/fixed. // Internal machinery of handles: // The handle consists of the block index in the arena and the offset // inside the block, encoded into a single unsigned uint32 value. // Note that, the rightmost alignment bits (controlled by // set_handle_alignment()) are shaved off the saved offset in the Handle, // to give some extra capacity :) // set_handle_alignment() can only be called when the arena is empty, // as changing it invalidates any handles that are still in flight. // // // PUTTING IT ALL TOGETHER // Here's a program that uses all of the above. Note almost all the // examples are the various ways to use "new" and STL. Using the // malloc-like features and the string type are much easier! // // Class A : public Gladiator { // public: // int i; // vector v1; // vector >* v3; // vector >* v4; // vector* v5; // vector vs; // vector va; // char *s; // A() : v1(), v3(NULL), v4(NULL), vs(), va(), s(NULL) { // // v1 is allocated on the arena whenever A is. Its ints never are. // v5 = new vector; // // v5 is not allocated on the arena, and neither are any of its ints. // } // ~A() { // delete v5; // needed since v5 wasn't allocated on the arena // printf("I'm done!\n"); // } // }; // // class B : public A { // we inherit from Gladiator, but indirectly // public: // UnsafeArena* arena_; // vector > v2; // vector va1; // vector > va2; // vector* pva; // vector >* pva2; // astring a; // // B(UnsafeArena * arena) // : arena_(arena), v2(arena_), va1(), va2(arena_), a("initval", arena_) { // v3 = new vector >(arena_); // v4 = new (AllocateInArena, arena_) vector >(arena_); // v5 = new (AllocateInArena, arena_) vector; // // v2 is allocated on the arena whenever B is. Its ints always are. // // v3 is not allocated on the arena, but the ints you give it are // // v4 is allocated on the arena, and so are the ints you give it // // v5 is allocated on the arena, but the ints you give it are not // // va1 is allocated on the arena whenever B is. No A ever is. // // va2 is allocated on the arena whenever B is. Its A's always are. // pva = new (AllocateInArena, arena_) vector; // pva2 = new (AllocateInArena, arena_) vector >(arena_); // // pva is allocated on the arena, but its A's are not // // pva2 is allocated on the arena, and so are its A's. // // a's value "initval" is stored on the arena. If we reassign a, // // the new value will be stored on the arena too. // } // ~B() { // delete v3; // necessary to free v3's memory, though not its ints' // // don't need to delete v4: arena_.Reset() will do as good // delete v5; // necessary to free v5's ints memory, though not v5 itself // delete pva; // necessary to make sure you reclaim space used by A's // delete pva2; // safe to call this; needed if you want to see the printfs // // pva2->clear() -- not necessary, arena_.Reset() will do just as good // } // }; // // main() { // UnsafeArena arena(1000); // A a1; // a1 is not on the arena // a1.vs.push_back(string("hello")); // hello is not copied onto the arena // a1.va.push_back(astring("hello", &arena)); // hello is on the arena, // // astring container isn't // a1.s = arena.Strdup("hello"); // hello is on the arena // // A* a2 = new (AllocateInArena, arena) A; // a2 is on the arena // a2.vs.push_back(string("hello")); // hello is *still* not on the arena // a2.s = arena.Strdup("world"); // world is on the arena. a1.s is ok // // B b1(&arena); // B is not allocated on the arena // b1.a.assign("hello"); // hello is on the arena // b1.pva2.push_back(a1); // our copy of a1 will be stored on // // the arena, though a1 itself wasn't // arena.Reset(); // all done with our memory! // } #ifndef BASE_ARENA_H_ #define BASE_ARENA_H_ #include #include "base/mutex.h" // must go first to get _XOPEN_SOURCE #include #include #include #include #include "base/thread_annotations.h" #include "base/util.h" // for CHECK, etc namespace ctemplate { // Annoying stuff for windows -- make sure clients (in this case // unittests) can import the class definitions and variables. #ifndef CTEMPLATE_DLL_DECL # ifdef _MSC_VER # define CTEMPLATE_DLL_DECL __declspec(dllimport) # else # define CTEMPLATE_DLL_DECL /* should be the empty string for non-windows */ # endif #endif // This class is "thread-compatible": different threads can access the // arena at the same time without locking, as long as they use only // const methods. class CTEMPLATE_DLL_DECL BaseArena { protected: // You can't make an arena directly; only a subclass of one BaseArena(char* first_block, const size_t block_size, bool align_to_page); public: virtual ~BaseArena(); virtual void Reset(); // A handle to a pointer in an arena. An opaque type, with default // copy and assignment semantics. class Handle { public: static const uint32_t kInvalidValue = 0xFFFFFFFF; // int32-max Handle() : handle_(kInvalidValue) { } // Default copy constructors are fine here. bool operator==(const Handle& h) const { return handle_ == h.handle_; } bool operator!=(const Handle& h) const { return handle_ != h.handle_; } uint32_t hash() const { return handle_; } bool valid() const { return handle_ != kInvalidValue; } private: // Arena needs to be able to access the internal data. friend class BaseArena; explicit Handle(uint32_t handle) : handle_(handle) { } uint32_t handle_; }; // they're "slow" only 'cause they're virtual (subclasses define "fast" ones) virtual char* SlowAlloc(size_t size) = 0; virtual void SlowFree(void* memory, size_t size) = 0; virtual char* SlowRealloc(char* memory, size_t old_size, size_t new_size) = 0; virtual char* SlowAllocWithHandle(const size_t size, Handle* handle) = 0; // Set the alignment to be used when Handles are requested. This can only // be set for an arena that is empty - it cannot be changed on the fly. // The alignment must be a power of 2 that the block size is divisable by. // The default alignment is 1. // Trying to set an alignment that does not meet the above constraints will // cause a CHECK-failure. void set_handle_alignment(int align); // Retrieve the memory pointer that the supplied handle refers to. // Calling this with an invalid handle will CHECK-fail. void* HandleToPointer(const Handle& h) const; class Status { private: friend class BaseArena; size_t bytes_allocated_; public: Status() : bytes_allocated_(0) { } size_t bytes_allocated() const { return bytes_allocated_; } }; // Accessors and stats counters // This accessor isn't so useful here, but is included so we can be // type-compatible with ArenaAllocator (in arena-inl.h). That is, // we define arena() because ArenaAllocator does, and that way you // can template on either of these and know it's safe to call arena(). virtual BaseArena* arena() { return this; } size_t block_size() const { return block_size_; } int block_count() const; bool is_empty() const { // must check block count in case we allocated a block larger than blksize return freestart_ == freestart_when_empty_ && 1 == block_count(); } // This should be the worst-case alignment for any type. This is // good for IA-32, SPARC version 7 (the last one I know), and // supposedly Alpha. i386 would be more time-efficient with a // default alignment of 8, but ::operator new() uses alignment of 4, // and an assertion will fail below after the call to MakeNewBlock() // if you try to use a larger alignment. #ifdef __i386__ static const int kDefaultAlignment = 4; #else static const int kDefaultAlignment = 8; #endif protected: void MakeNewBlock(); void* GetMemoryFallback(const size_t size, const int align); void* GetMemory(const size_t size, const int align) { assert(remaining_ <= block_size_); // an invariant if ( size > 0 && size < remaining_ && align == 1 ) { // common case last_alloc_ = freestart_; freestart_ += size; remaining_ -= size; return reinterpret_cast(last_alloc_); } return GetMemoryFallback(size, align); } // This doesn't actually free any memory except for the last piece allocated void ReturnMemory(void* memory, const size_t size) { if ( memory == last_alloc_ && size == freestart_ - last_alloc_ ) { remaining_ += size; freestart_ = last_alloc_; } } // This is used by Realloc() -- usually we Realloc just by copying to a // bigger space, but for the last alloc we can realloc by growing the region. bool AdjustLastAlloc(void* last_alloc, const size_t newsize); // Since using different alignments for different handles would make // the handles incompatible (e.g., we could end up with the same handle // value referencing two different allocations, the alignment is not passed // as an argument to GetMemoryWithHandle, and handle_alignment_ is used // automatically for all GetMemoryWithHandle calls. void* GetMemoryWithHandle(const size_t size, Handle* handle); Status status_; size_t remaining_; private: struct AllocatedBlock { char *mem; size_t size; }; // The returned AllocatedBlock* is valid until the next call to AllocNewBlock // or Reset (i.e. anything that might affect overflow_blocks_). AllocatedBlock *AllocNewBlock(const size_t block_size); const AllocatedBlock *IndexToBlock(int index) const; const int first_block_we_own_; // 1 if they pass in 1st block, 0 else const size_t block_size_; char* freestart_; // beginning of the free space in most recent block char* freestart_when_empty_; // beginning of the free space when we're empty char* last_alloc_; // used to make sure ReturnBytes() is safe // STL vector isn't as efficient as it could be, so we use an array at first int blocks_alloced_; // how many of the first_blocks_ have been alloced AllocatedBlock first_blocks_[16]; // the length of this array is arbitrary // if the first_blocks_ aren't enough, expand into overflow_blocks_. std::vector* overflow_blocks_; const bool page_aligned_; // when true, all blocks need to be page aligned int handle_alignment_; // Alignment to be used when Handles are requested. int handle_alignment_bits_; // log2(handle_alignment_). // The amount of bits required to keep block_size_ (ceil(log2(block_size_))). size_t block_size_bits_; void FreeBlocks(); // Frees all except first block // This subclass needs to alter permissions for all allocated blocks. friend class ProtectableUnsafeArena; DISALLOW_COPY_AND_ASSIGN(BaseArena); }; class CTEMPLATE_DLL_DECL UnsafeArena : public BaseArena { public: // Allocates a thread-compatible arena with the specified block size. explicit UnsafeArena(const size_t block_size) : BaseArena(NULL, block_size, false) { } UnsafeArena(const size_t block_size, bool align) : BaseArena(NULL, block_size, align) { } // Allocates a thread-compatible arena with the specified block // size. "first_block" must have size "block_size". Memory is // allocated from "first_block" until it is exhausted; after that // memory is allocated by allocating new blocks from the heap. UnsafeArena(char* first_block, const size_t block_size) : BaseArena(first_block, block_size, false) { } UnsafeArena(char* first_block, const size_t block_size, bool align) : BaseArena(first_block, block_size, align) { } char* Alloc(const size_t size) { return reinterpret_cast(GetMemory(size, 1)); } void* AllocAligned(const size_t size, const int align) { return GetMemory(size, align); } char* Calloc(const size_t size) { void* return_value = Alloc(size); memset(return_value, 0, size); return reinterpret_cast(return_value); } void* CallocAligned(const size_t size, const int align) { void* return_value = AllocAligned(size, align); memset(return_value, 0, size); return return_value; } // Free does nothing except for the last piece allocated. void Free(void* memory, size_t size) { ReturnMemory(memory, size); } typedef BaseArena::Handle Handle; char* AllocWithHandle(const size_t size, Handle* handle) { return reinterpret_cast(GetMemoryWithHandle(size, handle)); } virtual char* SlowAlloc(size_t size) { // "slow" 'cause it's virtual return Alloc(size); } virtual void SlowFree(void* memory, size_t size) { // "slow" 'cause it's virt Free(memory, size); } virtual char* SlowRealloc(char* memory, size_t old_size, size_t new_size) { return Realloc(memory, old_size, new_size); } virtual char* SlowAllocWithHandle(const size_t size, Handle* handle) { return AllocWithHandle(size, handle); } char* Memdup(const char* s, size_t bytes) { char* newstr = Alloc(bytes); memcpy(newstr, s, bytes); return newstr; } char* MemdupPlusNUL(const char* s, size_t bytes) { // like "string(s, len)" char* newstr = Alloc(bytes+1); memcpy(newstr, s, bytes); newstr[bytes] = '\0'; return newstr; } Handle MemdupWithHandle(const char* s, size_t bytes) { Handle handle; char* newstr = AllocWithHandle(bytes, &handle); memcpy(newstr, s, bytes); return handle; } char* Strdup(const char* s) { return Memdup(s, strlen(s) + 1); } // Unlike libc's strncpy, I always NUL-terminate. libc's semantics are dumb. // This will allocate at most n+1 bytes (+1 is for the NULL terminator). char* Strndup(const char* s, size_t n) { // Use memchr so we don't walk past n. // We can't use the one in //strings since this is the base library, // so we have to reinterpret_cast from the libc void *. const char* eos = reinterpret_cast(memchr(s, '\0', n)); // if no null terminator found, use full n const size_t bytes = (eos == NULL) ? n + 1 : eos - s + 1; char* ret = Memdup(s, bytes); ret[bytes-1] = '\0'; // make sure the string is NUL-terminated return ret; } // You can realloc a previously-allocated string either bigger or smaller. // We can be more efficient if you realloc a string right after you allocate // it (eg allocate way-too-much space, fill it, realloc to just-big-enough) char* Realloc(char* s, size_t oldsize, size_t newsize); // If you know the new size is smaller (or equal), you don't need to know // oldsize. We don't check that newsize is smaller, so you'd better be sure! char* Shrink(char* s, size_t newsize) { AdjustLastAlloc(s, newsize); // reclaim space if we can return s; // never need to move if we go smaller } // We make a copy so you can keep track of status at a given point in time Status status() const { return status_; } // Number of bytes remaining before the arena has to allocate another block. size_t bytes_until_next_allocation() const { return remaining_; } private: DISALLOW_COPY_AND_ASSIGN(UnsafeArena); }; // we inherit from BaseArena instead of UnsafeArena so that we don't need // virtual methods for allocation/deallocation. This means, however, // I have to copy the definitions of strdup, strndup, etc. :-( class CTEMPLATE_DLL_DECL SafeArena : public BaseArena { public: // Allocates a thread-safe arena with the specified block size. explicit SafeArena(const size_t block_size) : BaseArena(NULL, block_size, false) { } // Allocates a thread-safe arena with the specified block size. // "first_block" must have size "block_size". Memory is allocated // from "first_block" until it is exhausted; after that memory is // allocated by allocating new blocks from the heap. SafeArena(char* first_block, const size_t block_size) : BaseArena(first_block, block_size, false) { } virtual void Reset() LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); // in case two threads Reset() at same time BaseArena::Reset(); } char* Alloc(const size_t size) LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); return reinterpret_cast(GetMemory(size, 1)); } void* AllocAligned(const size_t size, const int align) LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); return GetMemory(size, align); } char* Calloc(const size_t size) { void* return_value = Alloc(size); memset(return_value, 0, size); return reinterpret_cast(return_value); } void* CallocAligned(const size_t size, const int align) { void* return_value = AllocAligned(size, align); memset(return_value, 0, size); return return_value; } // Free does nothing except for the last piece allocated. void Free(void* memory, size_t size) LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); ReturnMemory(memory, size); } typedef BaseArena::Handle Handle; char* AllocWithHandle(const size_t size, Handle* handle) LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); return reinterpret_cast(GetMemoryWithHandle(size, handle)); } virtual char* SlowAlloc(size_t size) { // "slow" 'cause it's virtual return Alloc(size); } virtual void SlowFree(void* memory, size_t size) { // "slow" 'cause it's virt Free(memory, size); } virtual char* SlowRealloc(char* memory, size_t old_size, size_t new_size) { return Realloc(memory, old_size, new_size); } virtual char* SlowAllocWithHandle(const size_t size, Handle* handle) { return AllocWithHandle(size, handle); } char* Memdup(const char* s, size_t bytes) { char* newstr = Alloc(bytes); memcpy(newstr, s, bytes); return newstr; } char* MemdupPlusNUL(const char* s, size_t bytes) { // like "string(s, len)" char* newstr = Alloc(bytes+1); memcpy(newstr, s, bytes); newstr[bytes] = '\0'; return newstr; } Handle MemdupWithHandle(const char* s, size_t bytes) { Handle handle; char* newstr = AllocWithHandle(bytes, &handle); memcpy(newstr, s, bytes); return handle; } char* Strdup(const char* s) { return Memdup(s, strlen(s) + 1); } // Unlike libc's strncpy, I always NUL-terminate. libc's semantics are dumb. // This will allocate at most n+1 bytes (+1 is for the NULL terminator). char* Strndup(const char* s, size_t n) { // Use memchr so we don't walk past n. // We can't use the one in //strings since this is the base library, // so we have to reinterpret_cast from the libc void *. const char* eos = reinterpret_cast(memchr(s, '\0', n)); // if no null terminator found, use full n const size_t bytes = (eos == NULL) ? n + 1 : eos - s + 1; char* ret = Memdup(s, bytes); ret[bytes-1] = '\0'; // make sure the string is NUL-terminated return ret; } // You can realloc a previously-allocated string either bigger or smaller. // We can be more efficient if you realloc a string right after you allocate // it (eg allocate way-too-much space, fill it, realloc to just-big-enough) char* Realloc(char* s, size_t oldsize, size_t newsize) LOCKS_EXCLUDED(mutex_); // If you know the new size is smaller (or equal), you don't need to know // oldsize. We don't check that newsize is smaller, so you'd better be sure! char* Shrink(char* s, size_t newsize) LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); AdjustLastAlloc(s, newsize); // reclaim space if we can return s; // we never need to move if we go smaller } Status status() LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); return status_; } // Number of bytes remaining before the arena has to allocate another block. size_t bytes_until_next_allocation() LOCKS_EXCLUDED(mutex_) { MutexLock lock(&mutex_); return remaining_; } protected: Mutex mutex_; private: DISALLOW_COPY_AND_ASSIGN(SafeArena); }; } #endif // BASE_ARENA_H_ ctemplate-ctemplate-2.4/src/base/fileutil.h000066400000000000000000000063741363122316600210150ustar00rootroot00000000000000// Copyright (c) 2011, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // A tiny wrapper around struct stat and FILE*. #ifndef TEMPLATE_OPENSOURCE_FILEUTIL_H_ #define TEMPLATE_OPENSOURCE_FILEUTIL_H_ #include #include #include #include #ifdef HAVE_UNISTD_H # include #endif #include namespace ctemplate { class FileStat { public: time_t mtime; off_t length; bool IsDirectory() { return S_ISDIR(internal_statbuf.st_mode); } private: friend class File; struct stat internal_statbuf; }; class File { public: static bool Stat(const std::string& filename, FileStat* statbuf) { if (stat(filename.c_str(), &statbuf->internal_statbuf) != 0) return false; statbuf->mtime = statbuf->internal_statbuf.st_mtime; statbuf->length = statbuf->internal_statbuf.st_size; return true; } static bool Readable(const char* filename) { return access(filename, R_OK) == 0; } static File* Open(const char* filename, const char* mode) { char binary_mode[3]; const char* mode_to_use = mode; if ((mode[0] == 'r' || mode[0] == 'w') && mode[1] == '\0') { // We add a 'b' to make sure we do the right thing even on // Windows. On unix, this will be a noop. binary_mode[0] = mode[0]; binary_mode[1] = 'b'; binary_mode[2] = '\0'; mode_to_use = binary_mode; } FILE* fp = fopen(filename, mode_to_use); if (!fp) return NULL; return new File(fp); } size_t Read(char* buf, size_t size) { return fread(buf, 1, size, fp_); } void Close() { fclose(fp_); delete this; // naughty naughty! } private: explicit File(FILE* fp) : fp_(fp) { } FILE* fp_; }; } #endif // TEMPLATE_OPENSOURCE_FILEUTIL_H_ ctemplate-ctemplate-2.4/src/base/macros.h000066400000000000000000000063201363122316600204530ustar00rootroot00000000000000// Copyright 2011 Google Inc. All Rights Reserved. // Author: csilvers@google.com (Craig Silverstein) // // Provides macros and typedefs based on config.h settings. // Provides the following macros: // UNALIGNED_LOAD32 (may be an inline function on some architectures) #ifndef CTEMPLATE_MACROS_H_ #define CTEMPLATE_MACROS_H_ #include #include // This is all to figure out endian-ness and byte-swapping on various systems #if defined(HAVE_ENDIAN_H) #include // for the __BYTE_ORDER use below #elif defined(HAVE_SYS_ENDIAN_H) #include // location on FreeBSD #elif defined(HAVE_MACHINE_ENDIAN_H) #include // location on OS X #endif #if defined(HAVE_SYS_BYTEORDER_H) #include // BSWAP_32 on Solaris 10 #endif #ifdef HAVE_SYS_ISA_DEFS_H #include // _BIG_ENDIAN/_LITTLE_ENDIAN on Solaris 10 #endif // MurmurHash does a lot of 4-byte unaligned integer access. It // interprets these integers in little-endian order. This is perfect // on x86, for which this is a natural memory access; for other systems // we do what we can to make this as efficient as possible. #if defined(HAVE_BYTESWAP_H) # include // GNU (especially linux) # define BSWAP32(x) bswap_32(x) #elif defined(HAVE_LIBKERN_OSBYTEORDER_H) # include // OS X # define BSWAP32(x) OSSwapInt32(x) #elif defined(bswap32) // FreeBSD // FreeBSD defines bswap32 as a macro in sys/endian.h (already #included) # define BSWAP32(x) bswap32(x) #elif defined(BSWAP_32) // Solaris 10 // Solaris defines BSWSAP_32 as a macro in sys/byteorder.h (already #included) # define BSWAP32(x) BSWAP_32(x) #elif !defined(BSWAP32) # define BSWAP32(x) ((((x) & 0x000000ff) << 24) | \ (((x) & 0x0000ff00) << 8) | \ (((x) & 0x00ff0000) >> 8) | \ (((x) & 0xff000000) >> 24)); #else # define CTEMPLATE_BSWAP32_ALREADY_DEFINED #endif #if defined(__i386__) || defined(__x86_64__) || defined(_M_IX86) || defined(_M_X64) // We know they allow unaligned memory access and are little-endian # define UNALIGNED_LOAD32(_p) (*reinterpret_cast(_p)) #elif defined(__ppc__) || defined(__ppc64__) // We know they allow unaligned memory access and are big-endian # define UNALIGNED_LOAD32(_p) BSWAP32(*reinterpret_cast(_p)) #elif (BYTE_ORDER == 1234) || (_BYTE_ORDER == 1234) || defined(_LITTLE_ENDIAN) // Use memcpy to align the memory properly inline uint32_t UNALIGNED_LOAD32(const void *p) { uint32_t t; memcpy(&t, p, sizeof(t)); return t; } #elif (BYTE_ORDER == 4321) || (_BYTE_ORDER == 4321) || defined(_BIG_ENDIAN) inline uint32_t UNALIGNED_LOAD32(const void *p) { uint32_t t; memcpy(&t, p, sizeof(t)); return BSWAP32(t); } #else // Means we can't find find endian.h on this machine: # error Need to define UNALIGNED_LOAD32 for this architecture #endif #ifndef CTEMPLATE_BSWAP32_ALREADY_DEFINED # undef BSWAP32 // don't leak outside this file #else # undef CTEMPLATE_BSWAP32_ALREADY_DEFINED // just cleaning up #endif #endif // CTEMPLATE_MACROS_H_ ctemplate-ctemplate-2.4/src/base/manual_constructor.h000066400000000000000000000211721363122316600231130ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // Author: kenton@google.com (Kenton Varda) // // ManualConstructor statically-allocates space in which to store some // object, but does not initialize it. You can then call the constructor // and destructor for the object yourself as you see fit. This is useful // for memory management optimizations, where you want to initialize and // destroy an object multiple times but only allocate it once. // // (When I say ManualConstructor statically allocates space, I mean that // the ManualConstructor object itself is forced to be the right size.) // // For example usage, check out util/gtl/small_map.h. #ifndef UTIL_GTL_MANUAL_CONSTRUCTOR_H_ #define UTIL_GTL_MANUAL_CONSTRUCTOR_H_ #include namespace ctemplate { namespace util { namespace gtl { namespace internal { // // Provides a char array with the exact same alignment as another type. The // first parameter must be a complete type, the second parameter is how many // of that type to provide space for. // // UTIL_GTL_ALIGNED_CHAR_ARRAY(struct stat, 16) storage_; // // Because MSVC and older GCCs require that the argument to their alignment // construct to be a literal constant integer, we use a template instantiated // at all the possible powers of two. #ifndef SWIG template struct AlignType { }; template struct AlignType<0, size> { typedef char result[size]; }; #if defined(_MSC_VER) #define UTIL_GTL_ALIGN_ATTRIBUTE(X) __declspec(align(X)) #define UTIL_GTL_ALIGN_OF(T) __alignof(T) #elif defined(__GNUC__) || defined(__APPLE__) || defined(__INTEL_COMPILER) \ || defined(__nacl__) #define UTIL_GTL_ALIGN_ATTRIBUTE(X) __attribute__((aligned(X))) #define UTIL_GTL_ALIGN_OF(T) __alignof__(T) #endif #if defined(UTIL_GTL_ALIGN_ATTRIBUTE) #define UTIL_GTL_ALIGNTYPE_TEMPLATE(X) \ template struct AlignType { \ typedef UTIL_GTL_ALIGN_ATTRIBUTE(X) char result[size]; \ } UTIL_GTL_ALIGNTYPE_TEMPLATE(1); UTIL_GTL_ALIGNTYPE_TEMPLATE(2); UTIL_GTL_ALIGNTYPE_TEMPLATE(4); UTIL_GTL_ALIGNTYPE_TEMPLATE(8); UTIL_GTL_ALIGNTYPE_TEMPLATE(16); UTIL_GTL_ALIGNTYPE_TEMPLATE(32); UTIL_GTL_ALIGNTYPE_TEMPLATE(64); UTIL_GTL_ALIGNTYPE_TEMPLATE(128); UTIL_GTL_ALIGNTYPE_TEMPLATE(256); UTIL_GTL_ALIGNTYPE_TEMPLATE(512); UTIL_GTL_ALIGNTYPE_TEMPLATE(1024); UTIL_GTL_ALIGNTYPE_TEMPLATE(2048); UTIL_GTL_ALIGNTYPE_TEMPLATE(4096); UTIL_GTL_ALIGNTYPE_TEMPLATE(8192); // Any larger and MSVC++ will complain. #define UTIL_GTL_ALIGNED_CHAR_ARRAY(T, Size) \ typename util::gtl::internal::AlignType::result #undef UTIL_GTL_ALIGNTYPE_TEMPLATE #undef UTIL_GTL_ALIGN_ATTRIBUTE #else // defined(UTIL_GTL_ALIGN_ATTRIBUTE) #error "You must define UTIL_GTL_ALIGNED_CHAR_ARRAY for your compiler." #endif // defined(UTIL_GTL_ALIGN_ATTRIBUTE) #else // !SWIG // SWIG can't represent alignment and doesn't care about alignment on data // members (it works fine without it). template struct AlignType { typedef char result[Size]; }; #define UTIL_GTL_ALIGNED_CHAR_ARRAY(T, Size) \ util::gtl::internal::AlignType::result #endif // !SWIG } // namespace internal } // namespace gtl } // namespace util template class ManualConstructor { public: // No constructor or destructor because one of the most useful uses of // this class is as part of a union, and members of a union cannot have // constructors or destructors. And, anyway, the whole point of this // class is to bypass these. inline Type* get() { return reinterpret_cast(space_); } inline const Type* get() const { return reinterpret_cast(space_); } inline Type* operator->() { return get(); } inline const Type* operator->() const { return get(); } inline Type& operator*() { return *get(); } inline const Type& operator*() const { return *get(); } // You can pass up to four constructor arguments as arguments of Init(). inline void Init() { new(space_) Type; } template inline void Init(const T1& p1) { new(space_) Type(p1); } template inline void Init(const T1& p1, const T2& p2) { new(space_) Type(p1, p2); } template inline void Init(const T1& p1, const T2& p2, const T3& p3) { new(space_) Type(p1, p2, p3); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4) { new(space_) Type(p1, p2, p3, p4); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, const T5& p5) { new(space_) Type(p1, p2, p3, p4, p5); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, const T5& p5, const T6& p6) { new(space_) Type(p1, p2, p3, p4, p5, p6); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, const T5& p5, const T6& p6, const T7& p7) { new(space_) Type(p1, p2, p3, p4, p5, p6, p7); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, const T5& p5, const T6& p6, const T7& p7, const T8& p8) { new(space_) Type(p1, p2, p3, p4, p5, p6, p7, p8); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, const T5& p5, const T6& p6, const T7& p7, const T8& p8, const T9& p9) { new(space_) Type(p1, p2, p3, p4, p5, p6, p7, p8, p9); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, const T5& p5, const T6& p6, const T7& p7, const T8& p8, const T9& p9, const T10& p10) { new(space_) Type(p1, p2, p3, p4, p5, p6, p7, p8, p9, p10); } template inline void Init(const T1& p1, const T2& p2, const T3& p3, const T4& p4, const T5& p5, const T6& p6, const T7& p7, const T8& p8, const T9& p9, const T10& p10, const T11& p11) { new(space_) Type(p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11); } inline void Destroy() { get()->~Type(); } private: UTIL_GTL_ALIGNED_CHAR_ARRAY(Type, 1) space_; }; #undef UTIL_GTL_ALIGNED_CHAR_ARRAY #undef UTIL_GTL_ALIGN_OF } #endif // UTIL_GTL_MANUAL_CONSTRUCTOR_H_ ctemplate-ctemplate-2.4/src/base/mutex.h000066400000000000000000000402061363122316600203320ustar00rootroot00000000000000// Copyright (c) 2007, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // // --- // // A simple mutex wrapper, supporting locks and read-write locks. // You should assume the locks are *not* re-entrant. // // To use: you should define the following macros in your configure.ac: // ACX_PTHREAD // AC_RWLOCK // The latter is defined in ../autoconf. // // This class is meant to be internal-only and should be wrapped by an // internal namespace. Before you use this module, please give the // name of your internal namespace for this module. Or, if you want // to expose it, you'll want to move it to the Google namespace. We // cannot put this class in global namespace because there can be some // problems when we have multiple versions of Mutex in each shared object. // // NOTE: by default, we have #ifdef'ed out the TryLock() method. // This is for two reasons: // 1) TryLock() under Windows is a bit annoying (it requires a // #define to be defined very early). // 2) TryLock() is broken for NO_THREADS mode, at least in NDEBUG // mode. // If you need TryLock(), and either these two caveats are not a // problem for you, or you're willing to work around them, then // feel free to #define GMUTEX_TRYLOCK, or to remove the #ifdefs // in the code below. // // CYGWIN NOTE: Cygwin support for rwlock seems to be buggy: // http://www.cygwin.com/ml/cygwin/2008-12/msg00017.html // Because of that, we might as well use windows locks for // cygwin. They seem to be more reliable than the cygwin pthreads layer. // // TRICKY IMPLEMENTATION NOTE: // This class is designed to be safe to use during // dynamic-initialization -- that is, by global constructors that are // run before main() starts. The issue in this case is that // dynamic-initialization happens in an unpredictable order, and it // could be that someone else's dynamic initializer could call a // function that tries to acquire this mutex -- but that all happens // before this mutex's constructor has run. (This can happen even if // the mutex and the function that uses the mutex are in the same .cc // file.) Basically, because Mutex does non-trivial work in its // constructor, it's not, in the naive implementation, safe to use // before dynamic initialization has run on it. // // The solution used here is to pair the actual mutex primitive with a // bool that is set to true when the mutex is dynamically initialized. // (Before that it's false.) Then we modify all mutex routines to // look at the bool, and not try to lock/unlock until the bool makes // it to true (which happens after the Mutex constructor has run.) // // This works because before main() starts -- particularly, during // dynamic initialization -- there are no threads, so a) it's ok that // the mutex operations are a no-op, since we don't need locking then // anyway; and b) we can be quite confident our bool won't change // state between a call to Lock() and a call to Unlock() (that would // require a global constructor in one translation unit to call Lock() // and another global constructor in another translation unit to call // Unlock() later, which is pretty perverse). // // That said, it's tricky, and can conceivably fail; it's safest to // avoid trying to acquire a mutex in a global constructor, if you // can. One way it can fail is that a really smart compiler might // initialize the bool to true at static-initialization time (too // early) rather than at dynamic-initialization time. To discourage // that, we set is_safe_ to true in code (not the constructor // colon-initializer) and set it to true via a function that always // evaluates to true, but that the compiler can't know always // evaluates to true. This should be good enough. // // A related issue is code that could try to access the mutex // after it's been destroyed in the global destructors (because // the Mutex global destructor runs before some other global // destructor, that tries to acquire the mutex). The way we // deal with this is by taking a constructor arg that global // mutexes should pass in, that causes the destructor to do no // work. We still depend on the compiler not doing anything // weird to a Mutex's memory after it is destroyed, but for a // static global variable, that's pretty safe. #ifndef GOOGLE_MUTEX_H_ #define GOOGLE_MUTEX_H_ #include #if defined(NO_THREADS) typedef int MutexType; // to keep a lock-count #elif defined(_WIN32) || defined(__CYGWIN32__) || defined(__CYGWIN64__) # ifndef WIN32_LEAN_AND_MEAN # define WIN32_LEAN_AND_MEAN // We only need minimal includes # endif # ifndef NOMINMAX # define NOMINMAX // Don't want windows to override min()/max() # endif # ifdef GMUTEX_TRYLOCK // We need Windows NT or later for TryEnterCriticalSection(). If you // don't need that functionality, you can remove these _WIN32_WINNT // lines, and change TryLock() to assert(0) or something. # ifndef _WIN32_WINNT # define _WIN32_WINNT 0x0400 # endif # endif # include typedef CRITICAL_SECTION MutexType; #elif defined(HAVE_PTHREAD) && defined(HAVE_RWLOCK) // Needed for pthread_rwlock_*. If it causes problems, you could take it // out, but then you'd have to unset HAVE_RWLOCK (at least on linux -- it // *does* cause problems for FreeBSD, or MacOSX, but isn't needed // for locking there.) # ifdef __linux__ # if _XOPEN_SOURCE < 500 // including not being defined at all # undef _XOPEN_SOURCE # define _XOPEN_SOURCE 500 // may be needed to get the rwlock calls # endif # endif #if defined(HAVE_PTHREAD) && !defined(NO_THREADS) # include #endif typedef pthread_rwlock_t MutexType; #elif defined(HAVE_PTHREAD) #if defined(HAVE_PTHREAD) && !defined(NO_THREADS) # include #endif typedef pthread_mutex_t MutexType; #else # error Need to implement mutex.h for your architecture, or #define NO_THREADS #endif #include #include // for abort() namespace ctemplate { namespace base { // This is used for the single-arg constructor enum LinkerInitialized { LINKER_INITIALIZED }; } class Mutex { public: // Create a Mutex that is not held by anybody. This constructor is // typically used for Mutexes allocated on the heap or the stack. inline Mutex(); // This constructor should be used for global, static Mutex objects. // It inhibits work being done by the destructor, which makes it // safer for code that tries to acqiure this mutex in their global // destructor. inline Mutex(base::LinkerInitialized); // Destructor inline ~Mutex(); inline void Lock(); // Block if needed until free then acquire exclusively inline void Unlock(); // Release a lock acquired via Lock() #ifdef GMUTEX_TRYLOCK inline bool TryLock(); // If free, Lock() and return true, else return false #endif // Note that on systems that don't support read-write locks, these may // be implemented as synonyms to Lock() and Unlock(). So you can use // these for efficiency, but don't use them anyplace where being able // to do shared reads is necessary to avoid deadlock. inline void ReaderLock(); // Block until free or shared then acquire a share inline void ReaderUnlock(); // Release a read share of this Mutex inline void WriterLock() { Lock(); } // Acquire an exclusive lock inline void WriterUnlock() { Unlock(); } // Release a lock from WriterLock() private: MutexType mutex_; // We want to make sure that the compiler sets is_safe_ to true only // when we tell it to, and never makes assumptions is_safe_ is // always true. volatile is the most reliable way to do that. volatile bool is_safe_; // This indicates which constructor was called. bool destroy_; inline void SetIsSafe() { is_safe_ = true; } // Catch the error of writing Mutex when intending MutexLock. Mutex(Mutex* /*ignored*/) {} // Disallow "evil" constructors Mutex(const Mutex&); void operator=(const Mutex&); }; // We will also define GoogleOnceType, GOOGLE_ONCE_INIT, and // GoogleOnceInit, which are portable versions of pthread_once_t, // PTHREAD_ONCE_INIT, and pthread_once. // Now the implementation of Mutex for various systems #if defined(NO_THREADS) // When we don't have threads, we can be either reading or writing, // but not both. We can have lots of readers at once (in no-threads // mode, that's most likely to happen in recursive function calls), // but only one writer. We represent this by having mutex_ be -1 when // writing and a number > 0 when reading (and 0 when no lock is held). // // In debug mode, we assert these invariants, while in non-debug mode // we do nothing, for efficiency. That's why everything is in an // assert. Mutex::Mutex() : mutex_(0) { } Mutex::Mutex(base::LinkerInitialized) : mutex_(0) { } Mutex::~Mutex() { assert(mutex_ == 0); } void Mutex::Lock() { assert(--mutex_ == -1); } void Mutex::Unlock() { assert(mutex_++ == -1); } #ifdef GMUTEX_TRYLOCK bool Mutex::TryLock() { if (mutex_) return false; Lock(); return true; } #endif void Mutex::ReaderLock() { assert(++mutex_ > 0); } void Mutex::ReaderUnlock() { assert(mutex_-- > 0); } typedef int GoogleOnceType; const GoogleOnceType GOOGLE_ONCE_INIT = 0; inline int GoogleOnceInit(GoogleOnceType* once_control, void (*init_routine)(void)) { if ((*once_control)++ == 0) (*init_routine)(); return 0; } #elif defined(_WIN32) || defined(__CYGWIN32__) || defined(__CYGWIN64__) Mutex::Mutex() : destroy_(true) { InitializeCriticalSection(&mutex_); SetIsSafe(); } Mutex::Mutex(base::LinkerInitialized) : destroy_(false) { InitializeCriticalSection(&mutex_); SetIsSafe(); } Mutex::~Mutex() { if (destroy_) DeleteCriticalSection(&mutex_); } void Mutex::Lock() { if (is_safe_) EnterCriticalSection(&mutex_); } void Mutex::Unlock() { if (is_safe_) LeaveCriticalSection(&mutex_); } #ifdef GMUTEX_TRYLOCK bool Mutex::TryLock() { return is_safe_ ? TryEnterCriticalSection(&mutex_) != 0 : true; } #endif void Mutex::ReaderLock() { Lock(); } // we don't have read-write locks void Mutex::ReaderUnlock() { Unlock(); } // We do a simple spinlock for pthread_once_t. See // http://www.ddj.com/cpp/199203083?pgno=3 #ifdef INTERLOCKED_EXCHANGE_NONVOLATILE typedef LONG GoogleOnceType; #else typedef volatile LONG GoogleOnceType; #endif const GoogleOnceType GOOGLE_ONCE_INIT = 0; inline int GoogleOnceInit(GoogleOnceType* once_control, void (*init_routine)(void)) { while (1) { LONG prev = InterlockedCompareExchange(once_control, 1, 0); if (prev == 2) { // We've successfully initted in the past. return 0; } else if (prev == 0) { // No init yet, but we have the lock. (*init_routine)(); InterlockedExchange(once_control, 2); return 0; } else { // Someone else is holding the lock, so wait. assert(1 == prev); Sleep(1); // sleep for 1ms } } return 1; // unreachable } #elif defined(HAVE_PTHREAD) && defined(HAVE_RWLOCK) #define SAFE_PTHREAD(fncall) do { /* run fncall if is_safe_ is true */ \ if (is_safe_ && fncall(&mutex_) != 0) abort(); \ } while (0) Mutex::Mutex() : destroy_(true) { SetIsSafe(); if (is_safe_ && pthread_rwlock_init(&mutex_, NULL) != 0) abort(); } Mutex::Mutex(base::LinkerInitialized) : destroy_(false) { SetIsSafe(); if (is_safe_ && pthread_rwlock_init(&mutex_, NULL) != 0) abort(); } Mutex::~Mutex() { if (destroy_) SAFE_PTHREAD(pthread_rwlock_destroy); } void Mutex::Lock() { SAFE_PTHREAD(pthread_rwlock_wrlock); } void Mutex::Unlock() { SAFE_PTHREAD(pthread_rwlock_unlock); } #ifdef GMUTEX_TRYLOCK bool Mutex::TryLock() { return is_safe_ ? pthread_rwlock_trywrlock(&mutex_) == 0 : true; } #endif void Mutex::ReaderLock() { SAFE_PTHREAD(pthread_rwlock_rdlock); } void Mutex::ReaderUnlock() { SAFE_PTHREAD(pthread_rwlock_unlock); } #undef SAFE_PTHREAD typedef pthread_once_t GoogleOnceType; const GoogleOnceType GOOGLE_ONCE_INIT = PTHREAD_ONCE_INIT; inline int GoogleOnceInit(GoogleOnceType* once_control, void (*init_routine)(void)) { return pthread_once(once_control, init_routine); } #elif defined(HAVE_PTHREAD) #define SAFE_PTHREAD(fncall) do { /* run fncall if is_safe_ is true */ \ if (is_safe_ && fncall(&mutex_) != 0) abort(); \ } while (0) Mutex::Mutex() : destroy_(true) { SetIsSafe(); if (is_safe_ && pthread_mutex_init(&mutex_, NULL) != 0) abort(); } Mutex::Mutex(base::LinkerInitialized) : destroy_(false) { SetIsSafe(); if (is_safe_ && pthread_mutex_init(&mutex_, NULL) != 0) abort(); } Mutex::~Mutex() { if (destroy_) SAFE_PTHREAD(pthread_mutex_destroy); } void Mutex::Lock() { SAFE_PTHREAD(pthread_mutex_lock); } void Mutex::Unlock() { SAFE_PTHREAD(pthread_mutex_unlock); } #ifdef GMUTEX_TRYLOCK bool Mutex::TryLock() { return is_safe_ ? pthread_mutex_trylock(&mutex_) == 0 : true; } #endif void Mutex::ReaderLock() { Lock(); } void Mutex::ReaderUnlock() { Unlock(); } #undef SAFE_PTHREAD typedef pthread_once_t GoogleOnceType; const GoogleOnceType GOOGLE_ONCE_INIT = PTHREAD_ONCE_INIT; inline int GoogleOnceInit(GoogleOnceType* once_control, void (*init_routine)(void)) { return pthread_once(once_control, init_routine); } #endif // -------------------------------------------------------------------------- // Some helper classes // MutexLock(mu) acquires mu when constructed and releases it when destroyed. class MutexLock { public: explicit MutexLock(Mutex *mu) : mu_(mu) { mu_->Lock(); } ~MutexLock() { mu_->Unlock(); } private: Mutex * const mu_; // Disallow "evil" constructors MutexLock(const MutexLock&); void operator=(const MutexLock&); }; // ReaderMutexLock and WriterMutexLock do the same, for rwlocks class ReaderMutexLock { public: explicit ReaderMutexLock(Mutex *mu) : mu_(mu) { mu_->ReaderLock(); } ~ReaderMutexLock() { mu_->ReaderUnlock(); } private: Mutex * const mu_; // Disallow "evil" constructors ReaderMutexLock(const ReaderMutexLock&); void operator=(const ReaderMutexLock&); }; class WriterMutexLock { public: explicit WriterMutexLock(Mutex *mu) : mu_(mu) { mu_->WriterLock(); } ~WriterMutexLock() { mu_->WriterUnlock(); } private: Mutex * const mu_; // Disallow "evil" constructors WriterMutexLock(const WriterMutexLock&); void operator=(const WriterMutexLock&); }; // Catch bug where variable name is omitted, e.g. MutexLock (&mu); #define MutexLock(x) COMPILE_ASSERT(0, mutex_lock_decl_missing_var_name) #define ReaderMutexLock(x) COMPILE_ASSERT(0, rmutex_lock_decl_missing_var_name) #define WriterMutexLock(x) COMPILE_ASSERT(0, wmutex_lock_decl_missing_var_name) } #endif /* #define GOOGLE_MUTEX_H__ */ ctemplate-ctemplate-2.4/src/base/small_map.h000066400000000000000000000411601363122316600211350ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // Author: kenton@google.com (Kenton Varda) // // small_map is a drop-in replacement for map or hash_map. It uses a fixed // array to store a certain number of elements, then reverts to using a // map or hash_map when it runs out of space. For maps that are typically // small, this can be considerably faster than using something like hash_map // directly, as hash_map is optimized for large data sets. Of course, in // order for this to be a significant win, you have to have a situation where // you are using lots and lots of these small maps. One such situation is // MessageSet: A set of search results may contain thousands of MessageSets, // each containing only a couple items. // // TODO(kenton): This is very minimal, and was originally written for a // very specific use (MessageSet). It only implements a few core methods // of the STL associative container interface, though you are welcome to // extend it. #ifndef UTIL_GTL_SMALL_MAP_H_ #define UTIL_GTL_SMALL_MAP_H_ #include #include #include // for make_pair() #include "base/manual_constructor.h" namespace ctemplate { template struct CompileAssert { }; #define COMPILE_ASSERT(expr, msg) \ typedef CompileAssert<(bool(expr))> msg[bool(expr) ? 1 : -1] // An STL-like associative container which starts out backed by a simple // array but switches to some other container type if it grows beyond a // fixed size. // // NormalMap: The map type to fall back to. This also defines the key // and value types for the small_map. // kArraySize: The size of the initial array of results. Once the map // grows beyond this size, the map type will be used instead. // EqualKey: A functor which tests two keys for equality. If the wrapped // map type has a "key_equal" member (hash_map does), then that // will be used by default. Otherwise you must specify this // manually. // MapInit: A functor that takes a ManualConstructor* and uses it to // initialize the map. This functor will be called at most once per // small_map, when the map exceeds the threshold of kArraySize and we // are about to copy values from the array to the map. The functor // *must* call one of the Init() methods provided by // ManualConstructor, since after it runs we assume that the NormalMap // has been initialized. // // example: // small_map > days; // days["sunday" ] = 0; // days["monday" ] = 1; // days["tuesday" ] = 2; // days["wednesday"] = 3; // days["thursday" ] = 4; // days["friday" ] = 5; // days["saturday" ] = 6; // // You should assume that small_map might invalidate all the iterators // on any call to erase(), insert() and operator[]. template class small_map_default_init { public: void operator ()(ManualConstructor* map) const { map->Init(); } }; template > class small_map { // We cannot rely on the compiler to reject array of size 0. In // particular, gcc 2.95.3 does it but later versions allow 0-length // arrays. Therefore, we explicitly reject non-positive kArraySize // here. COMPILE_ASSERT(kArraySize > 0, default_initial_size_should_be_positive); public: typedef typename NormalMap::key_type key_type; typedef typename NormalMap::mapped_type data_type; typedef typename NormalMap::mapped_type mapped_type; typedef typename NormalMap::value_type value_type; typedef EqualKey key_equal; small_map() : size_(0), functor_(MapInit()) {} explicit small_map(const MapInit& functor) : size_(0), functor_(functor) {} // Allow copy-constructor and assignment, since STL allows them too. small_map(const small_map& src) { // size_ and functor_ are initted in InitFrom() InitFrom(src); } void operator=(const small_map& src) { if (&src == this) return; // This is not optimal. If src and dest are both using the small // array, we could skip the teardown and reconstruct. One problem // to be resolved is that the value_type itself is pair, and const K is not assignable. Destroy(); InitFrom(src); } ~small_map() { Destroy(); } class const_iterator; class iterator { public: typedef typename NormalMap::iterator::iterator_category iterator_category; typedef typename NormalMap::iterator::value_type value_type; typedef typename NormalMap::iterator::difference_type difference_type; typedef typename NormalMap::iterator::pointer pointer; typedef typename NormalMap::iterator::reference reference; inline iterator(): array_iter_(NULL) {} inline iterator& operator++() { if (array_iter_ != NULL) { ++array_iter_; } else { ++hash_iter_; } return *this; } inline iterator operator++(int) { iterator result(*this); ++(*this); return result; } inline iterator& operator--() { if (array_iter_ != NULL) { --array_iter_; } else { --hash_iter_; } return *this; } inline iterator operator--(int) { iterator result(*this); --(*this); return result; } inline value_type* operator->() const { if (array_iter_ != NULL) { return array_iter_->get(); } else { return hash_iter_.operator->(); } } inline value_type& operator*() const { if (array_iter_ != NULL) { return *array_iter_->get(); } else { return *hash_iter_; } } inline bool operator==(const iterator& other) const { if (array_iter_ != NULL) { return array_iter_ == other.array_iter_; } else { return other.array_iter_ == NULL && hash_iter_ == other.hash_iter_; } } inline bool operator!=(const iterator& other) const { return !(*this == other); } bool operator==(const const_iterator& other) const; bool operator!=(const const_iterator& other) const; private: friend class small_map; friend class const_iterator; inline explicit iterator(ManualConstructor* init) : array_iter_(init) {} inline explicit iterator(const typename NormalMap::iterator& init) : array_iter_(NULL), hash_iter_(init) {} ManualConstructor* array_iter_; typename NormalMap::iterator hash_iter_; }; class const_iterator { public: typedef typename NormalMap::const_iterator::iterator_category iterator_category; typedef typename NormalMap::const_iterator::value_type value_type; typedef typename NormalMap::const_iterator::difference_type difference_type; typedef typename NormalMap::const_iterator::pointer pointer; typedef typename NormalMap::const_iterator::reference reference; inline const_iterator(): array_iter_(NULL) {} inline const_iterator(const iterator& other) : array_iter_(other.array_iter_), hash_iter_(other.hash_iter_) {} inline const_iterator& operator++() { if (array_iter_ != NULL) { ++array_iter_; } else { ++hash_iter_; } return *this; } inline const_iterator operator++(int) { const_iterator result(*this); ++(*this); return result; } inline const_iterator& operator--() { if (array_iter_ != NULL) { --array_iter_; } else { --hash_iter_; } return *this; } inline const_iterator operator--(int) { const_iterator result(*this); --(*this); return result; } inline const value_type* operator->() const { if (array_iter_ != NULL) { return array_iter_->get(); } else { return hash_iter_.operator->(); } } inline const value_type& operator*() const { if (array_iter_ != NULL) { return *array_iter_->get(); } else { return *hash_iter_; } } inline bool operator==(const const_iterator& other) const { if (array_iter_ != NULL) { return array_iter_ == other.array_iter_; } else { return other.array_iter_ == NULL && hash_iter_ == other.hash_iter_; } } inline bool operator!=(const const_iterator& other) const { return !(*this == other); } private: friend class small_map; inline explicit const_iterator( const ManualConstructor* init) : array_iter_(init) {} inline explicit const_iterator( const typename NormalMap::const_iterator& init) : array_iter_(NULL), hash_iter_(init) {} const ManualConstructor* array_iter_; typename NormalMap::const_iterator hash_iter_; }; iterator find(const key_type& key) { key_equal compare; if (size_ >= 0) { for (int i = 0; i < size_; i++) { if (compare(array_[i]->first, key)) { return iterator(array_ + i); } } return iterator(array_ + size_); } else { return iterator(map()->find(key)); } } const_iterator find(const key_type& key) const { key_equal compare; if (size_ >= 0) { for (int i = 0; i < size_; i++) { if (compare(array_[i]->first, key)) { return const_iterator(array_ + i); } } return const_iterator(array_ + size_); } else { return const_iterator(map()->find(key)); } } // Invalidates iterators. data_type& operator[](const key_type& key) { key_equal compare; if (size_ >= 0) { // operator[] searches backwards, favoring recently-added // elements. for (int i = size_-1; i >= 0; --i) { if (compare(array_[i]->first, key)) { return array_[i]->second; } } if (size_ == kArraySize) { ConvertToRealMap(); return (*map_)[key]; } else { array_[size_].Init(key, data_type()); return array_[size_++]->second; } } else { return (*map_)[key]; } } // Invalidates iterators. std::pair insert(const value_type& x) { key_equal compare; if (size_ >= 0) { for (int i = 0; i < size_; i++) { if (compare(array_[i]->first, x.first)) { return std::make_pair(iterator(array_ + i), false); } } if (size_ == kArraySize) { ConvertToRealMap(); // Invalidates all iterators! std::pair ret = map_->insert(x); return std::make_pair(iterator(ret.first), ret.second); } else { array_[size_].Init(x); return std::make_pair(iterator(array_ + size_++), true); } } else { std::pair ret = map_->insert(x); return std::make_pair(iterator(ret.first), ret.second); } } // Invalidates iterators. template void insert(InputIterator f, InputIterator l) { while (f != l) { insert(*f); ++f; } } iterator begin() { if (size_ >= 0) { return iterator(array_); } else { return iterator(map_->begin()); } } const_iterator begin() const { if (size_ >= 0) { return const_iterator(array_); } else { return const_iterator(map_->begin()); } } iterator end() { if (size_ >= 0) { return iterator(array_ + size_); } else { return iterator(map_->end()); } } const_iterator end() const { if (size_ >= 0) { return const_iterator(array_ + size_); } else { return const_iterator(map_->end()); } } void clear() { if (size_ >= 0) { for (int i = 0; i < size_; i++) { array_[i].Destroy(); } } else { map_.Destroy(); } size_ = 0; } // Invalidates iterators. void erase(const iterator& position) { if (size_ >= 0) { int i = position.array_iter_ - array_; array_[i].Destroy(); --size_; if (i != size_) { array_[i].Init(*array_[size_]); array_[size_].Destroy(); } } else { map_->erase(position.hash_iter_); } } int erase(const key_type& key) { iterator iter = find(key); if (iter == end()) return 0; erase(iter); return 1; } int count(const key_type& key) const { return (find(key) == end()) ? 0 : 1; } int size() const { if (size_ >= 0) { return size_; } else { return map_->size(); } } bool empty() const { if (size_ >= 0) { return (size_ == 0); } else { return map_->empty(); } } // Returns true if we have fallen back to using the underlying map // representation. bool using_full_map() const { return size_ < 0; } inline NormalMap* map() { assert(using_full_map()); return map_.get(); } inline const NormalMap* map() const { assert(using_full_map()); return map_.get(); } private: int size_; // negative = using hash_map MapInit functor_; // We want to call constructors and destructors manually, but we don't // want to allocate and deallocate the memory used for them separately. // So, we use this crazy ManualConstructor class. // // Since array_ and map_ are mutually exclusive, we'll put them in a // union, too. We add in a dummy_ value which quiets MSVC (both // 7.1 and 8.0) from otherwise giving an erroneous "union member has // copy constructor" error message (C2621). This dummy member has // to come before array_ to quiet the compiler. Shrug. union { ManualConstructor dummy_; ManualConstructor array_[kArraySize]; ManualConstructor map_; }; void ConvertToRealMap() { // Move the current elements into a temporary array. ManualConstructor temp_array[kArraySize]; for (int i = 0; i < kArraySize; i++) { temp_array[i].Init(*array_[i]); array_[i].Destroy(); } // Initialize the map. size_ = -1; functor_(&map_); // Insert elements into it. for (int i = 0; i < kArraySize; i++) { map_->insert(*temp_array[i]); temp_array[i].Destroy(); } } // Helpers for constructors and destructors. void InitFrom(const small_map& src) { functor_ = src.functor_; size_ = src.size_; if (src.size_ >= 0) { for (int i = 0; i < size_; i++) { array_[i].Init(*src.array_[i]); } } else { functor_(&map_); (*map_.get()) = (*src.map_.get()); } } void Destroy() { if (size_ >= 0) { for (int i = 0; i < size_; i++) { array_[i].Destroy(); } } else { map_.Destroy(); } } }; template inline bool small_map::iterator::operator==( const const_iterator& other) const { return other == *this; } template inline bool small_map::iterator::operator!=( const const_iterator& other) const { return other != *this; } } #endif // UTIL_GTL_SMALL_MAP_H_ ctemplate-ctemplate-2.4/src/base/thread_annotations.h000066400000000000000000000131651363122316600230600ustar00rootroot00000000000000// Copyright (c) 2008, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // // This header file contains the macro definitions for thread safety // annotations that allow the developers to document the locking policies // of their multi-threaded code. The annotations can also help program // analysis tools to identify potential thread safety issues. // // // The annotations are implemented using GCC's "attributes" extension. // Using the macros defined here instead of the raw GCC attributes allows // for portability and future compatibility. // #ifndef BASE_THREAD_ANNOTATIONS_H_ #define BASE_THREAD_ANNOTATIONS_H_ #include #if defined(__GNUC__) && defined(__SUPPORT_TS_ANNOTATION__) && !defined(SWIG) #define THREAD_ANNOTATION_ATTRIBUTE__(x) __attribute__((x)) #else #define THREAD_ANNOTATION_ATTRIBUTE__(x) // no-op #endif // Document if a shared variable/field needs to be protected by a lock. // GUARDED_BY allows the user to specify a particular lock that should be // held when accessing the annotated variable, while GUARDED_VAR only // indicates a shared variable should be guarded (by any lock). GUARDED_VAR // is primarily used when the client cannot express the name of the lock. #define GUARDED_BY(x) THREAD_ANNOTATION_ATTRIBUTE__(guarded_by(x)) #define GUARDED_VAR THREAD_ANNOTATION_ATTRIBUTE__(guarded) // Document if the memory location pointed to by a pointer should be guarded // by a lock when dereferencing the pointer. Similar to GUARDED_VAR, // PT_GUARDED_VAR is primarily used when the client cannot express the name // of the lock. Note that a pointer variable to a shared memory location // could itself be a shared variable. For example, if a shared global pointer // q, which is guarded by mu1, points to a shared memory location that is // guarded by mu2, q should be annotated as follows: // int *q GUARDED_BY(mu1) PT_GUARDED_BY(mu2); #define PT_GUARDED_BY(x) \ THREAD_ANNOTATION_ATTRIBUTE__(point_to_guarded_by(x)) #define PT_GUARDED_VAR \ THREAD_ANNOTATION_ATTRIBUTE__(point_to_guarded) // Document the acquisition order between locks that can be held // simultaneously by a thread. For any two locks that need to be annotated // to establish an acquisition order, only one of them needs the annotation. // (i.e. You don't have to annotate both locks with both ACQUIRED_AFTER // and ACQUIRED_BEFORE.) #define ACQUIRED_AFTER(x) \ THREAD_ANNOTATION_ATTRIBUTE__(acquired_after(x)) #define ACQUIRED_BEFORE(x) \ THREAD_ANNOTATION_ATTRIBUTE__(acquired_before(x)) // The following three annotations document the lock requirements for // functions/methods. // Document if a function expects certain locks to be held before it is called #define EXCLUSIVE_LOCKS_REQUIRED(x) \ THREAD_ANNOTATION_ATTRIBUTE__(exclusive_locks_required(x)) #define SHARED_LOCKS_REQUIRED(x) \ THREAD_ANNOTATION_ATTRIBUTE__(shared_locks_required(x)) // Document the locks acquired in the body of the function. These locks // non-reentrant). #define LOCKS_EXCLUDED(x) \ THREAD_ANNOTATION_ATTRIBUTE__(locks_excluded(x)) // Document the lock the annotated function returns without acquiring it. #define LOCK_RETURNED(x) THREAD_ANNOTATION_ATTRIBUTE__(lock_returned(x)) // Document if a class/type is a lockable type (such as the Mutex class). #define LOCKABLE THREAD_ANNOTATION_ATTRIBUTE__(lockable) // Document if a class is a scoped lockable type (such as the MutexLock class). #define SCOPED_LOCKABLE THREAD_ANNOTATION_ATTRIBUTE__(scoped_lockable) // The following annotations specify lock and unlock primitives. #define EXCLUSIVE_LOCK_FUNCTION(x) \ THREAD_ANNOTATION_ATTRIBUTE__(exclusive_lock(x)) #define SHARED_LOCK_FUNCTION(x) \ THREAD_ANNOTATION_ATTRIBUTE__(shared_lock(x)) #define EXCLUSIVE_TRYLOCK_FUNCTION(x) \ THREAD_ANNOTATION_ATTRIBUTE__(exclusive_trylock(x)) #define SHARED_TRYLOCK_FUNCTION(x) \ THREAD_ANNOTATION_ATTRIBUTE__(shared_trylock(x)) #define UNLOCK_FUNCTION(x) \ THREAD_ANNOTATION_ATTRIBUTE__(unlock(x)) // An escape hatch for thread safety analysis to ignore the annotated function. #define NO_THREAD_SAFETY_ANALYSIS \ THREAD_ANNOTATION_ATTRIBUTE__(no_thread_safety_analysis) #endif // BASE_THREAD_ANNOTATIONS_H_ ctemplate-ctemplate-2.4/src/base/util.h000066400000000000000000000222661363122316600201530ustar00rootroot00000000000000// Copyright (c) 2011, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // Some generically useful utility routines that in google-land would // be their own projects. We make a shortened version here. #ifndef TEMPLATE_UTIL_H_ #define TEMPLATE_UTIL_H_ #include #include #include #include #include #include #include #include #include #include // -- utility macros --------------------------------------------------------- #define DISALLOW_COPY_AND_ASSIGN(TypeName) \ TypeName(const TypeName&); \ void operator=(const TypeName&) // Starting with Visual C++ 2005, WinNT.h includes ARRAYSIZE. #if !defined(_MSC_VER) || _MSC_VER < 1400 #define ARRAYSIZE(a) \ ((sizeof(a) / sizeof(*(a))) / \ static_cast(!(sizeof(a) % sizeof(*(a))))) #endif template // use like this: down_cast(foo); inline To down_cast(From* f) { // so we only accept pointers return static_cast(f); } // -- CHECK macros --------------------------------------------------------- // CHECK dies with a fatal error if condition is not true. It is *not* // controlled by NDEBUG, so the check will be executed regardless of // compilation mode. Therefore, it is safe to do things like: // CHECK(fp->Write(x) == 4) // We allow stream-like objects after this for debugging, but they're ignored. #define CHECK(condition) \ if (true) { \ if (!(condition)) { \ fprintf(stderr, "Check failed: %s\n", #condition); \ exit(1); \ } \ } else std::cerr << "" #define CHECK_OP(op, val1, val2) \ if (true) { \ if (!((val1) op (val2))) { \ fprintf(stderr, "Check failed: %s %s %s\n", #val1, #op, #val2); \ exit(1); \ } \ } else std::cerr << "" #define CHECK_EQ(val1, val2) CHECK_OP(==, val1, val2) #define CHECK_NE(val1, val2) CHECK_OP(!=, val1, val2) #define CHECK_LE(val1, val2) CHECK_OP(<=, val1, val2) #define CHECK_LT(val1, val2) CHECK_OP(< , val1, val2) #define CHECK_GE(val1, val2) CHECK_OP(>=, val1, val2) #define CHECK_GT(val1, val2) CHECK_OP(> , val1, val2) // Synonyms for CHECK_* that are used in some unittests. #define EXPECT_EQ(val1, val2) CHECK_EQ(val1, val2) #define EXPECT_NE(val1, val2) CHECK_NE(val1, val2) #define EXPECT_LE(val1, val2) CHECK_LE(val1, val2) #define EXPECT_LT(val1, val2) CHECK_LT(val1, val2) #define EXPECT_GE(val1, val2) CHECK_GE(val1, val2) #define EXPECT_GT(val1, val2) CHECK_GT(val1, val2) #define EXPECT_TRUE(cond) CHECK(cond) #define EXPECT_FALSE(cond) CHECK(!(cond)) #define EXPECT_STREQ(a, b) CHECK(strcmp(a, b) == 0) #define ASSERT_TRUE(cond) EXPECT_TRUE(cond) // LOG(FATAL) is an alias for CHECK(FALSE). We define FATAL, but no // other value that is reasonable inside LOG(), so the compile will // fail if someone tries to use LOG(DEBUG) or the like. #define LOG(x) INTERNAL_DO_LOG_ ## x #define INTERNAL_DO_LOG_FATAL CHECK(false) // These are used only in debug mode. #ifdef NDEBUG #define DCHECK(condition) CHECK(condition) #define DCHECK_EQ(val1, val2) CHECK_EQ(val1, val2) #define DCHECK_NE(val1, val2) CHECK_NE(val1, val2) #define DCHECK_LE(val1, val2) CHECK_LE(val1, val2) #define DCHECK_LT(val1, val2) CHECK_LT(val1, val2) #define DCHECK_GE(val1, val2) CHECK_GE(val1, val2) #define DCHECK_GT(val1, val2) CHECK_GT(val1, val2) #else #define DCHECK(condition) if (true) {} else std::cerr << "" #define DCHECK_EQ(val1, val2) if (true) {} else std::cerr << "" #define DCHECK_NE(val1, val2) if (true) {} else std::cerr << "" #define DCHECK_LE(val1, val2) if (true) {} else std::cerr << "" #define DCHECK_LT(val1, val2) if (true) {} else std::cerr << "" #define DCHECK_GE(val1, val2) if (true) {} else std::cerr << "" #define DCHECK_GT(val1, val2) if (true) {} else std::cerr << "" #endif #define PCHECK(cond) CHECK(cond) << ": " << strerror(errno) #define PFATAL(s) do { perror(s); exit(1); } while (0) // -- testing-related macros -------------------------------------------------- // Call this in a .cc file where you will later call RUN_ALL_TESTS in main(). #define TEST_INIT \ static std::vector g_testlist; /* the tests to run */ \ static int RUN_ALL_TESTS() { \ std::vector::const_iterator it; \ for (it = g_testlist.begin(); it != g_testlist.end(); ++it) { \ (*it)(); /* The test will error-exit if there's a problem. */ \ } \ fprintf(stderr, "\nPassed %d tests\n\nPASS\n", \ static_cast(g_testlist.size())); \ return 0; \ } #define TEST(a, b) \ class Test_##a##_##b { \ public: \ Test_##a##_##b() { g_testlist.push_back(&Run); } \ static void Run(); \ }; \ static Test_##a##_##b g_test_##a##_##b; \ void Test_##a##_##b::Run() // This is a dummy class that eases the google->opensource transition. namespace testing { class Test {}; } // -- template-related macros ---------------------------------------------- #ifndef DEFAULT_TEMPLATE_ROOTDIR # define DEFAULT_TEMPLATE_ROOTDIR "." #endif // -- string-related functions ---------------------------------------------- inline bool safe_strto32(const std::string& s, int* i) { char* error_pos; if (s.empty()) return false; // no input at all errno = 0; // just to be sure *i = strtol(s.c_str(), &error_pos, 10); return *error_pos == '\0' && errno == 0; } inline int atoi32(const char* s) { return atoi(s); } inline void StripWhiteSpace(std::string* str) { int str_length = str->length(); // Strip off leading whitespace. int first = 0; while (first < str_length && isspace(str->at(first))) { ++first; } // If entire string is white space. if (first == str_length) { str->clear(); return; } if (first > 0) { str->erase(0, first); str_length -= first; } // Strip off trailing whitespace. int last = str_length - 1; while (last >= 0 && isspace(str->at(last))) { --last; } if (last != (str_length - 1) && last >= 0) { str->erase(last + 1, std::string::npos); } } inline void SplitStringIntoKeyValuePairs( const std::string& s, const char* kv_split, // For instance: "=" const char* pair_split, // For instance: "," std::vector< std::pair > *pairs) { std::string key, value; std::string* add_to = &key; for (std::string::size_type i = 0; i < s.length(); ++i) { if (s[i] == kv_split[0]) { add_to = &value; } else if (s[i] == pair_split[0]) { if (!key.empty()) pairs->push_back(std::pair(key, value)); key.clear(); value.clear(); add_to = &key; } else { *add_to += s[i]; } } if (!key.empty()) pairs->push_back(std::pair(key, value)); } #endif // TEMPLATE_UTIL_H_ ctemplate-ctemplate-2.4/src/ctemplate/000077500000000000000000000000001363122316600200615ustar00rootroot00000000000000ctemplate-ctemplate-2.4/src/ctemplate/find_ptr.h.in000066400000000000000000000052221363122316600224450ustar00rootroot00000000000000// Copyright (c) 2012, Olaf van der Spek // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: Olaf van der Spek #ifndef TEMPLATE_FIND_PTR_H_ #define TEMPLATE_FIND_PTR_H_ #include @ac_windows_dllexport_defines@ namespace ctemplate { template const typename T::value_type* find_ptr0(const T& c, U v) { typename T::const_iterator i = c.find(v); return i == c.end() ? NULL : &*i; } template typename T::value_type::second_type* find_ptr(T& c, U v) { typename T::iterator i = c.find(v); return i == c.end() ? NULL : &i->second; } template const typename T::value_type::second_type* find_ptr(const T& c, U v) { typename T::const_iterator i = c.find(v); return i == c.end() ? NULL : &i->second; } template typename T::value_type::second_type find_ptr2(T& c, U v) { typename T::iterator i = c.find(v); return i == c.end() ? NULL : i->second; } template const typename T::value_type::second_type find_ptr2(const T& c, U v) { typename T::const_iterator i = c.find(v); return i == c.end() ? NULL : i->second; } } #endif // TEMPLATE_FIND_PTR_H_ ctemplate-ctemplate-2.4/src/ctemplate/per_expand_data.h.in000066400000000000000000000133241363122316600237600ustar00rootroot00000000000000// Copyright (c) 2008, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // In addition to a TemplateDictionary, there is also a PerExpandData // dictionary. This dictionary holds information that applies to one // call to Expand, such as whether to annotate the template expansion // output. A template dictionary is associated with a template (.tpl) // file; a per-expand dictionary is associated to a particular call to // Expand() in a .cc file. // // For (many) more details, see the doc/ directory. #ifndef TEMPLATE_PER_EXPAND_DATA_H_ #define TEMPLATE_PER_EXPAND_DATA_H_ #include // for NULL #include // for strcmp #include #include @ac_cv_cxx_hash_map@ #include // for StringHash @ac_windows_dllexport_defines@ namespace ctemplate { class TemplateModifier; class TemplateAnnotator; class @ac_windows_dllexport@ PerExpandData { public: PerExpandData() : annotate_path_(NULL), annotator_(NULL), expand_modifier_(NULL), map_(NULL) { } ~PerExpandData(); // Indicate that annotations should be inserted during template expansion. // template_path_start - the start of a template path. When // printing the filename for template-includes, anything before and // including template_path_start is elided. This can make the // output less dependent on filesystem location for template files. void SetAnnotateOutput(const char* template_path_start) { annotate_path_ = template_path_start; } // Whether to annotate the expanded output. bool annotate() const { return annotate_path_ != NULL; } // The annotate-path; undefined if annotate() != true const char* annotate_path() const { return annotate_path_; } // This sets the TemplateAnnotator to be used when annotating is on. // This allows you to override the default text-based annotator // that will be used if you do not call this. The passed annotator // will be aliased by this object and returned by annotator(). // Passing NULL has the special behavior of causing annotator() to // revert to returning its built-in instance. void SetAnnotator(TemplateAnnotator* annotator) { annotator_ = annotator; } // This returns the TemplateAnnotator to be used when annotating is on. // The value returned will be either an instance previously provided // to SetAnnotator() or the callable built-in text-based annotator. TemplateAnnotator* annotator() const; // This is a TemplateModifier to be applied to all templates // expanded via this call to Expand(). That is, this modifier is // applies to the template (.tpl) file we expand, as well as // sub-templates that are expanded due to {{>INCLUDE}} directives. // Caller is responsible for ensuring that modifier exists for the // lifetime of this object. void SetTemplateExpansionModifier(const TemplateModifier* modifier) { expand_modifier_ = modifier; } const TemplateModifier* template_expansion_modifier() const { return expand_modifier_; } // Store data in this structure, to be used by template modifiers // (see template_modifiers.h). Call with value set to NULL to clear // any value previously set. Caller is responsible for ensuring key // and value point to valid data for the lifetime of this object. void InsertForModifiers(const char* key, const void* value); // Retrieve data specific to this Expand call. Returns NULL if key // is not found. This should only be used by template modifiers. const void* LookupForModifiers(const char* key) const; // Same as Lookup, but casts the result to a c string. const char* LookupForModifiersAsString(const char* key) const { return static_cast(LookupForModifiers(key)); } private: struct DataEq { bool operator()(const char* s1, const char* s2) const; }; typedef @ac_cv_cxx_hash_namespace@::unordered_map DataMap; const char* annotate_path_; TemplateAnnotator* annotator_; const TemplateModifier* expand_modifier_; DataMap* map_; PerExpandData(const PerExpandData&); // disallow evil copy constructor void operator=(const PerExpandData&); // disallow evil operator= }; } #endif // TEMPLATE_PER_EXPAND_DATA_H_ ctemplate-ctemplate-2.4/src/ctemplate/str_ref.h.in000066400000000000000000000056621363122316600223140ustar00rootroot00000000000000// Copyright (c) 2012, Olaf van der Spek // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: Olaf van der Spek #ifndef TEMPLATE_STR_REF_H_ #define TEMPLATE_STR_REF_H_ #include @ac_windows_dllexport_defines@ namespace ctemplate { template class str_ref_basic { public: str_ref_basic() { clear(); } template str_ref_basic(const U& c) { if (c.end() != c.begin()) assign(&*c.begin(), c.end() - c.begin() + &*c.begin()); else clear(); } str_ref_basic(const void* b, const void* e) { assign(b, e); } str_ref_basic(const void* b, size_t sz) { assign(b, sz); } str_ref_basic(const char* b) { if (b) assign(b, strlen(b)); else clear(); } void clear() { begin_ = end_ = NULL; } void assign(const void* b, const void* e) { begin_ = reinterpret_cast(b); end_ = reinterpret_cast(e); } void assign(const void* b, size_t sz) { begin_ = reinterpret_cast(b); end_ = begin_ + sz; } T begin() const { return begin_; } T end() const { return end_; } T data() const { return begin(); } size_t size() const { return end() - begin(); } bool empty() const { return begin() == end(); } private: T begin_; T end_; }; typedef str_ref_basic data_ref; typedef str_ref_basic str_ref; } #endif // TEMPLATE_STR_REF_H_ ctemplate-ctemplate-2.4/src/ctemplate/template.h.in000066400000000000000000000530061363122316600224560ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // This file implements the Template class. For information about // how to use this class, and to write the templates it takes as input, // see the doc/ directory. #ifndef CTEMPLATE_TEMPLATE_H_ #define CTEMPLATE_TEMPLATE_H_ #include // for time_t #include #include #include #include // We include this just so folks don't have to include both template.h // and template_dictionary.h, or template_namelist.h etc, to use the // template system; we don't actually use anything in these files // ourselves. #if 1 #include #include #include #else namespace ctemplate { class TemplateDictionaryInterface; class PerExpandData; } #endif namespace ctemplate_htmlparser { class HtmlParser; } @ac_windows_dllexport_defines@ namespace ctemplate { // These free functions form the "simple" template API, and support // the most common operations (expanding a template from a file, and // from a string). They all just delegate to a default instance of // the TemplateCache object. // // For more sophisticated use of the template system, you may need // to create your own TemplateCache object, and work directly with // it. See template_cache.h for details. extern @ac_windows_dllexport@ const TemplateCache* default_template_cache(); extern @ac_windows_dllexport@ TemplateCache* mutable_default_template_cache(); // ---- EXPANDING A TEMPLATE ------- // ExpandTemplate // ExpandWithData // Loads the template named filename from disk if necessary -- it // gets it from the cache instead, if the template had been loaded // before or if it had been put explicitly in the cache via a call // to StringToTemplateCache() -- and expands it using the given // dictionary. // The first version is the most general, followed by common-case code. inline bool ExpandTemplate(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface *dictionary, ExpandEmitter* output) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, NULL, output); } inline bool ExpandTemplate(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface* dictionary, std::string* output_buffer) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, NULL, output_buffer); } // If you want any per-expand data to be used at expand time, call // this routine instead of Expand. You pass in an extra // PerExpandData structure (see per_expand_data.h) which sets this // data: whether or not you want the template to be annotated, and // any data you want to pass in to template modifers. If // per_expand_data is NULL, this is exactly the same as Expand(). // The first version is the most general, followed by common-case code. inline bool ExpandWithData(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface *dictionary, PerExpandData* per_expand_data, ExpandEmitter* output) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, per_expand_data, output); } inline bool ExpandWithData(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface* dictionary, PerExpandData* per_expand_data, std::string* output_buffer) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, per_expand_data, output_buffer); } // ---- INSERTING INTO THE CACHE ------- // LoadTemplate // StringToTemplateCache // Reads a file from disk and inserts it into the template, if it's // not already there. Returns true on success or false if the // template could not be found, or could not be parsed. It's never // necessary to call this -- Expand() will load templates lazily if // needed -- but you may want to if you want to make sure templates // exist before trying to expand them, or because you want to // control disk access patterns, or for some other reason. inline bool LoadTemplate(const TemplateString& filename, Strip strip) { return mutable_default_template_cache()->LoadTemplate(filename, strip); } // Inserts the given string into the default template cache, as if // it were a file read from disk. You can call Expand() with its // first arg (filename) the same as the key you use here. You can // also use this key as the 'filename' for sub-included templates, // in TemplateDictionary::SetFilename(). inline bool StringToTemplateCache(const TemplateString& key, const TemplateString& content, Strip strip) { return mutable_default_template_cache()->StringToTemplateCache( key, content, strip); } inline bool StringToTemplateCache(const TemplateString& key, const char* content, size_t content_len, Strip strip) { return mutable_default_template_cache()->StringToTemplateCache( key, content, content_len, strip); } // --------------------------------------------------------------------- // The follow are deprecated. // TODO(csilvers): move to parsed_template.h // TemplateState of a template is: // - TS_EMPTY before parsing is complete, // - TS_ERROR if a syntax error was found during parsing, and // - TS_READY if parsing has completed successfully // (TS_UNUSED is not used) enum TemplateState { TS_UNUSED, TS_EMPTY, TS_ERROR, TS_READY }; // Used for Auto-Escape. It represents the different contexts a template may // be initialized in via the AUTOESCAPE pragma in the template file // (or string). It is only public for testing. The contexts are: // - TC_HTML: The template contains HTML code. Need not be a complete HTML // page just content the browser interprets in the context of // HTML parsing. This should be the most common context to use. // This mode activates our HTML parser. // - TC_JS: The template contains raw javascript. If your template // starts with OR // // For variables that are quoted (i.e. string literals) use javascript_escape. // // Limitations: // . NaN, +/-Infinity and null are not recognized. // . Output is not guaranteed to be a valid literal, // e.g: +55+-e34 will output as is. // e.g: trueeee will output nothing as it is not a valid boolean. // // Details: // . For Hex numbers, it checks for case-insensitive 0x[0-9A-F]+ // that should be a proper check. // . For other numbers, it checks for case-insensitive [0-9eE+-.]* // so can also accept invalid numbers such as the number 5..45--10. // . "true" and "false" (without quotes) are also accepted and that's it. // class @ac_windows_dllexport@ JavascriptNumber : public TemplateModifier { MODIFY_SIGNATURE_; }; extern @ac_windows_dllexport@ JavascriptNumber javascript_number; // Escapes characters not in [0-9a-zA-Z.,_:*/~!()-] as %-prefixed hex. // Space is encoded as a +. class @ac_windows_dllexport@ UrlQueryEscape : public TemplateModifier { MODIFY_SIGNATURE_; }; extern @ac_windows_dllexport@ UrlQueryEscape url_query_escape; // Escapes " \ / to \" \\ \/ \f \r \n \b \t // Also escapes < > & to their corresponding \uXXXX representation // (\u003C, \u003E, \u0026 respectively). class @ac_windows_dllexport@ JsonEscape : public TemplateModifier { MODIFY_SIGNATURE_; }; extern @ac_windows_dllexport@ JsonEscape json_escape; // Inserts the given prefix (given as the argument to this modifier) // after every newline in the text. Note that it does *not* insert // prefix at the very beginning of the text -- in its expected use, // that prefix will already be present before this text, in the // template. This is meant to be used internally, and is not exported // via the g_modifiers list. class @ac_windows_dllexport@ PrefixLine : public TemplateModifier { MODIFY_SIGNATURE_; }; extern @ac_windows_dllexport@ PrefixLine prefix_line; #undef MODIFY_SIGNATURE_ // Registers a new template modifier. // long_name must start with "x-". // If the modifier takes a value (eg "{{VAR:x-name=value}}"), then // long_name should end with "=". This is similar to getopt(3) syntax. // We also allow value-specializations, with specific values specified // as part of long-name. For instance: // AddModifier("x-mod=", &my_modifierA); // AddModifier("x-mod=bar", &my_modifierB); // AddModifier("x-mod2", &my_modifierC); // For the template // {{VAR1:x-mod=foo}} {{VAR2:x-mod=bar}} {{VAR3:x-mod=baz}} {{VAR4:x-mod2}} // VAR1 and VAR3 would get modified by my_modifierA, VAR2 by my_modifierB, // and VAR4 by my_modifierC. The order of the AddModifier calls is not // significant. extern @ac_windows_dllexport@ bool AddModifier(const char* long_name, const TemplateModifier* modifier); // Same as AddModifier() above except that the modifier is considered // to produce safe output that can be inserted in any context without // the need for additional escaping. This difference only impacts // the Auto-Escape mode: In that mode, when a variable (or template-include) // has a modifier added via AddXssSafeModifier(), it is excluded from // further escaping, effectively treated as though it had the :none modifier. // Because Auto-Escape is disabled for any variable and template-include // that includes such a modifier, use this function with care and ensure // that it may not emit harmful output that could lead to XSS. // // Some valid uses of AddXssSafeModifier: // . A modifier that converts a string to an integer since // an integer is generally safe in any context. // . A modifier that returns one of a fixed number of safe values // depending on properties of the input. // // Some not recommended uses of AddXssSafeModifier: // . A modifier that applies some extra formatting to the input // before returning it since the output will still contain // harmful content if the input does. // . A modifier that applies one type of escaping to the input // (say HTML-escape). This may be dangerous when the modifier // is used in a different context (say Javascript) where this // escaping may be inadequate. extern @ac_windows_dllexport@ bool AddXssSafeModifier(const char* long_name, const TemplateModifier* modifier); } #endif // TEMPLATE_TEMPLATE_MODIFIERS_H_ ctemplate-ctemplate-2.4/src/ctemplate/template_namelist.h.in000066400000000000000000000161741363122316600243570ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // This class implements some template helper classes, that manage // template files and make it easier to monitor them. // // For information about how to use these classes and macros, and to // write the templates it takes as input, // see the doc/ directory. #ifndef TEMPLATE_TEMPLATE_NAMELIST_H_ #define TEMPLATE_TEMPLATE_NAMELIST_H_ #include // for time_t #include @ac_cv_cxx_hash_set@ #include #include #include // for Strip #include // for StringHash @ac_windows_dllexport_defines@ namespace ctemplate { // RegisterTemplateFilename // Used to define a reference variable for the name of a template file. E.g: // RegisterTemplateFilename(EXAMPLE_FN, "example.tpl"); // This should be used rather than the seemingly equivalent: // #define EXAMPLE_FN "example.tpl" // As in the latter, any call to GetTemplate may then reference the name // via the first argument. In the example, that would be: // Template::GetTemplate(EXAMPLE_FN, DO_NOT_STRIP); // By using this macro, rather than the #define, all templates can // later be tested for either existence or correct syntax after program // start-up. // TODO (we wish): Make this macro produce the #include for the auto-generated // header files, when and if the macro pre-processor supports that #define RegisterTemplateFilename(var, name) \ const char* const var = ctemplate::TemplateNamelist::RegisterTemplate(name); // Class: TemplateNamelist // Each time this class is instantiated, the name passed to // the constructor is added to the class's static list of names. The // entire list may be retrieved later by calling the GetList accessor // method. Or they may all be checked for existence or for correct // syntax via the other methods. We use this in our // sanity-checking code to make sure all the templates used by a program // exist and are syntactically correct. class @ac_windows_dllexport@ TemplateNamelist { friend class TemporaryRegisterTemplate; public: // These types should be taken as 'generic' containers. The only // thing you should do with them is call size() and/or iterate // between begin() and end(), and the only operations we promise // the iterators will support are operator* and operator++. typedef @ac_cv_cxx_hash_namespace@::unordered_set NameListType; typedef std::vector MissingListType; typedef std::vector SyntaxListType; public: // Takes a name and pushes it onto the static namelist // Returns: a pointer to the entry in namelist which holds the name static const char* RegisterTemplate(const char* name); // GetList // Description: Returns the collected list of names. static const NameListType& GetList(); // GetMissingList // If refresh is true or if it is the first time the function is called // in the execution of the program, it creates (or clears) the missing // list and then fills it with the list of // templates that the program knows about but are missing from // the template directory. // If refresh is false and it is not the first time the function is // called, it merely returns the list created in the // call when the last refresh was done. // NOTE: The templates are NOT read, parsed, or cached // by this function. static const MissingListType& GetMissingList(bool refresh); // GetBadSyntaxList // If refresh is true or if it is the first time the function is called // in the execution of the program, it creates (or clears) the "bad // syntax" list and then fills it with the list of // templates that the program knows about but contain syntax errors. // A missing file is not considered a syntax error, and thus is // not included in this list. // If refresh is false and it is not the first time the function is // called, it merely returns the list created in the // call when the last refresh was done. // NOTE: The side effect of calling this the first time or // with refresh equal true is that all templates are parsed and cached. // Hence they need to be retrieved with the flags that // the program needs them loaded with (i.e, the strip parameter // passed to Template::GetTemplate.) static const SyntaxListType& GetBadSyntaxList(bool refresh, Strip strip); // GetLastmodTime // Iterates through all non-missing templates, and returns the latest // last-modification time for the template files, as returned by stat(). // This can be used to make sure template files are getting refreshed. static time_t GetLastmodTime(); // AllDoExist // Retrieves the missing list (always refreshing the list) // and returns true if it contains any names. // Else, returns false. static bool AllDoExist(); // IsAllSyntaxOkay // Retrieves the "bad syntax" list (always refreshing the list) // and returns true if it contains any names. // Else, returns false. // NOTE: The side effect of calling this is that all templates are parsed // and cached, hence they need to be retrieved with the flags that // the program needs them loaded with. (I.e, the strip parameter // ultimately passed to Template::GetTemplate.) static bool IsAllSyntaxOkay(Strip strip); protected: // The static list of names static NameListType *namelist_; static MissingListType *missing_list_; static SyntaxListType *bad_syntax_list_; private: TemplateNamelist(const TemplateNamelist&); // disallow copying void operator=(const TemplateNamelist&); }; } #endif // TEMPLATE_TEMPLATE_NAMELIST_H_ ctemplate-ctemplate-2.4/src/ctemplate/template_pathops.h.in000066400000000000000000000055661363122316600242240ustar00rootroot00000000000000// Copyright (c) 2007, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) #ifndef TEMPLATE_TEMPLATE_PATHOPS_H_ #define TEMPLATE_TEMPLATE_PATHOPS_H_ #include @ac_windows_dllexport_defines@ namespace ctemplate { extern @ac_windows_dllexport@ const char kCWD[]; // a string that's equivalent to "./" extern @ac_windows_dllexport@ const char kRootdir[]; // a string that's equivalent to "/" extern @ac_windows_dllexport@ std::string PathJoin(const std::string& a, const std::string& b); extern @ac_windows_dllexport@ bool IsAbspath(const std::string& path); extern @ac_windows_dllexport@ bool IsDirectory(const std::string& path); // checks if path ends with / extern @ac_windows_dllexport@ void NormalizeDirectory(std::string* dir); // adds trailing / if needed extern @ac_windows_dllexport@ std::string Basename(const std::string& path); // everything after last / // Returns true iff text contains the word as a full word, i.e. delimited by one // of [.,_-#*?:] on both the sides. // This is used while loading a template, to check that the file's name matches // the auto-escape mode specified by it. // NOTE: This assumes that the word doesn't contain any of the delimiter // characters. extern @ac_windows_dllexport@ bool ContainsFullWord(const std::string& text, const std::string& word); } #endif // TEMPLATE_TEMPLATE_PATHOPS_H_ ctemplate-ctemplate-2.4/src/ctemplate/template_string.h.in000066400000000000000000000320571363122316600240470ustar00rootroot00000000000000// Copyright (c) 2008, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // // --- // Author: csilvers@google.com (Craig Silerstein) #ifndef TEMPLATE_TEMPLATE_STRING_H_ #define TEMPLATE_TEMPLATE_STRING_H_ #include // for memcmp() and size_t #include @ac_cv_cxx_hash_map@ #include #include #include #include class TemplateStringTest; // needed for friendship declaration class StaticTemplateStringTest; #if @ac_have_attribute_weak@ extern char _start[] __attribute__((weak)); // linker emits: start of .text extern char data_start[] __attribute__((weak)); // start of .data #endif @ac_windows_dllexport_defines@ namespace ctemplate { // Most methods of TemplateDictionary take a TemplateString rather than a // C++ string. This is for efficiency: it can avoid extra string copies. // For any argument that takes a TemplateString, you can pass in any of: // * A C++ string // * A char* // * A StringPiece // * TemplateString(char*, length) // The last of these is the most efficient, though it requires more work // on the call site (you have to create the TemplateString explicitly). class TemplateString; // If you have a string constant (e.g. the string literal "foo") that // you need to pass into template routines repeatedly, it is more // efficient if you convert it into a TemplateString only once. The // way to do this is to use a global StaticTemplateString via STS_INIT // (note: do this at global scope *only*!): // static const StaticTemplateString kMyVar = STS_INIT(kMyVar, "MY_VALUE"); struct StaticTemplateString; #define STS_INIT(name, str) STS_INIT_WITH_HASH(name, str, 0) // Let's define a convenient hash function for hashing 'normal' // strings: char* and string. We'll use MurmurHash, which is probably // better than the STL default. We don't include TemplateString or // StaticTemplateString here, since they are hashed more efficiently // based on their id. struct @ac_windows_dllexport@ StringHash { inline size_t operator()(const char* s) const { return Hash(s, strlen(s)); }; inline size_t operator()(const std::string& s) const { return Hash(s.data(), s.size()); } inline bool operator()(const char* a, const char* b) const { return (a != b) && (strcmp(a, b) < 0); // <, for MSVC } inline bool operator()(const std::string& a, const std::string& b) const { return a < b; } static const size_t bucket_size = 4; // These are required by MSVC static const size_t min_buckets = 8; // 4 and 8 are the defaults private: size_t Hash(const char* s, size_t slen) const; }; // ----------------------- THE CLASSES ------------------------------- typedef uint64_t TemplateId; const TemplateId kIllegalTemplateId = 0; struct @ac_windows_dllexport@ StaticTemplateString { // Do not define a constructor! We use only brace-initialization, // so the data is constructed at static-initialization time. // Anything you want to put in a constructor, put in // StaticTemplateStringInitializer instead. // These members shouldn't be accessed directly, except in the // internals of the template code. They are public because that is // the only way we can brace-initialize them. struct { const char* ptr_; size_t length_; mutable TemplateId id_; // sometimes lazily-initialized. } do_not_use_directly_; // This class is a good hash functor to pass in as the third // argument to unordered_map<>, when creating a map whose keys are // StaticTemplateString. NOTE: This class isn't that safe to use, // because it requires that StaticTemplateStringInitializer has done // its job. Unfortunately, even when you use the STS_INIT macro // (which is always, right??), dynamic initialiation does not happen // in a particular order, and objects in different .cc files may // reference a StaticTemplateString before the corresponding // StaticTemplateStringInitializer sets the id. struct Hasher { inline size_t operator()(const StaticTemplateString& sts) const; inline bool operator()(const StaticTemplateString& a, // <, for MSVC const StaticTemplateString& b) const; static const size_t bucket_size = 4; // These are required by MSVC static const size_t min_buckets = 8; // 4 and 8 are the defaults }; inline bool empty() const { return do_not_use_directly_.length_ == 0; } // Allows comparisons of StaticTemplateString objects as if they were // strings. This is useful for STL. inline bool operator==(const StaticTemplateString& x) const; }; class @ac_windows_dllexport@ TemplateString { public: TemplateString(const char* s) : ptr_(s ? s : ""), length_(strlen(ptr_)), is_immutable_(InTextSegment(ptr_)), id_(kIllegalTemplateId) { } TemplateString(const std::string& s) : ptr_(s.data()), length_(s.size()), is_immutable_(false), id_(kIllegalTemplateId) { } TemplateString(const char* s, size_t slen) : ptr_(s), length_(slen), is_immutable_(InTextSegment(s)), id_(kIllegalTemplateId) { } TemplateString(const StaticTemplateString& s) : ptr_(s.do_not_use_directly_.ptr_), length_(s.do_not_use_directly_.length_), is_immutable_(true), id_(s.do_not_use_directly_.id_) { } const char* begin() const { return ptr_; } const char* end() const { return ptr_ + length_; } const char* data() const { return ptr_; } size_t size() const { return length_; } inline bool empty() const { return length_ == 0; }; inline bool is_immutable() const { return is_immutable_; } // STL requires this to be public for hash_map, though I'd rather not. inline bool operator==(const TemplateString& x) const { return GetGlobalId() == x.GetGlobalId(); } private: // Only TemplateDictionaries and template expansion code can read these. friend class TemplateDictionary; friend class TemplateCache; // for GetGlobalId friend class StaticTemplateStringInitializer; // for AddToGlo... friend struct TemplateStringHasher; // for GetGlobalId friend TemplateId GlobalIdForTest(const char* ptr, int len); friend TemplateId GlobalIdForSTS_INIT(const TemplateString& s); TemplateString(const char* s, size_t slen, bool is_immutable, TemplateId id) : ptr_(s), length_(slen), is_immutable_(is_immutable), id_(id) { } // This returns true if s is in the .text segment of the binary. // (Note this only checks .text of the main executable, not of // shared libraries. So it may not be all that useful.) // This requires the gnu linker (and probably elf), to define // _start and data_start. static bool InTextSegment(const char* s) { #if @ac_have_attribute_weak@ return (s >= _start && s < data_start); // in .text #else return false; // the conservative choice: assume it's not static memory #endif } protected: inline void CacheGlobalId() { // used by HashedTemplateString id_ = GetGlobalId(); }; private: // Returns the global id, computing it for the first time if // necessary. Note that since this is a const method, we don't // store the computed value in id_, even if id_ is 0. TemplateId GetGlobalId() const; // Adds this TemplateString to the map from global-id to name. void AddToGlobalIdToNameMap(); // Use sparingly. Converting to a string loses information about the // id of the template string, making operations require extra hash // computations. std::string ToString() const { return std::string(ptr_, length_); } // Does the reverse map from TemplateId to TemplateString contents. // Returns a TemplateString(kStsEmpty) if id isn't found. Note that // the TemplateString returned is not necessarily NUL terminated. static TemplateString IdToString(TemplateId id); const char* ptr_; size_t length_; // Do we need to manage memory for this string? bool is_immutable_; // Id for hash lookups. If 0, we don't have one and it should be // computed as-needed. TemplateId id_; }; // ----------------------- THE CODE ------------------------------- // Use the low-bit from TemplateId as the "initialized" flag. Note // that since all initialized TemplateId have the lower bit set, it's // safe to have used 0 for kIllegalTemplateId, as we did above. const TemplateId kTemplateStringInitializedFlag = 1; inline bool IsTemplateIdInitialized(TemplateId id) { return id & kTemplateStringInitializedFlag; } // This is a helper struct used in TemplateString::Hasher/TemplateStringHasher struct TemplateIdHasher { size_t operator()(TemplateId id) const { // The shift has two effects: it randomizes the "initialized" flag, // and slightly improves the randomness of the low bits. This is // slightly useful when size_t is 32 bits, or when using a small // hash tables with power-of-2 sizes. return static_cast(id ^ (id >> 33)); } bool operator()(TemplateId a, TemplateId b) const { // <, for MSVC return a < b; } static const size_t bucket_size = 4; // These are required by MSVC static const size_t min_buckets = 8; // 4 and 8 are the defaults }; inline size_t StaticTemplateString::Hasher::operator()( const StaticTemplateString& sts) const { TemplateId id = sts.do_not_use_directly_.id_; assert(IsTemplateIdInitialized(id)); return TemplateIdHasher()(id); } inline bool StaticTemplateString::Hasher::operator()( const StaticTemplateString& a, const StaticTemplateString& b) const { TemplateId id_a = a.do_not_use_directly_.id_; TemplateId id_b = b.do_not_use_directly_.id_; assert(IsTemplateIdInitialized(id_a)); assert(IsTemplateIdInitialized(id_b)); return TemplateIdHasher()(id_a, id_b); } inline bool StaticTemplateString::operator==( const StaticTemplateString& x) const { return (do_not_use_directly_.length_ == x.do_not_use_directly_.length_ && (do_not_use_directly_.ptr_ == x.do_not_use_directly_.ptr_ || memcmp(do_not_use_directly_.ptr_, x.do_not_use_directly_.ptr_, do_not_use_directly_.length_) == 0)); } // We set up as much of StaticTemplateString as we can at // static-initialization time (using brace-initialization), but some // things can't be set up then. This class is for those things; it // runs at dynamic-initialization time. If you add logic here, only // do so as an optimization: this may be called rather late (though // before main), so other code should not depend on this being called // before them. class @ac_windows_dllexport@ StaticTemplateStringInitializer { public: // This constructor operates on a const StaticTemplateString - we should // only change those things that are mutable. explicit StaticTemplateStringInitializer(const StaticTemplateString* sts); }; // Don't use this. This is used only in auto-generated .varnames.h files. #define STS_INIT_WITH_HASH(name, str, hash) \ { { str, sizeof("" str "")-1, hash } }; \ namespace ctemplate_sts_init { \ static const ctemplate::StaticTemplateStringInitializer name##_init(&name); \ } // We computed this hash value for the empty string online. In debug // mode, we verify it's correct during runtime (that is, that we // verify the hash function used by make_tpl_varnames_h hasn't changed // since we computed this number). Note this struct is logically // static, but since it's in a .h file, we don't say 'static' but // instead rely on the linker to provide the POD-with-internal-linkage // magic. const StaticTemplateString kStsEmpty = STS_INIT_WITH_HASH(kStsEmpty, "", 1457976849674613049ULL); } #endif // TEMPLATE_TEMPLATE_STRING_H_ ctemplate-ctemplate-2.4/src/diff_tpl_auto_escape.cc000077500000000000000000000321541363122316600225610ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Heavily inspired from make_tpl_varnames_h.cc // // A utility for evaluating the changes in escaping modifiers // applied to variables between two versions of a template file. // This may come in handy when converting a template to Auto-Escape: // If the template previously had escaping modifiers, this tool will show // the variables for which Auto-Escaped determined a different escaping. // // How it works: // . You provide two template files, assumed to be identical in content // (same variables in the same order) except for escaping modifiers // and possibly the AUTOESCAPE pragma. You also provide the Strip mode // or a default of STRIP_WHITESPACE is assumed. // // . The tool loads both files and invokes DumpToString on both. It then // compares the escaping modifiers for each variable and when they do // not match, it prints a line with the variable name as well as // the differing modifiers. // // . We accept some command-line flags, the most notable are: // --template_dir to set a template root directory other than cwd // --strip to set the Strip mode to other than STRIP_WHITESPACE. // For correct operation of Auto-Escape, ensure this matches // the Strip mode you normally use on these templates. // // // Exit code is zero if there were no differences. It is non-zero // if we failed to load the templates or we found one or more // differences. // // TODO(jad): Add flag to optionally report differences when a variable // does not have modifiers in either template. // This is for opensource ctemplate on windows. Even though we // #include config.h, just like the files used to compile the dll, we // are actually a *client* of the dll, so we don't get to decl anything. #include #undef CTEMPLATE_DLL_DECL #include #include #ifdef HAVE_UNISTD_H # include #endif #include #ifdef HAVE_GETOPT_H # include #endif #include #include #include #include using std::string; using std::vector; using ctemplate::Template; using ctemplate::TemplateContext; using ctemplate::Strip; using ctemplate::STRIP_WHITESPACE; using ctemplate::STRIP_BLANK_LINES; using ctemplate::DO_NOT_STRIP; enum {LOG_VERBOSE, LOG_INFO, LOG_WARNING, LOG_ERROR, LOG_FATAL}; // A variable name and optional modifiers. // For example: in {{NAME:j:x-bla}} // variable_name is "NAME" and modifiers is "j:x-bla". struct VariableAndMod { VariableAndMod(string name, string mods) : variable_name(name), modifiers(mods) { } string variable_name; string modifiers; }; typedef vector VariableAndMods; static string FLAG_template_dir(ctemplate::kCWD); // "./" static string FLAG_strip = ""; // cmd-line arg -s static bool FLAG_verbose = false; // cmd-line arg -v static void LogPrintf(int severity, const char* pat, ...) { if (severity == LOG_VERBOSE && !FLAG_verbose) return; if (severity == LOG_FATAL) fprintf(stderr, "FATAL ERROR: "); if (severity == LOG_VERBOSE) fprintf(stdout, "[VERBOSE] "); va_list ap; va_start(ap, pat); vfprintf(severity == LOG_INFO || severity == LOG_VERBOSE ? stdout: stderr, pat, ap); va_end(ap); if (severity == LOG_FATAL) exit(1); } // Prints to outfile -- usually stdout or stderr -- and then exits static int Usage(const char* argv0, FILE* outfile) { fprintf(outfile, "USAGE: %s [-t] [-v] [-b] [-s] \n", argv0); fprintf(outfile, " -t --template_dir= Root directory of templates\n" " -s --strip= STRIP_WHITESPACE [default],\n" " STRIP_BLANK_LINES, DO_NOT_STRIP\n" " -h --help This help\n" " -v --verbose For a bit more output\n" " -V --version Version information\n"); fprintf(outfile, "\n" "This program reports changes to modifiers between two template\n" "files assumed to be identical except for modifiers applied\n" "to variables. One use case is converting a template to\n" "Auto-Escape and using this program to obtain the resulting\n" "changes in escaping modifiers.\n" "The Strip value should match what you provide in\n" "Template::GetTemplate.\n" "NOTE: Variables that do not have escaping modifiers in one of\n" "two templates are ignored and do not count in the differences.\n"); exit(0); } static int Version(FILE* outfile) { fprintf(outfile, "diff_tpl_auto_escape (part of google-template 0.9x)\n" "\n" "Copyright 2008 Google Inc.\n" "\n" "This is BSD licensed software; see the source for copying conditions\n" "and license information.\n" "There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A\n" "PARTICULAR PURPOSE.\n" ); exit(0); } // Populates the vector of VariableAndMods from the DumpToString // representation of the template file. // // Each VariableAndMod represents a variable node found in the template // along with the optional modifiers attached to it (or empty string). // The parsing is very simple. It looks for lines of the form: // "Variable Node: [:]\n" // as outputted by DumpToString() and extracts from each such line the // variable name and modifiers when present. // Because DumpToString also outputs text nodes, it is possible // to trip this function. Probably ok since this is just a helper tool. bool LoadVariables(const char* filename, Strip strip, VariableAndMods& vars_and_mods) { const string kVariablePreambleText = "Variable Node: "; Template *tpl; tpl = Template::GetTemplate(filename, strip); if (tpl == NULL) { LogPrintf(LOG_FATAL, "Could not load file: %s\n", filename); return false; } string output; tpl->DumpToString(filename, &output); string::size_type index = 0; string::size_type delim, end; // TODO(jad): Switch to using regular expressions. while((index = output.find(kVariablePreambleText, index)) != string::npos) { index += kVariablePreambleText.length(); end = output.find('\n', index); if (end == string::npos) { // Should never happen but no need to LOG_FATAL. LogPrintf(LOG_ERROR, "%s: Did not find terminating newline...\n", filename); end = output.length(); } string name_and_mods = output.substr(index, end - index); delim = name_and_mods.find(":"); if (delim == string::npos) // no modifiers. delim = name_and_mods.length(); VariableAndMod var_mod(name_and_mods.substr(0, delim), name_and_mods.substr(delim)); vars_and_mods.push_back(var_mod); } return true; } // Returns true if the difference in the modifier strings // is non-significant and can be safely omitted. This is the // case when one is ":j:h" and the other is ":j" since // the :h is a no-op after a :j. bool SuppressLameDiff(string modifiers_a, string modifiers_b) { if ((modifiers_a == ":j:h" && modifiers_b == ":j") || (modifiers_a == ":j" && modifiers_b == ":j:h")) return true; return false; } // Main function to analyze differences in escaping modifiers between // two template files. These files are assumed to be identical in // content [strictly speaking: same number of variables in the same order]. // If that is not the case, we fail. // We return true if there were no differences, false if we failed // or we found one or more differences. bool DiffTemplates(const char* filename_a, const char* filename_b, Strip strip) { vector vars_and_mods_a, vars_and_mods_b; if (!LoadVariables(filename_a, strip, vars_and_mods_a) || !LoadVariables(filename_b, strip, vars_and_mods_b)) return false; if (vars_and_mods_a.size() != vars_and_mods_b.size()) LogPrintf(LOG_FATAL, "Templates differ: %s [%d vars] vs. %s [%d vars].\n", filename_a, vars_and_mods_a.size(), filename_b, vars_and_mods_b.size()); int mismatch_count = 0; // How many differences there were. int no_modifiers_count = 0; // How many variables without modifiers. VariableAndMods::const_iterator iter_a, iter_b; for (iter_a = vars_and_mods_a.begin(), iter_b = vars_and_mods_b.begin(); iter_a != vars_and_mods_a.end() && iter_b != vars_and_mods_b.end(); ++iter_a, ++iter_b) { // The templates have different variables, we fail! if (iter_a->variable_name != iter_b->variable_name) LogPrintf(LOG_FATAL, "Variable name mismatch: %s vs. %s\n", iter_a->variable_name.c_str(), iter_b->variable_name.c_str()); // Variables without modifiers are ignored from the diff. They simply // get counted and the count is shown in verbose logging/ if (iter_a->modifiers == "" || iter_b->modifiers == "") { no_modifiers_count++; } else { if (iter_a->modifiers != iter_b->modifiers && !SuppressLameDiff(iter_a->modifiers, iter_b->modifiers)) { mismatch_count++; LogPrintf(LOG_INFO, "Difference for variable %s -- %s vs. %s\n", iter_a->variable_name.c_str(), iter_a->modifiers.c_str(), iter_b->modifiers.c_str()); } } } LogPrintf(LOG_VERBOSE, "Variables Found: Total=%d; Diffs=%d; NoMods=%d\n", vars_and_mods_a.size(), mismatch_count, no_modifiers_count); return (mismatch_count == 0); } int main(int argc, char **argv) { #if defined(HAVE_GETOPT_LONG) static struct option longopts[] = { {"help", 0, NULL, 'h'}, {"strip", 1, NULL, 's'}, {"template_dir", 1, NULL, 't'}, {"verbose", 0, NULL, 'v'}, {"version", 0, NULL, 'V'}, {0, 0, 0, 0} }; int option_index; # define GETOPT(argc, argv) getopt_long(argc, argv, "t:s:hvV", \ longopts, &option_index) #elif defined(HAVE_GETOPT_H) # define GETOPT(argc, argv) getopt(argc, argv, "t:s:hvV") #else // TODO(csilvers): implement something reasonable for windows/etc # define GETOPT(argc, argv) -1 int optind = 1; // first non-opt argument const char* optarg = ""; // not used #endif int r = 0; while (r != -1) { // getopt()/getopt_long() return -1 upon no-more-input r = GETOPT(argc, argv); switch (r) { case 's': FLAG_strip.assign(optarg); break; case 't': FLAG_template_dir.assign(optarg); break; case 'v': FLAG_verbose = true; break; case 'V': Version(stdout); break; case -1: break; // means 'no more input' default: Usage(argv[0], stderr); } } Template::SetTemplateRootDirectory(FLAG_template_dir); if (argc != (optind + 2)) LogPrintf(LOG_FATAL, "Must specify exactly two template files on the command line.\n"); // Validate the Strip value. Default is STRIP_WHITESPACE. Strip strip = STRIP_WHITESPACE; // To avoid compiler warnings. if (FLAG_strip == "STRIP_WHITESPACE" || FLAG_strip == "") strip = STRIP_WHITESPACE; else if (FLAG_strip == "STRIP_BLANK_LINES") strip = STRIP_BLANK_LINES; else if (FLAG_strip == "DO_NOT_STRIP") strip = DO_NOT_STRIP; else LogPrintf(LOG_FATAL, "Unrecognized Strip: %s. Must be one of: " "STRIP_WHITESPACE, STRIP_BLANK_LINES or DO_NOT_STRIP\n", FLAG_strip.c_str()); const char* filename_a = argv[optind]; const char* filename_b = argv[optind + 1]; LogPrintf(LOG_VERBOSE, "------ Diff of [%s, %s] ------\n", filename_a, filename_b); if (DiffTemplates(filename_a, filename_b, strip)) return 0; else return 1; } ctemplate-ctemplate-2.4/src/htmlparser/000077500000000000000000000000001363122316600202645ustar00rootroot00000000000000ctemplate-ctemplate-2.4/src/htmlparser/README000066400000000000000000000014471363122316600211520ustar00rootroot00000000000000This directory contains an implementation of an html context scanner with no lookahead. Its purpose is to scan an html stream and provide context information at any point within the input stream. This is used in the auto escaping functionality of the templating system, which would require html context information at very specific points within the html stream. The implementation is based on a simplified state machine of HTML4.1. The implementation is written in C, with the idea of making is usable in other contexts besides this html paresr; however, we compile all the files as c++ files in order to be able to use a namespace (and avoid global namespace pollution). There's also a c++ shim to give an object-oriented look at the html parser state, which is what the ctemplate system actually uses. ctemplate-ctemplate-2.4/src/htmlparser/fsm_config.py000077500000000000000000000155021363122316600227560ustar00rootroot00000000000000# Copyright (c) 2008, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # # Create a state machine object based on a definition file. # __author__ = 'falmeida@google.com (Filipe Almeida)' class OrderedDict: """Ordered dictionary implementation.""" # Define the minimum functionality we need for our application. # Easiser would be to subclass from UserDict.DictMixin, and only # define __getitem__, __setitem__, __delitem__, and keys, but that's # not as portable. We don't need to define much more, so we just do. def __init__(self): self._dict = {} self._keys = [] def __getitem__(self, key): return self._dict[key] def __setitem__(self, key, value): if key not in self._keys: self._keys.append(key) self._dict[key] = value def __delitem__(self, key): self._keys.remove(key) del self._dict[key] def keys(self): return self._keys # Below are all we have to define in addition to what DictMixin would need def __len__(self): return len(self.keys()) def __contains__(self, key): return self.has_key(key) def __iter__(self): # It's not as portable -- though it would be more space-efficient -- to do # for k in self.keys(): yield k return iter(self.keys()) class State(object): """Contains information about a specific state.""" def __init__(self): pass name = None external_name = None transitions = [] class Transition(object): """Contains information about a specific transition.""" def __init__(self, condition, source, destination): self.condition = condition self.source = source self.destination = destination class FSMConfig(object): """Container for the statemachine definition.""" sm = {} # dictionary that contains the finite state machine definition # loaded from a config file. transitions = [] # List of transitions. conditions = {} # Mapping between the condition name and the bracket # expression. states = OrderedDict() # Ordered dictionary of states. name = None comment = None def AddState(self, **dic): """Called from the definition file with the description of the state. Receives a dictionary and populates internal structures based on it. The dictionary is in the following format: {'name': state_name, 'external': exposed state name, 'transitions': [ [condition, destination_state ], [condition, destination_state ] ] } """ state = State() state.name = dic['name'] state.external_name = dic['external'] state_transitions = [] for (condition, destination) in dic['transitions']: transition = Transition(condition, state.name, destination) state_transitions.append(transition) self.transitions.extend(state_transitions) state.transitions = state_transitions self.states[state.name] = state def AddCondition(self, name, expression): """Called from the definition file with the definition of a condition. Receives the name of the condition and it's expression. """ self.conditions[name] = expression def Load(self, filename): """Load the state machine definition file. In the definition file, which is based on the python syntax, the following variables and functions are defined. name: Name of the state machine comment: Comment line on the generated file. condition(): A mapping between condition names and bracket expressions. state(): Defines a state and it's transitions. It accepts the following attributes: name: name of the state external: exported name of the state. The exported name can be used multiple times in order to create a super state. transitions: List of pairs containing the condition for the transition and the destination state. Transitions are ordered so if a default rule is used, it must be the last one in the list. Example: name = 'c comment parser' condition('/', '/') condition('*', '*') condition('linefeed', '\\n') condition('default', '[:default:]') state(name = 'text', external = 'comment', transitions = [ [ '/', 'comment_start' ], [ 'default', 'text' ] ]) state(name = 'comment_start', external = 'comment', transitions = [ [ '/', 'comment_line' ], [ '*', 'comment_multiline' ], [ 'default', 'text' ] ]) state(name = 'comment_line', external = 'comment', transitions = [ [ 'linefeed', 'text' ], [ 'default', 'comment_line' ] ]) state(name = 'comment_multiline', external = 'comment', transitions = [ [ '*', 'comment_multiline_close' ], [ 'default', 'comment_multiline' ] ]) state(name = 'comment_multiline_close', external = 'comment', transitions = [ [ '/', 'text' ], [ 'default', 'comment_multiline' ] ]) """ self.sm['state'] = self.AddState self.sm['condition'] = self.AddCondition exec(open(filename).read(), self.sm) self.name = self.sm['name'] if not self.name.isalnum(): raise Exception("State machine name must consist of only alphanumeric" "characters.") self.comment = self.sm['comment'] def __init__(self): pass ctemplate-ctemplate-2.4/src/htmlparser/generate_fsm.py000077500000000000000000000247711363122316600233130ustar00rootroot00000000000000#!/usr/bin/env python3 # # Copyright (c) 2008, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # # Generate a C include file from a finite state machine definition. # # Right now the form is the one expected by htmlparser.c so this file is pretty # tightly coupled with htmlparser.c. # __author__ = 'falmeida@google.com (Filipe Almeida)' import sys from fsm_config import FSMConfig class FSMGenerateAbstract(object): def __init__(self, config): self._config = config def Generate(self): """Returns the generated FSM description for the specified language. Raises a TypeError, because abstract methods can not be called. Raises: TypeError """ raise TypeError('Abstract method %s.%s called' % (self._class.__name__, self._function)) class FSMGenerateC(FSMGenerateAbstract): """Generate the C definition from a statemachien configuration object.""" TABSTOP_ = 2 def _Prefix(self): """Return a c declaration prefix.""" return self._config.name.lower() + '_' def _StateInternalC(self, st): """Return the internal name of the state.""" return '%sSTATE_INT_%s' % (self._Prefix().upper(), st.upper()) def _StateExternalC(self, st): """Return the external name of the state.""" return '%sSTATE_%s' % (self._Prefix().upper(), st.upper()) def _MakeTuple(self, data): """Converts data to a string representation of a C tuple.""" return '{ %s }' % ', '.join(data) def _CreateHeader(self): """Print the include file header.""" out = [] if self._config.comment: out.append('/* ' + self._config.comment) else: out.append('/* State machine definition for ' + self._config.name) out.append(' * Auto generated by generate_fsm.py. Please do not edit.') out.append(' */') return '\n'.join(out) def _ListToIndentedString(self, list): indented_list = [' ' + e for e in list] return ',\n'.join(indented_list) def _CreateEnum(self, name, data): """Print a c enum definition.""" return 'enum %s {\n%s\n};\n' % (name, self._ListToIndentedString(data)) def _CreateStructList(self, name, type, data): """Print a c flat list. Generic function to print list in c in the form of a struct. Args: name: name of the structure. type: type of the struct. data: contents of the struct as a list of elements Returns: String with the generated list. """ return "static const %s %s[] = {\n%s\n};\n" % ( type, name, self._ListToIndentedString(data)) def _CreateStatesEnum(self): """Print the internal states enum. Prints an enum containing all the valid states. Returns: String containing a C enumeration of the states. """ list = [] # output list for state in self._config.states: list.append(self._StateInternalC(state)) return self._CreateEnum(self._Prefix() + 'state_internal_enum', list) def _CreateStatesExternal(self): """Print a struct with a mapping from internal to external states.""" list = [] # output list for state_name in self._config.states: list.append(self._StateExternalC( self._config.states[state_name].external_name)) return self._CreateStructList(self._Prefix() + 'states_external', 'int', list) def _CreateStatesInternalNames(self): """Return a struct mapping internal states to a strings.""" out = [] # output list for state_name in self._config.states: out.append('"' + state_name + '"') return self._CreateStructList(self._Prefix() + 'states_internal_names', 'char *', out) def _CreateNumStates(self): """Print a Macro defining the number of states.""" return "#define %s_NUM_STATES %s" % (self._config.name.upper(), str(len(self._config.states) + 1)) def _ExpandBracketExpression(self, expression): """Expand ranges in a regexp bracket expression. Returns a string with the ranges in a bracket expression expanded. The bracket expression is similar to grep(1) or regular expression bracket expressions but it does not support the negation (^) modifier or named character classes like [:alpha:] or [:alnum:]. The especial character class [:default:] will expand to all elements in the ascii range. For example, the expression 'a-c13A-D' will expand to 'abc13ABCD'. Args: expression: A regexp bracket expression. Ie: 'A-Z0-9'. Returns: A string with the ranges in the bracket expression expanded. """ def ExpandRange(start, end): """Return a sequence of characters between start and end. Args: start: first character of the sequence. end: last character of the sequence. Returns: string containing the sequence of characters between start and end. """ return [chr(c) for c in range(ord(start), ord(end) + 1)] def ListNext(input_list): """Pop the first element of a list. Args: input_list: python list object. Returns: First element of the list or None if the list is empty. """ if input_list: return input_list.pop(0) else: return None out = [] # List containing the output # Special case for the character class [:default:] if expression == '[:default:]': out = [chr(c) for c in range(0, 255)] return ''.join(out) chars = [c for c in expression] # list o characters in the expression. current = ListNext(chars) while current: next = ListNext(chars) if next == '-': next = ListNext(chars) if next: out.extend(ExpandRange(current, next)) else: out.append(current) out.append('-') current = ListNext(chars) else: out.append(current) current = next return ''.join(out) def _CreateTransitionTable(self): """Print the state transition list. Returns a set of C structures that define the transition table for the state machine. This structure is a list of lists of ints (int **). The outer list indexes the source state and the inner list contains the destination state for each of the possible input characters: const int * const* transitions[source][input] == destination. The conditions are mapped from the conditions variable. Returns: String containing the generated transition table in a C struct. """ out = [] # output list default_state = 'STATEMACHINE_ERROR' state_table = {} for state in self._config.states: state_table[state] = [default_state for col in range(255)] # We process the transition in reverse order while updating the table. for i_transition in range(len(self._config.transitions) - 1, -1, -1): transition = self._config.transitions[i_transition] (condition_name, src, dst) = (transition.condition, transition.source, transition.destination) condition = self._config.conditions[condition_name] char_list = self._ExpandBracketExpression(condition) for c in char_list: state_table[src][ord(c)] = self._StateInternalC(dst) # Create the inner lists which map input characters to destination states. for state in self._config.states: transition_row = [] for c in range(0, 255): ch_repr = str(repr(chr(c)).encode("ASCII", "backslashreplace"), "ASCII") transition_row.append(' /* %06s */ %s' % (ch_repr, state_table[state][c])) out.append(self._CreateStructList('%stransition_row_%s' % (self._Prefix(), state), 'int', transition_row)) out.append('\n') # Create the outer list, which map source states to input characters. out.append('static const %s %s[] = {\n' % ('int *', self._Prefix() + 'state_transitions')) row_list = [' %stransition_row_%s' % (self._Prefix(), row) for row in self._config.states] out.append(',\n'.join(row_list)) out.append('\n};\n') return ''.join(out) def Generate(self): """Returns the generated the C include statements for the statemachine.""" print('\n'.join((self._CreateHeader(), self._CreateNumStates(), self._CreateStatesEnum(), self._CreateStatesExternal(), self._CreateStatesInternalNames(), self._CreateTransitionTable()))) def main(): if len(sys.argv) != 2: print("usage: generate_fsm.py config_file") sys.exit(1) config = FSMConfig() config.Load(sys.argv[1]) gen = FSMGenerateC(config) gen.Generate() if __name__ == "__main__": main() ctemplate-ctemplate-2.4/src/htmlparser/htmlparser.cc000066400000000000000000001000171363122316600227530ustar00rootroot00000000000000/* * Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * --- * * Author: falmeida@google.com (Filipe Almeida) */ /* TODO(falmeida): Breaks on NULL characters in the stream. fix. */ #include #include #include #include #include #include "htmlparser/statemachine.h" #include "htmlparser/htmlparser.h" #include "htmlparser/jsparser.h" /* So we can support both C and C++ compilers, we use the CAST() macro instead * of using C style casts or static_cast<>() directly. */ #ifdef __cplusplus #define CAST(type, expression) (static_cast(expression)) #else #define CAST(type, expression) ((type)(expression)) #endif #ifdef __cplusplus namespace ctemplate_htmlparser { #endif /* Generated state machine definition. */ #include "htmlparser/htmlparser_fsm.h" #define is_js_attribute(attr) ((attr)[0] == 'o' && (attr)[1] == 'n') #define is_style_attribute(attr) (strcmp((attr), "style") == 0) /* html entity filter */ static struct entityfilter_table_s { const char *entity; const char *value; } entityfilter_table[] = { { "lt", "<" }, { "gt", ">" }, { "quot", "\"" }, { "amp", "&" }, { "apos", "\'" }, { NULL, NULL } }; /* Utility functions */ /* Similar to strncpy() but avoids the NULL padding. */ static inline void nopad_strncpy(char *dst, const char *src, size_t dst_size, size_t src_size) { size_t size; /* size = min(dst_size, src_size) */ size = dst_size > src_size ? src_size : dst_size; strncpy(dst, src, size); if (size > 0) dst[size - 1] = '\0'; } /* Converts the internal state into the external superstate. */ static int state_external(int st) { if (st == STATEMACHINE_ERROR) return HTMLPARSER_STATE_ERROR; else return htmlparser_states_external[st]; } /* Returns true if the character is considered an html whitespace character. * * From: http://www.w3.org/TR/html401/struct/text.html#h-9.1 */ static inline int html_isspace(char chr) { if (chr == ' ' || chr == '\t' || chr == '\n' || chr == '\r') { return 1; } else { return 0; } } /* Returns true if the attribute is expected to contain a url * This list was taken from: http://www.w3.org/TR/html4/index/attributes.html */ static int is_uri_attribute(char *attr) { if (attr == NULL) return 0; switch (attr[0]) { case 'a': if (strcmp(attr, "action") == 0) return 1; /* TODO(falmeida): This is a uri list. Should we treat it diferently? */ if (strcmp(attr, "archive") == 0) /* This is a uri list */ return 1; break; case 'b': if (strcmp(attr, "background") == 0) return 1; break; case 'c': if (strcmp(attr, "cite") == 0) return 1; if (strcmp(attr, "classid") == 0) return 1; if (strcmp(attr, "codebase") == 0) return 1; break; case 'd': if (strcmp(attr, "data") == 0) return 1; if (strcmp(attr, "dynsrc") == 0) /* from msdn */ return 1; break; case 'h': if (strcmp(attr, "href") == 0) return 1; break; case 'l': if (strcmp(attr, "longdesc") == 0) return 1; break; case 's': if (strcmp(attr, "src") == 0) return 1; break; case 'u': if (strcmp(attr, "usemap") == 0) return 1; break; } return 0; } /* Convert a string to lower case characters inplace. */ static void tolower_str(char *s) { while (*s != '\0') { *s = CAST(char, tolower(CAST(unsigned char,*s))); s++; } } static const char *ignore_spaces_or_digits(const char *value) { while (html_isspace(*value) || ((*value >= '0' && *value <= '9'))) value++; return value; } static const char *ignore_spaces(const char *value) { while (html_isspace(*value)) value++; return value; } /* Return type of the function meta_redirect_type. */ enum meta_redirect_type_enum { META_REDIRECT_TYPE_NONE, META_REDIRECT_TYPE_URL_START, META_REDIRECT_TYPE_URL }; /* Analyzes a string for the presence of a meta refresh type url. * * This function receives the value of the content attribute of a meta tag and * parses it in order to identify if a url is going to be present. This is the * format of such tag: * * * * Using a regular expression library would be the most obvious way to implement * this functionality, but introducing such a dependency is undesirable. We * opted instead to parse programmaticly since the expression is simple enough. * * For reference, this is the spec on the meta http refresh tag: * http://dev.w3.org/html5/spec/Overview.html#attr-meta-http-equiv-refresh * * If the value has no content after the expression, we know we are at the start * of the URL. Otherwise we are past the start of the URL. * * * Returns: * * This functions returns one of the following values: * META_REDIRECT_TYPE_NONE - A url was not identified in the input string. * META_REDIRECT_TYPE_URL_START - The input string ends exactly at the start * of the url. * META_REDIRECT_TYPE_URL - The input string ends somewhere in the middle or * the end of the url. * * A few examples: * "5" * Returns META_REDIRECT_TYPE_NONE since we don't expect a url to follow. * * "5; URL = " * The function returns META_REDIRECT_TYPE_URL_START since we expect a url to * follow. * * "5; URL = http://www.google.com/?" * Returns META_REDIRECT_TYPE_URL since the input value terminates in the * middle or end of a url. * * * Caveats: We are only recording up to 256 characters of attribute values, so * our analysis is limited to that. This shouldn't be an issue in practice * though as it would be unexpected for the part of the string that we are * matching to be so long. */ enum meta_redirect_type_enum meta_redirect_type(const char *value) { if (value == NULL) return META_REDIRECT_TYPE_NONE; /* Match while [ \t\r\n0-9]* */ value = ignore_spaces_or_digits(value); /* Verify that we got a semi-colon character */ if (*value != ';') return META_REDIRECT_TYPE_NONE; value++; /* Match while [ \t\r\n]* */ value = ignore_spaces(value); /* Validate that we have 'URL' */ if (strncasecmp(value, "url", strlen("url")) != 0) return META_REDIRECT_TYPE_NONE; value += strlen("url"); /* Match while [ \t\r\n]* */ value = ignore_spaces(value); if (*value != '=') return META_REDIRECT_TYPE_NONE; value++; /* Match while [ \t\r\n]* */ value = ignore_spaces(value); /* The HTML5 spec allows for the url to be quoted, so we skip a single or * double quote if we find one. */ if (*value == '"' || *value == '\'') value++; if (*value == '\0') return META_REDIRECT_TYPE_URL_START; else return META_REDIRECT_TYPE_URL; } /* Resets the entityfilter to it's initial state so it can be reused. */ void entityfilter_reset(entityfilter_ctx *ctx) { ctx->buffer[0] = 0; ctx->buffer_pos = 0; ctx->in_entity = 0; } /* Initializes a new entity filter object. */ entityfilter_ctx *entityfilter_new() { entityfilter_ctx *ctx; ctx = CAST(entityfilter_ctx *, malloc(sizeof(entityfilter_ctx))); if (ctx == NULL) return NULL; ctx->buffer[0] = 0; ctx->buffer_pos = 0; ctx->in_entity = 0; return ctx; } /* Copies the context of the entityfilter pointed to by src to the entityfilter * dst. */ void entityfilter_copy(entityfilter_ctx *dst, entityfilter_ctx *src) { assert(src != NULL); assert(dst != NULL); assert(src != dst); memcpy(dst, src, sizeof(entityfilter_ctx)); } /* Deallocates an entity filter object. */ void entityfilter_delete(entityfilter_ctx *ctx) { free(ctx); } /* Converts a string containing an hexadecimal number to a string containing * one character with the corresponding ascii value. * * The provided output char array must be at least 2 chars long. */ static const char *parse_hex(const char *s, char *output) { int n; n = strtol(s, NULL, 16); output[0] = n; output[1] = 0; /* TODO(falmeida): Make this function return void */ return output; } /* Converts a string containing a decimal number to a string containing one * character with the corresponding ascii value. * * The provided output char array must be at least 2 chars long. */ static const char *parse_dec(const char *s, char *output) { int n; n = strtol(s, NULL, 10); output[0] = n; output[1] = 0; return output; } /* Converts a string with an html entity to it's encoded form, which is written * to the output string. */ static const char *entity_convert(const char *s, char *output, char terminator) { /* TODO(falmeida): Handle wide char encodings */ struct entityfilter_table_s *t = entityfilter_table; if (s[0] == '#') { if (s[1] == 'x' || s[1] == 'X') { /* hex */ return parse_hex(s + 2, output); } else { /* decimal */ return parse_dec(s + 1, output); } } while (t->entity != NULL) { if (strcasecmp(t->entity, s) == 0) return t->value; t++; } snprintf(output, HTMLPARSER_MAX_ENTITY_SIZE, "&%s%c", s, terminator); output[HTMLPARSER_MAX_ENTITY_SIZE - 1] = '\0'; return output; } /* Processes a character from the input stream and decodes any html entities * in the processed input stream. */ const char *entityfilter_process(entityfilter_ctx *ctx, char c) { if (ctx->in_entity) { if (c == ';' || html_isspace(c)) { ctx->in_entity = 0; ctx->buffer[ctx->buffer_pos] = '\0'; ctx->buffer_pos = 0; return entity_convert(ctx->buffer, ctx->output, c); } else { ctx->buffer[ctx->buffer_pos++] = c; if (ctx->buffer_pos >= HTMLPARSER_MAX_ENTITY_SIZE - 2) { /* No more buffer to use, finalize and return. * We need two characters left, one for the '&' character and * another for the NULL termination. */ ctx->buffer[ctx->buffer_pos] = '\0'; ctx->in_entity=0; ctx->buffer_pos = 0; snprintf(ctx->output, HTMLPARSER_MAX_ENTITY_SIZE, "&%s", ctx->buffer); ctx->output[HTMLPARSER_MAX_ENTITY_SIZE - 1] = '\0'; return ctx->output; } } } else { if (c == '&') { ctx->in_entity = 1; ctx->buffer_pos = 0; } else { ctx->output[0] = c; ctx->output[1] = 0; return ctx->output; } } return ""; } /* Called when the parser enters a new tag. Starts recording it's name into * html->tag. */ static void enter_tag_name(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); html->tag[0] = '\0'; statemachine_start_record(ctx); } /* Called when the parser exits the tag name in order to finalize the recording. * * It converts the tag name to lowercase, and if the tag was closed, just * clears html->tag. */ static void exit_tag_name(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); nopad_strncpy(html->tag, statemachine_stop_record(ctx), HTMLPARSER_MAX_STRING, statemachine_record_length(ctx)); tolower_str(html->tag); if (html->tag[0] == '/') html->tag[0] = '\0'; } /* Called when the parser enters a new tag. Starts recording it's name into * html->attr */ static void enter_attr(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); html->attr[0] = '\0'; statemachine_start_record(ctx); } /* Called when the parser exits the attribute name in order to finalize the * recording. * * It converts the tag name to lowercase. */ static void exit_attr(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); nopad_strncpy(html->attr, statemachine_stop_record(ctx), HTMLPARSER_MAX_STRING, statemachine_record_length(ctx)); tolower_str(html->attr); } /* Called when we enter an attribute value. * * Keeps track of a position index inside the value and initializes the * javascript state machine for attributes that accept javascript. */ static void enter_value(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); html->value_index = 0; if (is_js_attribute(html->attr)) { entityfilter_reset(html->entityfilter); jsparser_reset(html->jsparser); html->in_js = 1; } else { html->in_js = 0; } } /* Called when we enter the contents of an attribute value. * * Initializes the recording of the contents of the value. */ static void enter_value_content(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); html->value[0] = '\0'; statemachine_start_record(ctx); } /* Called when we exit the contents of an attribute value. * * Finalizes the recording of the contents of the value. */ static void exit_value_content(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); nopad_strncpy(html->value, statemachine_stop_record(ctx), HTMLPARSER_MAX_STRING, statemachine_record_length(ctx)); html->in_js = 0; } /* Called for every character inside an attribute value. * * Used to process javascript and keep track of the position index inside the * attribute value. */ static void in_state_value(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); html->value_index++; if (html->in_js == 1) { const char *output; output = entityfilter_process(html->entityfilter, chr); jsparser_parse_str(html->jsparser, output); } } /* Called everytime the parser leaves a tag definition. * * When we encounter a script tag, we initialize the js parser and switch the * state to cdata. We also switch to the cdata state when we encounter any * other CDATA/RCDATA tag (style, title or textarea) except that we do not * initialize the js parser. * * To simplify the code, we treat RCDATA and CDATA sections the same since the * differences between them don't affect the context we are in. */ static void tag_close(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); if (strcmp(html->tag, "script") == 0) { ctx->next_state = HTMLPARSER_STATE_INT_CDATA_TEXT; jsparser_reset(html->jsparser); html->in_js = 1; } else if (strcmp(html->tag, "style") == 0 || strcmp(html->tag, "title") == 0 || strcmp(html->tag, "textarea") == 0) { ctx->next_state = HTMLPARSER_STATE_INT_CDATA_TEXT; html->in_js = 0; } } /* Called inside cdata blocks in order to parse the javascript. * * Calls the javascript parser if currently in a script tag. */ static void in_state_cdata(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); assert(html != NULL); if (html->in_js) jsparser_parse_chr(html->jsparser, chr); } /* Called if we encounter a '<' character in a cdata section. * * When encountering a '<' character inside cdata, we need to find the closing * tag name in order to know if the tag is going to be closed or not, so we * start recording the name of what could be the closing tag. */ static void enter_state_cdata_may_close(statemachine_ctx *ctx, int start, char chr, int end) { statemachine_start_record(ctx); } /* Called when we finish reading what could be a closing cdata tag. * * Checks if the closing tag name matches the current entity, and if so closes * the element. */ static void exit_state_cdata_may_close(statemachine_ctx *ctx, int start, char chr, int end) { htmlparser_ctx *html = CAST(htmlparser_ctx *, ctx->user); const char *cdata_close_tag; assert(html != NULL); cdata_close_tag = statemachine_stop_record(ctx); assert(cdata_close_tag[0] == '/'); if (strcasecmp(&cdata_close_tag[1], html->tag) == 0 && (chr == '>' || html_isspace(chr))) { /* Make sure we have a delimiter */ html->tag[0] = '\0'; /* Empty tag mimicking exit_tag_name(). */ html->in_js = 0; /* In case this was a script tag. */ } else { /* Does not close the CDATA section. Go back to CDATA. */ ctx->next_state = HTMLPARSER_STATE_INT_CDATA_TEXT; } } /* Resets the parser to it's initial state and changes the parser mode. */ void htmlparser_reset_mode(htmlparser_ctx *ctx, int mode) { assert(ctx != NULL); statemachine_reset(ctx->statemachine); ctx->in_js = 0; ctx->tag[0] = '\0'; ctx->attr[0] = '\0'; ctx->value[0] = '\0'; jsparser_reset(ctx->jsparser); switch (mode) { case HTMLPARSER_MODE_HTML: ctx->statemachine->current_state = HTMLPARSER_STATE_INT_TEXT; break; case HTMLPARSER_MODE_JS: ctx->statemachine->current_state = HTMLPARSER_STATE_INT_JS_FILE; ctx->in_js = 1; break; case HTMLPARSER_MODE_CSS: ctx->statemachine->current_state = HTMLPARSER_STATE_INT_CSS_FILE; break; case HTMLPARSER_MODE_HTML_IN_TAG: ctx->statemachine->current_state = HTMLPARSER_STATE_INT_TAG_SPACE; break; default: assert("Invalid mode in htmlparser_reset_mode()." && 0); } } /* Resets the parser to it's initial state and to the default mode, which * is MODE_HTML. */ void htmlparser_reset(htmlparser_ctx *ctx) { assert(ctx != NULL); htmlparser_reset_mode(ctx, HTMLPARSER_MODE_HTML); } /* Creates a new state machine definition and initializes the events for the * state transitions. * * Although each instance of the parser has it's own private instance of a * statemachine definition, they are still identical across html parser objects * and are never modified after creation. As such, changes to this definition * should not occur outside this function and should not depend on properties * of this particular parser instance as in the future we may opt to use a * single public definition across parser objects. */ static statemachine_definition *create_statemachine_definition() { statemachine_definition *def; def = statemachine_definition_new(HTMLPARSER_NUM_STATES); if (def == NULL) return NULL; statemachine_definition_populate(def, htmlparser_state_transitions, htmlparser_states_internal_names); statemachine_enter_state(def, HTMLPARSER_STATE_INT_TAG_NAME, enter_tag_name); statemachine_exit_state(def, HTMLPARSER_STATE_INT_TAG_NAME, exit_tag_name); statemachine_enter_state(def, HTMLPARSER_STATE_INT_ATTR, enter_attr); statemachine_exit_state(def, HTMLPARSER_STATE_INT_ATTR, exit_attr); statemachine_enter_state(def, HTMLPARSER_STATE_INT_TAG_CLOSE, tag_close); /* CDATA states. We must list all cdata and javascript states here. */ /* TODO(falmeida): Declare this list in htmlparser_fsm.config so it doesn't * go out of sync. */ statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_TEXT, in_state_cdata); statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_COMMENT_START, in_state_cdata); statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_COMMENT_START_DASH, in_state_cdata); statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_COMMENT_BODY, in_state_cdata); statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_COMMENT_DASH, in_state_cdata); statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_COMMENT_DASH_DASH, in_state_cdata); statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_LT, in_state_cdata); statemachine_in_state(def, HTMLPARSER_STATE_INT_CDATA_MAY_CLOSE, in_state_cdata); /* For simplification, we treat the javascript mode as if it were cdata. */ statemachine_in_state(def, HTMLPARSER_STATE_INT_JS_FILE, in_state_cdata); statemachine_enter_state(def, HTMLPARSER_STATE_INT_CDATA_MAY_CLOSE, enter_state_cdata_may_close); statemachine_exit_state(def, HTMLPARSER_STATE_INT_CDATA_MAY_CLOSE, exit_state_cdata_may_close); /* value states */ statemachine_enter_state(def, HTMLPARSER_STATE_INT_VALUE, enter_value); /* Called when we enter the content of the value */ statemachine_enter_state(def, HTMLPARSER_STATE_INT_VALUE_TEXT, enter_value_content); statemachine_enter_state(def, HTMLPARSER_STATE_INT_VALUE_Q, enter_value_content); statemachine_enter_state(def, HTMLPARSER_STATE_INT_VALUE_DQ, enter_value_content); /* Called when we exit the content of the value */ statemachine_exit_state(def, HTMLPARSER_STATE_INT_VALUE_TEXT, exit_value_content); statemachine_exit_state(def, HTMLPARSER_STATE_INT_VALUE_Q, exit_value_content); statemachine_exit_state(def, HTMLPARSER_STATE_INT_VALUE_DQ, exit_value_content); statemachine_in_state(def, HTMLPARSER_STATE_INT_VALUE_TEXT, in_state_value); statemachine_in_state(def, HTMLPARSER_STATE_INT_VALUE_Q, in_state_value); statemachine_in_state(def, HTMLPARSER_STATE_INT_VALUE_DQ, in_state_value); return def; } /* Initializes a new htmlparser instance. * * Returns a pointer to the new instance or NULL if the initialization fails. * Initialization failure is fatal, and if this function fails it may not * deallocate all previsouly allocated memory. */ htmlparser_ctx *htmlparser_new() { htmlparser_ctx *html; html = CAST(htmlparser_ctx *, calloc(1, sizeof(htmlparser_ctx))); if (html == NULL) return NULL; html->statemachine_def = create_statemachine_definition(); if (html->statemachine_def == NULL) return NULL; html->statemachine = statemachine_new(html->statemachine_def, html); if (html->statemachine == NULL) return NULL; html->jsparser = jsparser_new(); if (html->jsparser == NULL) return NULL; html->entityfilter = entityfilter_new(); if (html->entityfilter == NULL) return NULL; htmlparser_reset(html); return html; } /* Copies the context of the htmlparser pointed to by src to the htmlparser dst. */ void htmlparser_copy(htmlparser_ctx *dst, const htmlparser_ctx *src) { dst->value_index = src->value_index; dst->in_js = src->in_js; strcpy(dst->tag, src->tag); strcpy(dst->attr, src->attr); strcpy(dst->value, src->value); statemachine_copy(dst->statemachine, src->statemachine, dst->statemachine_def, dst); jsparser_copy(dst->jsparser, src->jsparser); entityfilter_copy(dst->entityfilter, src->entityfilter); } /* Receives an htmlparser context and Returns the current html state. */ int htmlparser_state(htmlparser_ctx *ctx) { return state_external(ctx->statemachine->current_state); } /* Parses the input html stream and returns the finishing state. */ int htmlparser_parse(htmlparser_ctx *ctx, const char *str, int size) { int internal_state; internal_state = statemachine_parse(ctx->statemachine, str, size); return state_external(internal_state); } /* Returns true if the parser is inside an attribute value and the value is * surrounded by single or double quotes. */ int htmlparser_is_attr_quoted(htmlparser_ctx *ctx) { int st = statemachine_get_state(ctx->statemachine); if (st == HTMLPARSER_STATE_INT_VALUE_Q_START || st == HTMLPARSER_STATE_INT_VALUE_Q || st == HTMLPARSER_STATE_INT_VALUE_DQ_START || st == HTMLPARSER_STATE_INT_VALUE_DQ) return 1; else return 0; } /* Returns true if the parser is currently in javascript. */ int htmlparser_in_js(htmlparser_ctx *ctx) { int st = statemachine_get_state(ctx->statemachine); /* CDATA states plus JS_FILE. We must list all cdata and javascript states * here. */ /* TODO(falmeida): Declare this list in htmlparser_fsm.config so it doesn't go * out of sync. */ if (ctx->in_js && (st == HTMLPARSER_STATE_INT_CDATA_TEXT || st == HTMLPARSER_STATE_INT_CDATA_COMMENT_START || st == HTMLPARSER_STATE_INT_CDATA_COMMENT_START_DASH || st == HTMLPARSER_STATE_INT_CDATA_COMMENT_BODY || st == HTMLPARSER_STATE_INT_CDATA_COMMENT_DASH || st == HTMLPARSER_STATE_INT_CDATA_COMMENT_DASH_DASH || st == HTMLPARSER_STATE_INT_CDATA_LT || st == HTMLPARSER_STATE_INT_CDATA_MAY_CLOSE || st == HTMLPARSER_STATE_INT_JS_FILE)) return 1; if (state_external(st) == HTMLPARSER_STATE_VALUE && ctx->in_js) return 1; else return 0; } /* Returns the current tag or NULL if not available or we haven't seen the * entire tag yet. */ const char *htmlparser_tag(htmlparser_ctx *ctx) { if (ctx->tag[0] != '\0') return ctx->tag; else return NULL; } /* Returns true if inside an attribute or a value */ int htmlparser_in_attr(htmlparser_ctx *ctx) { int ext_state = state_external(statemachine_get_state(ctx->statemachine)); return ext_state == HTMLPARSER_STATE_ATTR || ext_state == HTMLPARSER_STATE_VALUE; } /* Returns the current attribute name if after an attribute name or in an * attribute value. Returns NULL otherwise. */ const char *htmlparser_attr(htmlparser_ctx *ctx) { if (htmlparser_in_attr(ctx)) return ctx->attr; else return NULL; } /* Returns true if the parser is currently inside a CSS construct. */ int htmlparser_in_css(htmlparser_ctx *ctx) { int state = statemachine_get_state(ctx->statemachine); const char *tag = htmlparser_tag(ctx); int external_state = state_external(state); if (state == HTMLPARSER_STATE_INT_CSS_FILE || (external_state == HTMLPARSER_STATE_VALUE && htmlparser_attr_type(ctx) == HTMLPARSER_ATTR_STYLE) || (tag && strcmp(tag, "style") == 0)) { return 1; } else { return 0; } } /* Returns the contents of the current attribute value. */ const char *htmlparser_value(htmlparser_ctx *ctx) { int ext_state = state_external(statemachine_get_state(ctx->statemachine)); if (ext_state == HTMLPARSER_STATE_VALUE) { strncpy(ctx->value, statemachine_record_buffer(ctx->statemachine), HTMLPARSER_MAX_STRING); ctx->value[HTMLPARSER_MAX_STRING - 1] = '\0'; return ctx->value; } else { return NULL; } } /* Returns the current state of the javascript state machine * * Currently only present for testing purposes. */ int htmlparser_js_state(htmlparser_ctx *ctx) { return jsparser_state(ctx->jsparser); } /* True is currently inside a javascript string literal */ int htmlparser_is_js_quoted(htmlparser_ctx *ctx) { if (htmlparser_in_js(ctx)) { int st = jsparser_state(ctx->jsparser); if (st == JSPARSER_STATE_Q || st == JSPARSER_STATE_DQ) return 1; } return 0; } /* True if currently inside an attribute value */ int htmlparser_in_value(htmlparser_ctx *ctx) { int ext_state = state_external(statemachine_get_state(ctx->statemachine)); return ext_state == HTMLPARSER_STATE_VALUE; } /* Returns the position inside the current attribute value */ int htmlparser_value_index(htmlparser_ctx *ctx) { if (htmlparser_in_value(ctx)) return ctx->value_index; return -1; } /* Returns true if this is the first character of a url inside an attribute. */ int htmlparser_is_url_start(htmlparser_ctx *ctx) { if (htmlparser_attr_type(ctx) == HTMLPARSER_ATTR_URI) { const char* tag = htmlparser_tag(ctx); /*const char* attr =*/ htmlparser_attr(ctx); if ((tag && strcmp(tag, "meta") == 0 && meta_redirect_type(htmlparser_value(ctx)) == META_REDIRECT_TYPE_URL_START) || htmlparser_value_index(ctx) == 0) return 1; } return 0; } /* Returns the current attribute type. */ int htmlparser_attr_type(htmlparser_ctx *ctx) { if (!htmlparser_in_attr(ctx)) return HTMLPARSER_ATTR_NONE; if (is_js_attribute(ctx->attr)) return HTMLPARSER_ATTR_JS; if (is_uri_attribute(ctx->attr)) return HTMLPARSER_ATTR_URI; if (is_style_attribute(ctx->attr)) return HTMLPARSER_ATTR_STYLE; const char* tag = htmlparser_tag(ctx); const char* attr = htmlparser_attr(ctx); /* Special logic to handle meta redirect type tags. */ if (tag && strcmp(tag, "meta") == 0 && attr && strcmp(attr, "content") == 0) { const char* value = htmlparser_value(ctx); meta_redirect_type_enum redirect_type = meta_redirect_type(value); if (redirect_type == META_REDIRECT_TYPE_URL || redirect_type == META_REDIRECT_TYPE_URL_START) return HTMLPARSER_ATTR_URI; } return HTMLPARSER_ATTR_REGULAR; } /* Return the current line number. */ int htmlparser_get_line_number(htmlparser_ctx *ctx) { return statemachine_get_line_number(ctx->statemachine); } /* Set the current line number. */ void htmlparser_set_line_number(htmlparser_ctx *ctx, int line) { statemachine_set_line_number(ctx->statemachine, line); } /* Return the current column number. */ int htmlparser_get_column_number(htmlparser_ctx *ctx) { return statemachine_get_column_number(ctx->statemachine); } /* Set the current column number. */ void htmlparser_set_column_number(htmlparser_ctx *ctx, int column) { statemachine_set_column_number(ctx->statemachine, column); } /* Retrieve a human readable error message in case an error occurred. * * NULL is returned if the parser didn't encounter an error. */ const char *htmlparser_get_error_msg(htmlparser_ctx *ctx) { return statemachine_get_error_msg(ctx->statemachine); } /* Invoked by the caller when text is expanded by the caller. */ int htmlparser_insert_text(htmlparser_ctx *ctx) { /* TODO(falmeida): Generalize and use a table for these values. */ if (statemachine_get_state(ctx->statemachine) == HTMLPARSER_STATE_INT_VALUE) { statemachine_set_state(ctx->statemachine, HTMLPARSER_STATE_INT_VALUE_TEXT); } return 1; } /* Deallocates an htmlparser context object. */ void htmlparser_delete(htmlparser_ctx *ctx) { assert(ctx != NULL); statemachine_definition_delete(ctx->statemachine_def); statemachine_delete(ctx->statemachine); jsparser_delete(ctx->jsparser); entityfilter_delete(ctx->entityfilter); free(ctx); } #ifdef __cplusplus } /* namespace security_streamhtmlparser */ #endif ctemplate-ctemplate-2.4/src/htmlparser/htmlparser.h000066400000000000000000000355271363122316600226320ustar00rootroot00000000000000/* * Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * --- * * Author: falmeida@google.com (Filipe Almeida) */ #ifndef SECURITY_STREAMHTMLPARSER_HTMLPARSER_H #define SECURITY_STREAMHTMLPARSER_HTMLPARSER_H #include #include "htmlparser/statemachine.h" #include "htmlparser/jsparser.h" // Annoying stuff for windows in opensource-land -- make sure clients // (in this case unittests) can import the functions. #ifndef CTEMPLATE_DLL_DECL # ifdef _MSC_VER # define CTEMPLATE_DLL_DECL __declspec(dllimport) # else # define CTEMPLATE_DLL_DECL /* should be the empty string for non-windows */ # endif #endif #ifdef __cplusplus namespace ctemplate_htmlparser { #endif /* entity filter */ /* String sizes used in htmlparser and entityfilter structures including the * NULL terminator. */ #define HTMLPARSER_MAX_STRING STATEMACHINE_RECORD_BUFFER_SIZE #define HTMLPARSER_MAX_ENTITY_SIZE 10 enum htmlparser_state_external_enum { HTMLPARSER_STATE_TEXT, HTMLPARSER_STATE_TAG, HTMLPARSER_STATE_ATTR, HTMLPARSER_STATE_VALUE, HTMLPARSER_STATE_COMMENT, HTMLPARSER_STATE_JS_FILE, HTMLPARSER_STATE_CSS_FILE, HTMLPARSER_STATE_ERROR }; enum htmlparser_mode { HTMLPARSER_MODE_HTML, HTMLPARSER_MODE_JS, HTMLPARSER_MODE_CSS, HTMLPARSER_MODE_HTML_IN_TAG }; enum htmlparser_attr_type { HTMLPARSER_ATTR_NONE, HTMLPARSER_ATTR_REGULAR, HTMLPARSER_ATTR_URI, HTMLPARSER_ATTR_JS, HTMLPARSER_ATTR_STYLE }; /* TODO(falmeida): Maybe move some of these declaration to the .c and only keep * a forward declaration in here, since these structures are private. */ /* entityfilter context structure. * * The entity filter collection of routines provide a way to decode html * entities from an html document in a streaming way. * * The html_process() function receives a character at a time from the input * stream and returns 0 or more characters which should be appended to the * resulting decoded document. * * Currently this collection of functions are only exported for testing purposes * and shouldn't be called from outside of htmlparser.c. * * Since we really only use these functions with the very specific purpose of * decoding html entities for javascript attributes, only a small subset of * entities are supported: <, >, "e;, &, &apos, and the numeric * character references for both decimal and hexadecimal. */ typedef struct entityfilter_ctx_s { /* Current position into the buffer. */ int buffer_pos; /* True if currently processing an html entity. */ int in_entity; /* Temporary character buffer that is used while processing html entities. */ char buffer[HTMLPARSER_MAX_ENTITY_SIZE]; /* String buffer returned to the application after we decoded an html * entity. */ char output[HTMLPARSER_MAX_ENTITY_SIZE]; } entityfilter_ctx; /* Resets the entityfilter to its initial state so it can be reused. */ void entityfilter_reset(entityfilter_ctx *ctx); /* Initializes a new entity filter object. */ entityfilter_ctx *entityfilter_new(void); /* Deallocates an entity filter object. */ void entityfilter_delete(entityfilter_ctx *ctx); /* Copies the context of the entityfilter pointed to by src to the entityfilter * dst. */ void entityfilter_copy(entityfilter_ctx *dst, entityfilter_ctx *src); /* Processes a character from the input stream and decodes any html entities * in the accumulated buffer. * * Returns a reference to a string that points to an internal buffer. This * buffer will be changed after every call to entityfilter_process(). As * such this string should be duplicated before subsequent calls to * entityfilter_process(). */ const char *entityfilter_process(entityfilter_ctx *ctx, char c); /* html parser */ /* Stores the context of the html parser. * If this structure is changed, htmlparser_new(), htmlparser_copy() and * htmlparser_reset() should be updated accordingly. */ typedef struct htmlparser_ctx_s { /* Holds a reference to the statemachine context. */ statemachine_ctx *statemachine; /* Holds a reference to the statemachine definition in use. Right now this is * only used so we can deallocate it at the end. * * It should be readonly and contain the same values across jsparser * instances. */ /* TODO(falmeida): Change statemachine_def to const. */ statemachine_definition *statemachine_def; /* Holds a reference to the javascript parser. */ jsparser_ctx *jsparser; /* Holds a reference to the entity filter. Used for decoding html entities * inside javascript attributes. */ entityfilter_ctx *entityfilter; /* Offset into the current attribute value where 0 is the first character in * the value. */ int value_index; /* True if currently processing javascript. */ int in_js; /* Current tag name. */ char tag[HTMLPARSER_MAX_STRING]; /* Current attribute name. */ char attr[HTMLPARSER_MAX_STRING]; /* Contents of the current value capped to HTMLPARSER_MAX_STRING. */ char value[HTMLPARSER_MAX_STRING]; } htmlparser_ctx; /* Resets the parser to its initial state and to the default mode, which * is MODE_HTML. * * All internal context like tag name, attribute name or the state of the * statemachine are reset to its original values as if the object was just * created. */ extern CTEMPLATE_DLL_DECL void htmlparser_reset(htmlparser_ctx *ctx); /* Resets the parser to its initial state and changes the parser mode. * All internal context like tag name, attribute name or the state of the * statemachine are reset to their original values as if the object was just * created. * * Available modes: * HTMLPARSER_MODE_HTML - Parses html text * HTMLPARSER_MODE_JS - Parses javascript files * HTMLPARSER_MODE_CSS - Parses CSS files. No actual parsing is actually done * but htmlparser_in_css() always returns true. * HTMLPARSER_MODE_HTML_IN_TAG - Parses an attribute list inside a tag. To * be used in a template expanded in the * following context: * */ extern CTEMPLATE_DLL_DECL void htmlparser_reset_mode(htmlparser_ctx *ctx, int mode); /* Initializes a new htmlparser instance. * * Returns a pointer to the new instance or NULL if the initialization fails. * Initialization failure is fatal, and if this function fails it may not * deallocate all previsouly allocated memory. */ extern CTEMPLATE_DLL_DECL htmlparser_ctx *htmlparser_new(void); /* Copies the context of the htmlparser pointed to by src to the htmlparser dst. * * Also copies over the instances of the state machine, the jsparser and the * entity filter but not the statemachine definition since this one is read * only. */ extern CTEMPLATE_DLL_DECL void htmlparser_copy(htmlparser_ctx *dst, const htmlparser_ctx *src); /* Receives an htmlparser context and returns the current html state. * * The return value will be one of the states of htmlparser_state_external_enum. */ extern CTEMPLATE_DLL_DECL int htmlparser_state(htmlparser_ctx *ctx); /* Parses the input html stream and returns the finishing state. * * Returns HTMLPARSER_ERROR if unable to parse the input. If htmlparser_parse() * is called after an error situation was encountered the behaviour is * unspecified. At this point, htmlparser_reset() or htmlparser_reset_mode() * can be called to reset the state. */ extern CTEMPLATE_DLL_DECL int htmlparser_parse(htmlparser_ctx *ctx, const char *str, int size); /* Returns true if the parser is inside an attribute value and the value is * surrounded by single or double quotes. */ extern CTEMPLATE_DLL_DECL int htmlparser_is_attr_quoted(htmlparser_ctx *ctx); /* Returns true if the parser is currently in javascript. This can be a * an attribute that takes javascript, a javascript block or the parser * can just be in MODE_JS. */ extern CTEMPLATE_DLL_DECL int htmlparser_in_js(htmlparser_ctx *ctx); /* Returns the current tag or NULL if not available or we haven't seen the * entire tag yet. * * There is no stack implemented because we currently don't have a need for * it, which means tag names are tracked only one level deep. * * This is better understood by looking at the following example: * * * [tag=b] * * [tag=i] * * [tag=NULL] * * * The tag is correctly filled inside the tag itself and before any new inner * tag is closed, at which point the tag will be null. * * For our current purposes this is not a problem, but we may implement a tag * tracking stack in the future for completeness. * */ extern CTEMPLATE_DLL_DECL const char *htmlparser_tag(htmlparser_ctx *ctx); /* Returns the current attribute name if after an attribute name or in an * attribute value. Returns NULL otherwise. */ extern CTEMPLATE_DLL_DECL const char *htmlparser_attr(htmlparser_ctx *ctx); /* Returns the contents of the current attribute value. * * Returns NULL if not inside an attribute value. */ extern CTEMPLATE_DLL_DECL const char *htmlparser_value(htmlparser_ctx *ctx); /* Returns true if the parser is currently inside a CSS construct. * * Currently this can be either a STYLE tag, a STYLE attribute or the fact that * the parser was reset in HTMLPARSER_MODE_CSS using * htmlparser_reset_mode(). */ extern CTEMPLATE_DLL_DECL int htmlparser_in_css(htmlparser_ctx *ctx); /* Returns the current state of the javascript state machine. * * Currently only present for testing purposes. */ extern CTEMPLATE_DLL_DECL int htmlparser_js_state(htmlparser_ctx *ctx); /* Returns non-zero if currently inside a javascript string literal and zero * otherwise. */ extern CTEMPLATE_DLL_DECL int htmlparser_is_js_quoted(htmlparser_ctx *ctx); /* Returns non-zero if currently inside an attribute value and zero otherwise. */ extern CTEMPLATE_DLL_DECL int htmlparser_value_index(htmlparser_ctx *ctx); /* Returns true if this is the first character of a url inside an attribute. * * This function can be used by an html sanitizer or auto escaping system as a * hint that it should validate the url for a whitelist of protocol handlers and * for well-formedness, or that it should just escape a component of it. * * For attributes that expect a URL, this will return true if we are at the * first character of the URL, false otherwise. * For most attributes, this is the same as checking that we are at the first * character of the attribute value but it also works correctly for the * "content" attribute of the "meta" tag where the URL follows some earlier * content. * e.g: * * For any other attributes, the result will always be false. */ extern CTEMPLATE_DLL_DECL int htmlparser_is_url_start(htmlparser_ctx *ctx); /* Returns the current attribute type. * * The attribute type can be one of: * HTMLPARSER_ATTR_NONE - not inside an attribute. * HTMLPARSER_ATTR_REGULAR - Inside a normal attribute. * HTMLPARSER_ATTR_URI - Inside an attribute that accepts a uri. * HTMLPARSER_ATTR_JS - Inside a javascript attribute. * HTMLPARSER_ATTR_STYLE - Inside a css style attribute. */ extern CTEMPLATE_DLL_DECL int htmlparser_attr_type(htmlparser_ctx *ctx); /* Return the current line number. */ extern CTEMPLATE_DLL_DECL int htmlparser_get_line_number(htmlparser_ctx *ctx); /* Set the current line number. */ extern CTEMPLATE_DLL_DECL void htmlparser_set_line_number(htmlparser_ctx *ctx, int line); /* Return the current column number. */ extern CTEMPLATE_DLL_DECL int htmlparser_get_column_number(htmlparser_ctx *ctx); /* Set the current column number. */ extern CTEMPLATE_DLL_DECL void htmlparser_set_column_number(htmlparser_ctx *ctx, int column); /* Retrieve a human readable error message in case an error occurred. * * NULL is returned if the parser didn't encounter an error. */ extern CTEMPLATE_DLL_DECL const char *htmlparser_get_error_msg(htmlparser_ctx *ctx); /* Invoked by the caller when text is expanded by the caller. * * Should be invoked when a template directive that expands to content is * executed but we don't provide this content to the parser itself. This changes * the current state by following the default rule, ensuring we stay in sync * with the template. * * Returns 1 if template directives are accepted for this state and 0 if they * are not, which should result in an error condition. * * Right now the only case being handled are unquoted attribute values and it * always returns 1. When insert_text() is called after the equals sign, we * assume some text was consumed and we are now in the middle of the attribute * value itself. Example: * * * * The template calls insert_text() when it encounters $HREF_VALUE. If it didn't * the parser would only have seen the following html: * * * * and would interpret alt=alternate_text as the value of the href attribute. */ extern CTEMPLATE_DLL_DECL int htmlparser_insert_text(htmlparser_ctx *ctx); /* Deallocates an htmlparser context object. */ extern CTEMPLATE_DLL_DECL void htmlparser_delete(htmlparser_ctx *ctx); #define htmlparser_parse_chr(a,b) htmlparser_parse(a, &(b), 1); #ifdef __cplusplus #define htmlparser_parse_str(a,b) htmlparser_parse(a, b, \ static_cast(strlen(b))); #else #define htmlparser_parse_str(a,b) htmlparser_parse(a, b, (int)strlen(b)); #endif #ifdef __cplusplus } /* namespace security_streamhtmlparser */ #endif #endif /* SECURITY_STREAMHTMLPARSER_HTMLPARSER_H */ ctemplate-ctemplate-2.4/src/htmlparser/htmlparser_cpp.h000066400000000000000000000251001363122316600234560ustar00rootroot00000000000000// Copyright (c) 2007, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: falmeida@google.com (Filipe Almeida) // // c++ bindings for htmlparser. #ifndef SECURITY_STREAMHTMLPARSER_HTMLPARSER_CPP_H__ #define SECURITY_STREAMHTMLPARSER_HTMLPARSER_CPP_H__ #include #include #include "htmlparser/htmlparser.h" #include "htmlparser/jsparser.h" #include "base/util.h" namespace ctemplate_htmlparser { class JavascriptParser { public: enum State { STATE_TEXT = JSPARSER_STATE_TEXT, STATE_Q = JSPARSER_STATE_Q, STATE_DQ = JSPARSER_STATE_DQ, STATE_REGEXP = JSPARSER_STATE_REGEXP, STATE_COMMENT = JSPARSER_STATE_COMMENT, }; }; class HtmlParser { public: /* html states */ enum State { STATE_TEXT = HTMLPARSER_STATE_TEXT, STATE_TAG = HTMLPARSER_STATE_TAG, STATE_ATTR = HTMLPARSER_STATE_ATTR, STATE_VALUE = HTMLPARSER_STATE_VALUE, STATE_COMMENT = HTMLPARSER_STATE_COMMENT, STATE_JS_FILE = HTMLPARSER_STATE_JS_FILE, STATE_CSS_FILE = HTMLPARSER_STATE_CSS_FILE, STATE_ERROR = HTMLPARSER_STATE_ERROR }; /* attribute types */ enum AttributeType { ATTR_NONE = HTMLPARSER_ATTR_NONE, ATTR_REGULAR = HTMLPARSER_ATTR_REGULAR, ATTR_URI = HTMLPARSER_ATTR_URI, ATTR_JS = HTMLPARSER_ATTR_JS, ATTR_STYLE = HTMLPARSER_ATTR_STYLE }; /* Parser modes */ enum Mode { MODE_HTML = HTMLPARSER_MODE_HTML, MODE_JS = HTMLPARSER_MODE_JS, MODE_CSS = HTMLPARSER_MODE_CSS, MODE_HTML_IN_TAG = HTMLPARSER_MODE_HTML_IN_TAG }; HtmlParser() { parser_ = htmlparser_new(); CHECK(parser_ != NULL); }; /* Parses the input html stream and returns the finishing state. * * Returns HtmlParser::STATE_ERROR if unable to parse the input. If * htmlparser_parse() is called after an error situation was encountered * the behaviour is unspecified. At this point, Reset() or ResetMode() * can be called to reset the state so it can be used to parse a new file. */ int Parse(const char *str, int len) { return htmlparser_parse(parser_, str, len); }; int Parse(const std::string &str) { return Parse(str.c_str(), static_cast(str.length())); }; /* Returns the current state the parser is in */ int state() const { return htmlparser_state(parser_); }; /* Returns the current tag or NULL if not available. * * There is no stack implemented because we currently don't have a need for * it, which means tag names are tracked only one level deep. * * This is better understood by looking at the following example: * * * [tag=b] * * [tag=i] * * [tag=NULL] * * * The tag is correctly filled inside the tag itself and before any new * inner tag is closed, at which point the tag will be set to NULL. * * For our current purposes this is not a problem, but we may implement a * tag tracking stack in the future for completeness. */ const char *tag() const { return htmlparser_tag(parser_); } /* Returns the current attribute name if inside an attribute name or an * attribute value. Returns NULL otherwise. */ const char *attribute() const { return htmlparser_attr(parser_); } /* Returns the contents of the current attribute value. */ const char *value() const { return htmlparser_value(parser_); } /* Returns true if inside javascript. This can be a javascript block, a * javascript attribute value or the parser may just be in javascript mode * (HtmlParser::MODE_JS) */ bool InJavascript() const { return static_cast(htmlparser_in_js(parser_)); } /* Returns true if the parser is currently inside a CSS construct. * * Currently this can be either a STYLE tag, a STYLE attribute or the fact * that the parser was reset using MODE_CSS using ResetMode(). */ bool InCss() const { return static_cast(htmlparser_in_css(parser_)); } /* Returns true if the current attribute is quoted */ bool IsAttributeQuoted() const { return static_cast(htmlparser_is_attr_quoted(parser_)); } /* Returns true if the parser is inside a js string literal. */ bool IsJavascriptQuoted() const { return static_cast(htmlparser_is_js_quoted(parser_)); } /* Returns the index within the current value or -1 if the parser is not * inside an attribute value */ int ValueIndex() const { return htmlparser_value_index(parser_); } /* Returns true if this is the first character of a url inside an attribute. * * This function can be used by an html sanitizer or auto escaping system as * a hint that it should validate the url for a whitelist of protocol * handlers and for well-formedness, or that it should just escape a * component of it. * * For attributes that expect a url this will return true if we are at the * first character of the attribute, but for the special case of a meta * redirect tag some analysis is made in order to verify if we are at the * start of a url or not. * * For any other attributes, the result will always be false. * */ bool IsUrlStart() const { return htmlparser_is_url_start(parser_); } /* Returns the current attribute type. * * The attribute type can be one of: * ATTR_NONE - not inside an attribute * ATTR_REGULAR - Inside a normal attribute * ATTR_URI - Inside an attribute that accepts a uri * ATTR_JS - Inside a javascript attribute * ATTR_STYLE - Inside a css style attribute * */ int AttributeType() const { return htmlparser_attr_type(parser_); } /* Return the current line number. */ int line_number() const { return htmlparser_get_line_number(parser_); } /* Set the current line number. */ void set_line_number(int line) { return htmlparser_set_line_number(parser_, line); } /* Return the current column number. */ int column_number() const { return htmlparser_get_column_number(parser_); } /* Set the current line number. */ void set_column_number(int column) { return htmlparser_set_column_number(parser_, column); } /* Retrieve a human readable error message in case an error occurred. * * NULL is returned if the parser didn't encounter an error. */ const char *GetErrorMessage() { return htmlparser_get_error_msg(parser_); } /* Returns the current state the javascript parser is in. * * Should only be used for testing. */ int javascript_state() const { return htmlparser_js_state(parser_); }; /* Resets the parser to it's initial state and changes the parser mode. * * Internal state (tag name, attribute name, state of statemachine) is * reset as * though the object was just created. * * Available modes: * MODE_HTML - Parses html text * MODE_JS - Parses javascript files * MODE_CSS - Parses CSS files. No actual parsing is actually done * but InCss() always returns true. * MODE_HTML_IN_TAG - Parses an attribute list inside a tag. To * be used in a template expanded in the * following context: */ void ResetMode(enum Mode mode) { return htmlparser_reset_mode(parser_, mode); } /* Resets the parser to it's initial state and to the default mode, which is * MODE_HTML. * * All internal context like tag name, attribute name or the state of the * statemachine are reset to it's original values as if the object was just * created. */ void Reset() { return htmlparser_reset(parser_); } /* Invoked when text is inserted by the caller. * * Should be called before a template directive that expands to content is * found. This changes the current state by following the default rule, * ensuring we stay in sync with template. * * Returns true if template directives are accepted for this state and * false if they are not, which should result in an error condition. * * Right now the only case being handled are unquoted attribute values and * it always returns true. In the future we can handle more cases and * restrict the states were we allow template directives by returning false * for those. */ bool InsertText() { return static_cast(htmlparser_insert_text(parser_)); } /* Copies the context of the HtmlParser object referenced in source to the * current object. */ void CopyFrom(const HtmlParser *source) { CHECK(this != source); CHECK(source != NULL); htmlparser_copy(parser_, source->parser_); } ~HtmlParser() { htmlparser_delete(parser_); }; private: htmlparser_ctx *parser_; DISALLOW_COPY_AND_ASSIGN(HtmlParser); }; } // namespace security_streamhtmlparser #endif // SECURITY_STREAMHTMLPARSER_HTMLPARSER_CPP_H__ ctemplate-ctemplate-2.4/src/htmlparser/htmlparser_fsm.config000066400000000000000000000243001363122316600245000ustar00rootroot00000000000000# Copyright (c) 2008, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # # Author: falmeida@google.com (Filipe Almeida) # TODO(falmeida): Add more descriptive names to the states and drop the # abbreviations. # TODO(falmeida): Reorder the states so that it's easier to read. # TODO(falmeida): Support CDATA blocks in the form: ', '>') condition('=', '=') # TODO(falmeida): This is not the correct expression. tag and attribute names # can only consist of alpha character. condition('id', 'A-Za-z0-9_:.-') condition('idtag', 'A-Za-z0-9/_:.-') # Whitespace according to: http://www.w3.org/TR/html401/struct/text.html#h-9.1 condition('space', ' \t\n\r') condition('!', '!') condition('q', '\'') condition('dq', '\"') condition('/', '/') condition('*', '*') condition('-', '-') condition('?', '?') condition('lf', '\n') condition('quote', '\\') # TODO(falmeida): This default rule is a hack and shouldn't be here. condition('default', '[:default:]') state(name = 'text', external = 'text', transitions = [ ['<', 'tag_start'], ['default', 'text'] ]) # When we found the < character in text. # Tag opening is defined in the HTML5 draft here: # http://www.whatwg.org/specs/web-apps/current-work/#tag-open-state # We don't exactly follow this and are much more loose in order to mimic the way # the major browsers behave. state(name = 'tag_start', external = 'tag', transitions = [ ['idtag', 'tag_name'], ['?', 'pi'], ['!', 'declaration_start'], ['<', 'tag_start'], ['default', 'text'] ]) # Name of the tag. Includes the closing tag character '/'. state(name = 'tag_name', external = 'tag', transitions = [ ['idtag', 'tag_name'], ['space', 'tag_space'], ['>', 'tag_close'] ]) # HTML declaration and comment parsing # # We don't expose declaration state because at this point we only want to # ensure that we are parsing them correctly so we don't get out of sync. # This is specifically made for DOCTYPE declarations and won't work if DTD's # are defined inside the declaration. # The HTML5 spec says we should specificly look for the string '', 'text'], ['default', 'declaration_body'] ]) # Inside a declaration. Ie: ' state(name = 'declaration_body', external = 'text', transitions = [ ['>', 'text'], ['default', 'declaration_body'] ]) # Got '' state(name = 'comment_body', external = 'comment', transitions = [ ['-', 'comment_dash'], ['default', 'comment_body'] ]) # Got '-' inside a comment. state(name = 'comment_dash', external = 'comment', transitions = [ ['-', 'comment_dash_dash'], ['default', 'comment_body'] ]) # Got '--' inside a comment. state(name = 'comment_dash_dash', external = 'comment', transitions = [ ['-', 'comment_dash_dash'], ['>', 'text'], ['default', 'comment_body'] ]) # XML Processing instruction parsing according to: # http://www.w3.org/TR/REC-xml/#sec-pi # # Everything between the characters is considered to be part of the # processing instruction. state(name = 'pi', external = 'text', transitions = [ ['?', 'pi_may_end'], ['default', 'pi'] ]) state(name = 'pi_may_end', external = 'text', transitions = [ ['>', 'text'], ['default', 'pi'] ]) # Whitespace between tag name, attributes. state(name = 'tag_space', external = 'tag', transitions = [ ['>', 'tag_close'], ['space', 'tag_space'], ['id', 'attr'], ['/', 'tag_space'] ]) state(name = 'tag_close', external = 'text', transitions = [ ['<', 'tag_start'], ['default', 'text'] ]) # Name of the attribute. state(name = 'attr', external = 'attr', transitions = [ ['id', 'attr'], ['>', 'tag_close'], ['/', 'tag_space'], ['=', 'value'], ['space', 'attr_space'] ]) # After the attribute name. state(name = 'attr_space', external = 'attr', transitions = [ ['>', 'tag_close'], ['space', 'attr_space'], ['id', 'attr'], ['/', 'tag_space'], ['=', 'value'] ]) # Expecting a value, after attribute= state(name = 'value', external = 'value', transitions = [ ['q', 'value_q_start'], ['dq', 'value_dq_start'], ['space', 'value'], ['>', 'tag_close'], ['default', 'value_text'] ]) # Unquoted attribute value. state(name = 'value_text', external = 'value', transitions = [ ['>', 'tag_close'], ['space', 'tag_space'], ['default', 'value_text'] ]) # First character of a single quoted attribute value. state(name = 'value_q_start', external = 'value', transitions = [ ['q', 'tag_space'], ['default', 'value_q'] ]) # In the middle of a single quoted attribute value. state(name = 'value_q', external = 'value', transitions = [ ['q', 'tag_space'], ['default', 'value_q'] ]) # First character of a double quoted attribute value. state(name = 'value_dq_start', external = 'value', transitions = [ ['dq', 'tag_space'], ['default', 'value_dq'] ]) # In the middle of a double quoted attribute value. state(name = 'value_dq', external = 'value', transitions = [ ['dq', 'tag_space'], ['default', 'value_dq'] ]) # CDATA escaping text spans. # TODO(falmeida): These states should go after cdata_text. # Got '', 'cdata_text'], ['default', 'cdata_comment_body'] ]) # CDATA processing # # To simplify the code, we treat RCDATA and CDATA sections the same since the # differences between them don't affect the context we are in. state(name = 'cdata_text', external = 'text', transitions = [ ['<', 'cdata_lt'], ['default', 'cdata_text'] ]) # Possible beginning of the closing tag. state(name = 'cdata_lt', external = 'text', transitions = [ ['/', 'cdata_may_close'], ['!', 'cdata_comment_start'], ['default', 'cdata_text'] ]) # If we encounter ', 'text'], ['space', 'tag_space'], ['default', 'cdata_text'] ]) # The next states are used for specialized parser modes. state(name = 'js_file', external = 'js_file', transitions = [ ['default', 'js_file'] ]) # TODO(falmeida): Having css_file and js_file as the external name doesn't make # sense. This should instead be text and the js/css state be # returned by # in_js() and in_css(). state(name = 'css_file', external = 'css_file', transitions = [ ['default', 'css_file'] ]) state(name = 'null', external = 'text', transitions = [ ['default', 'null'] ]) ctemplate-ctemplate-2.4/src/htmlparser/jsparser.cc000066400000000000000000000457101363122316600224330ustar00rootroot00000000000000/* Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * --- * * Author: falmeida@google.com (Filipe Almeida) */ #include #include #include #include #include "htmlparser/statemachine.h" #include "htmlparser/jsparser.h" /* So we can support both C and C++ compilers, we use the CAST() macro instead * of using C style casts or static_cast<>() directly. */ #ifdef __cplusplus #define CAST(type, expression) (static_cast(expression)) #else #define CAST(type, expression) ((type)(expression)) #endif #ifdef __cplusplus namespace ctemplate_htmlparser { #endif /* __cplusplus */ /* Generated state machine definition. */ #include "htmlparser/jsparser_fsm.h" /* List of keywords that can precede a regular expression literal. Taken from: * http://www.mozilla.org/js/language/js20-2000-07/rationale/syntax.html * * 'void' was added to this list. * The list is used as input to a binary search, so it must be kept in a sorted * form. * There are a large number of keywords in here that don't exist in * ecmascript 3, either because they are reserved or because they are part of * ecmascript 4. However they weren't removed in order to keep the list in sync * with the previous document. */ static const char *regexp_token_prefix[] = { "abstract", "break", "case", "catch", "class", "const", "continue", "debugger", "default", "delete", "do", "else", "enum", "eval", "export", "extends", "field", "final", "finally", "for", "function", "goto", "if", "implements", "import", "in", "instanceof", "native", "new", "package", "private", "protected", "public", "return", "static", "switch", "synchronized", "throw", "throws", "transient", "try", "typeof", "var", "void", "volatile", "while", "with" }; /* Utility functions */ /* Converts the internal state into the external superstate. */ static inline int state_external(int state) { assert(state < JSPARSER_NUM_STATES); assert(state >= 0); return jsparser_states_external[state]; } /* Returns true if the character is an ecmascript whitespace or line terminator * character. Includes the characters in sections 7.2 and 7.3 of ECMAScript 3 * with the exception of unicode space and line terminators: * http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf */ static inline int js_is_whitespace(char c) { return c == '\t' || /* Tab 0x09 */ c == '\v' || /* Vertical Tab 0x0B */ c == '\f' || /* Form Feed 0x0C */ c == ' ' || /* Space 0x20 */ c == '\xa0' || /* No-Break Space 0xA0 */ c == '\n' || /* line Feed 0x0A */ c == '\r'; /* Carriage Return 0x0D */ } /* Returns true if the character is part of a javascript identifier. The rules * are pretty relaxed here since we don't accept unicode and don't care if the * first character doesn't contain numbers or not. * * For more detail on the limitations of having this relaxed set of characters * please see the comments in_state_js_text(). */ static inline int js_is_identifier(char c) { return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_' || c == '$'; } /* Appends a character to the ring buffer. Sequences of whitespace and newlines * are folded into one. * * js->buffer_start points to the first character in the buffer and * js->buffer_end points to the position of the next character to be written, * or one plus the last character written. If the buffer is full there will be * an empty slot position so we can diferentiate an empty buffer from a full * buffer. * * If the buffer is empty, then: * js->buffer_start == js->buffer_end. * If the buffer is full, then: * (js->buffer_end + 1) % JSPARSER_RING_BUFFER_SIZE == js->buffer_start. * */ void jsparser_buffer_append_chr(jsparser_ctx *js, char chr) { /* Fold whitespace so we have enough space in the buffer. */ if (js_is_whitespace(chr) && js_is_whitespace(jsparser_buffer_get(js, -1))) { return; } js->buffer[js->buffer_end] = chr; js->buffer_end = (js->buffer_end + 1) % JSPARSER_RING_BUFFER_SIZE; if (js->buffer_end == js->buffer_start) { js->buffer_start = (js->buffer_end + 1) % JSPARSER_RING_BUFFER_SIZE; } } /* Appends a string to the ring buffer. Sequences of whitespace and newlines * are folded into one. */ void jsparser_buffer_append_str(jsparser_ctx *js, const char *str) { assert(js != NULL); assert(str != NULL); while(*str != '\0') { jsparser_buffer_append_chr(js, *str++); } } /* Returns the position relative to the start of the buffer or -1 if past the * size of the buffer.. */ static inline int jsparser_buffer_absolute_pos(jsparser_ctx *js, int pos) { int absolute_pos; int buffer_len; assert(pos < 0); if(pos <= -JSPARSER_RING_BUFFER_SIZE) { return -1; } buffer_len = js->buffer_end - js->buffer_start; if (buffer_len < 0) { buffer_len += JSPARSER_RING_BUFFER_SIZE; } if (pos < -buffer_len) { return -1; } absolute_pos = (pos + js->buffer_end) % JSPARSER_RING_BUFFER_SIZE; if (absolute_pos < 0) { absolute_pos += JSPARSER_RING_BUFFER_SIZE; } return absolute_pos; } /* Returns the last appended character and removes it from the buffer. If the * buffer is empty, then it returns ASCII 0 ('\0'). */ char jsparser_buffer_pop(jsparser_ctx *js) { if (js->buffer_start == js->buffer_end) { return '\0'; } js->buffer_end--; if (js->buffer_end < 0) { js->buffer_end += JSPARSER_RING_BUFFER_SIZE; } return js->buffer[js->buffer_end]; } /* Returns the value of the character at a certain index in the buffer or an * ASCII 0 ('\0') character if the index is outside the buffer boundaries. * * Index positions are negative, were -1 is the last character appended to the * buffer. Using positive integers for the index will result in undefined * behaviour. */ char jsparser_buffer_get(jsparser_ctx *js, int pos) { int absolute_pos; assert(pos < 0); absolute_pos = jsparser_buffer_absolute_pos(js, pos); if (absolute_pos < 0) { return '\0'; } return js->buffer[absolute_pos]; } /* Sets the value of the character at a certain index in the buffer. Returns * true if the write was successful or false if there was an attempt to write * outside of the buffer boundaries. * * Index positions are negative, were -1 is the last character appended to the * buffer. Using positive integers for the index will result in undefined * behaviour. */ int jsparser_buffer_set(jsparser_ctx *js, int pos, char value) { int absolute_pos; assert(pos < 0); absolute_pos = jsparser_buffer_absolute_pos(js, pos); if (absolute_pos < 0) { return 0; } js->buffer[absolute_pos] = value; return 1; } /* Copies a slice of the buffer to the string pointed to by output. start and * end are the indexes of the sliced region. If start extends beyond the * beginning of the buffer, the slice will only contain character from the * beginning of the buffer. */ void jsparser_buffer_slice(jsparser_ctx *js, char *output, int start, int end) { int pos; assert(start <= end); assert(start < 0); assert(end < 0); for (pos = start; pos <= end; ++pos) { char c; c = jsparser_buffer_get(js, pos); if (c != '\0') { *output++ = jsparser_buffer_get(js, pos); } } *output++ = '\0'; } /* Copy the last javascript identifier or keyword found in the buffer to the * string pointed by identifier. * * For simplicity, we consider an identifier to be a sequence of alphanumeric * characters (as opposed to a digit followed by an alphanumeric character. * * Returns 0 if no identifier was matched, in which case the identifier * argument is replaced with an empty string, or non 0 if the identifier was * found. */ int jsparser_buffer_last_identifier(jsparser_ctx *js, char *identifier) { int end; int pos; assert(identifier != NULL); end = -1; /* Ignore the optional whitespace delimiter */ if (js_is_whitespace(jsparser_buffer_get(js, -1))) { --end; } /* Find the beginning of the identifier. This loop ends either when we find a * character that doesn't belong to an identifier, or when we find a '\0' * character, which means we reached the end of the buffer. */ for(pos = end; js_is_identifier(jsparser_buffer_get(js, pos)); --pos) { } if (pos + 1 >= end) { identifier[0] = '\0'; return 0; } jsparser_buffer_slice(js, identifier, pos + 1, end); return 1; } /* Callback used in bsearch() for comparing a string against an array of * strings. */ static int bsearch_strcmp(const void *a, const void *b) { return strcmp(CAST(const char*, a), *CAST(const char * const *, b)); } /* Returns true if the token argument can be a token prefix to a javascript * regular expression. * * The token argument is compared against a list of identifiers that can * precede a regular expression in the javascript grammar, and returns true if * the argument is found on that list. */ static inline int is_regexp_token_prefix(char *token) { assert(token != NULL); return bsearch(token, regexp_token_prefix, sizeof(regexp_token_prefix) / sizeof(char *), sizeof(char *), bsearch_strcmp) != NULL; } /* Called for every character in state text. * * We copy every character we find when we are in state text to the ring * buffer. This has the side effect of also pushing slash characters that are * part of comments into the buffer, although for parsing purposes these should * be treated as whitespace. This issue is addressed in * enter_state_js_comment_ml_after(). */ static void in_state_js_text(statemachine_ctx *ctx, int start, char chr, int end) { jsparser_ctx *js = CAST(jsparser_ctx *, ctx->user); assert(js != NULL); jsparser_buffer_append_chr(js, chr); } /* This function is called every time we find a slash ('/') character in the * javascript text (except for slashes that close comments or regexp literals). * * Implements the logic to figure out if this slash character is a division * operator or if it opens a regular expression literal. This is heavily * inspired by the syntactic resynchronization for javascript 2.0: * http://www.mozilla.org/js/language/js20-2000-07/rationale/syntax.html * * When we receive a '/', we look at the previous non space character to figure * out if it's the ending of a punctuator that can precede a regexp literal, in * which case we assume the current '/' is part of a regular expression literal * (or the opening of a javascript comment, but that part is dealt with in the * state machine). The exceptions to this are unary operators, so we look back * a second character to rule out '++' and '--'. Although it is not * straightforward to figure out if the binary operator is a postfix of the * previous expression or a prefix of the regular expression, we rule out the * later as it is an uncommon practice. * * If we ruled out the previous token to be a valid regexp preceding * punctuator, we extract the last identifier in the buffer and match against a * list of keywords that are known to precede expressions in the grammar. If we * get a match on any of these keywords, then we are opening a regular * expression, if not, then we have a division operator. * * Known cases that are accepted by the grammar but we handle differently, * although I don't believe there is a legitimate usage for those: * * Division of a regular expression: * var result = /test/ / 5; * * Prefix unary increment of a regular expression: * var result = ++/test/; * * Division of an object literal: * { a: 1 } /x/.exec('x'); * * We only support ascii right now, so unicode characters in identifiers will * be treated as delimiters, effectively breaking the identifier name where * they appear, and this may cause issues in very specific situations. Namely, * if we have a unicode character in an identifier directly preceding a suffix * that matches one of the keywords in regexp_token_prefix[], if this * identifier precedes a / (slash) character: * * var x = testreturn / 5; * * We will interpret that slash as the start of a regular expression, when in * reality it is a division operator. */ static void enter_state_js_slash(statemachine_ctx *ctx, int start, char chr, int end) { jsparser_ctx *js; char buffer[JSPARSER_RING_BUFFER_SIZE]; int pos; assert(ctx != NULL); assert(ctx->user != NULL); js = CAST(jsparser_ctx *, ctx->user); assert(js != NULL); pos = -1; /* Consume the last whitespace. */ if (js_is_whitespace(jsparser_buffer_get(js, pos))) { --pos; } switch (jsparser_buffer_get(js, pos)) { /* Ignore unary increment */ case '+': if (jsparser_buffer_get(js, pos - 1) != '+') { ctx->next_state = JSPARSER_STATE_INT_JS_REGEXP_SLASH; } break; /* Ignore unary decrement */ case '-': if (jsparser_buffer_get(js, pos - 1) != '-') { ctx->next_state = JSPARSER_STATE_INT_JS_REGEXP_SLASH; } break; /* List of punctuator endings except ), ], }, + and - */ case '=': case '<': case '>': case '&': case '|': case '!': case '%': case '*': case '/': case ',': case ';': case '?': case ':': case '^': case '~': case '{': case '(': case '[': case '}': case '\0': ctx->next_state = JSPARSER_STATE_INT_JS_REGEXP_SLASH; break; default: if (jsparser_buffer_last_identifier(js, buffer) && is_regexp_token_prefix(buffer)) { ctx->next_state = JSPARSER_STATE_INT_JS_REGEXP_SLASH; } } jsparser_buffer_append_chr(js, chr); } /* Called at the end of a javascript comment. * * When we open a comment, the initial '/' was inserted into the ring buffer, * but it is not a token and should be considered whitespace for parsing * purposes. * * When we first saw the '/' character, we didn't yet know if it was the * beginning of a comment, a division operator, or a regexp. * * In this function we just replace the inital '/' with a whitespace character, * unless we had a preceding whitespace character, in which case we just remove * the '/'. This is needed to ensure all spaces in the buffer are correctly * folded. */ static void enter_state_js_comment_after(statemachine_ctx *ctx, int start, char chr, int end) { jsparser_ctx *js; assert(ctx != NULL); assert(ctx->user != NULL); js = CAST(jsparser_ctx *, ctx->user); if (js_is_whitespace(jsparser_buffer_get(js, -2))) { (void)jsparser_buffer_pop(js); } else { jsparser_buffer_set(js, -1, ' '); } } static statemachine_definition *create_statemachine_definition() { statemachine_definition *def; def = statemachine_definition_new(JSPARSER_NUM_STATES); if (def == NULL) return NULL; /* TODO(falmeida): Check return value. */ statemachine_definition_populate(def, jsparser_state_transitions, jsparser_states_internal_names); statemachine_in_state(def, JSPARSER_STATE_INT_JS_TEXT, in_state_js_text); statemachine_enter_state(def, JSPARSER_STATE_INT_JS_SLASH, enter_state_js_slash); statemachine_enter_state(def, JSPARSER_STATE_INT_JS_COMMENT_AFTER, enter_state_js_comment_after); return def; } /* Initializes a new jsparser instance. * * Returns a pointer to the new instance or NULL if the initialization * fails. * * Initialization failure is fatal, and if this function fails it may not * deallocate all previsouly allocated memory. */ jsparser_ctx *jsparser_new() { jsparser_ctx *js; js = CAST(jsparser_ctx *, calloc(1, sizeof(jsparser_ctx))); if (js == NULL) return NULL; js->statemachine_def = create_statemachine_definition(); if (js->statemachine_def == NULL) return NULL; js->statemachine = statemachine_new(js->statemachine_def, js); if (js->statemachine == NULL) return NULL; jsparser_reset(js); return js; } /* Returns a pointer to a context which is a duplicate of the jsparser src. */ jsparser_ctx *jsparser_duplicate(jsparser_ctx *src) { jsparser_ctx *dst; assert(src != NULL); dst = jsparser_new(); if (dst == NULL) return NULL; jsparser_copy(dst, src); return dst; } /* Copies the context of the jsparser pointed to by src to the jsparser dst. * * The state machine definition is preserved since it is read only. */ void jsparser_copy(jsparser_ctx *dst, jsparser_ctx *src) { dst->buffer_start = src->buffer_start; dst->buffer_end = src->buffer_end; memcpy(dst->buffer, src->buffer, sizeof(src->buffer)); statemachine_copy(dst->statemachine, src->statemachine, dst->statemachine_def, dst); } void jsparser_reset(jsparser_ctx *ctx) { assert(ctx != NULL); ctx->statemachine->current_state = 0; ctx->buffer_start = 0; ctx->buffer_end = 0; } int jsparser_state(jsparser_ctx *ctx) { return state_external(ctx->statemachine->current_state); } int jsparser_parse(jsparser_ctx *ctx, const char *str, int size) { int internal_state; internal_state = statemachine_parse(ctx->statemachine, str, size); return state_external(internal_state); } void jsparser_delete(jsparser_ctx *ctx) { assert(ctx != NULL); statemachine_delete(ctx->statemachine); statemachine_definition_delete(ctx->statemachine_def); free(ctx); } #ifdef __cplusplus } /* namespace security_streamhtmlparser */ #endif /* __cplusplus */ ctemplate-ctemplate-2.4/src/htmlparser/jsparser.h000066400000000000000000000140611363122316600222700ustar00rootroot00000000000000/* Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * --- * * Author: falmeida@google.com (Filipe Almeida) */ #ifndef SECURITY_STREAMHTMLPARSER_JSPARSER_H #define SECURITY_STREAMHTMLPARSER_JSPARSER_H #include #include "htmlparser/statemachine.h" #ifdef __cplusplus namespace ctemplate_htmlparser { #endif /* __cplusplus */ /* Size of the ring buffer used to lookup the last token in the javascript * stream. The size is pretty much arbitrary at this point but must be bigger * than the biggest token we want to lookup plus 3: Two delimiters plus an empty * ring buffer slot. */ #define JSPARSER_RING_BUFFER_SIZE 18 enum js_state_external_enum { JSPARSER_STATE_TEXT, JSPARSER_STATE_Q, JSPARSER_STATE_DQ, JSPARSER_STATE_REGEXP, JSPARSER_STATE_COMMENT }; /* Stores the context of the javascript parser. * * If this structure is changed, jsparser_new(), jsparser_copy() and * jsparser_reset() should be updated accordingly. */ typedef struct jsparser_ctx_s { /* Reference to the statemachine context. */ statemachine_ctx *statemachine; /* Reference to the statemachine definition. * * It should be readonly and contain the same values across jsparser * instances. */ /* TODO(falmeida): Change statemachine_def to const. */ statemachine_definition *statemachine_def; /* Index to the start of the buffer. */ int buffer_start; /* Index the current writing position (end of the buffer plus one). */ int buffer_end; /* Ring buffer used to lookup the last token. */ char buffer[JSPARSER_RING_BUFFER_SIZE]; } jsparser_ctx; void jsparser_reset(jsparser_ctx *ctx); jsparser_ctx *jsparser_new(void); /* Returns a pointer to a context which is a duplicate of the jsparser src. */ jsparser_ctx *jsparser_duplicate(jsparser_ctx *src); /* Copies the context of the jsparser pointed to by src to the jsparser dst. */ void jsparser_copy(jsparser_ctx *dst, jsparser_ctx *src); int jsparser_state(jsparser_ctx *ctx); int jsparser_parse(jsparser_ctx *ctx, const char *str, int size); void jsparser_delete(jsparser_ctx *ctx); /** * Ring buffer functions. * * These functions are only exported for testing and should not be called from * outside of jsparser.c in production code. */ /* Appends a character to the ring buffer. * * Sequences of whitespaces and newlines are folded into one character. */ void jsparser_buffer_append_chr(jsparser_ctx *js, char chr); /* Appends a string to the ring buffer. * * Sequences of whitespaces and newlines are folded into one character. */ void jsparser_buffer_append_str(jsparser_ctx *js, const char *str); /* Returns the last appended character and removes it from the buffer. If the * buffer is empty, then it returns ASCII 0 ('\0'). */ char jsparser_buffer_pop(jsparser_ctx *js); /* Returns the value of the character at a certain index in the buffer or an * ASCII 0 ('\0') character if the index is extends beyond the size of the * buffer, either because we don't have as many characters in the buffer, or * because the index points to a place bigger than the size of the buffer.. * * Index positions must be negative, where -1 is the last character appended to * the buffer. */ char jsparser_buffer_get(jsparser_ctx *js, int pos); /* Sets the value of the character at a certain index in the buffer. Returns * true if the write was successful or false if there was an attempt to write * outside of the buffer boundaries. * * Index positions are negative, were -1 is the last character appended to the * buffer. Using positive integers for the index will result in undefined * behaviour. */ int jsparser_buffer_set(jsparser_ctx *js, int pos, char value); /* Copies a slice of the buffer to the string pointed to by output. start and * end are the indexes of the sliced region. If the start argument extends * beyond the beginning of the buffer, the slice will only contain characters * starting from beginning of the buffer. */ void jsparser_buffer_slice(jsparser_ctx *js, char *buffer, int start, int end); /* Copy the last javascript identifier or keyword found in the buffer to the * string pointed by identifier. */ int jsparser_buffer_last_identifier(jsparser_ctx *js, char *identifier); #define jsparser_parse_chr(a,b) jsparser_parse(a, &(b), 1); #ifdef __cplusplus #define jsparser_parse_str(a,b) jsparser_parse(a, b, \ static_cast(strlen(b))); #else #define jsparser_parse_str(a,b) jsparser_parse(a, b, (int)strlen(b)); #endif #ifdef __cplusplus } /* namespace security_streamhtmlparser */ #endif /* __cplusplus */ #endif /* SECURITY_STREAMHTMLPARSER_JSPARSER_H */ ctemplate-ctemplate-2.4/src/htmlparser/jsparser_fsm.config000066400000000000000000000130601363122316600241510ustar00rootroot00000000000000# Copyright (c) 2008, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # # Author: falmeida@google.com (Filipe Almeida) name = 'jsparser' comment = 'Simplified finite state machine for tracking of javascript states' condition('q', '\''), condition('dq', '\"'), condition('/', '/'), condition('*', '*'), condition('[', '['), condition(']', ']'), condition('lf', '\n'), condition('backslash', '\\'), condition('default', '[:default:]') # Main javascript body. state(name = 'js_text', external = 'text', transitions = [ ['q', 'js_q'], ['dq', 'js_dq'], ['/', 'js_slash'], ['default', 'js_text'] ]) # Single quoted string literal. state(name = 'js_q', external = 'q', transitions = [ ['backslash', 'js_q_e'], ['q', 'js_text'], ['default', 'js_q'] ]) # Javascript escaped character in a single quoted string literal. state(name = 'js_q_e', external = 'q', transitions = [ ['default', 'js_q'] ]) # Double quoted string literal state(name = 'js_dq', external = 'dq', transitions = [ ['backslash', 'js_dq_e'], ['dq', 'js_text'], ['default', 'js_dq'] ]) # Javascript escaped character in a double quoted string literal. state(name = 'js_dq_e', external = 'dq', transitions = [ ['default', 'js_dq'] ]) # Possible start of a javascript comment. state(name = 'js_slash', external = 'text', transitions = [ ['/', 'js_comment_ln'], ['*', 'js_comment_ml'], ['default', 'js_text'] ]) # Possible start of a regular expression literal. # # The state diagram does not reach this state directly. When js_slash is # reached, the function enter_state_js_slash() is called, which checks if the # last token belongs to the set of tokens that can precede a regular # expression, in which case it changes the state to js_regexp_slash. # # For more information please read the comments in # jsparser.c:enter_state_js_slash(). state(name = 'js_regexp_slash', external = 'text', transitions = [ ['/', 'js_comment_ln'], ['*', 'js_comment_ml'], ['backslash', 'js_regexp_e'], ['[', 'js_regexp_bracket'], ['default', 'js_regexp'] ]) # Regular expression literal. state(name = 'js_regexp', external = 'regexp', transitions = [ ['backslash', 'js_regexp_e'], ['[', 'js_regexp_bracket'], ['/', 'js_text'], ['default', 'js_regexp'] ]) # Regexp bracket expression state(name = 'js_regexp_bracket', external = 'regexp', transitions = [ ['backslash', 'js_regexp_bracket_e'], [']', 'js_regexp'], ['default', 'js_regexp_bracket'] ]) # Backslash escaped regexp bracket expression state(name = 'js_regexp_bracket_e', external = 'regexp', transitions = [ ['default', 'js_regexp_bracket'] ]) # Escaped regular expression char. state(name = 'js_regexp_e', external = 'regexp', transitions = [ ['default', 'js_regexp'] ]) # Start of a single line javascript comment (//). state(name = 'js_comment_ln', external = 'comment', transitions = [ ['lf', 'js_comment_after'], ['default', 'js_comment_ln'] ]) # Start of a multiline javascript comment (/*). state(name = 'js_comment_ml', external = 'comment', transitions = [ ['*', 'js_comment_ml_close'], ['default', 'js_comment_ml'] ]) # Close of a multiline javascript comment (*/). state(name = 'js_comment_ml_close', external = 'comment', transitions = [ ['/', 'js_comment_after'], ['default', 'js_comment_ml'] ]) # Ending character of a javascript comment. # In can either be a '/ in the case of a multiline comment, or a line # terminator in the case of a single line comment. # This is needed so we don't insert the '/' or the new line character into the # ring buffer. state(name = 'js_comment_after', external = 'text', transitions = [ ['q', 'js_q'], ['dq', 'js_dq'], ['/', 'js_slash'], ['default', 'js_text'] ]) ctemplate-ctemplate-2.4/src/htmlparser/statemachine.cc000066400000000000000000000346651363122316600232560ustar00rootroot00000000000000/* Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * --- * * Author: falmeida@google.com (Filipe Almeida) */ #include #include #include #include #include "htmlparser/statemachine.h" /* So we can support both C and C++ compilers, we use the CAST() macro instead * of using C style casts or static_cast<>() directly. */ #ifdef __cplusplus #define CAST(type, expression) (static_cast(expression)) #else #define CAST(type, expression) ((type)(expression)) #endif #ifdef __cplusplus namespace ctemplate_htmlparser { #endif #define MAX_CHAR_8BIT 256 /* Populates the statemachine definition. */ void statemachine_definition_populate(statemachine_definition *def, const int* const* transition_table, const char* const* state_names) { assert(def != NULL); assert(transition_table != NULL); def->transition_table = transition_table; def->state_names = state_names; } /* Add's the callback for the event in_state that is called when the * statemachine is in state st. * * This event is called everytime the the statemachine is in the specified * state forevery character in the input stream even if the state remains * the same. * * This is event is the last event to be called and is fired after both events * exit_state and enter_state. */ void statemachine_in_state(statemachine_definition *def, int st, state_event_function func) { assert(def != NULL); assert(st < def->num_states); def->in_state_events[st] = func; } /* Add's the callback for the event enter_state that is called when the * statemachine enters state st. * * This event is fired after the event exit_state but before the event * in_state. */ void statemachine_enter_state(statemachine_definition *def, int st, state_event_function func) { assert(def != NULL); assert(st < def->num_states); def->enter_state_events[st] = func; } /* Add's the callback for the event exit_state that is called when the * statemachine exits from state st. * * This is the first event to be called and is fired before both the events * enter_state and in_state. */ void statemachine_exit_state(statemachine_definition *def, int st, state_event_function func) { assert(def != NULL); assert(st < def->num_states); def->exit_state_events[st] = func; } /* Initializes a new statemachine definition with a defined number of states. * * Returns NULL if initialization fails. * * Initialization failure is fatal, and if this function fails it may not * deallocate all previsouly allocated memory. */ statemachine_definition *statemachine_definition_new(int states) { statemachine_definition *def; def = CAST(statemachine_definition *, malloc(sizeof(statemachine_definition))); if (def == NULL) return NULL; def->in_state_events = CAST(state_event_function *, calloc(states, sizeof(state_event_function))); if (def->in_state_events == NULL) return NULL; def->enter_state_events =CAST(state_event_function *, calloc(states, sizeof(state_event_function))); if (def->enter_state_events == NULL) return NULL; def->exit_state_events = CAST(state_event_function *, calloc(states, sizeof(state_event_function))); if (def->exit_state_events == NULL) return NULL; def->num_states = states; def->state_names = NULL; return def; } /* Deallocates a statemachine definition object */ void statemachine_definition_delete(statemachine_definition *def) { assert(def != NULL); free(def->in_state_events); free(def->enter_state_events); free(def->exit_state_events); free(def); } /* Returns the current state. */ int statemachine_get_state(statemachine_ctx *ctx) { return ctx->current_state; } /* Sets the current state. * * It calls the exit event for the old state and the enter event for the state * we intend to move into. * * Since this state change was not initiated by a character in the input stream * we pass a null char to the event functions. */ void statemachine_set_state(statemachine_ctx *ctx, int state) { statemachine_definition *def; assert(ctx != NULL); assert(ctx->definition != NULL); def = ctx->definition; assert(state < def->num_states); ctx->next_state = state; if (ctx->current_state != ctx->next_state) { if (def->exit_state_events[ctx->current_state]) def->exit_state_events[ctx->current_state](ctx, ctx->current_state, '\0', ctx->next_state); if (def->enter_state_events[ctx->next_state]) def->enter_state_events[ctx->next_state](ctx, ctx->current_state, '\0', ctx->next_state); } ctx->current_state = state; } /* Reset the statemachine. * * The state is set to the initialization values. This includes setting the * state to the default state (0), stopping recording and setting the line * number to 1. */ void statemachine_reset(statemachine_ctx *ctx) { ctx->current_state = 0; ctx->next_state = 0; ctx->record_buffer[0] = '\0'; ctx->record_pos = 0; ctx->recording = 0; ctx->line_number = 1; ctx->column_number = 1; } /* Initializes a new statemachine. Receives a statemachine definition object * that should have been initialized with statemachine_definition_new() and a * user reference to be used by the caller. * * The user reference is used by the caller to store any instance specific data * the caller may need and is typically used to propagate context information * to the event callbacks. The user pointer can just be set to NULL if the * caller doesn't need it. * * Returns NULL if initialization fails. * * Initialization failure is fatal, and if this function fails it may not * deallocate all previously allocated memory. */ statemachine_ctx *statemachine_new(statemachine_definition *def, void *user) { statemachine_ctx *ctx; assert(def != NULL); ctx = CAST(statemachine_ctx *, malloc(sizeof(statemachine_ctx))); if (ctx == NULL) return NULL; statemachine_reset(ctx); ctx->definition = def; ctx->user = user; return ctx; } /* Returns a pointer to a context which is a duplicate of the statemachine src. * The statemachine definition and the user pointer have to be provided since * these references are not owned by the statemachine itself, but this will be * shallow copies as they point to data structures we do not own. */ statemachine_ctx *statemachine_duplicate(statemachine_ctx *src, statemachine_definition *def, void *user) { statemachine_ctx *dst; assert(src != NULL); dst = statemachine_new(def, user); if (dst == NULL) return NULL; statemachine_copy(dst, src, def, user); return dst; } /* Copies the context of the statemachine pointed to by src to the statemachine * provided by dst. * The statemachine definition and the user pointer have to be provided since * these references are not owned by the statemachine itself. */ void statemachine_copy(statemachine_ctx *dst, statemachine_ctx *src, statemachine_definition *def, void *user) { memcpy(dst, src, sizeof(statemachine_ctx)); dst->definition = def; dst->user = user; } /* Deallocates a statemachine object */ void statemachine_delete(statemachine_ctx *ctx) { assert(ctx != NULL); free(ctx); } /* Starts recording the current input stream into an internal buffer. * The current input character is included in the recording. */ void statemachine_start_record(statemachine_ctx *ctx) { assert(ctx != NULL); ctx->record_buffer[0] = '\0'; ctx->record_pos = 0; ctx->recording = 1; } /* Stops recording the current input stream. * The last input character is not included in the recording. * This function returns a pointer to the recorded string buffer. */ const char *statemachine_stop_record(statemachine_ctx *ctx) { assert(ctx != NULL); assert(ctx->recording); ctx->record_buffer[ctx->record_pos] = '\0'; ctx->recording = 0; return ctx->record_buffer; } /* Returns a pointer to the record string buffer. */ const char *statemachine_record_buffer(statemachine_ctx *ctx) { return ctx->record_buffer; } void statemachine_encode_char(char schr, char *output, size_t len) { unsigned char chr = schr; if (chr == '\'') { strncpy(output, "\\'", len); } else if (chr == '\\') { strncpy(output, "\\\\", len); /* Like isprint() but not dependent on locale. */ } else if (chr >= 32 && chr <= 126) { snprintf(output, len, "%c", chr); } else if (chr == '\n') { strncpy(output, "\\n", len); } else if (chr == '\r') { strncpy(output, "\\r", len); } else if (chr == '\t') { strncpy(output, "\\t", len); } else { snprintf(output, len, "\\x%.2x", chr); } output[len - 1] = '\0'; } /* Sets the error message in case of a transition error. * * Called from statemachine_parse to set the error message in case of a * transition error. */ static void statemachine_set_transition_error_message(statemachine_ctx *ctx) { char encoded_char[10]; statemachine_encode_char(ctx->current_char, encoded_char, sizeof(encoded_char)); if (ctx->definition->state_names) { snprintf(ctx->error_msg, STATEMACHINE_MAX_STR_ERROR, "Unexpected character '%s' in state '%s'", encoded_char, ctx->definition->state_names[ctx->current_state]); } else { snprintf(ctx->error_msg, STATEMACHINE_MAX_STR_ERROR, "Unexpected character '%s'", encoded_char); } } /* Parses the input html stream and returns the finishing state. * * Returns STATEMACHINE_ERROR if unable to parse the input. If * statemachine_parse() is called after an error situation was encountered * the behaviour is unspecified. */ /* TODO(falmeida): change int size to size_t size */ int statemachine_parse(statemachine_ctx *ctx, const char *str, int size) { int i; const int* const* state_table = ctx->definition->transition_table; statemachine_definition *def; assert(ctx !=NULL && ctx->definition != NULL && ctx->definition->transition_table != NULL); if (size < 0) { snprintf(ctx->error_msg, STATEMACHINE_MAX_STR_ERROR, "%s", "Negative size in statemachine_parse()."); return STATEMACHINE_ERROR; } def = ctx->definition; for (i = 0; i < size; i++) { ctx->current_char = *str; ctx->next_state = state_table[ctx->current_state][CAST(unsigned char, *str)]; if (ctx->next_state == STATEMACHINE_ERROR) { statemachine_set_transition_error_message(ctx); return STATEMACHINE_ERROR; } if (ctx->current_state != ctx->next_state) { if (def->exit_state_events[ctx->current_state]) def->exit_state_events[ctx->current_state](ctx, ctx->current_state, *str, ctx->next_state); } if (ctx->current_state != ctx->next_state) { if (def->enter_state_events[ctx->next_state]) def->enter_state_events[ctx->next_state](ctx, ctx->current_state, *str, ctx->next_state); } if (def->in_state_events[ctx->next_state]) def->in_state_events[ctx->next_state](ctx, ctx->current_state, *str, ctx->next_state); /* We need two bytes left so we can NULL terminate the string. */ if (ctx->recording && STATEMACHINE_RECORD_BUFFER_SIZE - 1 > ctx->record_pos) { ctx->record_buffer[ctx->record_pos++] = *str; ctx->record_buffer[ctx->record_pos] = '\0'; } /* TODO(falmeida): Should clarify the contract here, since an event can change * ctx->next_state and we need this functionality */ ctx->current_state = ctx->next_state; ctx->column_number++; if (*str == '\n') { ctx->line_number++; ctx->column_number = 1; } str++; } return ctx->current_state; } #ifdef __cplusplus } /* namespace security_streamhtmlparser */ #endif ctemplate-ctemplate-2.4/src/htmlparser/statemachine.h000066400000000000000000000203271363122316600231060ustar00rootroot00000000000000/* Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * --- * * Author: falmeida@google.com (Filipe Almeida) */ #ifndef SECURITY_STREAMHTMLPARSER_STATEMACHINE_H #define SECURITY_STREAMHTMLPARSER_STATEMACHINE_H #include #ifdef __cplusplus namespace ctemplate_htmlparser { #endif /* TODO(falmeida): I'm not sure about these limits, but since right now we only * have 24 states it should be fine */ enum { STATEMACHINE_ERROR = 127 }; #define STATEMACHINE_RECORD_BUFFER_SIZE 256 #define STATEMACHINE_MAX_STR_ERROR 80 struct statemachine_ctx_s; typedef void(*state_event_function)(struct statemachine_ctx_s *, int, char, int); typedef struct statemachine_definition_s { int num_states; const int* const* transition_table; /* Array containing the name of the states as a C string. * This field is optional and if not in use it should be set to NULL. */ const char* const* state_names; state_event_function *in_state_events; state_event_function *enter_state_events; state_event_function *exit_state_events; } statemachine_definition; typedef struct statemachine_ctx_s { int current_state; int next_state; statemachine_definition *definition; char current_char; /* Current line number. */ int line_number; /* Current column number. */ int column_number; char record_buffer[STATEMACHINE_RECORD_BUFFER_SIZE]; size_t record_pos; /* True if we are recording the stream to record_buffer. */ int recording; /* In case there was an error (we are in state STATEMACHINE_ERROR), it will * contain a human readable description of the error. */ char error_msg[STATEMACHINE_MAX_STR_ERROR]; /* Storage space for the layer above. */ void *user; } statemachine_ctx; /* Populates the statemachine definition. * * Receives a transition table and an optional array of state names. It uses * this data to populate the state machine definition. * * The transition table structure is a list of lists of ints (int **). The * outer list indexes the source state and the inner list contains the * destination state for each of the possible input characters: * * const int* const* transitions[source][input] == destination. * * The optional argument state_names points to a list of strings containing * human readable state names. These strings are used when reporting error * messages. */ void statemachine_definition_populate(statemachine_definition *def, const int* const* transition_table, const char* const* state_names); void statemachine_in_state(statemachine_definition *def, int st, state_event_function func); void statemachine_enter_state(statemachine_definition *def, int st, state_event_function func); void statemachine_exit_state(statemachine_definition *def, int st, state_event_function func); statemachine_definition *statemachine_definition_new(int states); void statemachine_definition_delete(statemachine_definition *def); int statemachine_get_state(statemachine_ctx *ctx); void statemachine_set_state(statemachine_ctx *ctx, int state); void statemachine_start_record(statemachine_ctx *ctx); const char *statemachine_stop_record(statemachine_ctx *ctx); const char *statemachine_record_buffer(statemachine_ctx *ctx); /* Returns the the number of characters currently stored in the record buffer. */ static inline size_t statemachine_record_length(statemachine_ctx *ctx) { return ctx->record_pos + 1; } /* Return the current line number. */ static inline int statemachine_get_line_number(statemachine_ctx *ctx) { return ctx->line_number; } /* Set the current line number. */ static inline void statemachine_set_line_number(statemachine_ctx *ctx, int line) { ctx->line_number = line; } /* Return the current column number. */ static inline int statemachine_get_column_number(statemachine_ctx *ctx) { return ctx->column_number; } /* Set the current column number. */ static inline void statemachine_set_column_number(statemachine_ctx *ctx, int column) { ctx->column_number = column; } /* Retrieve a human readable error message in case an error occurred. * * NULL is returned if the parser didn't encounter an error. */ static inline const char *statemachine_get_error_msg(statemachine_ctx *ctx) { if (ctx->next_state == STATEMACHINE_ERROR) { return ctx->error_msg; } else { return NULL; } } /* Reset the statemachine. * * The state is set to the initialization values. This includes setting the * state to the default state (0), stopping recording and setting the line * number to 1. */ void statemachine_reset(statemachine_ctx *ctx); /* Initializes a new statemachine. Receives a statemachine definition object * that should have been initialized with statemachine_definition_new() and a * user reference to be used by the caller. * * Returns NULL if initialization fails. * * Initialization failure is fatal, and if this function fails it may not * deallocate all previsouly allocated memory. */ statemachine_ctx *statemachine_new(statemachine_definition *def, void *user); /* Returns a pointer to a context which is a duplicate of the statemachine src. * The statemachine definition and the user pointer have to be provided since * these references are not owned by the statemachine itself. */ statemachine_ctx *statemachine_duplicate(statemachine_ctx *ctx, statemachine_definition *def, void *user); /* Copies the context of the statemachine pointed to by src to the statemachine * provided by dst. * The statemachine definition and the user pointer have to be provided since * these references are not owned by the statemachine itself. */ void statemachine_copy(statemachine_ctx *dst, statemachine_ctx *src, statemachine_definition *def, void *user); int statemachine_parse(statemachine_ctx *ctx, const char *str, int size); void statemachine_delete(statemachine_ctx *ctx); /***** * The following functions are only exported for testing purposes and should * be treated as private. */ /* Encode the character as an escaped C string. * * Encode the character chr into the string output. Writes at most len * characters to the output string but makes sure output is NULL terminated. */ void statemachine_encode_char(char chr, char *output, size_t len); #ifdef __cplusplus } /* namespace security_streamhtmlparser */ #endif #endif /* SECURITY_STREAMHTMLPARSER_STATEMACHINE_H */ ctemplate-ctemplate-2.4/src/indented_writer.h000066400000000000000000000112031363122316600214370ustar00rootroot00000000000000// Copyright (c) 2009, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: williasr@google.com (Scott Williams) #ifndef TEMPLATE_INDENTED_WRITER_H_ #define TEMPLATE_INDENTED_WRITER_H_ #include #include namespace ctemplate { using std::string; // An indented writer is a wrapper around a string buffer. It takes care of // tracking and applying leading whitespace to the buffer at the beginning of // new lines. class IndentedWriter { public: IndentedWriter(string* out, int starting_indentation) : out_(out), current_indentation_(starting_indentation), original_indentation_(starting_indentation), line_state_(AT_BEGINNING) { } ~IndentedWriter() { assert(original_indentation_ == current_indentation_); } // Append some output to the buffer. If the string ends with a newline, then // the output buffer will be indented before the next Write() call. If the // output contains embedded newlines, these won't have proper indentation, so // call Write() at least once per physical line of output. void Write(string s1, string s2 = string(), string s3 = string(), string s4 = string(), string s5 = string(), string s6 = string(), string s7 = string()) { DoWrite(s1); if (!s2.empty()) DoWrite(s2); if (!s3.empty()) DoWrite(s3); if (!s4.empty()) DoWrite(s4); if (!s5.empty()) DoWrite(s5); if (!s6.empty()) DoWrite(s6); if (!s7.empty()) DoWrite(s7); } // Increment the indentation level. This only has a meaning after outputting a // complete line (otherwise, are you saying you want to modify the indentation // of the current line or the next line?) void Indent() { assert(line_state_ == AT_BEGINNING); current_indentation_ += kIndent; } // Decrement the indentation level. This only has a meaning after outputting a // complete line (otherwise, are you saying you want to modify the indentation // of the current line or the next line?) void Dedent() { assert(line_state_ == AT_BEGINNING); current_indentation_ -= kIndent; assert(current_indentation_ >= original_indentation_); } // Get access to the underlying indentation level and string buffer. Most // useful for interfacing with non-IndentedWriter printing code. int GetIndent() const { return current_indentation_; } string* GetBuffer() { return out_; } private: void DoWrite(const string& line) { if (line_state_ == AT_BEGINNING) { IndentLine(); } out_->append(line); if (EndsWithNewline(line)) { line_state_ = AT_BEGINNING; } else { line_state_ = MID_LINE; } } static bool EndsWithNewline(const string& line) { return !line.empty() && (*(line.end() - 1) == '\n'); } void IndentLine() { assert(line_state_ == AT_BEGINNING); out_->append(string(current_indentation_, ' ') + (current_indentation_ ? " " : "")); } string* out_; int current_indentation_; int original_indentation_; enum LineState { AT_BEGINNING, MID_LINE } line_state_; const static int kIndent = 2; // num spaces to indent each level }; } #endif // TEMPLATE_INDENTED_WRITER_H_ ctemplate-ctemplate-2.4/src/make_tpl_varnames_h.cc000077500000000000000000000436601363122316600224250ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // A utility for checking syntax and generating headers to // use with Google Templates. // // For example: // // > /make_tpl_varnames_h some_template_file.tpl // // This creates the header file some_template_file.tpl.varnames.h. If // there are any syntax errors they are reported to stderr (in which // case, no header file is created). // // // Exit code is the number of templates we were unable to parse. // // Headers can be all written to one output file (via --outputfile) // or written to one output file per template processed (via --header_dir). // As such, we have a first stage where we load each template and generate // its headers and a second stage where we write the headers to disk. // // TODO(jad): Prevent -f and -o from being used together. // Previously -o would be silently ignored. // This is for windows. Even though we #include config.h, just like // the files used to compile the dll, we are actually a *client* of // the dll, so we don't get to decl anything. #include #undef CTEMPLATE_DLL_DECL #include // for toupper(), isalnum() #include #ifdef HAVE_GETOPT_H # include #endif #include #include #include #include #ifdef HAVE_UNISTD_H # include #endif #include #include #include #include #include using std::set; using std::string; using std::vector; using ctemplate::Template; enum {LOG_INFO, LOG_WARNING, LOG_ERROR, LOG_FATAL}; // Holds information on each template we process. struct TemplateRecord { const string name; // filename given on cmd-line (may be relative bool error; // true iff an error occurred during template loading string header_entries; // output of tpl->WriteHeaderEntries() explicit TemplateRecord(const string& aname) : name(aname), error(false) { } }; static void LogPrintf(int severity, int should_log_info, const char* pat, ...) { if (severity == LOG_INFO && !should_log_info) return; if (severity == LOG_FATAL) fprintf(stderr, "FATAL ERROR: "); va_list ap; va_start(ap, pat); vfprintf(stderr, pat, ap); va_end(ap); fprintf(stderr, "\n"); if (severity == LOG_FATAL) exit(1); } // prints to outfile -- usually stdout or stderr static void Usage(const char* argv0, FILE* outfile) { fprintf(outfile, "USAGE: %s [-t] [-o] [-s] [-f]" " [-n] [-d] [-q] ...\n", argv0); fprintf(outfile, " -t --template_dir= Root directory of templates\n" " -o --header_dir= Where to place output files\n" " -s --outputfile_suffix=\n" " outname = inname + suffix\n" " -f --outputfile=\n" " outname = filename (when given, \n" " --header_dir is ignored)\n" " -n --noheader Just check syntax, no output\n" " -d --dump_templates Cause templates dump contents\n" " -q --nolog_info Only log on error\n" " --v=-1 Obsolete, confusing synonym for -q\n" " -h --help This help\n" " -V --version Version information\n"); fprintf(outfile, "\n" "This program checks the syntax of one or more google templates.\n" "By default (without -n) it also emits a header file to an output\n" "directory that defines all valid template keys. This can be used\n" "in programs to minimze the probability of typos in template code.\n"); } static void Version(FILE* outfile) { fprintf(outfile, "make_tpl_varnames_h " " (part of " PACKAGE_STRING ")" "\n\n" "Copyright 1998 Google Inc.\n" "\n" "This is BSD licensed software; see the source for copying conditions\n" "and license information.\n" "There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A\n" "PARTICULAR PURPOSE.\n" ); } // Removes all non alphanumeric characters from a string to form a // valid C identifier to use as a double-inclusion guard. static void ConvertToIdentifier(string* s) { for (string::size_type i = 0; i < s->size(); i++) { if (!isalnum((*s)[i])) (*s)[i] = '_'; else (*s)[i] = toupper((*s)[i]); } } // Returns the given header entries wrapped with a compiler guard // whose name is generated from the output_file name. static string WrapWithGuard(const string& output_file, const string& header_entries) { string guard(string("TPL_") + output_file); ConvertToIdentifier(&guard); guard.append("_H_"); string out; out.append(string("#ifndef ") + guard + "\n"); out.append(string("#define ") + guard + "\n\n"); // Now append the header-entry info to the intro above out.append(header_entries); out.append(string("\n#endif // ") + guard + "\n"); return out; } // Generates a multi-line comment that will go at the top of the output file. // The comment includes the filename(s) that produced the output, one per line. static string Boilerplate(const string& progname, const vector& filenames) { string out(string("//\n")); if (filenames.size() > 1) out.append("// This header file auto-generated for the templates\n"); else out.append("// This header file auto-generated for the template\n"); for (vector::size_type i = 0; i < filenames.size(); ++i) out.append("// " + filenames[i] + "\n"); out.append("// by " + progname + "\n" + "// DO NOT MODIFY THIS FILE DIRECTLY\n" + "//\n"); return out; } // Returns true iff line is empty or only contains whitespace // (space, horizontal tab, vertical tab, form feed, carriage return). static bool LineIsAllWhitespace(const string& input) { static const string kWhitespace(" \f\t\v\r"); return input.find_first_not_of(kWhitespace) == string::npos; } // Splits the input string into lines using the newline (\n) // as delimiter. The newlines are discarded. // An empty string input results in one empty line output. // // Examples: "Hello\nWorld\n" input results in two lines, // "Hello" and "World". // Same result for "Hello\nWorld" (not newline terminated). // static vector SplitIntoLines(const string &input) { vector lines; string::size_type begin_index = 0; string::size_type input_len = input.length(); while (1) { string::size_type end_index = input.find_first_of('\n', begin_index); if (end_index == string::npos) { lines.push_back(input.substr(begin_index)); break; } lines.push_back(input.substr(begin_index, (end_index - begin_index))); begin_index = end_index + 1; if (begin_index >= input_len) // To avoid adding a trailing empty line. break; } return lines; } // Receives header entries concatenated together from one or more // templates and returns a string with the duplicate lines removed. // // Duplicate lines that contain only whitespace are not removed, // all other duplicate lines (identical #include directives and // identical variable definitions) are removed. If the last // (or only) input line did not terminate with newline, we add one. // // Consider the following two templates: // ex1.tpl:

{{USER}}

// ex2.tpl: {{USER}} // // The header entries for ex1.tpl are: // #include "template/template_string.h" // static const ::ctemplate::StaticTemplateString ke_USER = // STS_INIT_WITH_HASH(ke_USER, "USER", 3254611514008215315LLU); // // The header entries for ex2.tpl are: // #include "template/template_string.h" // static const ::ctemplate::StaticTemplateString ke_URL = // STS_INIT_WITH_HASH(ke_URL, "URL", 1026025273225241985LLU); // static const ::ctemplate::StaticTemplateString ke_USER = // STS_INIT_WITH_HASH(ke_USER, "USER", 3254611514008215315LLU); // // Simply concatenating both header entries will result in // duplicate #include directives and duplicate definitions of // the ke_USER variable. This function instead outputs: // // #include "template/template_string.h" // static const ::ctemplate::StaticTemplateString ke_USER = // STS_INIT_WITH_HASH(ke_USER, "USER", 3254611514008215315LLU); // static const ::ctemplate::StaticTemplateString ke_URL = // STS_INIT_WITH_HASH(ke_URL, "URL", 1026025273225241985LLU); // static string TextWithDuplicateLinesRemoved(const string& header_entries) { string output; set lines_seen; vector lines = SplitIntoLines(header_entries); const int lines_len = lines.size(); for (int i = 0; i < lines_len; ++i) { const string& line = lines[i]; if (LineIsAllWhitespace(line) || // Blank lines always go in !lines_seen.count(line)) { // So do new lines output.append(line); output.append("\n"); lines_seen.insert(line); } } return output; } // Writes the given text to the filename header_file. // Returns true if it succeeded, false otherwise. static bool WriteToDisk(bool log_info, const string& output_file, const string& text) { FILE* outfile = fopen(output_file.c_str(), "wb"); if (!outfile) { LogPrintf(LOG_ERROR, log_info, "Can't open %s", output_file.c_str()); return false; } LogPrintf(LOG_INFO, log_info, "Creating %s", output_file.c_str()); if (fwrite(text.c_str(), 1, text.length(), outfile) != text.length()) { LogPrintf(LOG_ERROR, log_info, "Can't write %s: %s", output_file.c_str(), strerror(errno)); } fclose(outfile); return true; } int main(int argc, char **argv) { string FLAG_template_dir(ctemplate::kCWD); // "./" string FLAG_header_dir(ctemplate::kCWD); ctemplate::NormalizeDirectory(&FLAG_header_dir); // adds trailing slash string FLAG_outputfile_suffix(".varnames.h"); string FLAG_outputfile(""); bool FLAG_header = true; bool FLAG_dump_templates = false; bool FLAG_log_info = true; #if defined(HAVE_GETOPT_LONG) static struct option longopts[] = { {"help", 0, NULL, 'h'}, {"version", 0, NULL, 'V'}, {"template_dir", 1, NULL, 't'}, {"header_dir", 1, NULL, 'o'}, {"outputfile_suffix", 1, NULL, 's'}, {"outputfile", 1, NULL, 'f'}, {"noheader", 0, NULL, 'n'}, {"dump_templates", 0, NULL, 'd'}, {"nolog_info", 0, NULL, 'q'}, {"v", 1, NULL, 'q'}, {0, 0, 0, 0} }; int option_index; # define GETOPT(argc, argv) getopt_long(argc, argv, "t:o:s:f:ndqhV", \ longopts, &option_index) #elif defined(HAVE_GETOPT_H) # define GETOPT(argc, argv) getopt(argc, argv, "t:o:s:f:ndqhV") #else // TODO(csilvers): implement something reasonable for windows # define GETOPT(argc, argv) -1 int optind = 1; // first non-opt argument const char* optarg = ""; // not used #endif int r = 0; while (r != -1) { // getopt()/getopt_long() return -1 upon no-more-input r = GETOPT(argc, argv); switch (r) { case 't': FLAG_template_dir.assign(optarg); break; case 'o': FLAG_header_dir.assign(optarg); break; case 's': FLAG_outputfile_suffix.assign(optarg); break; case 'f': FLAG_outputfile.assign(optarg); break; case 'n': FLAG_header = false; break; case 'd': FLAG_dump_templates = true; break; case 'q': FLAG_log_info = false; break; case 'V': Version(stdout); return 0; break; case 'h': Usage(argv[0], stderr); return 0; break; case -1: break; // means 'no more input' default: Usage(argv[0], stderr); return 1; break; } } if (optind >= argc) { LogPrintf(LOG_FATAL, FLAG_log_info, "Must specify at least one template file on the command line."); } Template::SetTemplateRootDirectory(FLAG_template_dir); // Initialize the TemplateRecord array. It holds one element per // template given on the command-line. vector template_records; for (int i = optind; i < argc; ++i) { TemplateRecord *template_rec = new TemplateRecord(argv[i]); template_records.push_back(template_rec); } // Iterate through each template and (unless -n is given), write // its header entries into the headers array. int num_errors = 0; for (vector::iterator it = template_records.begin(); it != template_records.end(); ++it) { const char* tplname = (*it)->name.c_str(); LogPrintf(LOG_INFO, FLAG_log_info, "\n------ Checking %s ------", tplname); // The last two arguments in the following call do not matter // since they control how the template gets expanded and we never // expand the template after loading it here Template * tpl = Template::GetTemplate(tplname, ctemplate::DO_NOT_STRIP); // The call to GetTemplate (above) loads the template from disk // and attempts to parse it. If it cannot find the file or if it // detects any template syntax errors, the parsing routines // report the error and GetTemplate returns NULL. Syntax errors // include such things as mismatched double-curly-bracket pairs, // e.g. '{{VAR}', Invalid characters in template variables or // section names, e.g. '{{BAD_VAR?}}' [the question mark is // illegal], improperly nested section/end section markers, // e.g. a section close marker with no section start marker or a // section start of a different name. // If that happens, since the parsing errors have already been reported // we just continue on to the next one. if (!tpl) { LogPrintf(LOG_ERROR, FLAG_log_info, "Could not load file: %s", tplname); num_errors++; (*it)->error = true; continue; } else { LogPrintf(LOG_INFO, FLAG_log_info, "No syntax errors detected in %s", tplname); if (FLAG_dump_templates) tpl->Dump(tpl->template_file()); } // The rest of the loop creates the header file if (!FLAG_header) continue; // They don't want header files tpl->WriteHeaderEntries(&((*it)->header_entries)); } // We have headers to emit: // . If --outputfile was given, we combine all the header entries and // write them to the given output file. If any template had errors, // we fail and do not generate an output file. // . Otherwise, we write one output file per template we processed. // . In both cases, we add proper boilerplate first. if (FLAG_header) { string progname = argv[0]; if (!FLAG_outputfile.empty()) { // All header entries written to one file. // If any template had an error, we do not produce an output file. if (num_errors == 0) { vector template_filenames; string all_header_entries; for (vector::const_iterator it = template_records.begin(); it != template_records.end(); ++it) { all_header_entries.append((*it)->header_entries); template_filenames.push_back((*it)->name); } string output = Boilerplate(progname, template_filenames); const string cleantext = TextWithDuplicateLinesRemoved(all_header_entries); output.append(WrapWithGuard(FLAG_outputfile, cleantext)); if (!WriteToDisk(FLAG_log_info, FLAG_outputfile, output)) num_errors++; } } else { // Each template will have its own output file. Skip any that had errors. for (vector::const_iterator it = template_records.begin(); it != template_records.end(); ++it) { if ((*it)->error) continue; string basename = ctemplate::Basename((*it)->name); string output_file = ctemplate::PathJoin(FLAG_header_dir, basename + FLAG_outputfile_suffix); vector template_filenames; // Contains one template filename. template_filenames.push_back((*it)->name); string output = Boilerplate(progname, template_filenames); output.append(WrapWithGuard(output_file, (*it)->header_entries)); if (!WriteToDisk(FLAG_log_info, output_file, output)) num_errors++; } } } // Free dynamic memory for (vector::iterator it = template_records.begin(); it != template_records.end(); ++it) { delete *it; } // Cap at 127 to avoid causing problems with return code return num_errors > 127 ? 127 : num_errors; } ctemplate-ctemplate-2.4/src/per_expand_data.cc000066400000000000000000000054561363122316600215420ustar00rootroot00000000000000// Copyright (c) 2009, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // This contains some implementation of PerExpandData that is still simple // but is not conveniently defined in the header file, e.g., because it would // introduce new include dependencies. #include #include #include #include namespace ctemplate { using std::string; bool PerExpandData::DataEq::operator()(const char* s1, const char* s2) const { return ((s1 == 0 && s2 == 0) || (s1 && s2 && *s1 == *s2 && strcmp(s1, s2) == 0)); } PerExpandData::~PerExpandData() { delete map_; } TemplateAnnotator* PerExpandData::annotator() const { if (annotator_ != NULL) { return annotator_; } // TextTemplateAnnotator has no static state. So direct static definition // should be safe. static TextTemplateAnnotator g_default_annotator; return &g_default_annotator; } void PerExpandData::InsertForModifiers(const char* key, const void* value) { if (!map_) map_ = new DataMap; (*map_)[key] = value; } // Retrieve data specific to this Expand call. Returns NULL if key // is not found. This should only be used by template modifiers. const void* PerExpandData::LookupForModifiers(const char* key) const { return map_ ? find_ptr2(*map_, key) : NULL; } } ctemplate-ctemplate-2.4/src/solaris/000077500000000000000000000000001363122316600175575ustar00rootroot00000000000000ctemplate-ctemplate-2.4/src/solaris/libstdc++.la000066400000000000000000000027551363122316600216600ustar00rootroot00000000000000# libstdc++.la - a libtool library file # Generated by ltmain.sh - GNU libtool 1.4a-GCC3.0 (1.641.2.256 2001/05/28 20:09:07 with GCC-local changes) # # Please DO NOT delete this file! # It is necessary for linking the library. # --- # NOTE: This file lives in /usr/sfw/lib on Solaris 10. Unfortunately, # due to an apparent bug in the Solaris 10 6/06 release, # /usr/sfw/lib/libstdc++.la is empty. Below is the correct content, # according to # http://forum.java.sun.com/thread.jspa?threadID=5073150 # By passing LDFLAGS='-Lsrc/solaris' to configure, make will pick up # this copy of the file rather than the empty copy in /usr/sfw/lib. # # Also see # http://www.technicalarticles.org/index.php/Compiling_MySQL_5.0_on_Solaris_10 # # Note: this is for 32-bit systems. If you have a 64-bit system, # uncomment the appropriate dependency_libs line below. # ---- # The name that we can dlopen(3). dlname='libstdc++.so.6' # Names of this library. library_names='libstdc++.so.6.0.3 libstdc++.so.6 libstdc++.so' # The name of the static archive. old_library='libstdc++.a' # Libraries that this one depends upon. # 32-bit version: dependency_libs='-lc -lm -L/usr/sfw/lib -lgcc_s' # 64-bit version: #dependency_libs='-L/lib/64 -lc -lm -L/usr/sfw/lib/64 -lgcc_s' # Version information for libstdc++. current=6 age=0 revision=3 # Is this an already installed library? installed=yes # Files to dlopen/dlpreopen dlopen='' dlpreopen='' # Directory that this library needs to be installed in: libdir='/usr/sfw/lib' ctemplate-ctemplate-2.4/src/template-converter000077500000000000000000000077631363122316600216660ustar00rootroot00000000000000#!/usr/bin/perl -w # # Copyright 2001 Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # Author: Andrew Fikes # # Tool to convert a template file (.tpl) into a C++ header # file with a const string defining the same template. This string # can then be used to create/retrieve a template using # Template::StringToTemplate()/StringToTemplateCache() (see template.h). # # Usage: template-converter < infile > outfile # # template_name is the name of the variable we export. A good choice # is to pass in the outfile name. To make that easier, we treat # as a pathname, and take the basename, strip the # suffix if it's .h, and sanitize the rest of the name to be a legal # C variable name. # MOE:insert # NOTE: See doc/index.html for a general description of Google ctemplate. # Store the input argv. my $argv = join(" ", $0, @ARGV); # Open template file (my $template_name = shift) || usage("Need to specify template variable name."); # If a second argument is supplied, treat it as an input filename. if (my $infile = shift) { open(STDIN, "<", $infile) or usage("Can't open $infile for reading."); } # If a third argument is supplied, treat it as an output filename. if (my $outfile = shift) { open(STDOUT, ">", $outfile) or usage("Can't open $outfile for writing."); } # Get base name of template file $base_name = $template_name; $base_name =~ s|^.*/([^/]*)$|$1|; # Strip out directory name $base_name =~ s|\.h$||; # Strip out suffix, if it's .h $base_name =~ tr|A-Za-z0-9_|_|c; # Sanitize name to remove non-letters/nums # Print header print "// This file automatically generated by template-converter:\n"; print "// $argv\n"; print "//\n"; print "// DO NOT EDIT!\n\n"; print "#ifndef " . uc($base_name) . "_H_\n"; print "#define " . uc($base_name) . "_H_\n\n"; print "#include \n\n"; # Read in template file and print template as a string # MOE:begin_strip print "const string ${base_name} (\n"; # MOE:end_strip_and_replace print "const std::string ${base_name} (\n"; while (<>) { chomp; my $escaped_line = escape_line($_); print "\"$escaped_line\\n\"\n"; } print ");\n\n"; # Print footer and exit print "#endif /* " . uc($base_name) . "_H_ */\n"; exit(0); # Prints usage message sub usage { my $msg = shift; print STDERR "\n$msg\n"; print STDERR "Usage: template-converter ", " [infile] [outfile]\n\n"; exit(1); } # Escapes line (adds a '\' to quotes and possible control characters) sub escape_line { (my $line) = (@_); $line =~ s|\\|\\\\|g; $line =~ s|\"|\\"|g; return $line; } ctemplate-ctemplate-2.4/src/template.cc000077500000000000000000003437111363122316600202410ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- #include #include "base/mutex.h" // This must go first so we get _XOPEN_SOURCE #include #include #include #include // for fwrite, fflush #include #include #include #include #ifdef HAVE_UNISTD_H # include #endif // for stat() and open() and getcwd() #include // for binary_search() #include // for binary_function() #include HASH_MAP_H #include #include #include #include // for pair #include #include "base/thread_annotations.h" #include "htmlparser/htmlparser_cpp.h" #include #include #include #include #include // also gets kIndent #include #include "template_modifiers_internal.h" #include #include #include "base/fileutil.h" #include #include #include // for ostringstream #ifndef PATH_MAX #ifdef MAXPATHLEN #define PATH_MAX MAXPATHLEN #else #define PATH_MAX 4096 // seems conservative for max filename len! #endif #endif #define arraysize(x) ( sizeof(x) / sizeof(*(x)) ) #define AS_STR1(x) #x #define AS_STR(x) AS_STR1(x) // A very simple logging system #undef LOG // a non-working version is provided in base/util.h; redefine it static int kVerbosity = 0; // you can change this by hand to get vlogs #define LOG(level) std::cerr << #level ": " #define VLOG(level) if (kVerbosity >= level) LOG(level) // TODO(csilvers): use our own tables for these? static bool ascii_isalnum(char c) { return ((c & 0x80) == 0) && isalnum(c); // 7-bit ascii, and an alnum } static bool ascii_isspace(char c) { return ((c & 0x80) == 0) && isspace(c); // 7-bit ascii, and a space } #define strsuffix(str, suffix) \ ( strlen(str) > (sizeof("" suffix "") - 1) && \ strcmp(str + strlen(str) - (sizeof(suffix) - 1), suffix) == 0 ) using std::endl; using std::string; using std::list; using std::vector; using std::pair; using std::binary_search; using HASH_NAMESPACE::unordered_map; namespace ctemplate { using ctemplate_htmlparser::HtmlParser; TemplateId GlobalIdForSTS_INIT(const TemplateString& s) { return s.GetGlobalId(); // normally this method is private } int Template::num_deletes_ = 0; namespace { // Mutex for protecting Expand calls against ReloadIfChanged, which // might change a template while it's being expanded. This mutex used // to be a per-template mutex, rather than a global mutex, which seems // like it would be strictly better, but we ran into subtle problems // with deadlocks when a template would sub-include itself (thus // requiring a recursive read-lock during Expand), and the template // was Expanded and ReloadIfChanged at the same time. Rather than // deal with that complication, we just go with a global mutex. Since // ReloadIfChanged is deprecated, in most applications all the mutex // uses will be as read-locks, so this shouldn't cause much contention. static Mutex g_template_mutex(base::LINKER_INITIALIZED); // Mutex for protecting vars_seen in WriteOneHeaderEntry, below. // g_template_mutex and g_header_mutex are never held at the same time. // TODO(csilvers): assert this in the codebase. static Mutex g_header_mutex(base::LINKER_INITIALIZED); // It's not great to have a global variable with a constructor, but // it's safe in this case: the constructor is trivial and does not // depend on any other global constructors running first, and the // variable is used in only one place below, always after main() has // started. // It is ok for this modifier to be in XssClass XSS_WEB_STANDARD because // it only adds indentation characters - typically whitespace - iff these // are already present in the text. If such characters were XSS-harmful // in a given context, they would have already been escaped or replaced // by earlier escaping such as H=attribute. static const ModifierInfo g_prefix_line_info("", '\0', XSS_WEB_STANDARD, &prefix_line); const char * const kDefaultTemplateDirectory = kCWD; // "./" // Note this name is syntactically impossible for a user to accidentally use. const char * const kMainSectionName = "__{{MAIN}}__"; // A TemplateString object that precomputes its hash. This can be // useful in places like template filling code, where we'd like to // hash the string once then reuse it many times. This should not be // used for filling any part of a template dictionary, since we don't // map the id to its corresponding string or manage memory for the // string - it is for lookups *only*. class HashedTemplateString : public TemplateString { public: HashedTemplateString(const char* s, size_t slen) : TemplateString(s, slen) { CacheGlobalId(); } }; #define LOG_TEMPLATE_NAME(severity, template) \ LOG(severity) << "Template " << template->template_file() << ": " #define LOG_AUTO_ESCAPE_ERROR(error_msg, my_template) do { \ LOG_TEMPLATE_NAME(ERROR, my_template); \ LOG(ERROR) << "Auto-Escape: " << error_msg << endl; \ } while (0) // We are in auto-escape mode. #define AUTO_ESCAPE_MODE(context) ((context) != TC_MANUAL) // Auto-Escape contexts which utilize the HTML Parser. #define AUTO_ESCAPE_PARSING_CONTEXT(context) \ ((context) == TC_HTML || (context) == TC_JS || (context) == TC_CSS) // ---------------------------------------------------------------------- // PragmaId // PragmaDefinition // PragmaMarker // Functionality to support the PRAGMA marker in the template, i.e // the {{%IDENTIFIER [name1="value1" [name2="value2"]...]}} syntax: // . IDENTIFIER as well as all attribute names are case-insensitive // whereas attribute values are case-sensitive. // . No extraneous whitespace is allowed (e.g. between name and '='). // . Double quotes inside an attribute value need to be backslash // escaped, i.e. " -> \". We unescape them during parsing. // // The only identifier currently supported is AUTOESCAPE which is // used to auto-escape a given template. Its syntax is: // {{%AUTOESCAPE context="context" [state="state"]}} where: // . context is one of: "HTML", "JAVASCRIPT", "CSS", "XML", "JSON". // . state may be omitted or equivalently, it may be set to "default". // It also accepts the value "IN_TAG" in the HTML context to // indicate the template contains HTML attribute name/value // pairs that are enclosed in a tag specified in a parent template. // e.g: Consider the parent template: // INC}}>text // and the included template: // class="{{CLASS}}" target="{{TARGET}}" // Then, for the included template to be auto-escaped properly, it // must have the pragma: {{%AUTOESCAPE context="HTML" state="IN_TAG"}}. // This is a very uncommon template structure. // // To add a new pragma identifier, you'll have to at least: // 1. Add a new id for it in PragmaId enum. // 2. Add the corresponding definition in static g_pragmas array // 3. If you accept more than 2 attributes, increase the size // of attribute_names in the PragmaDefinition struct. // 4. Add handling of that pragma in SectionTemplateNode::GetNextToken() // and possibly SectionTemplateNode::AddPragmaNode() // ---------------------------------------------------------------------- // PragmaId // Identify all the pragma identifiers we support. Currently only // one (for AutoEscape). PI_ERROR is only for internal error reporting, // and is not a valid pragma identifier. enum PragmaId { PI_UNUSED, PI_ERROR, PI_AUTOESCAPE, NUM_PRAGMA_IDS }; // Each pragma definition has a unique identifier as well as a list of // attribute names it accepts. This allows initial error checking while // parsing a pragma definition. Such error checking will need supplementing // with more pragma-specific logic in SectionTemplateNode::GetNextToken(). static struct PragmaDefinition { PragmaId pragma_id; const char* identifier; const char* attribute_names[2]; // Increase as needed. } g_pragmas[NUM_PRAGMA_IDS] = { /* PI_UNUSED */ { PI_UNUSED, NULL, {} }, /* PI_ERROR */ { PI_ERROR, NULL, {} }, /* PI_AUTOESCAPE */ { PI_AUTOESCAPE, "AUTOESCAPE", {"context", "state"} } }; // PragmaMarker // Functionality to parse the {{%...}} syntax and extract the // provided attribute values. We store the PragmaId as well // as a vector of all the attribute names and values provided. class PragmaMarker { public: // Constructs a PragmaMarker object from the PRAGMA marker // {{%ID [[name1=\"value1"] ...]}}. On error (unable to parse // the marker), returns an error description in error_msg. On // success, error_msg is cleared. PragmaMarker(const char* token_start, const char* token_end, string* error_msg); // Returns the attribute value for the corresponding attribute name // or NULL if none is found (as is the case with optional attributes). // Ensure you only call it on attribute names registered in g_pragmas // for that PragmaId. const string* GetAttributeValue(const char* attribute_name) const; private: // Checks that the identifier given matches one of the pragma // identifiers we know of, in which case returns the corresponding // PragmaId. In case of error, returns PI_ERROR. static PragmaId GetPragmaId(const char* id, size_t id_len); // Parses an attribute value enclosed in double quotes and updates // value_end to point at ending double quotes. Returns the attribute // value. If an error occurred, error_msg is set with information. // It is cleared on success. // Unescapes backslash-escaped double quotes ('\"' -> '"') if present. static string ParseAttributeValue(const char* value_start, const char** value_end, string* error_msg); // Returns true if the attribute name is an accepted one for that // given PragmaId. Otherwise returns false. static bool IsValidAttribute(PragmaId pragma_id, const char* name, size_t namelen); PragmaId pragma_id_; // A vector of attribute (name, value) pairs. vector > names_and_values_; }; PragmaId PragmaMarker::GetPragmaId(const char* id, size_t id_len) { for (int i = 0; i < NUM_PRAGMA_IDS; ++i) { if (g_pragmas[i].identifier == NULL) // PI_UNUSED, PI_ERROR continue; if ((strlen(g_pragmas[i].identifier) == id_len) && (strncasecmp(id, g_pragmas[i].identifier, id_len) == 0)) return g_pragmas[i].pragma_id; } return PI_ERROR; } bool PragmaMarker::IsValidAttribute(PragmaId pragma_id, const char* name, size_t namelen) { const int kMaxAttributes = sizeof(g_pragmas[0].attribute_names) / sizeof(*g_pragmas[0].attribute_names); for (int i = 0; i < kMaxAttributes; ++i) { const char* attr_name = g_pragmas[pragma_id].attribute_names[i]; if (attr_name == NULL) break; if ((strlen(attr_name) == namelen) && (strncasecmp(attr_name, name, namelen) == 0)) // We found the given name in our accepted attribute list. return true; } return false; // We did not find the name. } const string* PragmaMarker::GetAttributeValue( const char* attribute_name) const { // Developer error if assert triggers. assert(IsValidAttribute(pragma_id_, attribute_name, strlen(attribute_name))); for (vector >::const_iterator it = names_and_values_.begin(); it != names_and_values_.end(); ++it) { if (strcasecmp(attribute_name, it->first.c_str()) == 0) return &it->second; } return NULL; } string PragmaMarker::ParseAttributeValue(const char* value_start, const char** value_end, string* error_msg) { assert(error_msg); if (*value_start != '"') { error_msg->append("Attribute value is not enclosed in double quotes."); return ""; } const char* current = ++value_start; // Advance past the leading '"' const char* val_end; do { if (current >= *value_end || ((val_end = (const char*)memchr(current, '"', *value_end - current)) == NULL)) { error_msg->append("Attribute value not terminated."); return ""; } current = val_end + 1; // Advance past the current '"' } while (val_end[-1] == '\\'); string attribute_value(value_start, val_end - value_start); // Now replace \" with " size_t found; while ((found = attribute_value.find("\\\"")) != string::npos) attribute_value.erase(found, 1); *value_end = val_end; error_msg->clear(); return attribute_value; } PragmaMarker::PragmaMarker(const char* token_start, const char* token_end, string* error_msg) { assert(error_msg); string error; const char* identifier_end = (const char*)memchr(token_start, ' ', token_end - token_start); if (identifier_end == NULL) identifier_end = token_end; pragma_id_ = PragmaMarker::GetPragmaId(token_start, identifier_end - token_start); if (pragma_id_ == PI_ERROR) { error = "Unrecognized pragma identifier."; } else { const char* val_end; // Loop through attribute name/value pairs. for (const char* nameval = identifier_end; nameval < token_end; nameval = val_end + 1) { // Either after identifier or afer a name/value pair. Must be whitespace. if (*nameval++ != ' ') { error = "Extraneous text."; break; } const char* val = (const char*)memchr(nameval, '=', token_end - nameval); if (val == NULL || val == nameval) { error = "Missing attribute name or value"; break; } const string attribute_name(nameval, val - nameval); if (!PragmaMarker::IsValidAttribute(pragma_id_, attribute_name.data(), attribute_name.length())) { error = "Unrecognized attribute name: " + attribute_name; break; } ++val; // Advance past '=' val_end = token_end; const string attribute_value = ParseAttributeValue(val, &val_end, &error); if (!error.empty()) // Failed to parse attribute value. break; names_and_values_.push_back(pair( attribute_name, attribute_value)); } } if (error.empty()) // Success error_msg->clear(); else // Error error_msg->append("In PRAGMA directive '" + string(token_start, token_end - token_start) + "' Error: " + error); } // ---------------------------------------------------------------------- // memmatch() // Return a pointer to the first occurrences of the given // length-denominated string, inside a bigger length-denominated // string, or NULL if not found. The mem version of strstr. // ---------------------------------------------------------------------- static const char *memmatch(const char *haystack, size_t haystack_len, const char *needle, size_t needle_len) { if (needle_len == 0) return haystack; // even if haystack_len is 0 else if (needle_len > haystack_len) return NULL; const char* match; const char* hayend = haystack + haystack_len - needle_len + 1; while ((match = (const char*)memchr(haystack, needle[0], hayend - haystack))) { if (memcmp(match, needle, needle_len) == 0) return match; else haystack = match + 1; } return NULL; } // ---------------------------------------------------------------------- // FilenameValidForContext() // GetTemplateContextFromPragma() // GetModifierForContext() // FindLongestMatch() // PrettyPrintTokenModifiers() // Static methods for the auto-escape mode specifically. // Perfoms matching of filename against the TemplateContext // and warns in the log on mismatch using "unwritten" filename // conventions below for templates in our codebase: // 1. If filename contains "css", "stylesheet" or "style" // check that it has type TC_CSS. // 2. If filename contains "js" or "javascript" check that // it has type TC_JS. // Returns false if there was a mismatch although currently // we ignore it and just rely on the LOG(WARNING) in the logs. static bool FilenameValidForContext(const string& filename, TemplateContext context) { string stripped_filename = Basename(filename); if (ctemplate::ContainsFullWord(stripped_filename, "css") || ctemplate::ContainsFullWord(stripped_filename, "stylesheet") || ctemplate::ContainsFullWord(stripped_filename, "style")) { if (context != TC_CSS) { LOG(WARNING) << "Template filename " << filename << " indicates CSS but given TemplateContext" << " was not TC_CSS." << endl; return false; } } else if (ctemplate::ContainsFullWord(stripped_filename, "js") || ctemplate::ContainsFullWord(stripped_filename, "javascript")) { if (context != TC_JS) { LOG(WARNING) << "Template filename " << filename << " indicates javascript but given TemplateContext" << " was not TC_JS." << endl; return false; } } return true; } // Returns a string containing a human-readable description of // the modifiers in the vector. The format is: // :modifier1[=val1][:modifier2][=val2]... static string PrettyPrintTokenModifiers( const vector& modvals) { string out; for (vector::const_iterator it = modvals.begin(); it != modvals.end(); ++it) { string one_mod = PrettyPrintOneModifier(*it); out.append(one_mod); } return out; } // Returns the TemplateContext corresponding to the "context" attribute // of the AUTOESCAPE pragma. Returns TC_MANUAL to indicate an error, // meaning an invalid context was given in the pragma. static TemplateContext GetTemplateContextFromPragma( const PragmaMarker& pragma) { const string* context = pragma.GetAttributeValue("context"); if (context == NULL) return TC_MANUAL; if (*context == "HTML" || *context == "html") return TC_HTML; else if (*context == "JAVASCRIPT" || *context == "javascript") return TC_JS; else if (*context == "CSS" || *context == "css") return TC_CSS; else if (*context == "JSON" || *context == "json") return TC_JSON; else if (*context == "XML" || *context == "xml") return TC_XML; return TC_MANUAL; } // Based on the state of the parser, determines the appropriate escaping // directive and returns a pointer to the corresponding // global ModifierAndValue vector. Called when a variable template node // is traversed. // Returns NULL if there is no suitable modifier for that context in // which the case the caller is expected to fail the template initialization. static const vector GetModifierForContext( TemplateContext my_context, HtmlParser *htmlparser, const Template* my_template) { assert(AUTO_ESCAPE_MODE(my_context)); vector modvals; string error_msg; switch (my_context) { case TC_XML: modvals = GetModifierForXml(htmlparser, &error_msg); break; case TC_JSON: modvals = GetModifierForJson(htmlparser, &error_msg); break; case TC_CSS: assert(htmlparser); // Parser is active in CSS modvals = GetModifierForCss(htmlparser, &error_msg); break; default: // Must be in TC_HTML or TC_JS. Parser is active in these modes. assert(AUTO_ESCAPE_PARSING_CONTEXT(my_context)); assert(htmlparser); modvals = GetModifierForHtmlJs(htmlparser, &error_msg); } // In any mode, there should be at least one modifier. if (modvals.empty()) LOG_AUTO_ESCAPE_ERROR(error_msg, my_template); return modvals; } // Returns the largest int N indicating how many XSS safe alternative // modifiers are in the in-template modifiers already. // . If N is equal to the number of modifiers determined by the Auto Escaper, // we have a full match and the in-template modifiers were safe. We leave // them untouched. // . Otherwise, N is less (or zero) and we have a partial match (or none). // The in-template modifiers are not XSS safe and need the missing ones, // i.e. those in the auto escape modifiers which are not in the first N. // // We allow in-template modifiers to have extra modifiers than we deem // necessary, for e.g. :j:h when :j would have sufficed. But to make sure // these modifiers do not introduce XSS concerns we require that they // be in the same XssClass as the modifier we had. // For example :h:x-bla is not safe in HTML context because x-bla is // in a different XssClass as our :h whereas :h:j would be safe. static size_t FindLongestMatch( const vector& modvals_man, const vector& modvals_auto) { if (modvals_auto.empty()) return 0; // See if modvals_auto is "consistent" with the modifiers that are // already present (modvals_man). This is true if all the // modifiers in auto also occur in man, and any gaps between them // (if any) are filled by "neutral" modifiers that do not affect // xss-safety. We go through the vectors backwards. // If all of modvals_auto is not consistent, maybe a prefix of it // is; that's better than nothing, since we only need to auto-apply // the suffix that's not already in modvals_man. typedef vector::const_reverse_iterator ModAutoIterator; typedef vector::const_reverse_iterator ModManIterator; for (ModAutoIterator end_of_prefix = modvals_auto.rbegin(); end_of_prefix != modvals_auto.rend(); ++end_of_prefix) { ModAutoIterator curr_auto = end_of_prefix; ModManIterator curr_man = modvals_man.rbegin(); while (curr_auto != modvals_auto.rend() && curr_man != modvals_man.rend()) { if (IsSafeXSSAlternative(*(*curr_auto)->modifier_info, *curr_man->modifier_info)) { ++curr_auto; ++curr_man; } else if ((curr_man->modifier_info->xss_class == (*curr_auto)->modifier_info->xss_class) && (curr_man->modifier_info->xss_class != XSS_UNIQUE)) { ++curr_man; // Ignore this modifier: it's harmless. } else { break; // An incompatible modifier; we've failed } } if (curr_auto == modvals_auto.rend()) // got through them all, full match! return curr_auto - end_of_prefix; } return 0; } // ---------------------------------------------------------------------- // WriteOneHeaderEntry() // This dumps information about a template that is useful to // make_tpl_varnames_h -- information about the variable and // section names used in a template, so we can define constants // to refer to them instead of having to type them in by hand. // Output is *appended* to outstring. // ---------------------------------------------------------------------- static void WriteOneHeaderEntry( string *outstring, const string& variable, const string& full_pathname) LOCKS_EXCLUDED(g_header_mutex) { MutexLock ml(&g_header_mutex); // we use hash_map instead of hash_set just to keep the stl size down static unordered_map vars_seen GUARDED_BY(g_header_mutex); static string current_file GUARDED_BY(g_header_mutex); static string prefix GUARDED_BY(g_header_mutex); if (full_pathname != current_file) { // changed files so re-initialize the static variables vars_seen.clear(); current_file = full_pathname; // remove the path before the filename string filename(Basename(full_pathname)); prefix = "k"; bool take_next = true; for (string::size_type i = 0; i < filename.length(); i++) { if (filename[i] == '.') { // stop when we find the dot break; } if (take_next) { if (filename.substr(i, 4) == "post") { // stop before we process post... break; } prefix = prefix + filename[i]; take_next = false; } if (filename[i] == '_') { take_next = true; } } prefix = prefix + "_"; } // print out the variable, but only if we haven't seen it before. if (!vars_seen.count(variable)) { if (variable == kMainSectionName || variable.find("BI_") == 0) { // We don't want to write entries for __MAIN__ or the built-ins } else { const TemplateId id = GlobalIdForSTS_INIT(TemplateString(variable)); std::ostringstream outstream; outstream << "static const " << "::ctemplate::StaticTemplateString " << prefix << variable << " = STS_INIT_WITH_HASH(" << prefix << variable << ", \"" << variable << "\", " << id << "ULL);\n"; outstring->append(outstream.str()); } vars_seen[variable] = true; } } // ---------------------------------------------------------------------- // TemplateToken // A TemplateToken is a string marked with a token type enum. The string // has different meanings for different token types. For text, the // string is the text itself. For variable and template types, the // string is the name of the variable holding the value or the // template name, resp. For section types, the string is the name // of the section, used to retrieve the hidden/visible state and // the associated list of dictionaries, if any. For pragma type, // the string is the full text of the marker and is only used for // debug information. // ---------------------------------------------------------------------- enum TemplateTokenType { TOKENTYPE_UNUSED, TOKENTYPE_TEXT, TOKENTYPE_VARIABLE, TOKENTYPE_SECTION_START, TOKENTYPE_SECTION_END, TOKENTYPE_TEMPLATE, TOKENTYPE_COMMENT, TOKENTYPE_SET_DELIMITERS, TOKENTYPE_PRAGMA, TOKENTYPE_NULL, TOKENTYPE_HIDDEN_DEFAULT_SECTION, }; } // unnamed namespace // A sorted array of Template variable names that Auto-Escape should // not escape. Variables that you may want to add here typically // satisfy all the following conditions: // 1. Are "trusted" variables, meaning variables you know to not // contain potentially harmful content. // 2. Contain some markup that gets broken when escaping is // applied to them. // 3. Are used often such that requiring developers to add // ":none" to each use is error-prone and inconvenient. // // Note: Keep this array sorted as you add new elements! // const char * const Template::kSafeWhitelistedVariables[] = { "" // a placekeeper element: replace with your real values! }; const size_t Template::kNumSafeWhitelistedVariables = arraysize(Template::kSafeWhitelistedVariables); // A TemplateToken is a typed string. The semantics of the string depends on the // token type, as follows: // TOKENTYPE_TEXT - the text // TOKENTYPE_VARIABLE - the name of the variable // TOKENTYPE_SECTION_START - the name of the section being started // TOKENTYPE_SECTION_END - the name of the section being ended // TOKENTYPE_TEMPLATE - the name of the variable whose value will be // the template filename // TOKENTYPE_COMMENT - the empty string, not used // TOKENTYPE_SET_DELIMITERS- the empty string, not used // TOKENTYPE_PRAGMA - identifier and optional set of name/value pairs // - exactly as given in the template // TOKENTYPE_NULL - the empty string // TOKENTYPE_HIDDEN_DEFAULT_SECTION // - like TOKENTYPE_SECTION_START, but defaults to // hidden // All non-comment tokens may also have modifiers, which follow the name // of the token: the syntax is {{:::...}} // The modifiers are also stored as a string, starting with the first : struct TemplateToken { TemplateTokenType type; const char* text; size_t textlen; vector modvals; TemplateToken(TemplateTokenType t, const char* txt, size_t len, const vector* m) : type(t), text(txt), textlen(len) { if (m) modvals = *m; } string ToString() const { // used for debugging (annotations) string retval(text, textlen); for (vector::const_iterator it = modvals.begin(); it != modvals.end(); ++it) { const string& modname = it->modifier_info->long_name; retval += string(":") + modname; if (!it->modifier_info->is_registered) retval += ""; } return retval; } // Updates the correct modifiers for the token (variable or template node) // based on our computed modifiers from the HTML parser context as well // as the in-template modifiers that may have been provided. // If the in-template modifiers are considered safe, we use them // without modification. This could happen in one of three cases: // 1. The token has the ":none" modifier as one of the modifiers. // 2. The token has a custom modifier considered XSS-Safe as one of // the modifiers. The modifier was added via AddXssSafeModifier() // and has the XSS_SAFE XssClass. // 3. The escaping modifiers are XSS-equivalent to the ones we computed. // // If the in-template modifiers are not found to be safe, we add // the escaping modifiers we determine missing. This is done based on a // longest match search between the two modifiers vectors, refer to comment // in FindLongestMatch. We also issue a warning in the log, unless the // in-template modifiers were all not escaping related (e.g. custom) // since that case is similar to that of not providing any modifiers. void UpdateModifier(const vector& auto_modvals) { // Common case: no modifiers given in template. Assign our own. No warning. if (modvals.empty()) { for (vector::const_iterator it = auto_modvals.begin(); it != auto_modvals.end(); ++it) { modvals.push_back(**it); } return; } // Look for any XSS-Safe modifiers (added via AddXssSafeModifier or :none). // If one is found anywhere in the vector, consider the variable safe. for (vector::const_iterator it = modvals.begin(); it != modvals.end(); ++it) { if (it->modifier_info->xss_class == XSS_SAFE) return; } size_t longest_match = FindLongestMatch(modvals, auto_modvals); if (longest_match == auto_modvals.size()) { return; // We have a complete match, nothing to do. } else { // Copy missing ones and issue warning. assert(longest_match >= 0 && longest_match < auto_modvals.size()); // We only log if one or more of the in-template modifiers was // escaping-related which we infer from the XssClass. Currently, // all escaping modifiers are in XSS_WEB_STANDARD except for 'none' // but that one is handled above. bool do_log = false; for (vector::const_iterator it = modvals.begin(); it != modvals.end(); ++it) { if (it->modifier_info->xss_class == XSS_WEB_STANDARD) { do_log = true; break; } } string before = PrettyPrintTokenModifiers(modvals); // for logging for (vector::const_iterator it = auto_modvals.begin() + longest_match; it != auto_modvals.end(); ++it) { modvals.push_back(**it); } if (do_log) LOG(ERROR) << "Token: " << string(text, textlen) << " has missing in-template modifiers. You gave " << before << " and we computed " << PrettyPrintModifiers(auto_modvals, "") << ". We changed to " << PrettyPrintTokenModifiers(modvals) << endl; } } }; static bool AnyMightModify(const vector& modifiers, const PerExpandData* data) { for (vector::const_iterator it = modifiers.begin(); it != modifiers.end(); ++it) { string value_string(it->value, it->value_len); if (it->modifier_info->modifier->MightModify(data, value_string)) { return true; } } return false; } // This applies the modifiers to the string in/inlen, and writes the end // result directly to the end of outbuf. Precondition: |modifiers| > 0. // // TODO(user): In the case of multiple modifiers, we are applying // all of them if any of them MightModify the output. We can do // better. We should store the MightModify values that we use to // compute AnyMightModify and respect them here. static void EmitModifiedString(const vector& modifiers, const char* in, size_t inlen, const PerExpandData* data, ExpandEmitter* outbuf) { string result; string value_string; if (modifiers.size() > 1) { // If there's more than one modifiers, we need to store the // intermediate results in a temp-buffer. We use a string. // We'll assume that each modifier adds about 12% to the input // size. result.reserve((inlen + inlen/8) + 16); StringEmitter scratchbuf(&result); value_string = string(modifiers.front().value, modifiers.front().value_len); modifiers.front().modifier_info->modifier->Modify(in, inlen, data, &scratchbuf, value_string); // Only used when modifiers.size() > 2 for (vector::const_iterator it = modifiers.begin() + 1; it != modifiers.end()-1; ++it) { string output_of_this_modifier; output_of_this_modifier.reserve(result.size() + result.size()/8 + 16); StringEmitter scratchbuf2(&output_of_this_modifier); value_string = string(it->value, it->value_len); it->modifier_info->modifier->Modify(result.c_str(), result.size(), data, &scratchbuf2, value_string); result.swap(output_of_this_modifier); } in = result.data(); inlen = result.size(); } // For the last modifier, we can write directly into outbuf assert(!modifiers.empty()); value_string = string(modifiers.back().value, modifiers.back().value_len); modifiers.back().modifier_info->modifier->Modify(in, inlen, data, outbuf, value_string); } static void AppendTokenWithIndent(int level, string *out, const string& before, const TemplateToken& token, const string& after) { out->append(string(level * kIndent, ' ')); string token_string(token.text, token.textlen); out->append(before + token_string + after); } // ---------------------------------------------------------------------- // TemplateNode // When we read a template, we decompose it into its components: // variables, sections, include-templates, and runs of raw text. // Each of these we see becomes one TemplateNode. TemplateNode // is the abstract base class; each component has its own type. // ---------------------------------------------------------------------- class TemplateNode { public: TemplateNode() {} virtual ~TemplateNode() {} // Expands the template node using the supplied dictionary. The // result is placed into output_buffer. If // per_expand_data->annotate() is true, the output is annotated. // Returns true iff all the template files load and parse correctly. virtual bool Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData *per_expand_data, const TemplateCache *cache) const = 0; // Writes entries to a header file to provide syntax checking at // compile time. virtual void WriteHeaderEntries(string *outstring, const string& filename) const = 0; // Appends a representation of the node and its subnodes to a string // as a debugging aid. virtual void DumpToString(int level, string *out) const = 0; protected: typedef list NodeList; private: TemplateNode(const TemplateNode&); // disallow copying void operator=(const TemplateNode&); }; // ---------------------------------------------------------------------- // TextTemplateNode // The simplest template-node: it holds runs of raw template text, // that should be emitted verbatim. The text points into // template_text_. // ---------------------------------------------------------------------- class TextTemplateNode : public TemplateNode { public: explicit TextTemplateNode(const TemplateToken& token) : token_(token) { VLOG(2) << "Constructing TextTemplateNode: " << string(token_.text, token_.textlen) << endl; } virtual ~TextTemplateNode() { VLOG(2) << "Deleting TextTemplateNode: " << string(token_.text, token_.textlen) << endl; } // Expands the text node by simply outputting the text string. This // virtual method does not use TemplateDictionaryInterface or PerExpandData. // Returns true iff all the template files load and parse correctly. virtual bool Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *, PerExpandData *, const TemplateCache *) const { output_buffer->Emit(token_.text, token_.textlen); return true; } // A noop for text nodes virtual void WriteHeaderEntries(string *outstring, const string& filename) const { return; } // Appends a representation of the text node to a string. virtual void DumpToString(int level, string *out) const { assert(out); AppendTokenWithIndent(level, out, "Text Node: -->|", token_, "|<--\n"); } private: TemplateToken token_; // The text held by this node. }; // ---------------------------------------------------------------------- // VariableTemplateNode // Holds a variable to be replaced when the template is expanded. // The variable is stored in a token object, which has a char* // that points into template_text_. There may also be modifiers, // which are applied at Expand time. // ---------------------------------------------------------------------- class VariableTemplateNode : public TemplateNode { public: explicit VariableTemplateNode(const TemplateToken& token) : token_(token), variable_(token_.text, token_.textlen) { VLOG(2) << "Constructing VariableTemplateNode: " << string(token_.text, token_.textlen) << endl; } virtual ~VariableTemplateNode() { VLOG(2) << "Deleting VariableTemplateNode: " << string(token_.text, token_.textlen) << endl; } // Expands the variable node by outputting the value (if there is one) // of the node variable which is retrieved from the dictionary // Returns true iff all the template files load and parse correctly. virtual bool Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData *per_expand_data, const TemplateCache *cache) const; virtual void WriteHeaderEntries(string *outstring, const string& filename) const { WriteOneHeaderEntry(outstring, string(token_.text, token_.textlen), filename); } // Appends a representation of the variable node to a string. We // also append the modifiers for that variable in the form: // :modifier1[=val1][:modifier2][=val2]...\n virtual void DumpToString(int level, string *out) const { assert(out); AppendTokenWithIndent(level, out, "Variable Node: ", token_, PrettyPrintTokenModifiers(token_.modvals) + "\n"); } private: const TemplateToken token_; const HashedTemplateString variable_; }; bool VariableTemplateNode::Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData* per_expand_data, const TemplateCache *cache) const { if (per_expand_data->annotate()) { per_expand_data->annotator()->EmitOpenVariable(output_buffer, token_.ToString()); } const TemplateString value = dictionary->GetValue(variable_); if (AnyMightModify(token_.modvals, per_expand_data)) { EmitModifiedString(token_.modvals, value.data(), value.size(), per_expand_data, output_buffer); } else { // No need to modify value, so just emit it. output_buffer->Emit(value.data(), value.size()); } if (per_expand_data->annotate()) { per_expand_data->annotator()->EmitCloseVariable(output_buffer); } return true; } // ---------------------------------------------------------------------- // PragmaTemplateNode // It simply stores the text given inside the pragma marker // {{%...}} for possible use in DumpToString(). // ---------------------------------------------------------------------- class PragmaTemplateNode : public TemplateNode { public: explicit PragmaTemplateNode(const TemplateToken& token) : token_(token) { VLOG(2) << "Constructing PragmaTemplateNode: " << string(token_.text, token_.textlen) << endl; } virtual ~PragmaTemplateNode() { VLOG(2) << "Deleting PragmaTemplateNode: " << string(token_.text, token_.textlen) << endl; } // A no-op for pragma nodes. virtual bool Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *, PerExpandData *, const TemplateCache *) const { return true; }; // A no-op for pragma nodes. virtual void WriteHeaderEntries(string *outstring, const string& filename) const { } // Appends a representation of the pragma node to a string. We output // the full text given in {{%...}} verbatim. virtual void DumpToString(int level, string *out) const { assert(out); AppendTokenWithIndent(level, out, "Pragma Node: -->|", token_, "|<--\n"); } private: TemplateToken token_; // The text of the pragma held by this node. }; // ---------------------------------------------------------------------- // TemplateTemplateNode // Holds a variable to be replaced by an expanded (included) // template whose filename is the value of the variable in the // dictionary. // Also holds the TemplateContext which it passes on to // GetTemplateCommon when this included template is initialized. // The indentation_ string is used by the PrefixLine modifier so be // careful not to perform any operation on it that might invalidate // its character array (indentation_.data()). // // In the Auto Escape mode, the PrefixLine modifier is added *after* // auto-escape has updated the modifiers that may be present for that // template include, but that is ok because PrefixLine does not invalidate // their XSS-safety. // ---------------------------------------------------------------------- class TemplateTemplateNode : public TemplateNode { public: explicit TemplateTemplateNode(const TemplateToken& token, Strip strip, const string& indentation) : token_(token), variable_(token_.text, token_.textlen), strip_(strip), indentation_(indentation) { VLOG(2) << "Constructing TemplateTemplateNode: " << string(token_.text, token_.textlen) << endl; // If this template is indented (eg, " {{>SUBTPL}}"), make sure // every line of the expanded template is indented, not just the // first one. We do this by adding a modifier that applies to // the entire template node, that inserts spaces after newlines. if (!indentation_.empty()) { token_.modvals.push_back(ModifierAndValue(&g_prefix_line_info, indentation_.data(), indentation_.length())); } } virtual ~TemplateTemplateNode() { VLOG(2) << "Deleting TemplateTemplateNode: " << string(token_.text, token_.textlen) << endl; } // Expands the template node by retrieving the name of a template // file from the supplied dictionary, expanding it (using this // dictionary if none other is provided in the TemplateDictionary), // and then outputting this newly expanded template in place of the // original variable. // Returns true iff all the template files load and parse correctly. virtual bool Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData *per_expand_data, const TemplateCache *cache) const; virtual void WriteHeaderEntries(string *outstring, const string& filename) const { WriteOneHeaderEntry(outstring, string(token_.text, token_.textlen), filename); } virtual void DumpToString(int level, string *out) const { assert(out); AppendTokenWithIndent(level, out, "Template Node: ", token_, "\n"); } private: TemplateToken token_; // text is the name of a template file. const HashedTemplateString variable_; Strip strip_; // Flag to pass from parent template to included template. const string indentation_; // Used by ModifierAndValue for g_prefix_line. // A helper used for expanding one child dictionary. bool ExpandOnce(ExpandEmitter *output_buffer, const TemplateDictionaryInterface &dictionary, const char* const filename, PerExpandData *per_expand_data, const TemplateCache *cache) const; }; // If no value is found in the dictionary for the template variable // in this node, then no output is generated in place of this variable. bool TemplateTemplateNode::Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData *per_expand_data, const TemplateCache *cache) const { if (dictionary->IsHiddenTemplate(variable_)) { // if this "template include" section is "hidden", do nothing return true; } TemplateDictionaryInterface::Iterator* di = dictionary->CreateTemplateIterator(variable_); if (!di->HasNext()) { // empty dict means 'expand once using containing dict' delete di; // TODO(csilvers): have this return a TemplateString instead? const char* const filename = dictionary->GetIncludeTemplateName(variable_, 0); // If the filename wasn't set then treat it as if it were "hidden", i.e, do // nothing if (filename && *filename) { return ExpandOnce(output_buffer, *dictionary, filename, per_expand_data, cache); } else { return true; } } bool error_free = true; for (int dict_num = 0; di->HasNext(); ++dict_num) { const TemplateDictionaryInterface& child = di->Next(); // We do this in the loop, because maybe one day we'll support // each expansion having its own template dictionary. That's also // why we pass in the dictionary-index as an argument. const char* const filename = dictionary->GetIncludeTemplateName( variable_, dict_num); // If the filename wasn't set then treat it as if it were "hidden", i.e, do // nothing if (filename && *filename) { error_free &= ExpandOnce(output_buffer, child, filename, per_expand_data, cache); } } delete di; return error_free; } static void EmitMissingInclude(const char* const filename, ExpandEmitter *output_buffer, PerExpandData *per_expand_data) { // if there was a problem retrieving the template, bail! if (per_expand_data->annotate()) { TemplateAnnotator* annotator = per_expand_data->annotator(); annotator->EmitFileIsMissing(output_buffer, filename); } LOG(ERROR) << "Failed to load included template: \"" << filename << "\"\n"; } bool TemplateTemplateNode::ExpandOnce( ExpandEmitter *output_buffer, const TemplateDictionaryInterface &dictionary, const char* const filename, PerExpandData *per_expand_data, const TemplateCache *cache) const { bool error_free = true; // NOTE: Although we do this const_cast here, if the cache is frozen // the expansion doesn't mutate the cache, and is effectively 'const'. TemplateCache* cache_ptr = const_cast(cache); // Expand the included template once for each "template specific" // dictionary. Normally this will only iterate once, but it's // possible to supply a list of more than one sub-dictionary and // then the template explansion will be iterative, just as though // the included template were an iterated section. if (per_expand_data->annotate()) { per_expand_data->annotator()->EmitOpenInclude(output_buffer, token_.ToString()); } // sub-dictionary NULL means 'just use the current dictionary instead'. // We force children to annotate the output if we have to. // If the include-template has modifiers, we need to expand to a string, // modify the string, and append to output_buffer. Otherwise (common // case), we can just expand into the output-buffer directly. if (AnyMightModify(token_.modvals, per_expand_data)) { string sub_template; StringEmitter subtemplate_buffer(&sub_template); if (!cache_ptr->ExpandLocked(filename, strip_, &subtemplate_buffer, &dictionary, per_expand_data)) { EmitMissingInclude(filename, output_buffer, per_expand_data); error_free = false; } else { EmitModifiedString(token_.modvals, sub_template.data(), sub_template.size(), per_expand_data, output_buffer); } } else { // No need to modify sub-template if (!cache_ptr->ExpandLocked(filename, strip_, output_buffer, &dictionary, per_expand_data)) { EmitMissingInclude(filename, output_buffer, per_expand_data); error_free = false; } } if (per_expand_data->annotate()) { per_expand_data->annotator()->EmitCloseInclude(output_buffer); } return error_free; } // ---------------------------------------------------------------------- // SectionTemplateNode // Holds the name of a section and a list of subnodes contained // in that section. // ---------------------------------------------------------------------- class SectionTemplateNode : public TemplateNode { public: SectionTemplateNode(const TemplateToken& token, bool hidden_by_default); virtual ~SectionTemplateNode(); // The highest level parsing method. Reads a single token from the // input -- taken from my_template->parse_state_ -- and adds the // corresponding type of node to the template's parse // tree. It may add a node of any type, whether text, variable, // section, or template to the list of nodes contained in this // section. Returns true iff we really added a node and didn't just // end a section or hit a syntax error in the template file. // You should hold the g_template_mutex write-lock when calling this // (unless you're calling it from a constructor). bool AddSubnode(Template *my_template); // Expands a section node as follows: // - Checks to see if the section is hidden and if so, does nothing but // return // - Tries to retrieve a list of dictionaries from the supplied dictionary // stored under this section's name // - If it finds a non-empty list of dictionaries, it iterates over the // list and calls itself recursively to expand the section once for // each dictionary // - If there is no dictionary list (or an empty dictionary list somehow) // is found, then the section is expanded once using the supplied // dictionary. (This is the mechanism used to expand each single // iteration of the section as well as to show a non-hidden section, // allowing the section template syntax to be used for both conditional // and iterative text). // Returns true iff all the template files load and parse correctly. virtual bool Expand(ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData* per_expand_data, const TemplateCache *cache) const; // Writes a header entry for the section name and calls the same // method on all the nodes in the section virtual void WriteHeaderEntries(string *outstring, const string& filename) const; virtual void DumpToString(int level, string *out) const; private: const TemplateToken token_; // text is the name of the section const HashedTemplateString variable_; NodeList node_list_; // The list of subnodes in the section // A sub-section named "OURNAME_separator" is special. If we see it // when parsing our section, store a pointer to it for ease of use. SectionTemplateNode* separator_section_; // When the last node read was literal text that ends with "\n? +" // (that is, leading whitespace on a line), this stores the leading // whitespace. This is used to properly indent included // sub-templates. string indentation_; // If true, hide sections that have not explicitly had their hidden/visible // state set. If false, use the underlying template dictionary's default // behavior for hiding. // This bool is currently always set to true. bool hidden_by_default_; // A protected method used in parsing the template file // Finds the next token in the file and return it. Anything not inside // a template marker is just text. Each template marker type, delimited // by "{{" and "}}" (or parser_state_->marker_delimiters.start_marker // and .end_marker, more precisely) is a different type of token. The // first character inside the opening curly braces indicates the type // of the marker, as follows: // # - Start a section // / - End a section // > - A template file variable (the "include" directive) // ! - A template comment // % - A pragma such as AUTOESCAPE // = - Change marker delimiters (from the default of '{{' and '}}') // - A scalar variable // One more thing. Before a name token is returned, if it happens to be // any type other than a scalar variable, and if the next character after // the closing curly braces is a newline, then the newline is eliminated // from the output. This reduces the number of extraneous blank // lines in the output. If the template author desires a newline to be // retained after a final marker on a line, they must add a space character // between the marker and the linefeed character. TemplateToken GetNextToken(Template* my_template); // Helper routine used by Expand virtual bool ExpandOnce( ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData* per_expand_data, bool is_last_child_dict, const TemplateCache *cache) const; // The specific methods called used by AddSubnode to add the // different types of nodes to this section node. // Currently only reasons to fail (return false) are if the // HTML parser failed to parse in auto-escape mode or the // PRAGMA marker was invalid in the template. bool AddTextNode(const TemplateToken* token, Template* my_template); bool AddVariableNode(TemplateToken* token, Template* my_template); bool AddPragmaNode(TemplateToken* token, Template* my_template); bool AddTemplateNode(TemplateToken* token, Template* my_template, const string& indentation); bool AddSectionNode(const TemplateToken* token, Template* my_template, bool hidden_by_default); bool AddSectionNode(const TemplateToken* token, Template* my_template); }; // --- constructor and destructor, Expand, Dump, and WriteHeaderEntries SectionTemplateNode::SectionTemplateNode(const TemplateToken& token, bool hidden_by_default) : token_(token), variable_(token_.text, token_.textlen), separator_section_(NULL), indentation_("\n"), hidden_by_default_(hidden_by_default) { VLOG(2) << "Constructing SectionTemplateNode: " << string(token_.text, token_.textlen) << endl; } SectionTemplateNode::~SectionTemplateNode() { VLOG(2) << "Deleting SectionTemplateNode: " << string(token_.text, token_.textlen) << " and its subnodes" << endl; // Need to delete the member of the list because the list is a list // of pointers to these instances. NodeList::iterator iter = node_list_.begin(); for (; iter != node_list_.end(); ++iter) { delete (*iter); } VLOG(2) << "Finished deleting subnodes of SectionTemplateNode: " << string(token_.text, token_.textlen) << endl; } bool SectionTemplateNode::ExpandOnce( ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData *per_expand_data, bool is_last_child_dict, const TemplateCache* cache) const { bool error_free = true; if (per_expand_data->annotate()) { per_expand_data->annotator()->EmitOpenSection(output_buffer, token_.ToString()); } // Expand using the section-specific dictionary. // We force children to annotate the output if we have to. NodeList::const_iterator iter = node_list_.begin(); for (; iter != node_list_.end(); ++iter) { error_free &= (*iter)->Expand(output_buffer, dictionary, per_expand_data, cache); // If this sub-node is a "separator section" -- a subsection // with the name "OURNAME_separator" -- expand it every time // through but the last. if (*iter == separator_section_ && !is_last_child_dict) { // We call ExpandOnce to make sure we always expand, // even if *iter would normally be hidden. error_free &= separator_section_->ExpandOnce(output_buffer, dictionary, per_expand_data, true, cache); } } if (per_expand_data->annotate()) { per_expand_data->annotator()->EmitCloseSection(output_buffer); } return error_free; } bool SectionTemplateNode::Expand( ExpandEmitter *output_buffer, const TemplateDictionaryInterface *dictionary, PerExpandData *per_expand_data, const TemplateCache *cache) const { // The section named __{{MAIN}}__ is special: you always expand it // exactly once using the containing (main) dictionary. if (token_.text == kMainSectionName) { return ExpandOnce(output_buffer, dictionary, per_expand_data, true, cache); } else if (hidden_by_default_ ? !dictionary->IsUnhiddenSection(variable_) : dictionary->IsHiddenSection(variable_)) { // Some dictionaries might have sections that can be explicitly hidden // and unhidden, so by default both IsHidden() and IsUnhidden() are false, // in which case hidden_by_default_ controls the behavior. return true; // if this section is "hidden", do nothing } TemplateDictionaryInterface::Iterator* di = dictionary->CreateSectionIterator(variable_); // If there are no child dictionaries, that means we should expand with the // current dictionary instead. This corresponds to the situation where // template variables within a section are set on the template-wide dictionary // instead of adding a dictionary to the section and setting them there. if (!di->HasNext()) { delete di; return ExpandOnce(output_buffer, dictionary, per_expand_data, true, cache); } // Otherwise, there's at least one child dictionary, and when expanding this // section, we should use the child dictionaries instead of the current one. bool error_free = true; while (di->HasNext()) { const TemplateDictionaryInterface& child = di->Next(); error_free &= ExpandOnce(output_buffer, &child, per_expand_data, !di->HasNext(), cache); } delete di; return error_free; } void SectionTemplateNode::WriteHeaderEntries(string *outstring, const string& filename) const { WriteOneHeaderEntry(outstring, string(token_.text, token_.textlen), filename); NodeList::const_iterator iter = node_list_.begin(); for (; iter != node_list_.end(); ++iter) { (*iter)->WriteHeaderEntries(outstring, filename); } } void SectionTemplateNode::DumpToString(int level, string *out) const { assert(out); AppendTokenWithIndent(level, out, "Section Start: ", token_, "\n"); NodeList::const_iterator iter = node_list_.begin(); for (; iter != node_list_.end(); ++iter) { (*iter)->DumpToString(level + 1, out); } AppendTokenWithIndent(level, out, "Section End: ", token_, "\n"); } // --- AddSubnode and its sub-routines // Under auto-escape (and parsing-enabled modes) advance the parser state. // TextTemplateNode is the only TemplateNode type that can change // the state of the parser. // Returns false only if the HTML parser failed to parse in // auto-escape mode. bool SectionTemplateNode::AddTextNode(const TemplateToken* token, Template* my_template) { assert(token); bool success = true; HtmlParser *htmlparser = my_template->htmlparser_; if (token->textlen > 0) { // ignore null text sections node_list_.push_back(new TextTemplateNode(*token)); if (AUTO_ESCAPE_PARSING_CONTEXT(my_template->initial_context_)) { assert(htmlparser); if (htmlparser->state() == HtmlParser::STATE_ERROR || htmlparser->Parse(token->text, static_cast(token->textlen)) == HtmlParser::STATE_ERROR) { string error_msg = "Failed parsing: " + string(token->text, token->textlen) + "\nIn: " + string(token_.text, token_.textlen); LOG_AUTO_ESCAPE_ERROR(error_msg, my_template); success = false; } } } return success; } // In Auto Escape mode, we update the variable modifiers based on what // modifiers are specified in the template and what Auto-Escape computes // for that context. Returns false only if the HTML parser failed to parse // in auto-escape mode. // // We also have special logic for BI_SPACE and BI_NEWLINE. // Even though they look like variables, they're really not: the user // is expected to use them in situations where they'd normally put // a space character or a newline character, but can't for technical // reasons (namely, that the template parser would strip these // characters because of the STRIP mode it's in). So unlike other // variables, we want to treat these variables as literal text. This // means that we never add modifiers to them, but we do let the // htmlparser know about them in order to update its state. Existing // modifiers will be honored. // // Finally, we check if the variable is whitelisted, in which case // Auto-Escape does not apply escaping to it. See comment for global // array kSafeWhitelistedVariables[]. bool SectionTemplateNode::AddVariableNode(TemplateToken* token, Template* my_template) { assert(token); bool success = true; HtmlParser *htmlparser = my_template->htmlparser_; TemplateContext initial_context = my_template->initial_context_; if (AUTO_ESCAPE_MODE(initial_context)) { // Determines modifiers for the variable in auto escape mode. string variable_name(token->text, token->textlen); // We declare in the documentation that if the user changes the // value of these variables, they must only change it to a value // that's "equivalent" from the point of view of an html parser. // So it's ok to hard-code in that these are " " and "\n", // respectively, even though in theory the user could change them // (to say, BI_NEWLINE == "\r\n"). if (variable_name == "BI_SPACE" || variable_name == "BI_NEWLINE") { if (AUTO_ESCAPE_PARSING_CONTEXT(initial_context)) { assert(htmlparser); if (htmlparser->state() == HtmlParser::STATE_ERROR || htmlparser->Parse(variable_name == "BI_SPACE" ? " " : "\n") == HtmlParser::STATE_ERROR) success = false; } } else if (binary_search(Template::kSafeWhitelistedVariables, Template::kSafeWhitelistedVariables + arraysize(Template::kSafeWhitelistedVariables), variable_name.c_str(), // Luckily, StringHash(a, b) is defined as "a < b" StringHash())) { // Do not escape the variable, it is whitelisted. } else { vector modvals = GetModifierForContext(initial_context, htmlparser, my_template); // There should always be at least one modifier in any Auto-Escape mode. if (modvals.empty()) success = false; else token->UpdateModifier(modvals); } } node_list_.push_back(new VariableTemplateNode(*token)); return success; } // AddPragmaNode // Create a pragma node from the given token and add it // to the node list. // The AUTOESCAPE pragma is only allowed at the top of a template // file (above any non-comment node) to minimize the chance of the // HTML parser being out of sync with the template text. So we check // that the section is the MAIN section and we are the first node. // Note: Since currently we only support one pragma, we apply the check // always but when other pragmas are added we'll need to propagate the // Pragma identifier from GetNextToken(). bool SectionTemplateNode::AddPragmaNode(TemplateToken* token, Template* my_template) { if (token_.text != kMainSectionName || !node_list_.empty()) return false; node_list_.push_back(new PragmaTemplateNode(*token)); return true; } // AddSectionNode bool SectionTemplateNode::AddSectionNode(const TemplateToken* token, Template* my_template, bool hidden_by_default) { assert(token); SectionTemplateNode *new_node = new SectionTemplateNode(*token, hidden_by_default); // Not only create a new section node, but fill it with all *its* // subnodes by repeatedly calling AddSubNode until it returns false // (indicating either the end of the section or a syntax error) while (new_node->AddSubnode(my_template)) { // Found a new subnode to add } node_list_.push_back(new_node); // Check the name of new_node. If it's "OURNAME_separator", store it // as a special "separator" section. if (token->textlen == token_.textlen + sizeof("_separator")-1 && memcmp(token->text, token_.text, token_.textlen) == 0 && memcmp(token->text + token_.textlen, "_separator", sizeof("_separator")-1) == 0) separator_section_ = new_node; return true; } // Note: indentation will be used in constructor of TemplateTemplateNode. // Note on Auto-Escape: Each template is Auto-Escaped independently of // the template it may be included from or templates it may include. // The context does not carry on and hence does not need to be provided // to the new TemplateNode. bool SectionTemplateNode::AddTemplateNode(TemplateToken* token, Template* my_template, const string& indentation) { assert(token); bool success = true; node_list_.push_back( new TemplateTemplateNode(*token, my_template->strip_, indentation)); return success; } // If "text" ends with a newline followed by whitspace, returns a // string holding that whitespace. Otherwise, returns the empty // string. If implicit_newline is true, also consider the text to be // an indentation if it consists entirely of whitespace; this is set // when we know that right before this text there was a newline, or // this text is the beginning of a document. static string GetIndentation(const char* text, size_t textlen, bool implicit_newline) { const char* nextline; // points to one char past the last newline for (nextline = text + textlen; nextline > text; --nextline) if (nextline[-1] == '\n') break; if (nextline == text && !implicit_newline) return ""; // no newline found, so no indentation bool prefix_is_whitespace = true; for (const char* p = nextline; p < text + textlen; ++p) { if (*p != ' ' && *p != '\t') { prefix_is_whitespace = false; break; } } if (prefix_is_whitespace && text + textlen > nextline) return string(nextline, text + textlen - nextline); else return ""; } bool SectionTemplateNode::AddSubnode(Template *my_template) { bool auto_escape_success = true; // Don't proceed if we already found an error if (my_template->state() == TS_ERROR) { return false; } // Stop when the buffer is empty. if (my_template->parse_state_.bufstart >= my_template->parse_state_.bufend) { // running out of file contents ends the section too if (token_.text != kMainSectionName) { // if we are not in the main section, we have a syntax error in the file LOG_TEMPLATE_NAME(ERROR, my_template); LOG(ERROR) << "File ended before all sections were closed" << endl; my_template->set_state(TS_ERROR); } return false; } TemplateToken token = GetNextToken(my_template); switch (token.type) { case TOKENTYPE_TEXT: auto_escape_success = this->AddTextNode(&token, my_template); // Store the indentation (trailing whitespace after a newline), if any. this->indentation_ = GetIndentation(token.text, token.textlen, indentation_ == "\n"); break; case TOKENTYPE_VARIABLE: auto_escape_success = this->AddVariableNode(&token, my_template); this->indentation_.clear(); // clear whenever last read wasn't whitespace break; case TOKENTYPE_SECTION_START: auto_escape_success = this->AddSectionNode(&token, my_template, false); this->indentation_.clear(); // clear whenever last read wasn't whitespace break; case TOKENTYPE_HIDDEN_DEFAULT_SECTION: auto_escape_success = this->AddSectionNode(&token, my_template, true); this->indentation_.clear(); // clear whenever last read wasn't whitespace break; case TOKENTYPE_SECTION_END: // Don't add a node. Just make sure we are ending the right section // and return false to indicate the section is complete if (token.textlen != token_.textlen || memcmp(token.text, token_.text, token.textlen)) { LOG_TEMPLATE_NAME(ERROR, my_template); LOG(ERROR) << "Found end of different section than the one I am in" << "\nFound: " << string(token.text, token.textlen) << "\nIn: " << string(token_.text, token_.textlen) << endl; my_template->set_state(TS_ERROR); } this->indentation_.clear(); // clear whenever last read wasn't whitespace return false; break; case TOKENTYPE_TEMPLATE: auto_escape_success = this->AddTemplateNode(&token, my_template, this->indentation_); this->indentation_.clear(); // clear whenever last read wasn't whitespace break; case TOKENTYPE_COMMENT: // Do nothing. Comments just drop out of the file altogether. break; case TOKENTYPE_SET_DELIMITERS: if (!Template::ParseDelimiters( token.text, token.textlen, &my_template->parse_state_.current_delimiters)) { LOG_TEMPLATE_NAME(ERROR, my_template); LOG(ERROR) << "Invalid delimiter-setting command." << "\nFound: " << string(token.text, token.textlen) << "\nIn: " << string(token_.text, token_.textlen) << endl; my_template->set_state(TS_ERROR); } break; case TOKENTYPE_PRAGMA: // We can do nothing and simply drop the pragma of the file as is done // for comments. But, there is value in keeping it for debug purposes // (via DumpToString) so add it as a pragma node. if (!this->AddPragmaNode(&token, my_template)) { LOG_TEMPLATE_NAME(ERROR, my_template); LOG(ERROR) << "Pragma marker must be at the top of the template: '" << string(token.text, token.textlen) << "'" << endl; my_template->set_state(TS_ERROR); } break; case TOKENTYPE_NULL: // GetNextToken either hit the end of the file or a syntax error // in the file. Do nothing more here. Just return false to stop // processing. return false; break; default: // This shouldn't happen. If it does, it's a programmer error. LOG_TEMPLATE_NAME(ERROR, my_template); LOG(ERROR) << "Invalid token type returned from GetNextToken" << endl; } if (!auto_escape_success) { // The error is logged where it happens. Here indicate // the initialization failed. my_template->set_state(TS_ERROR); return false; } // for all the cases where we did not return false return true; } // --- GetNextToken and its subroutines // A valid marker name is made up of alphanumerics and underscores... // nothing else. static bool IsValidName(const char* name, int namelen) { for (const char *cur_char = name; cur_char - name < namelen; ++cur_char) { if (!ascii_isalnum(*cur_char) && *cur_char != '_') return false; } return true; } // If we're pointing to the end of a line, and in a high enough strip mode, // pass over the newline. If the line ends in a \, we skip over the \ and // keep the newline. Returns a pointer to the new 'start' location, which // is either 'start' or after a newline. static const char* MaybeEatNewline(const char* start, const char* end, Strip strip) { // first, see if we have the escaped linefeed sequence if (end - start >= 2 && start[0] == '\\' && start[1] == '\n') { ++start; // skip over the \, which keeps the \n } else if (end - start >= 1 && start[0] == '\n' && strip >= STRIP_WHITESPACE) { ++start; // skip over the \n in high strip_ modes } return start; } // When the parse fails, we take several actions. msg is a stream #define FAIL(msg) do { \ LOG_TEMPLATE_NAME(ERROR, my_template); \ LOG(ERROR) << msg << endl; \ my_template->set_state(TS_ERROR); \ /* make extra-sure we never try to parse anything more */ \ my_template->parse_state_.bufstart = my_template->parse_state_.bufend; \ return TemplateToken(TOKENTYPE_NULL, "", 0, NULL); \ } while (0) // Parses the text of the template file in the input_buffer as // follows: If the buffer is empty, return the null token. If getting // text, search for the next "{{" sequence (more precisely, for // parse_state_->marker_delimiters.start_marker). If one is found, // return all the text collected up to that sequence in a TextToken // and change the token-parsing phase variable to GETTING_NAME, so the // next call will know to look for a named marker, instead of more // text. If getting a name, read the next character to learn what // kind of marker it is. Then collect the characters of the name up // to the "}}" sequence. If the "name" is a template comment, then we // do not return the text of the comment in the token. If it is any // other valid type of name, we return the token with the appropriate // type and the name. If any syntax errors are discovered (like // inappropriate characters in a name, not finding the closing curly // braces, etc.) an error message is logged, the error state of the // template is set, and a NULL token is returned. Updates // parse_state_. You should hold the g_template_mutex write-lock // when calling this (unless you're calling it from a constructor). TemplateToken SectionTemplateNode::GetNextToken(Template *my_template) { Template::ParseState* ps = &my_template->parse_state_; // short abbrev. const char* token_start = ps->bufstart; if (ps->bufstart >= ps->bufend) { // at end of buffer return TemplateToken(TOKENTYPE_NULL, "", 0, NULL); } switch (ps->phase) { case Template::ParseState::GETTING_TEXT: { const char* token_end = memmatch(ps->bufstart, ps->bufend - ps->bufstart, ps->current_delimiters.start_marker, ps->current_delimiters.start_marker_len); if (!token_end) { // Didn't find the start-marker ('{{'), so just grab all the // rest of the buffer. token_end = ps->bufend; ps->bufstart = ps->bufend; // next token will start at EOF } else { // If we see code like this: "{{{VAR}}, we want to match the // second "{{", not the first. while ((token_end + 1 + ps->current_delimiters.start_marker_len <= ps->bufend) && memcmp(token_end + 1, ps->current_delimiters.start_marker, ps->current_delimiters.start_marker_len) == 0) token_end++; ps->phase = Template::ParseState::GETTING_NAME; ps->bufstart = token_end + ps->current_delimiters.start_marker_len; } return TemplateToken(TOKENTYPE_TEXT, token_start, token_end - token_start, NULL); } case Template::ParseState::GETTING_NAME: { TemplateTokenType ttype; const char* token_end = NULL; // Find out what type of name we are getting switch (token_start[0]) { case '#': ttype = TOKENTYPE_SECTION_START; ++token_start; break; case '/': ttype = TOKENTYPE_SECTION_END; ++token_start; break; case '!': ttype = TOKENTYPE_COMMENT; ++token_start; break; case '=': ttype = TOKENTYPE_SET_DELIMITERS; // Keep token_start the same; the token includes the leading '='. // But we have to figure token-end specially: it should be "=}}". if (ps->bufend > (token_start + 1)) token_end = (char*)memchr(token_start + 1, '=', ps->bufend - (token_start + 1)); if (!token_end || token_end + ps->current_delimiters.end_marker_len > ps->bufend || memcmp(token_end + 1, ps->current_delimiters.end_marker, ps->current_delimiters.end_marker_len) != 0) token_end = NULL; // didn't find it, fall through to code below else token_end++; // advance past the "=" to the "}}". break; case '>': ttype = TOKENTYPE_TEMPLATE; ++token_start; break; case '%': ttype = TOKENTYPE_PRAGMA; ++token_start; break; default: // the assumption that the next char is alnum or _ will be // tested below in the call to IsValidName(). ttype = TOKENTYPE_VARIABLE; } // Now get the name (or the comment, as the case may be) if (!token_end) // that is, it wasn't set in special-case code above token_end = memmatch(token_start, ps->bufend - token_start, ps->current_delimiters.end_marker, ps->current_delimiters.end_marker_len); if (!token_end) { // Didn't find the '}}', so name never ended. Error! FAIL("No ending '" << string(ps->current_delimiters.end_marker, ps->current_delimiters.end_marker_len) << "' when parsing name starting with " << "'" << string(token_start, ps->bufend-token_start) << "'"); } if (ttype == TOKENTYPE_PRAGMA) { string error_msg; const PragmaMarker pragma(token_start, token_end, &error_msg); if (!error_msg.empty()) FAIL(error_msg); TemplateContext context = GetTemplateContextFromPragma(pragma); if (context == TC_MANUAL) // TC_MANUAL is used to indicate error. FAIL("Invalid context in Pragma directive."); const string* parser_state = pragma.GetAttributeValue("state"); bool in_tag = false; if (parser_state != NULL) { if (context == TC_HTML && (*parser_state == "IN_TAG" || *parser_state == "in_tag")) in_tag = true; else if (*parser_state != "default") FAIL("Unsupported state '" + *parser_state + "'in Pragma directive."); } // Only an AUTOESCAPE pragma can change the initial_context // away from TC_MANUAL and we do not support multiple such pragmas. assert(my_template->initial_context_ == TC_MANUAL); my_template->initial_context_ = context; my_template->MaybeInitHtmlParser(in_tag); // ParseState change will happen below. } // Comments are a special case, since they don't have a name or action. // The set-delimiters command is the same way. if (ttype == TOKENTYPE_COMMENT || ttype == TOKENTYPE_SET_DELIMITERS || ttype == TOKENTYPE_PRAGMA) { ps->phase = Template::ParseState::GETTING_TEXT; ps->bufstart = token_end + ps->current_delimiters.end_marker_len; // If requested, remove any unescaped linefeed following a comment ps->bufstart = MaybeEatNewline(ps->bufstart, ps->bufend, my_template->strip_); // For comments, don't bother returning the text if (ttype == TOKENTYPE_COMMENT) token_start = token_end; return TemplateToken(ttype, token_start, token_end - token_start, NULL); } // Now we have the name, possibly with following modifiers. // Find the modifier-start. const char* mod_start = (const char*)memchr(token_start, ':', token_end - token_start); if (mod_start == NULL) mod_start = token_end; // Make sure the name is legal. if (!IsValidName(token_start, mod_start - token_start)) { FAIL("Illegal name in template '" << string(token_start, mod_start-token_start) << "'"); } // Figure out what all the modifiers are. Mods are colon-separated. vector modifiers; const char* mod_end; for (const char* mod = mod_start; mod < token_end; mod = mod_end) { assert(*mod == ':'); ++mod; // skip past the starting colon mod_end = (const char*)memchr(mod, ':', token_end - mod); if (mod_end == NULL) mod_end = token_end; // Modifiers can be of the form :modname=value. Extract out value const char* value = (const char*)memchr(mod, '=', mod_end - mod); if (value == NULL) value = mod_end; string value_string(value, mod_end - value); // Convert the string to a functor, and error out if we can't. const ModifierInfo* modstruct = FindModifier(mod, value - mod, value, mod_end - value); // There are various ways a modifier syntax can be illegal. if (modstruct == NULL) { FAIL("Unknown modifier for variable " << string(token_start, mod_start - token_start) << ": " << "'" << string(mod, value - mod) << "'"); } else if (!modstruct->modval_required && value < mod_end) { FAIL("Modifier for variable " << string(token_start, mod_start - token_start) << ":" << string(mod, value - mod) << " " << "has illegal mod-value '" << value_string << "'"); } else if (modstruct->modval_required && value == mod_end) { FAIL("Modifier for variable " << string(token_start, mod_start - token_start) << ":" << string(mod, value - mod) << " " << "is missing a required mod-value"); } // We rely on the fact that the memory pointed to by 'value' // remains valid throughout the life of this token since // ModifierAndValue does not itself manage its memory. modifiers.push_back( ModifierAndValue(modstruct, value, mod_end - value)); } // For now, we only allow variable and include nodes to have // modifiers. I think it's better not to have this for // sections, but instead to modify all the text and vars in the // section appropriately, but I could be convinced otherwise. if (!modifiers.empty() && ttype != TOKENTYPE_VARIABLE && ttype != TOKENTYPE_TEMPLATE) { FAIL(string(token_start, token_end - token_start) << "malformed: only variables and template-includes " << "are allowed to have modifiers"); } // Whew! We passed the gauntlet. Get ready for the next token ps->phase = Template::ParseState::GETTING_TEXT; ps->bufstart = token_end + ps->current_delimiters.end_marker_len; // If requested, remove any linefeed following a comment, // or section start or end, or template marker, unless // it is escaped by '\' if (ttype != TOKENTYPE_VARIABLE) { ps->bufstart = MaybeEatNewline(ps->bufstart, ps->bufend, my_template->strip_); } // create and return the TEXT token that we found return TemplateToken(ttype, token_start, mod_start - token_start, &modifiers); } default: { FAIL("Programming error: Unexpected parse phase while " << "parsing template: " << ps->phase); } } } // ---------------------------------------------------------------------- // CreateTemplateCache() // default_template_cache() // mutable_default_template_cache() // These create the default TemplateCache object, that Template // often just delegates (deprecated) operations to. // ---------------------------------------------------------------------- static TemplateCache* g_default_template_cache = NULL; GoogleOnceType g_default_cache_init_once = GOOGLE_ONCE_INIT; static void CreateTemplateCache() { g_default_template_cache = new TemplateCache(); } const TemplateCache* default_template_cache() { GoogleOnceInit(&g_default_cache_init_once, &CreateTemplateCache); return g_default_template_cache; } TemplateCache* mutable_default_template_cache() { GoogleOnceInit(&g_default_cache_init_once, &CreateTemplateCache); return g_default_template_cache; } // ---------------------------------------------------------------------- // Template::StringToTemplate() // StringToTemplate reads a string representing a template (eg // "Hello {{WORLD}}"), and parses it to a Template*. It returns // the parsed template, or NULL if there was a parsing error. // StringToTemplateCache does the same, but then inserts the // resulting Template* into the template cache, for future retrieval // via GetTemplate. You pass in the key to use with GetTemplate. // It returns a bool indicating success or failure of template // creation/insertion. (Insertion will fail if a string or file // with that key already exists in the cache.) // RemoveStringFromTemplateCache() lets you remove a string that // you had previously interned via StringToTemplateCache(). // ---------------------------------------------------------------------- Template* Template::StringToTemplate(const TemplateString& content, Strip strip) { // An empty original_filename_ keeps ReloadIfChangedLocked from performing // file operations. Template *tpl = new Template("", strip, NULL); // But we have to do the "loading" and parsing ourselves: // BuildTree deletes the buffer when done, so we need a copy for it. char* buffer = new char[content.size()]; size_t content_len = content.size(); memcpy(buffer, content.data(), content_len); tpl->StripBuffer(&buffer, &content_len); if ( tpl->BuildTree(buffer, buffer + content_len) ) { assert(tpl->state() == TS_READY); } else { assert(tpl->state() != TS_READY); delete tpl; return NULL; } return tpl; } // ---------------------------------------------------------------------- // Template::Template() // Template::~Template() // Template::MaybeInitHtmlParser() // Calls ReloadIfChanged to load the template the first time. // The constructor is private; GetTemplate() is the factory // method used to actually construct a new template if needed. // GetTemplateCommon() first looks in the two caches -- the // cache of parsed template trees, and the cache of raw // template-file contents -- before trying to load the // template-file from disk. // ---------------------------------------------------------------------- Template::Template(const TemplateString& filename, Strip strip, TemplateCache* owner) // TODO(csilvers): replace ToString() with an is_immutable() check : original_filename_(filename.data(), filename.size()), resolved_filename_(), filename_mtime_(0), strip_(strip), state_(TS_EMPTY), template_cache_(owner), template_text_(NULL), template_text_len_(0), tree_(NULL), parse_state_(), initial_context_(TC_MANUAL), htmlparser_(NULL) { VLOG(2) << "Constructing Template for " << template_file() << "; with context " << initial_context_ << "; and strip " << strip_ << endl; // Preserve whitespace in Javascript files because carriage returns // can convey meaning for comment termination and closures if (strsuffix(original_filename_.c_str(), ".js") && strip_ == STRIP_WHITESPACE) { strip_ = STRIP_BLANK_LINES; } ReloadIfChangedLocked(); } Template::~Template() { VLOG(2) << endl << "Deleting Template for " << template_file() << "; with context " << initial_context_ << "; and strip " << strip_ << endl; // Since this is only used by tests, we don't bother with locking num_deletes_++; delete tree_; // Delete this last, since tree has pointers into template_text_ delete[] template_text_; delete htmlparser_; } // In TemplateContexts where the HTML parser is needed, we initialize it in // the appropriate mode. Also we do a sanity check (cannot fail) on the // template filename. This function is invoked when an AUTOESCAPE pragma is // found during template parsing and should at most be called once per template. // // In_tag is only meaningful for TC_HTML: It is true for templates that // start inside an HTML tag and hence are expected to contain HTML attribute // name/value pairs only. It is false for standard HTML templates. void Template::MaybeInitHtmlParser(bool in_tag) { assert(!htmlparser_); if (AUTO_ESCAPE_PARSING_CONTEXT(initial_context_)) { htmlparser_ = new HtmlParser(); switch (initial_context_) { case TC_JS: htmlparser_->ResetMode(HtmlParser::MODE_JS); break; case TC_CSS: htmlparser_->ResetMode(HtmlParser::MODE_CSS); break; default: if (in_tag) htmlparser_->ResetMode(HtmlParser::MODE_HTML_IN_TAG); break; } FilenameValidForContext(original_filename_, initial_context_); } } // ---------------------------------------------------------------------- // Template::BuildTree() // Template::WriteHeaderEntry() // Template::Dump() // These kick off their various parsers -- BuildTree for the // main task of parsing a Template when it's read from memory, // WriteHeaderEntry for parsing for make_tpl_varnames_h, and // Dump() for when Dump() is called by the caller. // ---------------------------------------------------------------------- // NOTE: BuildTree takes over ownership of input_buffer, and will delete it. // It should have been created via new[]. // You should hold a write-lock on g_template_mutex before calling this // (unless you're calling it from a constructor). // In auto-escape mode, the HTML context is tracked as the tree is being // built, in a single pass. When this function completes, all variables // will have the proper modifiers set. bool Template::BuildTree(const char* input_buffer, const char* input_buffer_end) { set_state(TS_EMPTY); parse_state_.bufstart = input_buffer; parse_state_.bufend = input_buffer_end; parse_state_.phase = ParseState::GETTING_TEXT; parse_state_.current_delimiters = Template::MarkerDelimiters(); // Assign an arbitrary name to the top-level node SectionTemplateNode *top_node = new SectionTemplateNode( TemplateToken(TOKENTYPE_SECTION_START, kMainSectionName, strlen(kMainSectionName), NULL), false); while (top_node->AddSubnode(this)) { // Add the rest of the template in. } // get rid of the old tree, whenever we try to build a new one. delete tree_; delete[] template_text_; tree_ = top_node; template_text_ = input_buffer; template_text_len_ = input_buffer_end - input_buffer; // TS_ERROR can also be set by the auto-escape mode, at the point // where the parser failed to parse. if (state() != TS_ERROR) { set_state(TS_READY); return true; } else { delete tree_; tree_ = NULL; delete[] template_text_; template_text_ = NULL; template_text_len_ = 0; return false; } } void Template::WriteHeaderEntries(string *outstring) const { if (state() == TS_READY) { // only write header entries for 'good' tpls outstring->append("#include \n"); tree_->WriteHeaderEntries(outstring, template_file()); } } // Dumps the parsed structure of the template for debugging assistance. // It goes to stdout instead of LOG to avoid possible truncation due to size. void Template::Dump(const char *filename) const { string out; DumpToString(filename, &out); fwrite(out.data(), 1, out.length(), stdout); fflush(stdout); } void Template::DumpToString(const char *filename, string *out) const { if (!out) return; out->append("------------Start Template Dump [" + string(filename) + "]--------------\n"); if (tree_) { tree_->DumpToString(1, out); } else { out->append("No parse tree has been produced for this template\n"); } out->append("------------End Template Dump----------------\n"); } // ------------------------------------------------------------------------- // Template::state() // Template::set_state() // Template::template_file() // Template::original_filename() // Template::strip() // Template::mtime() // Various introspection methods. state() is the parse-state // (success, error). template_file() is the resolved filename of a // given template object's input. original_filename() is the unresolved, // original filename, strip() is the Strip type. mtime() is // the lastmod time. For string-based templates, not backed by a file, // mtime() returns 0. // ------------------------------------------------------------------------- void Template::set_state(TemplateState new_state) { state_ = new_state; } TemplateState Template::state() const { return state_; } const char *Template::template_file() const { return resolved_filename_.c_str(); } const char *Template::original_filename() const { return original_filename_.c_str(); } Strip Template::strip() const { return strip_; } time_t Template::mtime() const { return filename_mtime_; } // ---------------------------------------------------------------------- // Template::GetTemplate() // Template::StringToTemplateCache() // Template::SetTemplateRootDirectory() // Template::AddAlternateTemplateRootDirectory() // Template::template_root_directory() // Template::FindTemplateFilename() // Template::RemoveStringFromTemplateCache() // Template::ClearCache() // Template::ReloadAllIfChanged() // These are deprecated static methods that have been moved to // template_cache.h. We just forward to them, using the global // default template cache. // ---------------------------------------------------------------------- Template *Template::GetTemplate(const TemplateString& filename, Strip strip) { // Until I've resolved the TODO that lets me return a const Template* // here, I have to do an ugly cast. :-( return const_cast( mutable_default_template_cache()->GetTemplate(filename, strip)); } // This method is deprecated (and slow). Instead, use the above // StringToTemplateCache method that takes a Strip argument. bool Template::StringToTemplateCache(const TemplateString& key, const TemplateString& content) { // We say the insert succeeded only if it succeded for all strip values. bool retval = true; for (int i = 0; i < static_cast(NUM_STRIPS); ++i) { if (!ctemplate::StringToTemplateCache(key, content, static_cast(i))) retval = false; } return retval; } // ---------------------------------------------------------------------- // Template::ParseDelimiters() // Given an input that looks like =XXX YYY=, set the // MarkerDelimiters to point to XXX and YYY. This is used to parse // {{=XXX YYY=}} markers, which reset the marker delimiters. // Returns true if successfully parsed (starts and ends with =, // exactly one space, no internal ='s), false else. // ---------------------------------------------------------------------- bool Template::ParseDelimiters(const char* text, size_t textlen, MarkerDelimiters* delim) { const char* space = (const char*)memchr(text, ' ', textlen); if (textlen < 3 || text[0] != '=' || text[textlen - 1] != '=' || // no = at ends memchr(text + 1, '=', textlen - 2) || // = in the middle !space || // no interior space memchr(space + 1, ' ', text + textlen - (space+1))) // too many spaces return false; delim->start_marker = text + 1; delim->start_marker_len = space - delim->start_marker; delim->end_marker = space + 1; delim->end_marker_len = text + textlen - 1 - delim->end_marker; return true; } // ---------------------------------------------------------------------- // StripTemplateWhiteSpace() // Template::IsBlankOrOnlyHasOneRemovableMarker() // Template::InsertLine() // Template::StripBuffer() // This mini-parser modifies an input buffer, replacing it with a // new buffer that is the same as the old, but with whitespace // removed as is consistent with the given strip-mode: // STRIP_WHITESPACE, STRIP_BLANK_LINES, DO_NOT_STRIP (the last // of these is a no-op). This parser may work by allocating // a new buffer and deleting the input buffer when it's done). // The trickiest bit if in STRIP_BLANK_LINES mode, if we see // a line that consits entirely of one "removable" marker on it, // and nothing else other than whitespace. ("Removable" markers // are comments, start sections, end sections, pragmas and // template-include.) In such a case, we elide the newline at // the end of that line. // ---------------------------------------------------------------------- // We define our own version rather than using the one in strutil, mostly // so we can take a size_t instead of an int. The code is simple enough. static void StripTemplateWhiteSpace(const char** str, size_t* len) { // Strip off trailing whitespace. while ((*len) > 0 && ascii_isspace((*str)[(*len)-1])) { (*len)--; } // Strip off leading whitespace. while ((*len) > 0 && ascii_isspace((*str)[0])) { (*len)--; (*str)++; } } // Adjusts line and length iff condition is met, and RETURNS true. // MarkerDelimiters are {{ and }}, or equivalent. bool Template::IsBlankOrOnlyHasOneRemovableMarker( const char** line, size_t* len, const Template::MarkerDelimiters& delim) { const char *clean_line = *line; size_t new_len = *len; StripTemplateWhiteSpace(&clean_line, &new_len); // If there was only white space on the line, new_len will now be zero. // In that case the line should be removed, so return true. if (new_len == 0) { *line = clean_line; *len = new_len; return true; } // The smallest removable marker is at least start_marker_len + // end_marker_len + 1 characters long. If there aren't enough // characters, then keep the line by returning false. if (new_len < delim.start_marker_len + delim.end_marker_len + 1) { return false; } // Only {{#...}}, {{/....}, {{>...}, {{!...}, {{%...}} and {{=...=}} // are "removable" if (memcmp(clean_line, delim.start_marker, delim.start_marker_len) != 0 || !strchr("#/>!%=", clean_line[delim.start_marker_len])) { return false; } const char *found_end_marker = memmatch(clean_line + delim.start_marker_len, new_len - delim.start_marker_len, delim.end_marker, delim.end_marker_len); // Make sure the end marker comes at the end of the line. if (!found_end_marker || found_end_marker + delim.end_marker_len != clean_line + new_len) { return false; } // else return the line stripped of its white space chars so when the // marker is removed in expansion, no white space is left from the line // that has now been removed *line = clean_line; *len = new_len; return true; } size_t Template::InsertLine(const char *line, size_t len, Strip strip, const MarkerDelimiters& delim, char* buffer) { bool add_newline = (len > 0 && line[len-1] == '\n'); if (add_newline) len--; // so we ignore the newline from now on if (strip >= STRIP_WHITESPACE) { StripTemplateWhiteSpace(&line, &len); add_newline = false; // IsBlankOrOnlyHasOneRemovableMarker may modify the two input // parameters if the line contains only spaces or only one input // marker. This modification must be done before the line is // written to the input buffer. Hence the need for the boolean flag // add_newline to be referenced after the Write statement. } else if (strip >= STRIP_BLANK_LINES && IsBlankOrOnlyHasOneRemovableMarker(&line, &len, delim)) { add_newline = false; } memcpy(buffer, line, len); if (add_newline) { buffer[len++] = '\n'; } return len; } void Template::StripBuffer(char **buffer, size_t* len) { if (strip_ == DO_NOT_STRIP) return; char* bufend = *buffer + *len; char* retval = new char[*len]; char* write_pos = retval; MarkerDelimiters delim; const char* next_pos = NULL; for (const char* prev_pos = *buffer; prev_pos < bufend; prev_pos = next_pos) { next_pos = (char*)memchr(prev_pos, '\n', bufend - prev_pos); if (next_pos) next_pos++; // include the newline else next_pos = bufend; // for the last line, when it has no newline write_pos += InsertLine(prev_pos, next_pos - prev_pos, strip_, delim, write_pos); assert(write_pos >= retval && static_cast(write_pos-retval) <= *len); // Before looking at the next line, see if the current line // changed the marker-delimiter. We care for // IsBlankOrOnlyHasOneRemovableMarker, so we don't need to be // perfect -- we don't have to handle the delimiter changing in // the middle of a line -- just make sure that the next time // there's only one marker on a line, we notice because we know // the right delim. const char* end_marker = NULL; for (const char* marker = prev_pos; marker; marker = end_marker) { marker = memmatch(marker, next_pos - marker, delim.start_marker, delim.start_marker_len); if (!marker) break; end_marker = memmatch(marker + delim.start_marker_len, next_pos - (marker + delim.start_marker_len), delim.end_marker, delim.end_marker_len); if (!end_marker) break; end_marker += delim.end_marker_len; // needed for the for loop // This tries to parse the marker as a set-delimiters marker. // If it succeeds, it updates delim. If not, it ignores it. assert(((end_marker - delim.end_marker_len) - (marker + delim.start_marker_len)) >= 0); Template::ParseDelimiters(marker + delim.start_marker_len, ((end_marker - delim.end_marker_len) - (marker + delim.start_marker_len)), &delim); } } assert(write_pos >= retval); // Replace the input retval with our new retval. delete[] *buffer; *buffer = retval; *len = static_cast(write_pos - retval); } // ---------------------------------------------------------------------- // Template::ReloadIfChanged() // Template::ReloadIfChangedLocked() // If one template, try immediately to reload it from disk. If all // templates, just set all their reload statuses to true, so next time // GetTemplate() is called on the template, it will be reloaded from disk if // the disk version is newer than the one currently in memory. // ReloadIfChanged() returns true if the file changed and disk *and* we // successfully reloaded and parsed it. It never returns true if // original_filename_ is "". // ---------------------------------------------------------------------- // Besides being called when locked, it's also ok to call this from // the constructor, when you know nobody else will be messing with // this object. bool Template::ReloadIfChangedLocked() EXCLUSIVE_LOCKS_REQUIRED(g_template_mutex) { // TODO(panicker): Remove this duplicate code when constructing the template, // after deprecating this method. // TemplateCache::GetTemplate() already checks if the template filename is // valid and resolvable. It also checks if the file needs to be reloaded // based on mtime. // NOTE(panicker): we should not be using original_filename_ to determine // if a template is string-based, instead use the boolean 'string_based' // in the template cache. if (original_filename_.empty()) { // string-based templates don't reload return false; } FileStat statbuf; if (resolved_filename_.empty()) { if (!template_cache_->ResolveTemplateFilename(original_filename_, &resolved_filename_, &statbuf)) { LOG(WARNING) << "Unable to locate file " << original_filename_ << endl; set_state(TS_ERROR); return false; } } else { if (!File::Stat(resolved_filename_, &statbuf)) { LOG(WARNING) << "Unable to stat file " << resolved_filename_ << endl; // We keep the old tree if there is one, otherwise we're in error set_state(TS_ERROR); return false; } } if (statbuf.IsDirectory()) { LOG(WARNING) << resolved_filename_ << "is a directory and thus not readable" << endl; // We keep the old tree if there is one, otherwise we're in error set_state(TS_ERROR); return false; } if (statbuf.mtime == filename_mtime_ && filename_mtime_ > 0 && tree_) { // force a reload if we don't already have a tree_ VLOG(1) << "Not reloading file " << resolved_filename_ << ": no new mod-time" << endl; set_state(TS_READY); return false; // file's timestamp hasn't changed, so no need to reload } File* fp = File::Open(resolved_filename_.c_str(), "r"); if (fp == NULL) { LOG(ERROR) << "Can't find file " << resolved_filename_ << "; skipping" << endl; // We keep the old tree if there is one, otherwise we're in error set_state(TS_ERROR); return false; } size_t buflen = statbuf.length; char* file_buffer = new char[buflen]; if (fp->Read(file_buffer, buflen) != buflen) { LOG(ERROR) << "Error reading file " << resolved_filename_ << ": " << strerror(errno) << endl; fp->Close(); delete[] file_buffer; // We could just keep the old tree, but probably safer to say 'error' set_state(TS_ERROR); return false; } fp->Close(); // Now that we know we've read the file ok, mark the new mtime filename_mtime_ = statbuf.mtime; // Parse the input one line at a time to get the "stripped" input. StripBuffer(&file_buffer, &buflen); // Re-initialize Auto-Escape data. Delete the parser and reset the template // context back to TC_MANUAL. If the new content has the AUTOESCAPE pragma, // the parser will then be re-created. initial_context_ = TC_MANUAL; delete htmlparser_; htmlparser_ = NULL; // Now parse the template we just read. BuildTree takes over ownership // of input_buffer in every case, and will eventually delete it. if ( BuildTree(file_buffer, file_buffer + buflen) ) { assert(state() == TS_READY); return true; } else { assert(state() != TS_READY); return false; } } // ---------------------------------------------------------------------- // Template::ExpandLocked() // Template::ExpandWithDataAndCache() // This is the main function clients call: it expands a template // by expanding its parse tree (which starts with a top-level // section node). For each variable/section/include-template it // sees, it replaces the name stored in the parse-tree with the // appropriate value from the passed-in dictionary. // ---------------------------------------------------------------------- bool Template::ExpandLocked(ExpandEmitter *expand_emitter, const TemplateDictionaryInterface *dict, PerExpandData *per_expand_data, const TemplateCache *cache) const SHARED_LOCKS_REQUIRED(g_template_mutex) { // Accumulator for the results of Expand for each sub-tree. bool error_free = true; // TODO(csilvers): could make this static if it's expensive to construct. PerExpandData empty_per_expand_data; if (per_expand_data == NULL) per_expand_data = &empty_per_expand_data; if (state() != TS_READY) { // We'd like to reload if reload status is true, but ExpandWD() is const return false; } if (per_expand_data->annotate()) { // Remove the machine dependent prefix from the template file name. const char* file = template_file(); const char* short_file = strstr(file, per_expand_data->annotate_path()); if (short_file != NULL) { file = short_file; } per_expand_data->annotator()->EmitOpenFile(expand_emitter, string(file)); } // If the client registered an expand-modifier, which is a modifier // meant to modify all templates after they are expanded, apply it // now. const TemplateModifier* modifier = per_expand_data->template_expansion_modifier(); if (modifier && modifier->MightModify(per_expand_data, template_file())) { // We found a expand TemplateModifier. Apply it. // // Since the expand-modifier doesn't ever have an arg (it doesn't // have a name and can't be applied in the text of a template), we // pass the template name in as the string arg in this case. string value; StringEmitter tmp_emitter(&value); error_free &= tree_->Expand(&tmp_emitter, dict, per_expand_data, cache); modifier->Modify(value.data(), value.size(), per_expand_data, expand_emitter, template_file()); } else { // No need to modify this template. error_free &= tree_->Expand(expand_emitter, dict, per_expand_data, cache); } if (per_expand_data->annotate()) { per_expand_data->annotator()->EmitCloseFile(expand_emitter); } return error_free; } bool Template::ExpandWithDataAndCache( ExpandEmitter *expand_emitter, const TemplateDictionaryInterface *dict, PerExpandData *per_expand_data, const TemplateCache *cache) const LOCKS_EXCLUDED(g_template_mutex) { // We hold g_template_mutex the entire time we expand, because // ReloadIfChanged(), which also holds template_mutex, is allowed to // delete tree_, and we want to make sure it doesn't do that (in another // thread) while we're expanding. We also protect state_, etc. // Note we only need a read-lock here, so many expands can go on at once. // TODO(csilvers): We can remove this once we delete ReloadIfChanged. // When we do that, ExpandLocked() can go away as well. ReaderMutexLock ml(&g_template_mutex); return ExpandLocked(expand_emitter, dict, per_expand_data, cache); } } ctemplate-ctemplate-2.4/src/template_annotator.cc000066400000000000000000000104041363122316600223110ustar00rootroot00000000000000// Copyright (c) 2009, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- #include #include #include #include // Emits an open annotation string. 'name' must be a string literal. #define EMIT_OPEN_ANNOTATION(emitter, name, value) \ (emitter)->Emit("{{#" name "=", 4 + sizeof(name)-1); \ (emitter)->Emit(value); \ (emitter)->Emit("}}", 2); // Emits a close annotation string. 'name' must be a string literal. #define EMIT_CLOSE_ANNOTATION(emitter, name) \ (emitter)->Emit("{{/" name "}}", 5 + sizeof(name)-1); #define EMIT_MISSING_ANNOTATION(emitter, name, value) \ (emitter)->Emit("{{" name "=", 3 + sizeof(name)-1); \ (emitter)->Emit(value); \ (emitter)->Emit("}}", 2); namespace ctemplate { using std::string; // Implementation note: TextTemplateAnnotator contains no state, and // code elsewhere is depending on this. E.g., a statically allocated // instance is used as the default annotator in the implementation of // PerExpandData. If you add state to this class, please revisit // the setup of such static instances. // This implements precisely the same annotation that was originally // built into the template.cc. Many upstream tools depend on the // exact formatting that this implementation happens to produce-- // so do not consider changes to this lightly. void TextTemplateAnnotator::EmitOpenInclude(ExpandEmitter* emitter, const string& value) { EMIT_OPEN_ANNOTATION(emitter, "INC", value); } void TextTemplateAnnotator::EmitCloseInclude(ExpandEmitter* emitter) { EMIT_CLOSE_ANNOTATION(emitter, "INC"); } void TextTemplateAnnotator::EmitOpenFile(ExpandEmitter* emitter, const string& value) { EMIT_OPEN_ANNOTATION(emitter, "FILE", value); } void TextTemplateAnnotator::EmitCloseFile(ExpandEmitter* emitter) { EMIT_CLOSE_ANNOTATION(emitter, "FILE"); } void TextTemplateAnnotator::EmitOpenSection(ExpandEmitter* emitter, const string& value) { EMIT_OPEN_ANNOTATION(emitter, "SEC", value); } void TextTemplateAnnotator::EmitCloseSection(ExpandEmitter* emitter) { EMIT_CLOSE_ANNOTATION(emitter, "SEC"); } void TextTemplateAnnotator::EmitOpenVariable(ExpandEmitter* emitter, const string& value) { EMIT_OPEN_ANNOTATION(emitter, "VAR", value); } void TextTemplateAnnotator::EmitCloseVariable(ExpandEmitter* emitter) { EMIT_CLOSE_ANNOTATION(emitter, "VAR"); } void TextTemplateAnnotator::EmitFileIsMissing(ExpandEmitter* emitter, const string& value) { EMIT_MISSING_ANNOTATION(emitter,"MISSING_FILE", value); } } ctemplate-ctemplate-2.4/src/template_cache.cc000066400000000000000000000702731363122316600213610ustar00rootroot00000000000000// Copyright (c) 2009, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- #include #include "base/mutex.h" // This must go first so we get _XOPEN_SOURCE #include #include // for assert() #include #include // for size_t #include // for strerror() #include #ifdef HAVE_UNISTD_H # include #endif // for getcwd() #include HASH_MAP_H // for hash_map<>::iterator, hash_map<>, etc #include // for pair<>, make_pair() #include // for vector<>::size_type, vector<>, etc #include "base/thread_annotations.h" // for GUARDED_BY #include #include // for Template, TemplateState #include // for Strip, DO_NOT_STRIP #include // for PathJoin(), IsAbspath(), etc #include // for StringHash #include "base/fileutil.h" #include // for cerr #ifndef PATH_MAX #ifdef MAXPATHLEN #define PATH_MAX MAXPATHLEN #else #define PATH_MAX 4096 // seems conservative for max filename len! #endif #endif using std::endl; using std::string; using std::vector; using std::pair; using std::make_pair; using HASH_NAMESPACE::unordered_map; static int kVerbosity = 0; // you can change this by hand to get vlogs #define LOG(level) std::cerr << #level ": " #define PLOG(level) std::cerr << #level ": [" << strerror(errno) << "] " #define VLOG(level) if (kVerbosity >= level) std::cerr << "V" #level ": " namespace ctemplate { // ---------------------------------------------------------------------- // TemplateCache::RefcountedTemplate // A simple refcounting class to keep track of templates, which // might be shared between caches. It also owns the pointer to // the template itself. // ---------------------------------------------------------------------- class TemplateCache::RefcountedTemplate { public: explicit RefcountedTemplate(const Template* ptr) : ptr_(ptr), refcount_(1) { } void IncRef() { MutexLock ml(&mutex_); assert(refcount_ > 0); ++refcount_; } void DecRefN(int n) { bool refcount_is_zero; { MutexLock ml(&mutex_); assert(refcount_ >= n); refcount_ -= n; refcount_is_zero = (refcount_ == 0); } // We can't delete this within the MutexLock, because when the // MutexLock tries to unlock Mutex at function-exit, the mutex // will have been deleted! This is just as safe as doing the // delete within the lock -- in either case, if anyone tried to do // anything to this class after the refcount got to 0, bad things // would happen. if (refcount_is_zero) delete this; } void DecRef() { DecRefN(1); } int refcount() const { MutexLock ml(&mutex_); // could be ReaderMutexLock, but whatever return refcount_; } const Template* tpl() const { return ptr_; } private: ~RefcountedTemplate() { delete ptr_; } const Template* const ptr_; int refcount_ GUARDED_BY(mutex_); mutable Mutex mutex_; }; // ---------------------------------------------------------------------- // TemplateCache::RefTplPtrHash // TemplateCache::TemplateCacheHash // TemplateCache::CachedTemplate // These are used for the cache-map. CachedTemplate is what is // actually stored in the map: the Template* and some information // about it (whether we need to reload it, etc.). Refcount is // a simple refcounting class, used to keep track of templates. // ---------------------------------------------------------------------- // This is needed just because many STLs (eg FreeBSD's) are unable to // hash pointers by default. class TemplateCache::RefTplPtrHash { public: size_t operator()(const RefcountedTemplate* p) const { return reinterpret_cast(p); } // Less operator for MSVC's hash containers. bool operator()(const RefcountedTemplate* a, const RefcountedTemplate* b) const { return a < b; } // These two public members are required by msvc. 4 and 8 are defaults. static const size_t bucket_size = 4; static const size_t min_buckets = 8; }; class TemplateCache::TemplateCacheHash { public: size_t operator()(const TemplateCacheKey& p) const { // Using + here is silly, but should work ok in practice. return p.first + p.second; } // Less operator for MSVC's hash containers. bool operator()(const TemplateCacheKey& a, const TemplateCacheKey& b) const { return (a.first == b.first ? a.second < b.second : a.first < b.first); } // These two public members are required by msvc. 4 and 8 are defaults. static const size_t bucket_size = 4; static const size_t min_buckets = 8; }; struct TemplateCache::CachedTemplate { enum TemplateType { UNUSED, FILE_BASED, STRING_BASED }; CachedTemplate() : refcounted_tpl(NULL), should_reload(false), template_type(UNUSED) { } CachedTemplate(const Template* tpl_ptr, TemplateType type) : refcounted_tpl(new TemplateCache::RefcountedTemplate(tpl_ptr)), should_reload(false), template_type(type) { } // we won't remove the template from the cache until refcount drops to 0 TemplateCache::RefcountedTemplate* refcounted_tpl; // shared across Clone() // reload status bool should_reload; // indicates if the template is string-based or file-based TemplateType template_type; }; // ---------------------------------------------------------------------- // TemplateCache::TemplateCache() // TemplateCache::~TemplateCache() // ---------------------------------------------------------------------- TemplateCache::TemplateCache() : parsed_template_cache_(new TemplateMap), is_frozen_(false), search_path_(), get_template_calls_(new TemplateCallMap), mutex_(new Mutex), search_path_mutex_(new Mutex) { } TemplateCache::~TemplateCache() { ClearCache(); delete parsed_template_cache_; delete get_template_calls_; delete mutex_; delete search_path_mutex_; } // ---------------------------------------------------------------------- // HasTemplateChangedOnDisk // Indicates whether the template has changed, based on the // backing file's last modtime. // ---------------------------------------------------------------------- bool HasTemplateChangedOnDisk(const char* resolved_filename, time_t mtime, FileStat* statbuf) { if (!File::Stat(resolved_filename, statbuf)) { LOG(WARNING) << "Unable to stat file " << resolved_filename << endl; // If we can't Stat the file then the file may have been deleted, // so reload the template. return true; } if (statbuf->mtime == mtime && mtime > 0) { // No need to reload yet. return false; } return true; } // ---------------------------------------------------------------------- // TemplateCache::LoadTemplate() // TemplateCache::GetTemplate() // TemplateCache::GetTemplateLocked() // TemplateCache::StringToTemplateCache() // The routines for adding a template to the cache. LoadTemplate // loads the template into the cache and returns true if the // template was successfully loaded or if it already exists in the // cache. GetTemplate loads the template into the cache from disk // and returns the parsed template. StringToTemplateCache parses // and loads the template from the given string into the parsed // cache, or returns false if an older version already exists in // the cache. // ---------------------------------------------------------------------- bool TemplateCache::LoadTemplate(const TemplateString& filename, Strip strip) { TemplateCacheKey cache_key = TemplateCacheKey(filename.GetGlobalId(), strip); WriterMutexLock ml(mutex_); return GetTemplateLocked(filename, strip, cache_key) != NULL; } const Template *TemplateCache::GetTemplate(const TemplateString& filename, Strip strip) { // No need to have the cache-mutex acquired for this step TemplateCacheKey cache_key = TemplateCacheKey(filename.GetGlobalId(), strip); CachedTemplate retval; WriterMutexLock ml(mutex_); RefcountedTemplate* refcounted_tpl = GetTemplateLocked(filename, strip, cache_key); if (!refcounted_tpl) return NULL; refcounted_tpl->IncRef(); // DecRef() is in DoneWithGetTemplatePtrs() (*get_template_calls_)[refcounted_tpl]++; // set up for DoneWith...() return refcounted_tpl->tpl(); } TemplateCache::RefcountedTemplate* TemplateCache::GetTemplateLocked( const TemplateString& filename, Strip strip, const TemplateCacheKey& template_cache_key) { // NOTE: A write-lock must be held on mutex_ when this method is called. CachedTemplate* it = find_ptr(*parsed_template_cache_, template_cache_key); if (!it) { // If the cache is frozen and the template doesn't already exist in cache, // do not load the template, return NULL. if (is_frozen_) { return NULL; } // TODO(panicker): Validate the filename here, and if the file can't be // resolved then insert a NULL in the cache. // If validation succeeds then pass in resolved filename, mtime & // file length (from statbuf) to the constructor. const Template* tpl = new Template(filename, strip, this); it = &(*parsed_template_cache_)[template_cache_key]; *it = CachedTemplate(tpl, CachedTemplate::FILE_BASED); assert(it); } if (it->should_reload) { // check if the template has changed on disk or if a new template with the // same name has been added earlier in the search path: const string resolved = FindTemplateFilename( it->refcounted_tpl->tpl()->original_filename()); FileStat statbuf; if (it->template_type == CachedTemplate::FILE_BASED && (resolved != it->refcounted_tpl->tpl()->template_file() || HasTemplateChangedOnDisk( it->refcounted_tpl->tpl()->template_file(), it->refcounted_tpl->tpl()->mtime(), &statbuf))) { // Create a new template, insert it into the cache under // template_cache_key, and DecRef() the old one to indicate // the cache no longer has a reference to it. const Template* tpl = new Template(filename, strip, this); // DecRef after creating the new template since DecRef may free up // the storage for filename, it->refcounted_tpl->DecRef(); *it = CachedTemplate(tpl, CachedTemplate::FILE_BASED); } it->should_reload = false; } // If the state is TS_ERROR, we leave the state as is, but return // NULL. We won't try to load the template file again until the // reload status is set to true by another call to ReloadAllIfChanged. return it->refcounted_tpl->tpl()->state() == TS_READY ? it->refcounted_tpl : NULL; } bool TemplateCache::StringToTemplateCache(const TemplateString& key, const TemplateString& content, Strip strip) { TemplateCacheKey template_cache_key = TemplateCacheKey(key.GetGlobalId(), strip); { ReaderMutexLock ml(mutex_); if (is_frozen_) { return false; } // If the key is already in the parsed-cache, we just return false. CachedTemplate* it = find_ptr(*parsed_template_cache_, template_cache_key); if (it && it->refcounted_tpl->tpl()->state() != TS_ERROR) { return false; } } Template* tpl = Template::StringToTemplate(content, strip); if (tpl == NULL) { return false; } if (tpl->state() != TS_READY) { delete tpl; return false; } WriterMutexLock ml(mutex_); // Double-check it wasn't just inserted. CachedTemplate* it = find_ptr(*parsed_template_cache_, template_cache_key); if (it) { if (it->refcounted_tpl->tpl()->state() == TS_ERROR) { // replace the old entry with the new one it->refcounted_tpl->DecRef(); } else { delete tpl; return false; } } // Insert into cache. (*parsed_template_cache_)[template_cache_key] = CachedTemplate(tpl, CachedTemplate::STRING_BASED); return true; } // ---------------------------------------------------------------------- // TemplateCache::ExpandWithData() // TemplateCache::ExpandFrozen() // TemplateCache::ExpandLocked() // ExpandWithData gets the template from the parsed-cache, possibly // loading the template on-demand, and then expands the template. // ExpandFrozen is for frozen caches only -- if the filename isn't // in the cache, the routine fails (returns false) rather than trying // to fetch the template. ExpandLocked is used for recursive // sub-template includes, and just tells template.cc it doesn't // need to recursively acquire any locks. // ---------------------------------------------------------------------- bool TemplateCache::ExpandWithData(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface *dict, PerExpandData *per_expand_data, ExpandEmitter *expand_emitter) { TemplateCacheKey template_cache_key(filename.GetGlobalId(), strip); // We make a local copy of this struct so we don't have to worry about // what happens to our cache while we don't hold the lock (during Expand). RefcountedTemplate* refcounted_tpl = NULL; { WriterMutexLock ml(mutex_); // Optionally load the template (depending on whether the cache is frozen, // the reload bit is set etc.) refcounted_tpl = GetTemplateLocked(filename, strip, template_cache_key); if (!refcounted_tpl) return false; refcounted_tpl->IncRef(); } const bool result = refcounted_tpl->tpl()->ExpandWithDataAndCache( expand_emitter, dict, per_expand_data, this); { WriterMutexLock ml(mutex_); refcounted_tpl->DecRef(); } return result; } bool TemplateCache::ExpandNoLoad( const TemplateString& filename, Strip strip, const TemplateDictionaryInterface *dict, PerExpandData *per_expand_data, ExpandEmitter *expand_emitter) const { TemplateCacheKey template_cache_key(filename.GetGlobalId(), strip); CachedTemplate cached_tpl; { ReaderMutexLock ml(mutex_); if (!is_frozen_) { LOG(DFATAL) << ": ExpandNoLoad() only works on frozen caches."; return false; } CachedTemplate* it = find_ptr(*parsed_template_cache_, template_cache_key); if (!it) { return false; } cached_tpl = *it; cached_tpl.refcounted_tpl->IncRef(); } const bool result = cached_tpl.refcounted_tpl->tpl()->ExpandWithDataAndCache( expand_emitter, dict, per_expand_data, this); { WriterMutexLock ml(mutex_); cached_tpl.refcounted_tpl->DecRef(); } return result; } // Note: "Locked" in this name refers to the template object, not to // use; we still need to acquire our locks as per normal. bool TemplateCache::ExpandLocked(const TemplateString& filename, Strip strip, ExpandEmitter *expand_emitter, const TemplateDictionaryInterface *dict, PerExpandData *per_expand_data) { TemplateCacheKey template_cache_key(filename.GetGlobalId(), strip); RefcountedTemplate* refcounted_tpl = NULL; { WriterMutexLock ml(mutex_); refcounted_tpl = GetTemplateLocked(filename, strip, template_cache_key); if (!refcounted_tpl) return false; refcounted_tpl->IncRef(); } const bool result = refcounted_tpl->tpl()->ExpandLocked( expand_emitter, dict, per_expand_data, this); { WriterMutexLock ml(mutex_); refcounted_tpl->DecRef(); } return result; } // ---------------------------------------------------------------------- // TemplateCache::SetTemplateRootDirectory() // TemplateCache::AddAlternateTemplateRootDirectory() // TemplateCache::template_root_directory() // TemplateCache::FindTemplateFilename() // The template-root-directory is where we look for template // files (in GetTemplate and include templates) when they're // given with a relative rather than absolute name. You can // set a 'main' root directory (where we look first), as well // as alternates. // ---------------------------------------------------------------------- bool TemplateCache::AddAlternateTemplateRootDirectoryHelper( const string& directory, bool clear_template_search_path) { { ReaderMutexLock ml(mutex_); if (is_frozen_) { // Cannot set root-directory on a frozen cache. return false; } } string normalized = directory; // make sure it ends with '/' NormalizeDirectory(&normalized); // Make the directory absolute if it isn't already. This makes code // safer if client later does a chdir. if (!IsAbspath(normalized)) { char* cwdbuf = new char[PATH_MAX]; // new to avoid stack overflow const char* cwd = getcwd(cwdbuf, PATH_MAX); if (!cwd) { // probably not possible, but best to be defensive PLOG(WARNING) << "Unable to convert '" << normalized << "' to an absolute path, with cwd=" << cwdbuf; } else { normalized = PathJoin(cwd, normalized); } delete[] cwdbuf; } VLOG(2) << "Setting Template directory to " << normalized << endl; { WriterMutexLock ml(search_path_mutex_); if (clear_template_search_path) { search_path_.clear(); } search_path_.push_back(normalized); } // NOTE(williasr): The template root is not part of the template // cache key, so we need to invalidate the cache contents. ReloadAllIfChanged(LAZY_RELOAD); return true; } bool TemplateCache::SetTemplateRootDirectory(const string& directory) { return AddAlternateTemplateRootDirectoryHelper(directory, true); } bool TemplateCache::AddAlternateTemplateRootDirectory( const string& directory) { return AddAlternateTemplateRootDirectoryHelper(directory, false); } string TemplateCache::template_root_directory() const { ReaderMutexLock ml(search_path_mutex_); if (search_path_.empty()) { return kCWD; } return search_path_[0]; } // Given an unresolved filename, look through the template search path // to see if the template can be found. If so, resolved contains the // resolved filename, statbuf contains the stat structure for the file // (to avoid double-statting the file), and the function returns // true. Otherwise, the function returns false. bool TemplateCache::ResolveTemplateFilename(const string& unresolved, string* resolved, FileStat* statbuf) const { ReaderMutexLock ml(search_path_mutex_); if (search_path_.empty() || IsAbspath(unresolved)) { *resolved = unresolved; if (File::Stat(*resolved, statbuf)) { VLOG(1) << "Resolved " << unresolved << " to " << *resolved << endl; return true; } } else { for (TemplateSearchPath::const_iterator path = search_path_.begin(); path != search_path_.end(); ++path) { *resolved = PathJoin(*path, unresolved); if (File::Stat(*resolved, statbuf)) { VLOG(1) << "Resolved " << unresolved << " to " << *resolved << endl; return true; } } } resolved->clear(); return false; } string TemplateCache::FindTemplateFilename(const string& unresolved) const { string resolved; FileStat statbuf; if (!ResolveTemplateFilename(unresolved, &resolved, &statbuf)) resolved.clear(); return resolved; } // ---------------------------------------------------------------------- // TemplateCache::Delete() // TemplateCache::ClearCache() // Delete deletes one entry from the cache. // ---------------------------------------------------------------------- bool TemplateCache::Delete(const TemplateString& key) { WriterMutexLock ml(mutex_); if (is_frozen_) { // Cannot delete from a frozen cache. return false; } vector to_erase; const TemplateId key_id = key.GetGlobalId(); for (TemplateMap::iterator it = parsed_template_cache_->begin(); it != parsed_template_cache_->end(); ++it) { if (it->first.first == key_id) { // We'll delete the content pointed to by the entry here, since // it's handy, but we won't delete the entry itself quite yet. it->second.refcounted_tpl->DecRef(); to_erase.push_back(it->first); } } for (vector::iterator it = to_erase.begin(); it != to_erase.end(); ++it) { parsed_template_cache_->erase(*it); } return !to_erase.empty(); } void TemplateCache::ClearCache() { // NOTE: We allow a frozen cache to be cleared with this method, although // no other changes can be made to the cache. // We clear the cache by swapping it with an empty cache. This lets // us delete the items in the cache at our leisure without needing // to hold mutex_. TemplateMap tmp_cache; { WriterMutexLock ml(mutex_); parsed_template_cache_->swap(tmp_cache); is_frozen_ = false; } for (TemplateMap::iterator it = tmp_cache.begin(); it != tmp_cache.end(); ++it) { it->second.refcounted_tpl->DecRef(); } // Do a decref for all templates ever returned by GetTemplate(). DoneWithGetTemplatePtrs(); } // ---------------------------------------------------------------------- // TemplateCache::DoneWithGetTemplatePtrs() // DoneWithGetTemplatePtrs() DecRefs every template in the // get_template_calls_ list. This is because the user of // GetTemplate() didn't have a pointer to the refcounted Template // to do this themselves. Note we only provide this as a batch // operation, so the user should be careful to only call this when // they are no longer using *any* template ever retrieved by // this cache's GetTemplate(). // ---------------------------------------------------------------------- void TemplateCache::DoneWithGetTemplatePtrs() { WriterMutexLock ml(mutex_); for (TemplateCallMap::iterator it = get_template_calls_->begin(); it != get_template_calls_->end(); ++it) { it->first->DecRefN(it->second); // it.second: # of times GetTpl was called } get_template_calls_->clear(); } // ---------------------------------------------------------------------- // TemplateCache::ReloadAllIfChanged() // IMMEDIATE_RELOAD attempts to immediately reload and parse // all templates if the corresponding template files have changed. // LAZY_RELOAD just sets the reload bit in the cache so that the next // GetTemplate will reload and parse the template, if it changed. // NOTE: Suppose the search path is "dira:dirb", and a template is // created with name "foo", which resolves to "dirb/foo" because // dira/foo does not exist. Then suppose dira/foo is created and then // ReloadAllIfChanged() is called. Then ReloadAllIfChanged() will replace // the contents of the template with dira/foo, *not* dirb/foo, even if // dirb/foo hasn't changed. // ---------------------------------------------------------------------- void TemplateCache::ReloadAllIfChanged(ReloadType reload_type) { WriterMutexLock ml(mutex_); if (is_frozen_) { // do not reload a frozen cache. return; } for (TemplateMap::iterator it = parsed_template_cache_->begin(); it != parsed_template_cache_->end(); ++it) { it->second.should_reload = true; if (reload_type == IMMEDIATE_RELOAD) { const Template* tpl = it->second.refcounted_tpl->tpl(); // Reload should always use the original filename. // For instance on reload, we may replace an existing template with a // new one that came earlier on the search path. GetTemplateLocked(tpl->original_filename(), tpl->strip(), it->first); } } } // ---------------------------------------------------------------------- // TemplateCache::Freeze() // This method marks the cache as 'frozen'. After this method is called, // the cache is immutable, and cannot be modified. New templates cannot be // loaded and existing templates cannot be reloaded. // ---------------------------------------------------------------------- void TemplateCache::Freeze() { { ReaderMutexLock ml(mutex_); if (is_frozen_) { // if already frozen, then this is a no-op. return; } } // A final reload before freezing the cache. ReloadAllIfChanged(IMMEDIATE_RELOAD); { WriterMutexLock ml(mutex_); is_frozen_ = true; } } // ---------------------------------------------------------------------- // TemplateCache::Clone() // Clone makes a shallow copy of the parsed cache by incrementing // templates' refcount. // The caller is responsible for deallocating the returned TemplateCache. // ---------------------------------------------------------------------- TemplateCache* TemplateCache::Clone() const { ReaderMutexLock ml(mutex_); TemplateCache* new_cache = new TemplateCache(); *(new_cache->parsed_template_cache_) = *parsed_template_cache_; for (TemplateMap::iterator it = parsed_template_cache_->begin(); it != parsed_template_cache_->end(); ++it) { it->second.refcounted_tpl->IncRef(); } return new_cache; } // ---------------------------------------------------------------------- // TemplateCache::Refcount() // This routine is DEBUG-only. It returns the refcount of a template, // given the TemplateCacheKey. // ---------------------------------------------------------------------- int TemplateCache::Refcount(const TemplateCacheKey template_cache_key) const { ReaderMutexLock ml(mutex_); CachedTemplate* it = find_ptr(*parsed_template_cache_, template_cache_key); return it ? it->refcounted_tpl->refcount() : 0; } // ---------------------------------------------------------------------- // TemplateCache::TemplateIsCached() // This routine is for testing only -- is says whether a given // template is already in the cache or not. // ---------------------------------------------------------------------- bool TemplateCache::TemplateIsCached(const TemplateCacheKey template_cache_key) const { ReaderMutexLock ml(mutex_); return parsed_template_cache_->count(template_cache_key); } // ---------------------------------------------------------------------- // TemplateCache::ValidTemplateFilename // Validates the filename before constructing the template. // ---------------------------------------------------------------------- bool TemplateCache::IsValidTemplateFilename(const string& filename, string* resolved_filename, FileStat* statbuf) const { if (!ResolveTemplateFilename(filename, resolved_filename, statbuf)) { LOG(WARNING) << "Unable to locate file " << filename << endl; return false; } if (statbuf->IsDirectory()) { LOG(WARNING) << *resolved_filename << "is a directory and thus not readable" << endl; return false; } return true; } } ctemplate-ctemplate-2.4/src/template_dictionary.cc000066400000000000000000001210241363122316600224520ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // Based on the 'old' TemplateDictionary by Frank Jernigan. #include #include "base/mutex.h" // This must go first so we get _XOPEN_SOURCE #include #include #include #include #include // for sort() #include HASH_MAP_H #include #include #include // for pair<> #include #include "base/arena-inl.h" #include "base/thread_annotations.h" #include "indented_writer.h" #include #include #include #include "base/small_map.h" #include "base/util.h" // for DCHECK using std::vector; using std::string; using std::map; using std::pair; using std::make_pair; namespace ctemplate { // Guards the initialization of the global dictionary. static GoogleOnceType g_once = GOOGLE_ONCE_INIT; // Guard access to the global dictionary. static Mutex g_static_mutex(base::LINKER_INITIALIZED); /*static*/ UnsafeArena* const TemplateDictionary::NO_ARENA = NULL; /*static*/ TemplateDictionary::GlobalDict* TemplateDictionary::global_dict_ GUARDED_BY(g_static_mutex) PT_GUARDED_BY(g_static_mutex) = NULL; /*static*/ TemplateString* TemplateDictionary::empty_string_ = NULL; static const char* const kAnnotateOutput = "__ctemplate_annotate_output__"; // ---------------------------------------------------------------------- // TemplateDictionary::map_arena_init // This class is what small_map<> uses to create a new // arena-allocated map<> when it decides it needs to do that. // ---------------------------------------------------------------------- class TemplateDictionary::map_arena_init { public: map_arena_init(UnsafeArena* arena) : arena_(arena) { } template void operator ()(ManualConstructor* map) const { map->Init(typename T::key_compare(), arena_); } private: UnsafeArena* arena_; }; // ---------------------------------------------------------------------- // TemplateDictionary::LazilyCreateDict() // TemplateDictionary::LazilyCreateTemplateGlobalDict() // TemplateDictionary::CreateDictVector() // TemplateDictionary::CreateTemplateSubdict() // These routines allocate the objects that TemplateDictionary // allocates (sub-dictionaries, variable maps, etc). Each // allocates memory on the arena, and instructs the STL objects // to use the arena for their own internal allocations as well. // ---------------------------------------------------------------------- template inline void TemplateDictionary::LazilyCreateDict(T** dict) { if (*dict != NULL) return; // Placement new: construct the map in the memory used by *dict. void* buffer = arena_->AllocAligned(sizeof(**dict), BaseArena::kDefaultAlignment); new (buffer) T(arena_); *dict = reinterpret_cast(buffer); } inline void TemplateDictionary::LazyCreateTemplateGlobalDict() { if (!template_global_dict_owner_->template_global_dict_) { template_global_dict_owner_->template_global_dict_ = CreateTemplateSubdict("Template Globals", arena_, template_global_dict_owner_, template_global_dict_owner_); } } inline TemplateDictionary::DictVector* TemplateDictionary::CreateDictVector() { void* buffer = arena_->AllocAligned(sizeof(DictVector), BaseArena::kDefaultAlignment); // Placement new: construct the vector in the memory used by buffer. new (buffer) DictVector(arena_); return reinterpret_cast(buffer); } inline TemplateDictionary* TemplateDictionary::CreateTemplateSubdict( const TemplateString& name, UnsafeArena* arena, TemplateDictionary* parent_dict, TemplateDictionary* template_global_dict_owner) { void* buffer = arena->AllocAligned(sizeof(TemplateDictionary), BaseArena::kDefaultAlignment); // Placement new: construct the sub-tpl in the memory used by tplbuf. new (buffer) TemplateDictionary(name, arena, parent_dict, template_global_dict_owner); return reinterpret_cast(buffer); } // ---------------------------------------------------------------------- // TemplateDictionary::HashInsert() // A convenience function that's equivalent to m[key] = value, but // converting the key to an id first, and without necessarily needing // key to have a default constructor like operator[] does. It also // inserts (key, id(key)) into a map to allow for id->key mapping. // ---------------------------------------------------------------------- // By default, prefer the m[key] = value construct. We do something // more complex for TemplateString, though, since m[key] requires a // zero-arg constructor, which TemplateString doesn't have. We could // do the more complex thing everywhere, but that seems to trigger a // bug in in gcc 4.1.2 (at least) when compiled with -O2. Shrug. namespace { template inline void DoHashInsert(MapType* m, TemplateId id, ValueType value) { (*m)[id] = value; } template inline void DoHashInsert(MapType* m, TemplateId id, TemplateString value) { pair r = m->insert(typename MapType::value_type(id, value)); // Unfortunately, insert() doesn't actually replace if key is // already in the map. Thus, in that case (insert().second == false), // we need to overwrite the old value. Since TemplateString // doesn't define operator=, the easiest legal way to overwrite is // to use the copy-constructor with placement-new. Note that since // TemplateString has no destructor, we don't need to call the // destructor to 'clear out' the old value. if (r.second == false) { new (&r.first->second) TemplateString(value); } } } template void TemplateDictionary::HashInsert(MapType* m, TemplateString key, ValueType value) { const TemplateId id = key.GetGlobalId(); DoHashInsert(m, id, value); AddToIdToNameMap(id, key); // allows us to do the hash-key -> name mapping } // ---------------------------------------------------------------------- // TemplateDictionary::SetupGlobalDict() // Must be called exactly once before accessing global_dict_. // GoogleOnceInit() is used to manage that initialization in a thread-safe // way. // ---------------------------------------------------------------------- /*static*/ void TemplateDictionary::SetupGlobalDict() NO_THREAD_SAFETY_ANALYSIS { global_dict_ = new TemplateDictionary::GlobalDict; // Initialize the built-ins HashInsert(global_dict_, TemplateString("BI_SPACE"), TemplateString(" ")); HashInsert(global_dict_, TemplateString("BI_NEWLINE"), TemplateString("\n")); // This is used for name-lookup misses. empty_string_ = new TemplateString(""); } // ---------------------------------------------------------------------- // TemplateDictionary::TemplateDictionary() // TemplateDictionary::~TemplateDictionary() // The only tricky thing is that we make sure the static vars are // set up properly. This must be done at each construct time, // because it's the responsibility of the first dictionary created // in the program to set up the globals, and that could be us. // The UnsafeArena() argument is how big to make each arena // block. Too big and space is wasted. Too small and we spend // a lot of time allocating new arena blocks. 32k seems right. // ---------------------------------------------------------------------- TemplateDictionary::TemplateDictionary(const TemplateString& name, UnsafeArena* arena) : arena_(arena ? arena : new UnsafeArena(32768)), should_delete_arena_(arena ? false : true), // true if we called new name_(Memdup(name)), // arena must have been set up first variable_dict_(NULL), section_dict_(NULL), include_dict_(NULL), template_global_dict_(NULL), template_global_dict_owner_(this), parent_dict_(NULL), filename_(NULL) { GoogleOnceInit(&g_once, &SetupGlobalDict); } TemplateDictionary::TemplateDictionary( const TemplateString& name, UnsafeArena* arena, TemplateDictionary* parent_dict, TemplateDictionary* template_global_dict_owner) : arena_(arena), should_delete_arena_(false), // parents own it name_(Memdup(name)), // arena must have been set up first variable_dict_(NULL), section_dict_(NULL), include_dict_(NULL), template_global_dict_(NULL), template_global_dict_owner_(template_global_dict_owner), parent_dict_(parent_dict), filename_(NULL) { assert(template_global_dict_owner_ != NULL); GoogleOnceInit(&g_once, &SetupGlobalDict); } TemplateDictionary::~TemplateDictionary() { // Everything we allocate, we allocate on the arena, so we // don't need to free anything here. if (should_delete_arena_) { delete arena_; } } // ---------------------------------------------------------------------- // TemplateDictionary::MakeCopy() // Makes a recursive copy: so we copy any include dictionaries and // section dictionaries we see as well. InternalMakeCopy() is // needed just so we can ensure that if we're doing a copy of a // subtree, it's due to a recursive call. Returns NULL if there // is an error copying. // ---------------------------------------------------------------------- TemplateDictionary* TemplateDictionary::InternalMakeCopy( const TemplateString& name_of_copy, UnsafeArena* arena, TemplateDictionary* parent_dict, TemplateDictionary* template_global_dict_owner) { TemplateDictionary* newdict; if (template_global_dict_owner_ == this) { // We're a root-level template. We want the copy to be just like // us, and have its own template_global_dict_, that it owns. // We use the normal global new, since newdict will be returned // to the user. newdict = new TemplateDictionary(name_of_copy, arena); } else { // recursive calls use private contructor // We're not a root-level template, so we want the copy to refer to the // same template_global_dict_ owner that we do. // Note: we always use our own arena, even when we have a parent // (though we have the same arena as our parent when we have one). assert(arena); assert(parent_dict ? arena == parent_dict->arena_ : true); newdict = CreateTemplateSubdict(name_of_copy, arena, parent_dict, template_global_dict_owner); } // Copy the variable dictionary if (variable_dict_) { newdict->LazilyCreateDict(&newdict->variable_dict_); for (VariableDict::const_iterator it = variable_dict_->begin(); it != variable_dict_->end(); ++it) { newdict->variable_dict_->insert(make_pair(it->first, newdict->Memdup(it->second))); } } // ...and the template-global-dict, if we have one (only root-level tpls do) if (template_global_dict_) { newdict->template_global_dict_ = template_global_dict_->InternalMakeCopy( template_global_dict_->name(), newdict->arena_, newdict, newdict->template_global_dict_owner_); } // Copy the section dictionary if (section_dict_) { newdict->LazilyCreateDict(&newdict->section_dict_); for (SectionDict::iterator it = section_dict_->begin(); it != section_dict_->end(); ++it) { DictVector* dicts = newdict->CreateDictVector(); newdict->section_dict_->insert(make_pair(it->first, dicts)); for (DictVector::iterator it2 = it->second->begin(); it2 != it->second->end(); ++it2) { TemplateDictionary* subdict = *it2; // In this case, we pass in newdict as the parent of our new dict. dicts->push_back(subdict->InternalMakeCopy( subdict->name(), newdict->arena_, newdict, newdict->template_global_dict_owner_)); } } } // Copy the includes-dictionary if (include_dict_) { newdict->LazilyCreateDict(&newdict->include_dict_); for (IncludeDict::iterator it = include_dict_->begin(); it != include_dict_->end(); ++it) { DictVector* dicts = newdict->CreateDictVector(); newdict->include_dict_->insert(make_pair(it->first, dicts)); for (DictVector::iterator it2 = it->second->begin(); it2 != it->second->end(); ++it2) { TemplateDictionary* subdict = *it2; // In this case, we pass in NULL as the parent of our new dict: // parents are not inherited across include-dictionaries. dicts->push_back(subdict->InternalMakeCopy( subdict->name(), newdict->arena_, NULL, newdict->template_global_dict_owner_)); } } } // Finally, copy everything else not set properly by the constructor newdict->filename_ = newdict->Memdup(filename_).ptr_; return newdict; } TemplateDictionary* TemplateDictionary::MakeCopy( const TemplateString& name_of_copy, UnsafeArena* arena) { if (template_global_dict_owner_ != this) { // We're not at the root, which is illegal. return NULL; } return InternalMakeCopy(name_of_copy, arena, NULL, template_global_dict_owner_); } // ---------------------------------------------------------------------- // TemplateDictionary::StringAppendV() // Does an snprintf to a string. Idea is to grow string as needed. // Writes to space if possible -- caller must ensure space has // size at least 1024 -- and if not allocates a buffer of its // own which the caller must free. Sets out to the buffer written // to (space or something else). Returns the number of bytes // written into out. // ---------------------------------------------------------------------- int TemplateDictionary::StringAppendV(char* space, char** out, const char* format, va_list ap) { const int kBufsize = 1024; // It's possible for methods that use a va_list to invalidate // the data in it upon use. The fix is to make a copy // of the structure before using it and use that copy instead. va_list backup_ap; va_copy(backup_ap, ap); int result = vsnprintf(space, kBufsize, format, backup_ap); va_end(backup_ap); if ((result >= 0) && (result < kBufsize)) { *out = space; return result; // It fit } // Repeatedly increase buffer size until it fits int length = kBufsize; while (true) { if (result < 0) { // Older snprintf() behavior. :-( Just try doubling the buffer size length *= 2; } else { // We need exactly "result+1" characters length = result+1; } char* buf = new char[length]; // Restore the va_list before we use it again va_copy(backup_ap, ap); result = vsnprintf(buf, length, format, backup_ap); va_end(backup_ap); if ((result >= 0) && (result < length)) { *out = buf; return result; } delete[] buf; } } // ---------------------------------------------------------------------- // TemplateDictionary::SetValue() // TemplateDictionary::SetIntValue() // TemplateDictionary::SetFormattedValue() // TemplateDictionary::SetEscapedValue() // TemplateDictionary::SetEscapedFormattedValue() // The functions to set the value of a variable. For each, // I first define the char*+length version. Then, after those // five definitions, I define a zillion alternate versions: // strings, char*s, etc. The only non-obvious thing about // each function is I make sure to copy both key and value to // the arena, so we have our own, persistent copy of them. // ---------------------------------------------------------------------- void TemplateDictionary::SetValue(const TemplateString variable, const TemplateString value) { LazilyCreateDict(&variable_dict_); HashInsert(variable_dict_, variable, Memdup(value)); } void TemplateDictionary::SetValueWithoutCopy(const TemplateString variable, const TemplateString value) { LazilyCreateDict(&variable_dict_); // Don't memdup value - the caller will manage memory. HashInsert(variable_dict_, variable, value); } void TemplateDictionary::SetIntValue(const TemplateString variable, long value) { char buffer[64]; // big enough for any int int valuelen = snprintf(buffer, sizeof(buffer), "%ld", value); LazilyCreateDict(&variable_dict_); HashInsert(variable_dict_, variable, Memdup(buffer, valuelen)); } void TemplateDictionary::SetFormattedValue(const TemplateString variable, const char* format, ...) { char* buffer; char* scratch = arena_->Alloc(1024); // StringAppendV requires >=1024 bytes va_list ap; va_start(ap, format); const int buflen = StringAppendV(scratch, &buffer, format, ap); va_end(ap); LazilyCreateDict(&variable_dict_); // If it fit into scratch, great, otherwise we need to copy into arena if (buffer == scratch) { scratch = arena_->Shrink(scratch, buflen+1); // from 1024 to |value+\0| HashInsert(variable_dict_, variable, TemplateString(scratch, buflen)); } else { arena_->Shrink(scratch, 0); // reclaim arena space we didn't use HashInsert(variable_dict_, variable, Memdup(buffer, buflen)); delete[] buffer; } } void TemplateDictionary::SetEscapedValue(TemplateString variable, TemplateString value, const TemplateModifier& escfn) { SetValue(variable, string(escfn(value.data(), value.size()))); } void TemplateDictionary::SetEscapedFormattedValue(TemplateString variable, const TemplateModifier& escfn, const char* format, ...) { char* buffer; char* scratch = arena_->Alloc(1024); // StringAppendV requires >=1024 bytes va_list ap; va_start(ap, format); const int buflen = StringAppendV(scratch, &buffer, format, ap); va_end(ap); string escaped_string(escfn(buffer, buflen)); // Reclaim the arena space: the value we care about is now in escaped_string arena_->Shrink(scratch, 0); // reclaim arena space we didn't use if (buffer != scratch) delete[] buffer; SetValue(variable, escaped_string); } // ---------------------------------------------------------------------- // TemplateDictionary::SetTemplateGlobalValue() // Sets a value in the template-global dict. Unlike normal // variable lookups, these persist across sub-includes. // ---------------------------------------------------------------------- void TemplateDictionary::SetTemplateGlobalValue(const TemplateString variable, const TemplateString value) { assert(template_global_dict_owner_ != NULL); LazyCreateTemplateGlobalDict(); template_global_dict_owner_->template_global_dict_->SetValue(variable, value); } void TemplateDictionary::SetTemplateGlobalValueWithoutCopy( const TemplateString variable, const TemplateString value) { assert(template_global_dict_owner_ != NULL); LazyCreateTemplateGlobalDict(); // Don't memdup value - the caller will manage memory. template_global_dict_owner_->template_global_dict_-> SetValueWithoutCopy(variable, value); } // ---------------------------------------------------------------------- // TemplateDictionary::SetGlobalValue() // Sets a value in the global dict. Note this is a static method. // ---------------------------------------------------------------------- /*static*/ void TemplateDictionary::SetGlobalValue( const TemplateString variable, const TemplateString value) LOCKS_EXCLUDED(g_static_mutex) { // We can't use memdup here, since we're a static method. We do a strdup, // which is fine, since global_dict_ lives the entire program anyway. // It's unnecessary to copy the variable, since HashInsert takes care of // that for us. char* value_copy = new char[value.length_ + 1]; memcpy(value_copy, value.ptr_, value.length_); value_copy[value.length_] = '\0'; GoogleOnceInit(&g_once, &SetupGlobalDict); MutexLock ml(&g_static_mutex); HashInsert(global_dict_, variable, TemplateString(value_copy, value.length_)); } // ---------------------------------------------------------------------- // TemplateDictionary::AddSectionDictionary() // TemplateDictionary::ShowSection() // TemplateDictionary::ShowTemplateGlobalSection() // The new dictionary starts out empty, with us as the parent. // It shares our arena. The name is constructed out of our // name plus the section name. ShowSection() is the equivalent // to AddSectionDictionary("empty_dict"). // ---------------------------------------------------------------------- /*static*/ string TemplateDictionary::CreateSubdictName( const TemplateString& dict_name, const TemplateString& sub_name, size_t index, const char* suffix) { char index_str[64]; snprintf(index_str, sizeof(index_str), "%" PRIuS, index); return (PrintableTemplateString(dict_name) + "/" + PrintableTemplateString(sub_name) + "#" + index_str + suffix); } TemplateDictionary* TemplateDictionary::AddSectionDictionary( const TemplateString section_name) { LazilyCreateDict(§ion_dict_); DictVector* dicts = find_ptr2(*section_dict_, section_name.GetGlobalId()); if (!dicts) { dicts = CreateDictVector(); // Since most lists will remain under 8 or 16 entries but will frequently // be more than four, this prevents copying from 1->2->4->8. dicts->reserve(8); HashInsert(section_dict_, section_name, dicts); } assert(dicts != NULL); const string newname(CreateSubdictName(name_, section_name, dicts->size() + 1, "")); TemplateDictionary* retval = CreateTemplateSubdict( newname, arena_, this, template_global_dict_owner_); dicts->push_back(retval); return retval; } void TemplateDictionary::ShowSection(const TemplateString section_name) { LazilyCreateDict(§ion_dict_); if (!section_dict_->count(section_name.GetGlobalId())) { TemplateDictionary* empty_dict = CreateTemplateSubdict( "empty dictionary", arena_, this, template_global_dict_owner_); DictVector* sub_dict = CreateDictVector(); sub_dict->push_back(empty_dict); HashInsert(section_dict_, section_name, sub_dict); } } void TemplateDictionary::ShowTemplateGlobalSection( const TemplateString section_name) { assert(template_global_dict_owner_ != NULL); LazyCreateTemplateGlobalDict(); template_global_dict_owner_->template_global_dict_-> ShowSection(section_name); } // ---------------------------------------------------------------------- // TemplateDictionary::SetValueAndShowSection() // TemplateDictionary::SetEscapedValueAndShowSection() // If value is "", do nothing. Otherwise, call AddSectionDictionary() // on the section and add exactly one entry to the sub-dictionary: // the given variable/value pair. // ---------------------------------------------------------------------- void TemplateDictionary::SetValueAndShowSection(const TemplateString variable, const TemplateString value, const TemplateString section_name) { if (value.length_ == 0) // no value: the do-nothing case return; TemplateDictionary* sub_dict = AddSectionDictionary(section_name); sub_dict->SetValue(variable, value); } // ---------------------------------------------------------------------- // TemplateDictionary::AddIncludeDictionary() // This is much like AddSectionDictionary(). One major difference // is that the new dictionary does not have a parent dictionary: // there's no automatic variable inclusion across template-file // boundaries. Note there is no ShowTemplate() -- you must always // specify the dictionary to use explicitly. // ---------------------------------------------------------------------- TemplateDictionary* TemplateDictionary::AddIncludeDictionary( const TemplateString include_name) { LazilyCreateDict(&include_dict_); DictVector* dicts = find_ptr2(*include_dict_, include_name.GetGlobalId()); if (!dicts) { dicts = CreateDictVector(); HashInsert(include_dict_, include_name, dicts); } assert(dicts != NULL); const string newname(CreateSubdictName(name_, include_name, dicts->size() + 1, "")); TemplateDictionary* retval = CreateTemplateSubdict( newname, arena_, NULL, template_global_dict_owner_); dicts->push_back(retval); return retval; } // ---------------------------------------------------------------------- // TemplateDictionary::SetFilename() // Sets the filename this dictionary is meant to be associated with. // When set, it's possible to expand a template with just the // template-dict; the template is loaded via SetFilename() (though // we'd have to assume a value for strip). This is required for // dictionaries that are meant to be used with an include-template. // ---------------------------------------------------------------------- void TemplateDictionary::SetFilename(const TemplateString filename) { filename_ = Memdup(filename).ptr_; } // ---------------------------------------------------------------------- // TemplateDictionary::AddToIdToNameMap() // We have a problem when we try to dump the contents of the // dictionary, because instead of storing the keys to global_dict_ // etc as strings, we store them as integer id's. We need this // map, from id->string, to be able to dump. This should be called // every time we add a string to a TemplateDictionary hashtable. // ---------------------------------------------------------------------- /*static*/ void TemplateDictionary::AddToIdToNameMap(TemplateId id, const TemplateString& str) { // If str.id_ is set, that means we were added to the id-to-name map // at TemplateString constructor time, when the id_ was set. So we // don't need to bother again here. if (str.id_ != 0) { return; } // Verify that if this id is already in the map, it's there with our // contents. If not, that would mean a hash collision (since our // id's are hash values). DCHECK(TemplateString::IdToString(id) == kStsEmpty || memcmp(str.ptr_, TemplateString::IdToString(id).ptr_, str.length_) == 0) << string(str.ptr_, str.length_) << " vs " << string(TemplateString::IdToString(id).ptr_, TemplateString::IdToString(id).length_); TemplateString str_with_id(str.ptr_, str.length_, str.is_immutable(), id); str_with_id.AddToGlobalIdToNameMap(); } // ---------------------------------------------------------------------- // TemplateDictionary::DumpToString() // TemplateDictionary::Dump() // The values are shown in the following order: // - Scalar values // - Sub-dictionaries and their associated section names. // - Sub-dictionaries and their associated template names, with filename. // ---------------------------------------------------------------------- // DictionaryPrinter knows how to dump a whole dictionary tree. class TemplateDictionary::DictionaryPrinter { public: DictionaryPrinter(string* out, int initial_indent) : writer_(out, initial_indent) { } void DumpToString(const TemplateDictionary& dict) { // Show globals if we're a top-level dictionary if (dict.parent_dict_ == NULL) { DumpGlobals(); } // Show template-globals if (dict.template_global_dict_ && !dict.template_global_dict_->Empty()) { DumpTemplateGlobals(*dict.template_global_dict_); } DumpDictionary(dict); } private: void FillSortedGlobalDictMap(map* sorted_global_dict) LOCKS_EXCLUDED(g_static_mutex) { ReaderMutexLock ml(&g_static_mutex); for (GlobalDict::const_iterator it = global_dict_->begin(); it != global_dict_->end(); ++it) { const TemplateString key = TemplateDictionary::IdToString(it->first); assert(!InvalidTemplateString(key)); // checks key.ptr_ != NULL (*sorted_global_dict)[PrintableTemplateString(key)] = PrintableTemplateString(it->second); } } void DumpGlobals() { writer_.Write("global dictionary {\n"); writer_.Indent(); // We could be faster than converting every TemplateString into a // string and inserted into an ordered data structure, but why bother? map sorted_global_dict; FillSortedGlobalDictMap(&sorted_global_dict); for (map::const_iterator it = sorted_global_dict.begin(); it != sorted_global_dict.end(); ++it) { writer_.Write(it->first + ": >" + it->second + "<\n"); } writer_.Dedent(); writer_.Write("};\n"); } void DumpTemplateGlobals(const TemplateDictionary& template_global_dict) { writer_.Write("template dictionary {\n"); writer_.Indent(); DumpDictionaryContent(template_global_dict); writer_.Dedent(); writer_.Write("};\n"); } void DumpDictionary(const TemplateDictionary& dict) { string intended_for = dict.filename_ && dict.filename_[0] ? string(" (intended for ") + dict.filename_ + ")" : ""; writer_.Write("dictionary '", PrintableTemplateString(dict.name_), intended_for, "' {\n"); writer_.Indent(); DumpDictionaryContent(dict); writer_.Dedent(); writer_.Write("}\n"); } void DumpDictionaryContent(const TemplateDictionary& dict) { if (dict.variable_dict_) { // Show variables DumpVariables(*dict.variable_dict_); } if (dict.section_dict_) { // Show section sub-dictionaries DumpSectionDict(*dict.section_dict_); } if (dict.include_dict_) { // Show template-include sub-dictionaries DumpIncludeDict(*dict.include_dict_); } } void DumpVariables(const VariableDict& dict) { map sorted_variable_dict; for (VariableDict::const_iterator it = dict.begin(); it != dict.end(); ++it) { const TemplateString key = TemplateDictionary::IdToString(it->first); assert(!InvalidTemplateString(key)); // checks key.ptr_ != NULL sorted_variable_dict[PrintableTemplateString(key)] = PrintableTemplateString(it->second); } for (map::const_iterator it = sorted_variable_dict.begin(); it != sorted_variable_dict.end(); ++it) { writer_.Write(it->first + ": >" + it->second + "<\n"); } } template void SortSections(MyMap* sorted_section_dict, const MySectionDict& section_dict) { typename MySectionDict::const_iterator it = section_dict.begin(); for (; it != section_dict.end(); ++it) { const TemplateString key = TemplateDictionary::IdToString(it->first); assert(!InvalidTemplateString(key)); // checks key.ptr_ != NULL (*sorted_section_dict)[PrintableTemplateString(key)] = it->second; } } void DumpSectionDict(const SectionDict& section_dict) { map sorted_section_dict; SortSections(&sorted_section_dict, section_dict); for (map::const_iterator it = sorted_section_dict.begin(); it != sorted_section_dict.end(); ++it) { for (DictVector::const_iterator it2 = it->second->begin(); it2 != it->second->end(); ++it2) { TemplateDictionary* dict = *it2; writer_.Write("section ", it->first, " (dict ", GetDictNum(it2 - it->second->begin() + 1, it->second->size()), ") -->\n"); writer_.Indent(); DumpToString(*dict); writer_.Dedent(); } } } void DumpIncludeDict(const IncludeDict& include_dict) { map sorted_include_dict; SortSections(&sorted_include_dict, include_dict); for (map::const_iterator it = sorted_include_dict.begin(); it != sorted_include_dict.end(); ++it) { for (vector::size_type i = 0; i < it->second->size(); ++i) { TemplateDictionary* dict = (*it->second)[i]; string from_name = (dict->filename_ && *dict->filename_) ? string(", from ") + dict->filename_ : string(", **NO FILENAME SET; THIS DICT WILL BE IGNORED**"); writer_.Write("include-template ", it->first, " (dict ", GetDictNum(static_cast(i + 1), it->second->size()), from_name, ") -->\n"); writer_.Indent(); DumpToString(*dict); writer_.Dedent(); } } } string GetDictNum(size_t index, size_t size) const { char buf[64]; // big enough for two ints snprintf(buf, sizeof(buf), "%" PRIuS " of %" PRIuS, index, size); return buf; } IndentedWriter writer_; }; void TemplateDictionary::DumpToString(string* out, int indent) const { DictionaryPrinter printer(out, indent); printer.DumpToString(*this); } void TemplateDictionary::Dump(int indent) const { string out; DumpToString(&out, indent); fwrite(out.data(), 1, out.length(), stdout); fflush(stdout); } // ---------------------------------------------------------------------- // TemplateDictionary::Memdup() // Copy the input into the arena, so we have a permanent copy of // it. Returns a pointer to the arena-copy, as a TemplateString // (in case the input has internal NULs). // ---------------------------------------------------------------------- TemplateString TemplateDictionary::Memdup(const char* s, size_t slen) { return TemplateString(arena_->MemdupPlusNUL(s, slen), slen); // add a \0 too } // ---------------------------------------------------------------------- // TemplateDictionary::GetSectionValue() // TemplateDictionary::IsHiddenSection() // TemplateDictionary::IsHiddenTemplate() // TemplateDictionary::GetIncludeTemplateName() // The 'introspection' routines that tell Expand() what's in the // template dictionary. GetSectionValue() does variable lookup: // first look in this dict, then in parent dicts, etc. IsHidden*() // returns true iff the name is not present in the appropriate // dictionary. None of these functions ever returns NULL. // ---------------------------------------------------------------------- TemplateString TemplateDictionary::GetValue( const TemplateString& variable) const LOCKS_EXCLUDED(g_static_mutex) { for (const TemplateDictionary* d = this; d; d = d->parent_dict_) { if (d->variable_dict_) { if (const TemplateString* it = find_ptr(*d->variable_dict_, variable.GetGlobalId())) return *it; } } // No match in the dict tree. Check the template-global dict. assert(template_global_dict_owner_ != NULL); if (template_global_dict_owner_->template_global_dict_ && template_global_dict_owner_->template_global_dict_->variable_dict_) { const VariableDict* template_global_vars = template_global_dict_owner_->template_global_dict_->variable_dict_; if (const TemplateString* it = find_ptr(*template_global_vars, variable.GetGlobalId())) return *it; } // No match in dict tree or template-global dict. Last chance: global dict. { ReaderMutexLock ml(&g_static_mutex); if (const TemplateString* it = find_ptr(*global_dict_, variable.GetGlobalId())) return *it; return *empty_string_; } } bool TemplateDictionary::IsHiddenSection(const TemplateString& name) const { for (const TemplateDictionary* d = this; d; d = d->parent_dict_) { if (d->section_dict_ && d->section_dict_->count(name.GetGlobalId())) return false; } assert(template_global_dict_owner_ != NULL); if (template_global_dict_owner_->template_global_dict_ && template_global_dict_owner_->template_global_dict_->section_dict_) { SectionDict* sections = template_global_dict_owner_->template_global_dict_->section_dict_; if (sections->count(name.GetGlobalId())) { return false; } } return true; } bool TemplateDictionary::IsHiddenTemplate(const TemplateString& name) const { for (const TemplateDictionary* d = this; d; d = d->parent_dict_) { if (d->include_dict_ && d->include_dict_->count(name.GetGlobalId())) return false; } return true; } const char *TemplateDictionary::GetIncludeTemplateName( const TemplateString& variable, int dictnum) const { for (const TemplateDictionary* d = this; d; d = d->parent_dict_) { if (d->include_dict_) { if (DictVector* it = find_ptr2(*d->include_dict_, variable.GetGlobalId())) { TemplateDictionary* dict = (*it)[dictnum]; return dict->filename_ ? dict->filename_ : ""; // map NULL to "" } } } assert("Call IsHiddenTemplate before GetIncludeTemplateName" && 0); abort(); } bool TemplateDictionary::Empty() const { if ((variable_dict_ && !variable_dict_->empty()) || (section_dict_ && section_dict_->empty()) || (include_dict_ && include_dict_->empty())) { return false; } return true; } // ---------------------------------------------------------------------- // TemplateDictionary::CreateSectionIterator() // TemplateDictionary::CreateTemplateIterator() // TemplateDictionary::Iterator::HasNext() // TemplateDictionary::Iterator::Next() // Iterator framework. // ---------------------------------------------------------------------- template bool TemplateDictionary::Iterator::HasNext() const { return begin_ != end_; } template const TemplateDictionaryInterface& TemplateDictionary::Iterator::Next() { return **(begin_++); } TemplateDictionaryInterface::Iterator* TemplateDictionary::CreateTemplateIterator( const TemplateString& section_name) const { for (const TemplateDictionary* d = this; d; d = d->parent_dict_) { if (d->include_dict_) { if (DictVector* it = find_ptr2(*d->include_dict_, section_name.GetGlobalId())) { // Found it! Return it as an Iterator return MakeIterator(*it); } } } assert("Call IsHiddenTemplate before CreateTemplateIterator" && 0); abort(); } TemplateDictionaryInterface::Iterator* TemplateDictionary::CreateSectionIterator( const TemplateString& section_name) const { for (const TemplateDictionary* d = this; d; d = d->parent_dict_) { if (d->section_dict_) { if (const DictVector* it = find_ptr2(*d->section_dict_, section_name.GetGlobalId())) { // Found it! Return it as an Iterator return MakeIterator(*it); } } } // Check the template global dictionary. assert(template_global_dict_owner_); const TemplateDictionary* template_global_dict = template_global_dict_owner_->template_global_dict_; if (template_global_dict && template_global_dict->section_dict_) { if (const DictVector* it = find_ptr2(*template_global_dict->section_dict_, section_name.GetGlobalId())) { return MakeIterator(*it); } } assert("Call IsHiddenSection before GetDictionaries" && 0); abort(); } } ctemplate-ctemplate-2.4/src/template_modifiers.cc000066400000000000000000001555651363122316600223070ustar00rootroot00000000000000// Copyright (c) 2007, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // template_modifiers.h has a description of what each escape-routine does. // // When creating a new modifier, you must subclass TemplateModifier // and define your own Modify() method. This method takes the string // to be modified as a char*/int pair. It then emits the modified // version of the string to outbuf. Outbuf is an ExpandEmitter, as // defined in template_modifiers.h. It's a very simple type that // supports appending to a data stream. // // Be very careful editing an existing modifier. Subtle changes can // introduce the possibility for cross-site scripting attacks. If you // do change a modifier, be careful that it does not affect // the list of Safe XSS Alternatives. // #include #include #include #include #include #include #include "htmlparser/htmlparser_cpp.h" #include #include "template_modifiers_internal.h" #include using std::string; using std::vector; #define strliterallen(s) (sizeof("" s "") - 1) // Really we should be using uint_16_t or something, but this is good // enough, and more portable... typedef unsigned int uint16; namespace URL { bool HasInsecureProtocol(const char* in, int inlen) { if (inlen > strliterallen("http://") && strncasecmp(in, "http://", strliterallen("http://")) == 0) { return false; // We're ok, it's an http protocol } if (inlen > strliterallen("https://") && strncasecmp(in, "https://", strliterallen("https://")) == 0) { return false; // https is ok as well } if (inlen > strliterallen("ftp://") && strncasecmp(in, "ftp://", strliterallen("ftp://")) == 0) { return false; // and ftp } return true; } } // namespace URL namespace ctemplate { using ctemplate_htmlparser::HtmlParser; // A most-efficient way to append a string literal to the var named 'out'. // The ""s ensure literal is actually a string literal #define APPEND(literal) out->Emit("" literal "", sizeof(literal)-1) // Check whether the string of length len is identical to the literal. // The ""s ensure literal is actually a string literal #define STR_IS(str, len, literal) \ ((len) == sizeof("" literal "") - 1 && \ memcmp(str, literal, sizeof("" literal "") - 1) == 0) TemplateModifier::~TemplateModifier() {} void NullModifier::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { out->Emit(in, inlen); } NullModifier null_modifier; static inline void EmitRun(const char* start, const char* limit, ExpandEmitter* out) { if (start < limit) { out->Emit(start, (limit - start)); } } void HtmlEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { const char* pos = in; const char* start = pos; const char* const limit = in + inlen; while (pos < limit) { switch (*pos) { default: // Increment our counter and look at the next character. ++pos; continue; case '&': EmitRun(start, pos, out); APPEND("&"); break; case '"': EmitRun(start, pos, out); APPEND("""); break; case '\'': EmitRun(start, pos, out); APPEND("'"); break; case '<': EmitRun(start, pos, out); APPEND("<"); break; case '>': EmitRun(start, pos, out); APPEND(">"); break; case '\r': case '\n': case '\v': case '\f': case '\t': EmitRun(start, pos, out); APPEND(" "); break; } start = ++pos; } EmitRun(start, pos, out); } HtmlEscape html_escape; void PreEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { const char* pos = in; const char* start = pos; const char* const limit = in + inlen; while (pos < limit) { switch (*pos) { default: // Increment our counter and look at the next character. ++pos; continue; // Unlike HtmlEscape, we leave whitespace as is. case '&': EmitRun(start, pos, out); APPEND("&"); break; case '"': EmitRun(start, pos, out); APPEND("""); break; case '\'': EmitRun(start, pos, out); APPEND("'"); break; case '<': EmitRun(start, pos, out); APPEND("<"); break; case '>': EmitRun(start, pos, out); APPEND(">"); break; } start = ++pos; } EmitRun(start, pos, out); } PreEscape pre_escape; // We encode the presence and ordering of unclosed tags in a string, using the // letters b, i, s, and e to stand for , , , and respectively. // The most recently opened tag is appended onto the end of the string, so in // the common case of properly nested tags, we need only look at the last // character. If we don't find it there, we need to continue looking at // everything until we find it, because tags may not necessarily be in order. // Similarly, when we add a tag, we need to check each existing tag for a match // so that we don't nest. class UnclosedSnippetTags { public: // We could use ordinary ints for the enum values, but using mnemonic // characters potentially makes debugging easier. typedef enum { TAG_B = 'b', TAG_I = 'i', TAG_EM = 'e', TAG_SPAN = 's', } Tag; UnclosedSnippetTags() : tag_length(0) { memset(tags, 0, 5); } // Adds a tag to the set of open tags if it's not already open, or otherwise // return false. inline bool MaybeAdd(Tag tag) { if (strchr(tags, tag)) { return false; } else { tags[tag_length++] = tag; return true; } } // Removes a tag from the set of open tags if it's open, or otherwise return // false. inline bool MaybeRemove(Tag tag) { char* tag_location = strchr(tags, tag); if (tag_location) { for (char* c = tag_location; *c; ++c) { // Have to copy all later tags down by one so we don't leave a gap in the // array. *c = *(c + 1); } --tag_length; return true; } else { return false; } } inline void PrintClosingTags(ExpandEmitter* out) { for (int i = tag_length; i >= 0; --i) { switch (tags[i]) { case TAG_B: out->Emit(""); break; case TAG_I: out->Emit(""); break; case TAG_EM: out->Emit(""); break; case TAG_SPAN: out->Emit(""); break; } } } private: char tags[5]; int tag_length; }; void SnippetEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { UnclosedSnippetTags unclosed; const char* pos = in; const char* start = pos; const char* const limit = in + inlen; while (pos < limit) { switch (*pos) { default: // Increment our counter and look at the next character. ++pos; continue; case '<': { // If there is a permissible tag, just advance pos past it to // make it part of the current run. Notice the use of // "continue" below. const char* const next_pos = pos + 1; const int chars_left = limit - next_pos; if ((chars_left >= 2) && !memcmp(next_pos, "b>", 2) && unclosed.MaybeAdd(UnclosedSnippetTags::TAG_B)) { pos += strliterallen(""); continue; } else if ((chars_left >= 2) && !memcmp(next_pos, "i>", 2) && unclosed.MaybeAdd(UnclosedSnippetTags::TAG_I)) { pos += strliterallen(""); continue; } else if ((chars_left >= 3) && !memcmp(next_pos, "em>", 3) && unclosed.MaybeAdd(UnclosedSnippetTags::TAG_EM)) { pos += strliterallen(""); continue; } else if ((chars_left >= 13) && !memcmp(next_pos, "span dir=", 9) && (!memcmp(next_pos + 9, "ltr>", 4) || !memcmp(next_pos + 9, "rtl>", 4)) && unclosed.MaybeAdd(UnclosedSnippetTags::TAG_SPAN)) { pos += strliterallen(""); continue; } else if ((chars_left >= 3) && !memcmp(next_pos, "/b>", 3) && unclosed.MaybeRemove(UnclosedSnippetTags::TAG_B)) { pos += strliterallen(""); continue; } else if ((chars_left >= 3) && !memcmp(next_pos, "/i>", 3) && unclosed.MaybeRemove(UnclosedSnippetTags::TAG_I)) { pos += strliterallen(""); continue; } else if ((chars_left >= 4) && !memcmp(next_pos, "/em>", 4) && unclosed.MaybeRemove(UnclosedSnippetTags::TAG_EM)) { pos += strliterallen(""); continue; } else if ((chars_left >= 6) && !memcmp(next_pos, "/span>", 6) && unclosed.MaybeRemove(UnclosedSnippetTags::TAG_SPAN)) { pos += strliterallen(""); continue; } else if ((chars_left >= 3) && !memcmp(next_pos, "br>", 3)) { pos += strliterallen("
"); continue; } else if ((chars_left >= 4) && !memcmp(next_pos, "wbr>", 4)) { pos += strliterallen(""); continue; } // Emit the entity and break out of the switch. EmitRun(start, pos, out); APPEND("<"); break; } case '&': EmitRun(start, pos, out); if (pos + 1 < limit && pos[1] == '{') { // Could be a javascript entity, so we need to escape. // (Javascript entities are an xss risk in Netscape 4.) APPEND("&"); } else { APPEND("&"); } break; case '"': EmitRun(start, pos, out); APPEND("""); break; case '\'': EmitRun(start, pos, out); APPEND("'"); break; case '>': EmitRun(start, pos, out); APPEND(">"); break; case '\r': case '\n': case '\v': case '\f': case '\t': // non-space whitespace EmitRun(start, pos, out); APPEND(" "); break; } start = ++pos; } EmitRun(start, pos, out); unclosed.PrintClosingTags(out); } SnippetEscape snippet_escape; void CleanseAttribute::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { for (size_t i = 0; i < inlen; ++i) { char c = in[i]; switch (c) { case '=': { if (i == 0 || i == (inlen - 1)) out->Emit('_'); else out->Emit(c); break; } case '-': case '.': case '_': case ':': { out->Emit(c); break; } default: { if ((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) { out->Emit(c); } else { APPEND("_"); } break; } } } } CleanseAttribute cleanse_attribute; void CleanseCss::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { for (size_t i = 0; i < inlen; ++i) { char c = in[i]; switch (c) { case ' ': case '_': case '.': case ',': case '!': case '#': case '%': case '-': { out->Emit(c); break; } default: { if ((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) { out->Emit(c); } break; } } } } CleanseCss cleanse_css; // CssUrlEscape is used as a chained modifier by ValidateUrl // (validate_url_and_css_escape) and is not directly exposed. class CssUrlEscape : public TemplateModifier { public: virtual void Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* outbuf, const string& arg) const; }; // URL-encodes the characters [\n\r\\'"()<>*] to ensure the URL can be safely // inserted in a CSS context, e.g: // . In an '@import url("URL");' statement // . In a CSS property such as 'background: url("URL");' // In both locations above, enclosing quotes are optional but parens are not. // We want to make sure the URL cannot exit the parens enclosure, close a // STYLE tag or reset the browser's CSS parser (via comments or newlines). // // References: // . CSS 2.1 URLs: http://www.w3.org/TR/CSS21/syndata.html#url // . CSS 1 URLs: http://www.w3.org/TR/REC-CSS1/#url void CssUrlEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { for (size_t i = 0; i < inlen; ++i) { char c = in[i]; switch (c) { case '\n': APPEND("%0A"); break; case '\r': APPEND("%0D"); break; case '"': APPEND("%22"); break; case '\'': APPEND("%27"); break; case '(': APPEND("%28"); break; case ')': APPEND("%29"); break; case '*': APPEND("%2A"); break; case '<': APPEND("%3C"); break; case '>': APPEND("%3E"); break; case '\\': APPEND("%5C"); break; default: out->Emit(c); break; } } } CssUrlEscape css_url_escape; // These URLs replace unsafe URLs for :U and :I url-escaping modes. const char* const ValidateUrl::kUnsafeUrlReplacement = "#"; const char* const ValidateUrl::kUnsafeImgSrcUrlReplacement = "/images/cleardot.gif"; void ValidateUrl::Modify(const char* in, size_t inlen, const PerExpandData* per_expand_data, ExpandEmitter* out, const string& arg) const { const char* slashpos = (char*)memchr(in, '/', inlen); if (slashpos == NULL) { slashpos = in + inlen; } const void* colonpos = memchr(in, ':', slashpos - in); // colon before first slash, could be a protocol if (colonpos != NULL && URL::HasInsecureProtocol(in, inlen)) { // It's a bad protocol, so return something safe chained_modifier_.Modify(unsafe_url_replacement_, unsafe_url_replacement_length_, per_expand_data, out, ""); return; } // If we get here, it's a valid url, so just escape it chained_modifier_.Modify(in, inlen, per_expand_data, out, ""); } ValidateUrl validate_url_and_html_escape( html_escape, ValidateUrl::kUnsafeUrlReplacement); ValidateUrl validate_url_and_javascript_escape( javascript_escape, ValidateUrl::kUnsafeUrlReplacement); ValidateUrl validate_url_and_css_escape( css_url_escape, ValidateUrl::kUnsafeUrlReplacement); ValidateUrl validate_img_src_url_and_html_escape( html_escape, ValidateUrl::kUnsafeImgSrcUrlReplacement); ValidateUrl validate_img_src_url_and_javascript_escape( javascript_escape, ValidateUrl::kUnsafeImgSrcUrlReplacement); ValidateUrl validate_img_src_url_and_css_escape( css_url_escape, ValidateUrl::kUnsafeImgSrcUrlReplacement); void XmlEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { const char* pos = in; const char* start = pos; const char* const limit = in + inlen; while (pos < limit) { char ch = *pos; // According to section 2.2 of the spec // http://www.w3.org/TR/REC-xml/#charsets control characters in range // 0x00-0x1F (except \t, \r and \n) are not valid XML characters. In // particular, conformant parsers are allowed to die when encountering a FF // char in PCDATA sections. These chars are replaced by a space. if (ch >= 0x00 && ch < 0x20 && ch != '\t' && ch != '\r' && ch != '\n') { EmitRun(start, pos, out); out->Emit(' '); start = ++pos; continue; } switch (ch) { default: // Increment our counter and look at the next character. ++pos; continue; case '&': EmitRun(start, pos, out); APPEND("&"); break; case '"': EmitRun(start, pos, out); APPEND("""); break; case '\'': EmitRun(start, pos, out); APPEND("'"); break; case '<': EmitRun(start, pos, out); APPEND("<"); break; case '>': EmitRun(start, pos, out); APPEND(">"); break; } start = ++pos; } EmitRun(start, pos, out); } XmlEscape xml_escape; // This table maps initial characters to code lengths. This could be // done with a 16-byte table and a shift, but there's a substantial // performance increase by eliminating the shift. static const char kCodeLengths[256] = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, }; // Returns the UTF-8 code-unit starting at start, or the special codepoint // 0xFFFD if the input ends abruptly or is not well-formed UTF-8. // start -- address of the start of the code unit which also receives the // address past the end of the code unit returned. // end -- exclusive end of the string static inline uint16 UTF8CodeUnit(const char** start, const char *end) { // Use kCodeLengths table to calculate the length of the code unit // from the first character. unsigned char first_char = static_cast(**start); size_t code_unit_len = kCodeLengths[first_char]; if (code_unit_len == 1) { // Return the current byte as a codepoint. // Either it is a valid single byte codepoint, or it's not part of a valid // UTF-8 sequence, and so has to be handled individually. ++*start; return first_char; } const char *code_unit_end = *start + code_unit_len; if (code_unit_end < *start || code_unit_end > end) { // Truncated code unit. ++*start; return 0xFFFDU; } const char* pos = *start; uint16 code_unit = *pos & (0xFFU >> code_unit_len); while (--code_unit_len) { uint16 tail_byte = *(++pos); if ((tail_byte & 0xC0U) != 0x80U) { // Malformed code unit. ++*start; return 0xFFFDU; } code_unit = (code_unit << 6) | (tail_byte & 0x3FU); } *start = code_unit_end; return code_unit; } // A good reference is the ECMA standard (3rd ed), section 7.8.4: // http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf void JavascriptEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { const char* pos = in; const char* start = pos; const char* const limit = in + inlen; if (limit < in) { return; } while (pos < limit) { const char* next_pos = pos; uint16 code_unit = UTF8CodeUnit(&next_pos, limit); // Test for 16-bit values outside the switch below, because gcc // will emit chained branches rather than a jump table for such a // wide range of values. if (code_unit & 0xFF00) { // Linebreaks according to EcmaScript 262 which cannot appear in strings. if (code_unit == 0x2028) { // Line separator EmitRun(start, pos, out); APPEND("\\u2028"); } else if (code_unit == 0x2029) { // Paragraph separator EmitRun(start, pos, out); APPEND("\\u2029"); } else { pos = next_pos; continue; } } else { switch (code_unit) { default: // Increment our counter and look at the next character. pos = next_pos; continue; case '\0': EmitRun(start, pos, out); APPEND("\\x00"); break; case '"': EmitRun(start, pos, out); APPEND("\\x22"); break; case '\'': EmitRun(start, pos, out); APPEND("\\x27"); break; case '\\': EmitRun(start, pos, out); APPEND("\\\\"); break; case '\t': EmitRun(start, pos, out); APPEND("\\t"); break; case '\r': EmitRun(start, pos, out); APPEND("\\r"); break; case '\n': EmitRun(start, pos, out); APPEND("\\n"); break; case '\b': EmitRun(start, pos, out); APPEND("\\b"); break; case '\f': EmitRun(start, pos, out); APPEND("\\f"); break; case '&': EmitRun(start, pos, out); APPEND("\\x26"); break; case '<': EmitRun(start, pos, out); APPEND("\\x3c"); break; case '>': EmitRun(start, pos, out); APPEND("\\x3e"); break; case '=': EmitRun(start, pos, out); APPEND("\\x3d"); break; case '\v': // Do not escape vertical tabs to "\\v" since it is interpreted as 'v' // by JScript according to section 2.1 of // http://wiki.ecmascript.org/lib/exe/fetch.php? // id=resources%3Aresources&cache=cache& // media=resources:jscriptdeviationsfromes3.pdf EmitRun(start, pos, out); APPEND("\\x0b"); break; } } start = pos = next_pos; } EmitRun(start, pos, out); } JavascriptEscape javascript_escape; void JavascriptNumber::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { if (inlen == 0) return; if (STR_IS(in, inlen, "true") || STR_IS(in, inlen, "false")) { out->Emit(in, inlen); return; } bool valid = true; if (in[0] == '0' && inlen > 2 && (in[1] == 'x' || in[1] == 'X')) { // There must be at least one hex digit after the 0x for it to be valid. // Hex number. Check that it is of the form 0(x|X)[0-9A-Fa-f]+ for (size_t i = 2; i < inlen; i++) { char c = in[i]; if (!((c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F') || (c >= '0' && c <= '9'))) { valid = false; break; } } } else { // Must be a base-10 (or octal) number. // Check that it has the form [0-9+-.eE]+ for (size_t i = 0; i < inlen; i++) { char c = in[i]; if (!((c >= '0' && c <= '9') || c == '+' || c == '-' || c == '.' || c == 'e' || c == 'E')) { valid = false; break; } } } if (valid) { out->Emit(in, inlen); // Number was valid, output it. } else { APPEND("null"); // Number was not valid, output null instead. } } JavascriptNumber javascript_number; static inline bool IsUrlQueryEscapeSafeChar(unsigned char c) { // Everything not matching [0-9a-zA-Z.,_*/~!()-] is escaped. static unsigned long _safe_characters[8] = { 0x00000000L, 0x03fff702L, 0x87fffffeL, 0x47fffffeL, 0x00000000L, 0x00000000L, 0x00000000L, 0x00000000L }; return (_safe_characters[(c)>>5] & (1 << ((c) & 31))); } void UrlQueryEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { const char* pos = in; const char* const limit = in + inlen; while (true) { // Peel off any initial runs of safe characters and emit them all // at once. const char* start = pos; while (pos < limit && IsUrlQueryEscapeSafeChar(*pos)) { pos++; } EmitRun(start, pos, out); // Now deal with a single unsafe character. if (pos < limit) { unsigned char c = *pos; if (c == ' ') { out->Emit('+'); } else { out->Emit('%'); out->Emit(((c>>4) < 10 ? ((c>>4) + '0') : (((c>>4) - 10) + 'A'))); out->Emit(((c&0xf) < 10 ? ((c&0xf) + '0') : (((c&0xf) - 10) + 'A'))); } pos++; } else { // We're done! break; } } } UrlQueryEscape url_query_escape; // For more information on escaping JSON, see section 2.5 in // http://www.ietf.org/rfc/rfc4627.txt. // Escaping '&', '<', '>' is optional in the JSON proposed RFC // but alleviates concerns with content sniffing if JSON is used // in a context where the browser may attempt to interpret HTML. void JsonEscape::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { const char* pos = in; const char* start = pos; const char* const limit = in + inlen; while (pos < limit) { switch (*pos) { default: // Increment our counter and look at the next character. ++pos; continue; case '"': EmitRun(start, pos, out); APPEND("\\\""); break; case '\\': EmitRun(start, pos, out); APPEND("\\\\"); break; case '/': EmitRun(start, pos, out); APPEND("\\/"); break; case '\b': EmitRun(start, pos, out); APPEND("\\b"); break; case '\f': EmitRun(start, pos, out); APPEND("\\f"); break; case '\n': EmitRun(start, pos, out); APPEND("\\n"); break; case '\r': EmitRun(start, pos, out); APPEND("\\r"); break; case '\t': EmitRun(start, pos, out); APPEND("\\t"); break; case '&': EmitRun(start, pos, out); APPEND("\\u0026"); break; case '<': EmitRun(start, pos, out); APPEND("\\u003C"); break; case '>': EmitRun(start, pos, out); APPEND("\\u003E"); break; } start = ++pos; } EmitRun(start, pos, out); } JsonEscape json_escape; void PrefixLine::Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* out, const string& arg) const { while (inlen > 0) { const char* nl = (const char*)memchr(in, '\n', inlen); const char* cr = (const char*)memchr(in, '\r', nl ? nl - in : inlen); size_t linelen; if (nl == NULL && cr == NULL) { // We're at the last line out->Emit(in, inlen); break; } else { // One or both of \r and \n is set; point to the first char past // the newline. Note for \r\n, that's the char after the \n, // otherwise, it's the char past the \r or the \n we see. if ((nl == NULL) != (cr == NULL)) // one is set, the other is NULL linelen = (nl ? nl : cr) + 1 - in; else if (nl == cr + 1 || nl < cr) // \r\n, or \n comes first linelen = nl + 1 - in; else linelen = cr + 1 - in; } out->Emit(in, linelen); out->Emit(arg); // a new line, so emit the prefix in += linelen; inlen -= linelen; assert(inlen >= 0); } } PrefixLine prefix_line; // Must be at least one more than the maximum number of alternative modifiers // specified in any given element of g_modifiers. # define MAX_SAFE_ALTERNATIVES 10 // If the compiler complains, increase it. // Use the empty string if you want a modifier not to have a long-name. // Use '\0' if you want a modifier not to have a short-name. // Note: not all modifiers are in this array: // 1) SnippetEscape: use html_escape_with_arg=snippet to get this // 2) CleanseAttribute: use html_escape_with_arg=attribute to get this // 3) ValidateUrl: use html_escape_with_arg=url to get this // // Some modifiers define other modifiers that are safe replacements // from an XSS perspective. Replacements are not commutative so for // example H=pre considers H=attribute a safe replacement to it // but H=attribute has no safe replacements. // This struct is not pretty but allows the definitions to be // done without the need for a global initialization method. // Be very careful making a change to g_modifiers as modifiers // point to other ones within that same array so elements // may not be re-ordered easily. Also you need to change // the global g_am_dirs correspondingly. // static struct ModifierWithAlternatives { ModifierInfo modifier_info; ModifierInfo* safe_alt_mods[MAX_SAFE_ALTERNATIVES]; } g_modifiers[] = { /* 0 */ { ModifierInfo("cleanse_css", 'c', XSS_WEB_STANDARD, &cleanse_css), {&g_modifiers[16].modifier_info, // url_escape_with_arg=css // img_src_url_escape_with_arg=css &g_modifiers[19].modifier_info} }, /* 1 */ { ModifierInfo("html_escape", 'h', XSS_WEB_STANDARD, &html_escape), {&g_modifiers[2].modifier_info, // html_escape_with_arg=snippet &g_modifiers[3].modifier_info, // html_escape_with_arg=pre &g_modifiers[4].modifier_info, // html_escape_with_arg=attribute &g_modifiers[5].modifier_info, // html_escape_with_arg=url &g_modifiers[8].modifier_info, // pre_escape &g_modifiers[9].modifier_info, // url_query_escape &g_modifiers[11].modifier_info, // url_escape_with_arg=html &g_modifiers[12].modifier_info, // url_escape_with_arg=query // img_src_url_escape_with_arg=html &g_modifiers[18].modifier_info} }, /* 2 */ { ModifierInfo("html_escape_with_arg=snippet", 'H', XSS_WEB_STANDARD, &snippet_escape), {&g_modifiers[1].modifier_info, // html_escape &g_modifiers[3].modifier_info, // html_escape_with_arg=pre &g_modifiers[4].modifier_info, // html_escape_with_arg=attribute &g_modifiers[8].modifier_info, // pre_escape &g_modifiers[9].modifier_info, // url_query_escape &g_modifiers[12].modifier_info} }, // url_escape_with_arg=query /* 3 */ { ModifierInfo("html_escape_with_arg=pre", 'H', XSS_WEB_STANDARD, &pre_escape), {&g_modifiers[1].modifier_info, // html_escape &g_modifiers[2].modifier_info, // html_escape_with_arg=snippet &g_modifiers[4].modifier_info, // html_escape_with_arg=attribute &g_modifiers[8].modifier_info, // pre_escape &g_modifiers[9].modifier_info, // url_query_escape &g_modifiers[12].modifier_info} }, // url_escape_with_arg=query /* 4 */ { ModifierInfo("html_escape_with_arg=attribute", 'H', XSS_WEB_STANDARD, &cleanse_attribute), {} }, /* 5 */ { ModifierInfo("html_escape_with_arg=url", 'H', XSS_WEB_STANDARD, &validate_url_and_html_escape), // img_src_url_escape_with_arg=html {&g_modifiers[18].modifier_info} }, /* 6 */ { ModifierInfo("javascript_escape", 'j', XSS_WEB_STANDARD, &javascript_escape), {&g_modifiers[7].modifier_info, // json_escape &g_modifiers[10].modifier_info, // url_escape_with_arg=javascript // img_src_url_escape_with_arg=javascript &g_modifiers[17].modifier_info} }, /* 7 */ { ModifierInfo("json_escape", 'o', XSS_WEB_STANDARD, &json_escape), {&g_modifiers[6].modifier_info} }, // javascript_escape /* 8 */ { ModifierInfo("pre_escape", 'p', XSS_WEB_STANDARD, &pre_escape), {&g_modifiers[1].modifier_info, // html_escape &g_modifiers[2].modifier_info, // html_escape_with_arg=snippet &g_modifiers[3].modifier_info, // html_escape_with_arg=pre &g_modifiers[4].modifier_info, // html_escape_with_arg=attr... &g_modifiers[9].modifier_info, // url_query_escape &g_modifiers[12].modifier_info} }, // url_escape_with_arg=query /* 9 */ { ModifierInfo("url_query_escape", 'u', XSS_WEB_STANDARD, &url_query_escape), {} }, /* 10 */ { ModifierInfo("url_escape_with_arg=javascript", 'U', XSS_WEB_STANDARD, &validate_url_and_javascript_escape), // img_src_url_escape_with_arg=javascript {&g_modifiers[17].modifier_info} }, /* 11 */ { ModifierInfo("url_escape_with_arg=html", 'U', XSS_WEB_STANDARD, &validate_url_and_html_escape), // img_src_url_escape_with_arg=html {&g_modifiers[18].modifier_info} }, /* 12 */ { ModifierInfo("url_escape_with_arg=query", 'U', XSS_WEB_STANDARD, &url_query_escape), {} }, /* 13 */ { ModifierInfo("none", '\0', XSS_SAFE, &null_modifier), {} }, /* 14 */ { ModifierInfo("xml_escape", '\0', XSS_WEB_STANDARD, &xml_escape), {&g_modifiers[1].modifier_info, // html_escape &g_modifiers[4].modifier_info,} }, // H=attribute /* 15 */ { ModifierInfo("javascript_escape_with_arg=number", 'J', XSS_WEB_STANDARD, &javascript_number), {} }, /* 16 */ { ModifierInfo("url_escape_with_arg=css", 'U', XSS_WEB_STANDARD, &validate_url_and_css_escape), {} }, /* 17 */ { ModifierInfo("img_src_url_escape_with_arg=javascript", 'I', XSS_WEB_STANDARD, &validate_img_src_url_and_javascript_escape), {} }, /* 18 */ { ModifierInfo("img_src_url_escape_with_arg=html", 'I', XSS_WEB_STANDARD, &validate_img_src_url_and_html_escape), {} }, /* 19 */ { ModifierInfo("img_src_url_escape_with_arg=css", 'I', XSS_WEB_STANDARD, &validate_img_src_url_and_css_escape), {} }, }; static vector g_extension_modifiers; static vector g_unknown_modifiers; // Returns whether or not candidate can be safely (w.r.t XSS) // used in lieu of our ModifierInfo. This is true iff: // 1. Both have the same modifier function OR // 2. Candidate's modifier function is in our ModifierInfo's // list (vector) of safe alternative modifier functions. // // This is used with the auto-escaping code, which automatically // figures out which modifier to apply to a variable based on the // variable's context (in an html "modifier_info.long_name == our.long_name) // We found our Modifier in the built-in array g_modifiers. for (int i = 0; mod_with_alts->safe_alt_mods[i] != NULL && i < MAX_SAFE_ALTERNATIVES; ++i) if (mod_with_alts->safe_alt_mods[i]->long_name == candidate.long_name) // We found candidate in our Modifier's list of safe alternatives. return true; } // our is not built-in or candidate is not a safe replacement to our. return false; } static inline bool IsExtensionModifier(const char* long_name) { return memcmp(long_name, "x-", 2) == 0; } static bool AddModifierCommon(const char* long_name, const TemplateModifier* modifier, bool xss_safe) { if (!IsExtensionModifier(long_name)) return false; // TODO(csilvers): store in a map or multimap, rather than a vector for (vector::const_iterator mod = g_extension_modifiers.begin(); mod != g_extension_modifiers.end(); ++mod) { // Check if mod has the same name as us. For modifiers that also take // values, this is everything before the =. The only time it's ok to // have the same name is when we have different modval specializations: // "foo=bar" and "foo=baz" are both valid names. Note "foo" and // "foo=bar" is not valid: foo has no modval, but "foo=bar" does. const size_t new_modifier_namelen = strcspn(long_name, "="); const size_t existing_modifier_namelen = strcspn((*mod)->long_name.c_str(), "="); if (new_modifier_namelen == existing_modifier_namelen && !memcmp(long_name, (*mod)->long_name.c_str(), new_modifier_namelen)) { if (long_name[new_modifier_namelen] == '=' && (*mod)->long_name[existing_modifier_namelen] == '=' && (*mod)->long_name != long_name) { // It's ok, we're different specializations! } else { // It's not ok: we have the same name and no good excuse. return false; } } } g_extension_modifiers.push_back( new ModifierInfo(long_name, '\0', xss_safe ? XSS_SAFE : XSS_UNIQUE, modifier)); return true; } // Modifier added with XSS_UNIQUE XssClass. bool AddModifier(const char* long_name, const TemplateModifier* modifier) { return AddModifierCommon(long_name, modifier, false); } // Modifier added with XSS_SAFE XssClass. bool AddXssSafeModifier(const char* long_name, const TemplateModifier* modifier) { return AddModifierCommon(long_name, modifier, true); } // If candidate_match is a better match for modname/modval than bestmatch, // update bestmatch. To be a better match, two conditions must be met: // 1) The candidate's name must match modname // 2) If the candidate is a specialization (that is, name is of the form // "foo=bar", then modval matches the specialization value). // 3) If the candidate is not a specialization, bestmatch isn't a // specialization either. // Condition (3) makes sure that if we match the ModifierInfo with name // "foo=bar", we don't claim the ModifierInfo "foo=" is a better match. // Recall that by definition, modval will always start with a '=' if present. static void UpdateBestMatch(const char* modname, size_t modname_len, const char* modval, size_t modval_len, const ModifierInfo* candidate_match, const ModifierInfo** best_match) { // It's easiest to handle the two case differently: (1) candidate_match // refers to a modifier that expects a modifier-value; (2) it doesn't. if (candidate_match->modval_required) { // To be a match, we have to fulfill three requirements: we have a // modval, our modname matches candidate_match's modname (either // shortname or longname), and our modval is consistent with the // value specified in the longname (whatever might follow the =). const char* const longname_start = candidate_match->long_name.c_str(); const char* const equals = strchr(longname_start, '='); assert(equals != NULL); if (modval_len > 0 && ((modname_len == 1 && *modname == candidate_match->short_name) || (modname_len == equals - longname_start && memcmp(modname, longname_start, modname_len) == 0)) && ((equals[1] == '\0') || // name is "foo=" (not a specialization) (modval_len == longname_start + candidate_match->long_name.size() - equals && memcmp(modval, equals, modval_len) == 0))) { // Condition (3) above is satisfied iff our longname is longer than // best-match's longname (so we prefer "foo=bar" to "foo="). if (*best_match == NULL || candidate_match->long_name.size() > (*best_match)->long_name.size()) *best_match = candidate_match; } } else { // In this case, to be a match: we must *not* have a modval. Our // modname still must match modinfo's modname (either short or long). if (modval_len == 0 && ((modname_len == 1 && *modname == candidate_match->short_name) || (modname_len == candidate_match->long_name.size() && !memcmp(modname, candidate_match->long_name.data(), modname_len)))) { // In the no-modval case, only one match should exist. assert(*best_match == NULL); *best_match = candidate_match; } } } const ModifierInfo* FindModifier(const char* modname, size_t modname_len, const char* modval, size_t modval_len) { // More than one modifier can match, in the case of modval specializations // (e.g., the modifier "foo=" and "foo=bar" will both match on input of // modname="foo", modval="bar"). In that case, we take the ModifierInfo // with the longest longname, since that's the most specialized match. const ModifierInfo* best_match = NULL; if (modname_len >= 2 && IsExtensionModifier(modname)) { for (vector::const_iterator mod = g_extension_modifiers.begin(); mod != g_extension_modifiers.end(); ++mod) { UpdateBestMatch(modname, modname_len, modval, modval_len, *mod, &best_match); } if (best_match != NULL) return best_match; for (vector::const_iterator mod = g_unknown_modifiers.begin(); mod != g_unknown_modifiers.end(); ++mod) { UpdateBestMatch(modname, modname_len, modval, modval_len, *mod, &best_match); } if (best_match != NULL) return best_match; // This is the only situation where we can pass in a modifier of NULL. // It means "we don't know about this modifier-name." string fullname(modname, modname_len); if (modval_len) { fullname.append(modval, modval_len); } // TODO(csilvers): store in a map or multimap, rather than a vector g_unknown_modifiers.push_back(new ModifierInfo(fullname, '\0', XSS_UNIQUE, NULL)); return g_unknown_modifiers.back(); } else { for (const ModifierWithAlternatives* mod_with_alts = g_modifiers; mod_with_alts < g_modifiers + sizeof(g_modifiers)/sizeof(*g_modifiers); ++mod_with_alts) { UpdateBestMatch(modname, modname_len, modval, modval_len, &mod_with_alts->modifier_info, &best_match); } return best_match; } } // For escaping variables under the auto-escape mode: // Each directive below maps to a distinct sequence of // escaping directives (i.e a vector) applied // to a variable during run-time substitution. // The directives are stored in a global array (g_mods_ae) // initialized under lock in InitializeGlobalModifiers. enum AutoModifyDirective { AM_EMPTY, // Unused, kept as marker. AM_HTML, AM_HTML_UNQUOTED, AM_JS, AM_JS_NUMBER, AM_URL_HTML, AM_URL_QUERY, AM_STYLE, AM_XML, NUM_ENTRIES_AM, }; // Populates the global vector of hard-coded modifiers that // Auto-Escape may pick. We point to the appropriate modifier in // the global g_modifiers. // Reference these globals via the global array g_am_dirs[] for consistency. // Note: We allow for more than one ModifierAndValue in the array hence // the need to terminate with a Null marker. However currently all the // escaping directives have exactly one ModifierAndValue. static const ModifierAndValue g_am_empty[] = { ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_html[] = { ModifierAndValue(&g_modifiers[1].modifier_info, "", 0), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_html_unquoted[] = { ModifierAndValue(&g_modifiers[4].modifier_info, "=attribute", 10), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_js[] = { ModifierAndValue(&g_modifiers[6].modifier_info, "", 0), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_js_number[] = { ModifierAndValue(&g_modifiers[15].modifier_info, "=number", 7), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_url_html[] = { ModifierAndValue(&g_modifiers[11].modifier_info, "=html", 5), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_url_query[] = { ModifierAndValue(&g_modifiers[9].modifier_info, "", 0), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_style[] = { ModifierAndValue(&g_modifiers[0].modifier_info, "", 0), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue g_am_xml[] = { ModifierAndValue(&g_modifiers[14].modifier_info, "", 0), ModifierAndValue(NULL, "", 0) }; static const ModifierAndValue* g_am_dirs[NUM_ENTRIES_AM] = { g_am_empty, /* AM_EMPTY */ g_am_html, /* AM_HTML */ g_am_html_unquoted, /* AM_HTML_UNQUOTED */ g_am_js, /* AM_JS */ g_am_js_number, /* AM_JS_NUMBER */ g_am_url_html, /* AM_URL_HTML */ g_am_url_query, /* AM_URL_QUERY */ g_am_style, /* AM_STYLE */ g_am_xml, /* AM_XML */ }; string PrettyPrintOneModifier(const ModifierAndValue& modval) { string out; out.append(":"); if (modval.modifier_info->short_name) // short_name is a char. out.append(1, modval.modifier_info->short_name); else out.append(modval.modifier_info->long_name); if (modval.value_len != 0) out.append(modval.value, modval.value_len); return out; } string PrettyPrintModifiers(const vector& modvals, const string& separator) { string out; for (vector::const_iterator it = modvals.begin(); it != modvals.end(); ++it) { if (it != modvals.begin()) out.append(separator); out.append(PrettyPrintOneModifier(**it)); } return out; } // Return the sequence of escaping directives to apply for the given context. // An empty vector indicates an error occurred. Currently we never need // to chain escaping directives hence on success, the vector is always of // size 1. This may change in the future. vector GetModifierForHtmlJs( HtmlParser* htmlparser, string* error_msg) { assert(htmlparser); assert(error_msg); vector modvals; // Two cases of being inside javascript: // 1. Inside raw javascript (within a ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/comments.html000066400000000000000000000033521363122316600260350ustar00rootroot00000000000000 "http://www.w3.org/TR/html4/strict.dtd"> test test --> ' ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/context.html000066400000000000000000000050741363122316600256770ustar00rootroot00000000000000

'"> blah= '"> xpto ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/google.html000066400000000000000000000137161363122316600254710ustar00rootroot00000000000000Google

Google

 
  Advanced Search
  Preferences
  Language Tools

New! The G1 phone is on sale now. Learn more.


Advertising Programs - Business Solutions - About Google

©2008 - Privacy

ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/javascript_attribute.html000066400000000000000000000013441363122316600304400ustar00rootroot00000000000000 test test test test ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/javascript_block.html000066400000000000000000000016721363122316600275330ustar00rootroot00000000000000 //--> --> ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/javascript_regexp.html000066400000000000000000000105051363122316600277260ustar00rootroot00000000000000 ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/position.html000066400000000000000000000011651363122316600260540ustar00rootroot00000000000000 ctemplate-ctemplate-2.4/src/tests/htmlparser_testdata/reset.html000066400000000000000000000020411363122316600253240ustar00rootroot00000000000000 "> <123 ?> ?q=tt>test test test test test test test color> color> alt> link >
<?state attr_type=regular?> >
>
< br >
<< > < alt=""> > <> <0 > <1 > > ctemplate-ctemplate-2.4/src/tests/make_tpl_varnames_h_unittest.sh000077500000000000000000000256131363122316600255510ustar00rootroot00000000000000#!/bin/sh # Copyright (c) 2006, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # Author: csilvers@google.com (Craig Silverstein) # die() { echo "Test failed: $@" 1>&2 exit 1 } MAKETPL=${1-"$TEST_SRCDIR/make_tpl_varnames_h"} # Optional second argument is tmpdir to use TMPDIR=${2-"$TEST_TMPDIR/maketpl"} rm -rf $TMPDIR mkdir $TMPDIR || die "$LINENO: Can't make $TMPDIR" # Let's make some templates: three that are ok, and three that are not echo '' > $TMPDIR/ok1.tpl echo '' > $TMPDIR/ok2.tpl echo '{{TITLE}}' > $TMPDIR/ok3.tpl echo ' $TMPDIR/bad1.tpl echo '' > $TMPDIR/bad2.tpl echo '{{TITLE?}}' > $TMPDIR/bad3.tpl # We'll make some templates with modifiers as well. echo '' > $TMPDIR/ok4.tpl echo '' > $TMPDIR/ok5.tpl echo '' > $TMPDIR/ok6.tpl echo '' > $TMPDIR/ok7.tpl echo '' > $TMPDIR/ok8.tpl # First, test commandline flags $MAKETPL >/dev/null 2>&1 \ && die "$LINENO: $MAKETPL with no args didn't give an error" $MAKETPL -o$TMPDIR/foo >/dev/null 2>&1 \ && die "$LINENO: $MAKETPL with no template didn't give an error" $MAKETPL -h >/dev/null 2>&1 \ || die "$LINENO: $MAKETPL -h failed" $MAKETPL --help >/dev/null 2>&1 \ || die "$LINENO: $MAKETPL --help failed" $MAKETPL --nonsense >/dev/null 2>&1 \ && die "$LINENO: $MAKETPL --nonsense didn't give an error" $MAKETPL -f$TMPDIR/bar.h >/dev/null 2>&1 \ && die "$LINENO: $MAKETPL -f with no templates didn't give an error" # Some weird (broken) shells leave the ending EOF in the here-document, # hence the grep. expected_ok1=`cat < static const ::ctemplate::StaticTemplateString ko_QCHAR = STS_INIT_WITH_HASH(ko_QCHAR, "QCHAR", 13739615363438531061ULL); static const ::ctemplate::StaticTemplateString ko_HREF = STS_INIT_WITH_HASH(ko_HREF, "HREF", 4441707909033668369ULL); static const ::ctemplate::StaticTemplateString ko_PARAMS = STS_INIT_WITH_HASH(ko_PARAMS, "PARAMS", 10755877064288701757ULL); #endif // %%%OUTPUT_NAME%%% EOF` expected_ok2=`cat < static const ::ctemplate::StaticTemplateString ko_ATTRIBUTES = STS_INIT_WITH_HASH(ko_ATTRIBUTES, "ATTRIBUTES", 11813232524653503831ULL); static const ::ctemplate::StaticTemplateString ko_ATTRIBUTE = STS_INIT_WITH_HASH(ko_ATTRIBUTE, "ATTRIBUTE", 14959290143384361001ULL); #endif // %%%OUTPUT_NAME%%% EOF` expected_ok3=`cat < static const ::ctemplate::StaticTemplateString ko_TITLE = STS_INIT_WITH_HASH(ko_TITLE, "TITLE", 8931122033088041025ULL); #endif // %%%OUTPUT_NAME%%% EOF` expected_ok4=`cat < static const ::ctemplate::StaticTemplateString ko_HREF = STS_INIT_WITH_HASH(ko_HREF, "HREF", 4441707909033668369ULL); static const ::ctemplate::StaticTemplateString ko_PARAMS = STS_INIT_WITH_HASH(ko_PARAMS, "PARAMS", 10755877064288701757ULL); #endif // %%%OUTPUT_NAME%%% EOF` expected_ok5=`echo "$expected_ok4" | sed s/ok4/ok5/g` expected_ok6=`echo "$expected_ok4" | sed s/ok4/ok6/g` expected_ok7=`echo "$expected_ok4" | sed s/ok4/ok7/g` expected_ok8=`echo "$expected_ok4" | sed s/ok4/ok8/g` # When -f (--output-file) is used on ok1.tpl and ok2.tpl # Note that there are no variables in common in these two templates. # All should be returned. expected_ok1and2=`cat < static const ::ctemplate::StaticTemplateString ko_QCHAR = STS_INIT_WITH_HASH(ko_QCHAR, "QCHAR", 13739615363438531061ULL); static const ::ctemplate::StaticTemplateString ko_HREF = STS_INIT_WITH_HASH(ko_HREF, "HREF", 4441707909033668369ULL); static const ::ctemplate::StaticTemplateString ko_PARAMS = STS_INIT_WITH_HASH(ko_PARAMS, "PARAMS", 10755877064288701757ULL); static const ::ctemplate::StaticTemplateString ko_ATTRIBUTES = STS_INIT_WITH_HASH(ko_ATTRIBUTES, "ATTRIBUTES", 11813232524653503831ULL); static const ::ctemplate::StaticTemplateString ko_ATTRIBUTE = STS_INIT_WITH_HASH(ko_ATTRIBUTE, "ATTRIBUTE", 14959290143384361001ULL); #endif // %%%OUTPUT_NAME%%% EOF` # When -f (--output-file) is used on ok1.tpl and ok4.tpl # Note that both variables in ok4.tpl will be duplicates and hence not returned. expected_ok1and4=`echo "$expected_ok1" | sed s/ok1/ok1and4/g` # Suppress unimportant aspects of the make_tpl_varnames_h output. Cleanse() { # Replace the file name guard with %%%OUTPUT_NAME%%% so we can use # the same expected_ok* variables for different file names. # Note that we only append 'H_' to the end of the string, instead # of '_H_'. This is because the first call to 'tr' is already # adding a '_' at the end of the converted $1 (due to the newline # emitted by echo). n="`basename $1 | sed -e 's/[^0-9a-zA-Z]/_/g' | tr 'abcdefghijklmnopqrstuvwxyz' 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'`" grep -v '^//' "$1" | sed -e "s:TPL_.*${n}_H_:%%%OUTPUT_NAME%%%:" > "$1.cleansed" } # syntax-check these templates $MAKETPL -n $TMPDIR/ok1.tpl $TMPDIR/ok2.tpl $TMPDIR/ok3.tpl >/dev/null 2>&1 \ || die "$LINENO: $MAKETPL gave error parsing good templates" $MAKETPL -n $TMPDIR/ok1.tpl $TMPDIR/ok2.tpl $TMPDIR/bad3.tpl >/dev/null 2>&1 \ && die "$LINENO: $MAKETPL gave no error parsing bad template" $MAKETPL -n $TMPDIR/ok1.tpl $TMPDIR/ok2.tpl $TMPDIR/ok100.tpl >/dev/null 2>&1 \ && die "$LINENO: $MAKETPL gave no error parsing non-existent template" # Now try the same thing, but use template-root so we don't need absdirs $MAKETPL -n --template_dir=$TMPDIR ok1.tpl ok2.tpl ok3.tpl >/dev/null 2>&1 \ || die "$LINENO: $MAKETPL gave error parsing good templates" # Parse the templates. Bad one in the middle should be ignored. $MAKETPL --header_dir=$TMPDIR $TMPDIR/ok1.tpl $TMPDIR/bad2.tpl $TMPDIR/ok3.tpl >/dev/null 2>&1 [ $? = 1 ] || die "$LINENO: $MAKETPL gave wrong error-code parsing 1 bad template: $?" Cleanse "$TMPDIR/ok1.tpl.varnames.h" echo "$expected_ok1" | diff - "$TMPDIR/ok1.tpl.varnames.h.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok1 output correctly" [ -f "$TMPDIR/bad2.tpl.varnames.h" ] \ && die "$LINENO: $MAKETPL >did< make bad2 output" Cleanse "$TMPDIR/ok3.tpl.varnames.h" echo "$expected_ok3" | diff - "$TMPDIR/ok3.tpl.varnames.h.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok3 output correctly" # Now try the same but with a different suffix. Try an alternate -t/-o form too. # Also test not being able to output the file for some reason. $MAKETPL -t$TMPDIR -o$TMPDIR -s.out ok1.tpl bad2.tpl ok3.tpl >/dev/null 2>&1 [ $? = 1 ] || die "$LINENO: $MAKETPL gave wrong error-code parsing 1 bad template: $?" Cleanse "$TMPDIR/ok1.tpl.out" echo "$expected_ok1" | diff - "$TMPDIR/ok1.tpl.out.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok1 output correctly" [ -f "$TMPDIR/bad2.tpl.out" ] && die "$LINENO: $MAKETPL >did< make bad2 output" Cleanse "$TMPDIR/ok3.tpl.out" echo "$expected_ok3" | diff - "$TMPDIR/ok3.tpl.out.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok3 output correctly" # Verify that -f generates the requested output file: # -f with one file $MAKETPL -t$TMPDIR -f$TMPDIR/ok1.h ok1.tpl >/dev/null 2>&1 Cleanse "$TMPDIR/ok1.h" echo "$expected_ok1" | diff - "$TMPDIR/ok1.h.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok1.h output correctly" # -f with two files - no common template variables $MAKETPL -t$TMPDIR -f$TMPDIR/ok1and2.h ok1.tpl ok2.tpl >/dev/null 2>&1 Cleanse "$TMPDIR/ok1and2.h" echo "$expected_ok1and2" | diff - "$TMPDIR/ok1and2.h.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok1and2.h output correctly" # -f with two files - two common template variables $MAKETPL -t$TMPDIR -f$TMPDIR/ok1and4.h ok1.tpl ok4.tpl >/dev/null 2>&1 Cleanse "$TMPDIR/ok1and4.h" echo "$expected_ok1and4" | diff - "$TMPDIR/ok1and4.h.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok1and4.h output correctly" # -f with a bad file should not produce an output $MAKETPL -t$TMPDIR -f$TMPDIR/bar.h ok1.tpl bad1.tpl >/dev/null 2>&1 \ && die "$LINENO: $MAKETPL -f gave no error parsing bad template" # Verify we don't give any output iff everything works, with -q flag. # Also test using a different output dir. Also, test *every* ok template. mkdir $TMPDIR/output # Normally I'd do {1,2,3,4,...}, but solaris sh doesn't understand that syntax out=`$MAKETPL -q -t$TMPDIR -o$TMPDIR/output -s"#" \ ok1.tpl ok2.tpl ok3.tpl ok4.tpl ok5.tpl ok6.tpl ok7.tpl ok8.tpl \ 2>&1` [ -z "$out" ] || die "$LINENO: $MAKETPL -q wasn't so quiet: '$out'" for i in 1 2 3 4 5 6 7 8; do Cleanse "$TMPDIR/output/ok$i.tpl#" eval "echo \"\$expected_ok$i\"" | diff - "$TMPDIR/output/ok$i.tpl#.cleansed" \ || die "$LINENO: $MAKETPL didn't make ok$i output correctly" done out=`$MAKETPL -q --outputfile_suffix=2 $TMPDIR/bad{1,2,3}.tpl 2>&1` [ -z "$out" ] && die "$LINENO: $MAKETPL -q was too quiet" for i in 1 2 3; do [ -f "$TMPDIR/output/bad$i.tpl2" ] && die "$LINENO: $MAKETPL made bad$i output" done echo "PASSED" ctemplate-ctemplate-2.4/src/tests/statemachine_test.c000066400000000000000000000260441363122316600231230ustar00rootroot00000000000000/* Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include #include #include #include "htmlparser/statemachine.h" enum { SIMPLE_STATE_A, SIMPLE_STATE_B, SIMPLE_STATE_C, SIMPLE_STATE_D, SIMPLE_STATE_ERROR_TEST }; /* Include the test state machine definition. */ #include "tests/statemachine_test_fsm.h" /* Taken from google templates */ #define ASSERT(cond) do { \ if (!(cond)) { \ printf("%s: %d: ASSERT FAILED: %s\n", __FILE__, __LINE__, \ #cond); \ assert(cond); \ exit(1); \ } \ } while (0) #define ASSERT_STREQ(a, b) do { \ if (strcmp((a), (b))) { \ printf("%s: %d: ASSERT FAILED: '%s' != '%s'\n", __FILE__, __LINE__, \ (a), (b)); \ assert(!strcmp((a), (b))); \ exit(1); \ } \ } while (0) #define ASSERT_STRSTR(text, substr) do { \ if (!strstr((text), (substr))) { \ printf("%s: %d: ASSERT FAILED: '%s' not in '%s'\n", \ __FILE__, __LINE__, (substr), (text)); \ assert(strstr((text), (substr))); \ exit(1); \ } \ } while (0) #define NUM_STATES 10 /* To simply the tests */ #define statemachine_parse_str(a,b) statemachine_parse(a, b, strlen(b)); /* Simple state machine test. */ int test_simple() { statemachine_definition *def; statemachine_ctx *sm; def = statemachine_definition_new(NUM_STATES); sm = statemachine_new(def, NULL); statemachine_definition_populate(def, simple_state_transitions, simple_states_internal_names); ASSERT(sm->current_state == SIMPLE_STATE_A); statemachine_parse(sm, "001", 3); ASSERT(sm->current_state == SIMPLE_STATE_B); statemachine_parse(sm, "001", 3); ASSERT(sm->current_state == SIMPLE_STATE_C); statemachine_parse(sm, "2", 1); ASSERT(sm->current_state == SIMPLE_STATE_B); statemachine_parse(sm, "11", 2); ASSERT(sm->current_state == SIMPLE_STATE_D); statemachine_delete(sm); return 0; } /* Tests error handling logic when we try to follow non existent transitions. */ int test_error() { statemachine_definition *def; statemachine_ctx *sm; int res; def = statemachine_definition_new(NUM_STATES); sm = statemachine_new(def, NULL); statemachine_definition_populate(def, simple_state_transitions, NULL); ASSERT(sm->current_state == SIMPLE_STATE_A); ASSERT(statemachine_get_error_msg(sm) == NULL); res = statemachine_parse_str(sm, "00E"); ASSERT(sm->current_state == SIMPLE_STATE_ERROR_TEST); ASSERT(sm->current_state == res); res = statemachine_parse(sm, "3", 1); ASSERT(res == STATEMACHINE_ERROR); ASSERT_STREQ(statemachine_get_error_msg(sm), "Unexpected character '3'"); statemachine_reset(sm); ASSERT(statemachine_get_error_msg(sm) == NULL); statemachine_delete(sm); def = statemachine_definition_new(NUM_STATES); sm = statemachine_new(def, NULL); statemachine_definition_populate(def, simple_state_transitions, simple_states_internal_names); ASSERT(sm->current_state == SIMPLE_STATE_A); res = statemachine_parse_str(sm, "00E"); ASSERT(sm->current_state == SIMPLE_STATE_ERROR_TEST); ASSERT(sm->current_state == res); res = statemachine_parse(sm, "3", 1); ASSERT(res == STATEMACHINE_ERROR); ASSERT_STREQ(statemachine_get_error_msg(sm), "Unexpected character '3' in state 'error_test'"); statemachine_delete(sm); return 0; } /* Tests htmlparser_start_record() and htmlparser_end_record() logic. */ int test_record() { statemachine_definition *def; statemachine_ctx *sm; const char *actual; char expected[STATEMACHINE_RECORD_BUFFER_SIZE]; int res; int counter; def = statemachine_definition_new(NUM_STATES); sm = statemachine_new(def, NULL); statemachine_definition_populate(def, simple_state_transitions, simple_states_internal_names); ASSERT(sm->current_state == SIMPLE_STATE_A); res = statemachine_parse_str(sm, "001"); ASSERT(sm->current_state == SIMPLE_STATE_B); ASSERT(sm->current_state == res); statemachine_start_record(sm); statemachine_parse_str(sm, "121212"); ASSERT_STREQ("121212", statemachine_stop_record(sm)); statemachine_parse_str(sm, "not recorded"); statemachine_start_record(sm); statemachine_parse_str(sm, "121212000"); ASSERT_STREQ("121212000", statemachine_stop_record(sm)); statemachine_start_record(sm); statemachine_parse_str(sm, "1234567890"); ASSERT_STREQ("1234567890", statemachine_record_buffer(sm)); statemachine_parse_str(sm, "test"); ASSERT_STREQ("1234567890test", statemachine_stop_record(sm)); statemachine_start_record(sm); /* Record 1000 chars + strlen("beginning-") */ statemachine_parse_str(sm, "beginning-"); for (counter = 0; counter < 100; counter++) { statemachine_parse_str(sm, "1234567890"); } /* Make sure we preserved the start of the buffer. */ ASSERT_STRSTR(statemachine_record_buffer(sm), "beginning-"); /* And make sure the size is what we expect. */ ASSERT(STATEMACHINE_RECORD_BUFFER_SIZE - 1 == strlen(statemachine_stop_record(sm))); statemachine_start_record(sm); for (counter = 0; counter < 100; counter++) { statemachine_parse_str(sm, "0123456789ABCDEF"); } expected[0] = '\0'; /* Fill the buffer with a pattern 255 chars long (16 * 15 + 15). */ for (counter = 0; counter < 15; counter++) { strcat(expected, "0123456789ABCDEF"); } strcat(expected, "0123456789ABCDE"); actual = statemachine_stop_record(sm); ASSERT_STREQ(expected, actual); statemachine_delete(sm); return 0; } /* Test with characters outside of the ascii range */ int test_no_ascii() { statemachine_definition *def; statemachine_ctx *sm; def = statemachine_definition_new(NUM_STATES); sm = statemachine_new(def, NULL); statemachine_definition_populate(def, simple_state_transitions, simple_states_internal_names); ASSERT(sm->current_state == SIMPLE_STATE_A); statemachine_parse(sm, "\xf0\xf0\xf1", 3); ASSERT(sm->current_state == SIMPLE_STATE_B); statemachine_parse(sm, "\xf0\xf0\xf1", 3); ASSERT(sm->current_state == SIMPLE_STATE_C); statemachine_parse(sm, "\xf2", 1); ASSERT(sm->current_state == SIMPLE_STATE_B); statemachine_parse(sm, "\xf1\xf1", 2); ASSERT(sm->current_state == SIMPLE_STATE_D); statemachine_delete(sm); return 0; } int test_copy() { statemachine_definition *def; statemachine_ctx *sm1; statemachine_ctx *sm2; statemachine_ctx *sm3; def = statemachine_definition_new(NUM_STATES); sm1 = statemachine_new(def, NULL); statemachine_definition_populate(def, simple_state_transitions, simple_states_internal_names); ASSERT(sm1->current_state == SIMPLE_STATE_A); sm2 = statemachine_duplicate(sm1, def, NULL); ASSERT(sm2->current_state == SIMPLE_STATE_A); statemachine_parse(sm1, "001", 3); ASSERT(sm1->current_state == SIMPLE_STATE_B); ASSERT(sm2->current_state == SIMPLE_STATE_A); statemachine_parse(sm1, "001", 3); statemachine_parse(sm2, "001", 3); ASSERT(sm1->current_state == SIMPLE_STATE_C); ASSERT(sm2->current_state == SIMPLE_STATE_B); sm3 = statemachine_duplicate(sm2, def, NULL); ASSERT(sm3->current_state == SIMPLE_STATE_B); statemachine_parse(sm1, "001", 3); statemachine_parse(sm2, "001", 3); statemachine_parse(sm3, "002", 3); ASSERT(sm1->current_state == SIMPLE_STATE_D); ASSERT(sm2->current_state == SIMPLE_STATE_C); ASSERT(sm3->current_state == SIMPLE_STATE_A); statemachine_delete(sm1); statemachine_delete(sm2); statemachine_delete(sm3); return 0; } /* Tests statemachine_encode_char(). */ int test_encode_char() { char encoded_char[10]; int i; struct { char chr; const char *result; } encode_map[] = { { 'x', "x" }, { '0', "0" }, { '\n', "\\n" }, { '\r', "\\r" }, { '\t', "\\t" }, { '\\', "\\\\" }, { '\0', "\\x00" }, { '\xF0', "\\xf0" }, { '\0', NULL} // Terminates when output == NULL }; for (i = 0; encode_map[i].result; i++) { statemachine_encode_char(encode_map[i].chr, encoded_char, sizeof(encoded_char) / sizeof(*encoded_char)); ASSERT_STREQ(encoded_char, encode_map[i].result); } statemachine_encode_char('\xFF', encoded_char, 1); ASSERT_STREQ(encoded_char, ""); statemachine_encode_char('\xFF', encoded_char, 2); ASSERT_STREQ(encoded_char, "\\"); statemachine_encode_char('\xFF', encoded_char, 3); ASSERT_STREQ(encoded_char, "\\x"); statemachine_encode_char('\xFF', encoded_char, 4); ASSERT_STREQ(encoded_char, "\\xf"); statemachine_encode_char('\xFF', encoded_char, 5); ASSERT_STREQ(encoded_char, "\\xff"); return 0; } int main(int argc, char **argv) { test_simple(); test_error(); test_record(); test_no_ascii(); test_copy(); test_encode_char(); printf("DONE.\n"); return 0; } ctemplate-ctemplate-2.4/src/tests/statemachine_test_fsm.config000066400000000000000000000046451363122316600250160ustar00rootroot00000000000000# Copyright (c) 2008, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # # Author: falmeida@google.com (Filipe Almeida) # # Simple state machine definition used in for testing the state machine. name = 'simple' comment = 'Simple state machine' condition('1', '1\xf1') condition('2', '2\xf2') condition('E', 'E') condition('default', '[:default:]') state(name = 'A', external = 'A', transitions = [ ['1', 'B'], ['E', 'error_test'], ['default', 'A'], ]) state(name = 'B', external = 'B', transitions = [ ['1', 'C'], ['2', 'A'], ['default', 'B'], ]) state(name = 'C', external = 'C', transitions = [ ['1', 'D'], ['2', 'B'], ['default', 'C'], ]) state(name = 'D', external = 'D', transitions = [ ['2', 'C'], ['default', 'D'], ]) state(name = 'error_test', external = 'error_test', transitions = [ ['2', 'A'], ]) ctemplate-ctemplate-2.4/src/tests/template_cache_test.cc000077500000000000000000001305351363122316600235630ustar00rootroot00000000000000// Copyright (c) 2009, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // #include "config_for_unittests.h" #include #include // for assert() #include // for printf() #include // for exit() #include // for strcmp() #include // for mode_t #ifdef HAVE_UNISTD_H # include #endif // for unlink() #include // for Template #include // for TemplateDictionary #include // for DO_NOT_STRIP, etc #include // for PathJoin(), kCWD #include // for TemplateString #include "tests/template_test_util.h" // for AssertExpandIs(), etc using std::string; using ctemplate::FLAGS_test_tmpdir; using ctemplate::AssertExpandIs; using ctemplate::CreateOrCleanTestDir; using ctemplate::CreateOrCleanTestDirAndSetAsTmpdir; using ctemplate::DO_NOT_STRIP; using ctemplate::PathJoin; using ctemplate::STRIP_BLANK_LINES; using ctemplate::STRIP_WHITESPACE; using ctemplate::StaticTemplateString; using ctemplate::StringToFile; using ctemplate::StringToTemplateCache; using ctemplate::StringToTemplateFile; using ctemplate::Template; using ctemplate::TemplateCache; using ctemplate::TemplateCachePeer; using ctemplate::TemplateDictionary; using ctemplate::kCWD; #define ASSERT(cond) do { \ if (!(cond)) { \ printf("ASSERT FAILED, line %d: %s\n", __LINE__, #cond); \ assert(cond); \ exit(1); \ } \ } while (0) #define ASSERT_STREQ(a, b) ASSERT(strcmp(a, b) == 0) static const StaticTemplateString kKey = STS_INIT(kKey, "MY_KEY"); static const StaticTemplateString kContent = STS_INIT(kContent, "content"); // It would be nice to use the TEST framework, but it makes friendship // more difficult. (TemplateCache befriends TemplateCacheUnittest.) class TemplateCacheUnittest { public: static void TestGetTemplate() { // Tests the cache TemplateCache cache1; const char* text = "{This is perfectly valid} yay!"; TemplateDictionary empty_dict("dict"); string filename = StringToTemplateFile(text); const Template* tpl1 = cache1.GetTemplate(filename, DO_NOT_STRIP); const Template* tpl2 = cache1.GetTemplate(filename.c_str(), DO_NOT_STRIP); const Template* tpl3 = cache1.GetTemplate(filename, STRIP_WHITESPACE); ASSERT(tpl1 && tpl2 && tpl3); ASSERT(tpl1 == tpl2); ASSERT(tpl1 != tpl3); AssertExpandIs(tpl1, &empty_dict, text, true); AssertExpandIs(tpl2, &empty_dict, text, true); AssertExpandIs(tpl3, &empty_dict, text, true); // Tests that a nonexistent template returns NULL const Template* tpl4 = cache1.GetTemplate("/yakakak", STRIP_WHITESPACE); ASSERT(!tpl4); // Make sure we get different results if we use a different cache. TemplateCache cache2; const Template* tpl5 = cache2.GetTemplate(filename, DO_NOT_STRIP); ASSERT(tpl5); ASSERT(tpl5 != tpl1); AssertExpandIs(tpl5, &empty_dict, text, true); // And different results yet if we use the default cache. const Template* tpl6 = Template::GetTemplate(filename, DO_NOT_STRIP); ASSERT(tpl6); ASSERT(tpl6 != tpl1); AssertExpandIs(tpl6, &empty_dict, text, true); } static void TestLoadTemplate() { // Tests the cache TemplateCache cache1; const char* text = "{This is perfectly valid} yay!"; TemplateDictionary empty_dict("dict"); string filename = StringToTemplateFile(text); ASSERT(cache1.LoadTemplate(filename, DO_NOT_STRIP)); // Tests that a nonexistent template returns false ASSERT(!cache1.LoadTemplate("/yakakak", STRIP_WHITESPACE)); } static void TestStringGetTemplate() { // If you use these same cache keys somewhere else, // call Template::ClearCache first. const string cache_key_a = "cache key a"; const string text = "Test template 1"; TemplateDictionary empty_dict("dict"); TemplateCache cache1; const Template *tpl1; ASSERT(cache1.StringToTemplateCache(cache_key_a, text, DO_NOT_STRIP)); tpl1 = cache1.GetTemplate(cache_key_a, DO_NOT_STRIP); AssertExpandIs(tpl1, &empty_dict, text, true); // A different cache should give different templates. TemplateCache cache2; const Template *tpl3; ASSERT(cache2.StringToTemplateCache(cache_key_a, text, DO_NOT_STRIP)); tpl3 = cache2.GetTemplate(cache_key_a, DO_NOT_STRIP); ASSERT(tpl3 != tpl1); AssertExpandIs(tpl3, &empty_dict, text, true); // And the main cache different still const Template *tpl4; ASSERT(StringToTemplateCache(cache_key_a, text, DO_NOT_STRIP)); tpl4 = Template::GetTemplate(cache_key_a, DO_NOT_STRIP); ASSERT(tpl4 != tpl1); AssertExpandIs(tpl4, &empty_dict, text, true); // If we register a new string with the same text, it should be ignored. ASSERT(!cache1.StringToTemplateCache(cache_key_a, "new text", DO_NOT_STRIP)); Template::ClearCache(); } static void TestStringToTemplateCacheWithStrip() { const string cache_key_a = "cache key a"; const string text = "Test template 1"; TemplateDictionary empty_dict("dict"); TemplateCache cache; ASSERT(cache.StringToTemplateCache(cache_key_a, text, DO_NOT_STRIP)); TemplateCachePeer cache_peer(&cache); TemplateCachePeer::TemplateCacheKey cache_key1(cache_key_a, DO_NOT_STRIP); ASSERT(cache_peer.TemplateIsCached(cache_key1)); const Template* tpl1 = cache_peer.GetTemplate(cache_key_a, DO_NOT_STRIP); ASSERT(tpl1); AssertExpandIs(tpl1, &empty_dict, text, true); // Different strip: when a string template is registered via // StringToTemplateCache with a strip, we cannot use a different // strip later to fetch the template. TemplateCachePeer::TemplateCacheKey cache_key2(cache_key_a, STRIP_WHITESPACE); ASSERT(!cache_peer.TemplateIsCached(cache_key2)); } static void TestExpandNoLoad() { TemplateCache cache; string filename = StringToTemplateFile("alone"); string top_filename = StringToTemplateFile("Hello, {{>WORLD}}"); string inc_filename = StringToTemplateFile("world"); TemplateDictionary dict("ExpandNoLoad"); dict.AddIncludeDictionary("WORLD")->SetFilename(inc_filename); string out; // This should fail because the cache is empty. cache.Freeze(); ASSERT(!cache.ExpandNoLoad(filename, DO_NOT_STRIP, &dict, NULL, &out)); cache.ClearCache(); // also clears the "frozen" state // This should succeed -- it loads inc_filename from disk. ASSERT(cache.ExpandWithData(filename, DO_NOT_STRIP, &dict, NULL, &out)); ASSERT(out == "alone"); out.clear(); // Now this should succeed -- it's in the cache. cache.Freeze(); ASSERT(cache.ExpandNoLoad(filename, DO_NOT_STRIP, &dict, NULL, &out)); ASSERT(out == "alone"); out.clear(); // This should fail because neither top nor inc are in the cache. cache.ClearCache(); cache.Freeze(); ASSERT(!cache.ExpandNoLoad(top_filename, DO_NOT_STRIP, &dict, NULL, &out)); cache.ClearCache(); ASSERT(cache.LoadTemplate(top_filename, DO_NOT_STRIP)); // This *should* fail, but because inc_filename isn't in the cache. cache.Freeze(); ASSERT(!cache.ExpandNoLoad(top_filename, DO_NOT_STRIP, &dict, NULL, &out)); // TODO(csilvers): this should not be necessary. But expand writes // to its output even before it fails. out.clear(); cache.ClearCache(); ASSERT(cache.LoadTemplate(top_filename, DO_NOT_STRIP)); ASSERT(cache.LoadTemplate(inc_filename, DO_NOT_STRIP)); cache.Freeze(); // *Now* it should succeed, with everything it needs loaded. ASSERT(cache.ExpandNoLoad(top_filename, DO_NOT_STRIP, &dict, NULL, &out)); ASSERT(out == "Hello, world"); out.clear(); // This should succeed too, of course. ASSERT(cache.ExpandWithData(top_filename, DO_NOT_STRIP, &dict, NULL, &out)); ASSERT(out == "Hello, world"); out.clear(); cache.ClearCache(); ASSERT(cache.ExpandWithData(top_filename, DO_NOT_STRIP, &dict, NULL, &out)); ASSERT(out == "Hello, world"); out.clear(); // Now everything NoLoad needs should be in the cache again. cache.Freeze(); ASSERT(cache.ExpandNoLoad(top_filename, DO_NOT_STRIP, &dict, NULL, &out)); ASSERT(out == "Hello, world"); out.clear(); cache.ClearCache(); ASSERT(cache.LoadTemplate(top_filename, DO_NOT_STRIP)); cache.Freeze(); // This fails, of course, because we're frozen. ASSERT(!cache.LoadTemplate(inc_filename, DO_NOT_STRIP)); // And thus, this fails too. ASSERT(!cache.ExpandNoLoad(top_filename, DO_NOT_STRIP, &dict, NULL, &out)); } static void TestTemplateSearchPath() { TemplateCache cache1; const string pathA = PathJoin(FLAGS_test_tmpdir, "a/"); const string pathB = PathJoin(FLAGS_test_tmpdir, "b/"); CreateOrCleanTestDir(pathA); CreateOrCleanTestDir(pathB); TemplateDictionary dict(""); cache1.SetTemplateRootDirectory(pathA); cache1.AddAlternateTemplateRootDirectory(pathB); ASSERT(cache1.template_root_directory() == pathA); // 1. Show that a template in the secondary path can be found. const string path_b_bar = PathJoin(pathB, "template_bar"); StringToFile("b/template_bar", path_b_bar); ASSERT_STREQ(path_b_bar.c_str(), cache1.FindTemplateFilename("template_bar").c_str()); const Template* b_bar = cache1.GetTemplate("template_bar", DO_NOT_STRIP); ASSERT(b_bar); AssertExpandIs(b_bar, &dict, "b/template_bar", true); // 2. Show that the search stops once the first match is found. // Create two templates in separate directories with the same name. const string path_a_foo = PathJoin(pathA, "template_foo"); const string path_b_foo = PathJoin(pathB, "template_foo"); StringToFile("a/template_foo", path_a_foo); StringToFile("b/template_foo", path_b_foo); ASSERT_STREQ(path_a_foo.c_str(), cache1.FindTemplateFilename("template_foo").c_str()); const Template* a_foo = cache1.GetTemplate("template_foo", DO_NOT_STRIP); ASSERT(a_foo); AssertExpandIs(a_foo, &dict, "a/template_foo", true); // 3. Show that attempting to find a non-existent template gives an // empty path. ASSERT(cache1.FindTemplateFilename("baz").empty()); // 4. If we make a new cache, its path will be followed. TemplateCache cache2; cache2.SetTemplateRootDirectory(pathB); ASSERT_STREQ(path_b_foo.c_str(), cache2.FindTemplateFilename("template_foo").c_str()); const Template* b_foo = cache2.GetTemplate("template_foo", DO_NOT_STRIP); ASSERT(b_foo); AssertExpandIs(b_foo, &dict, "b/template_foo", true); // 5. Neither path will work for the default cache, which has no path. ASSERT(Template::template_root_directory() == kCWD); ASSERT(Template::FindTemplateFilename("template_foo").empty()); ASSERT(!Template::GetTemplate("template_foo", DO_NOT_STRIP)); CreateOrCleanTestDir(pathA); CreateOrCleanTestDir(pathB); } static void TestDelete() { Template::ClearCache(); // just for exercise. const string cache_key = "TestRemoveStringFromTemplateCache"; const string text = "here today..."; const string text2 = "on disk tomorrow"; TemplateDictionary dict("test"); TemplateCache cache1; ASSERT(cache1.StringToTemplateCache(cache_key, text, DO_NOT_STRIP)); const Template* tpl = cache1.GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(tpl); AssertExpandIs(tpl, &dict, text, true); cache1.Delete(cache_key); tpl = cache1.GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(!tpl); tpl = cache1.GetTemplate(cache_key, STRIP_WHITESPACE); ASSERT(!tpl); tpl = cache1.GetTemplate(cache_key, STRIP_BLANK_LINES); ASSERT(!tpl); // Try delete on a file-based template as well. string filename = StringToTemplateFile(text2); tpl = cache1.GetTemplate(filename, DO_NOT_STRIP); ASSERT(tpl); AssertExpandIs(tpl, &dict, text2, true); cache1.Delete(filename); tpl = cache1.GetTemplate(filename, DO_NOT_STRIP); ASSERT(tpl); AssertExpandIs(tpl, &dict, text2, true); // Try re-adding a cache key after deleting it. ASSERT(cache1.StringToTemplateCache(cache_key, text, DO_NOT_STRIP)); tpl = cache1.GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(tpl); AssertExpandIs(tpl, &dict, text, true); // Try ClearCache while we're at it. cache1.ClearCache(); tpl = cache1.GetTemplate(cache_key, STRIP_BLANK_LINES); ASSERT(!tpl); // Test on the Template class, which has a different function name. ASSERT(StringToTemplateCache(cache_key, text, DO_NOT_STRIP)); tpl = Template::GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(tpl); AssertExpandIs(tpl, &dict, text, true); Template::RemoveStringFromTemplateCache(cache_key); tpl = Template::GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(!tpl); tpl = Template::GetTemplate(cache_key, STRIP_WHITESPACE); ASSERT(!tpl); tpl = Template::GetTemplate(cache_key, STRIP_BLANK_LINES); ASSERT(!tpl); } static void TestTemplateCache() { const string filename_a = StringToTemplateFile("Test template 1"); const string filename_b = StringToTemplateFile("Test template 2."); TemplateCache cache1; const Template *tpl, *tpl2; ASSERT(tpl = cache1.GetTemplate(filename_a, DO_NOT_STRIP)); ASSERT(tpl2 = cache1.GetTemplate(filename_b, DO_NOT_STRIP)); ASSERT(tpl2 != tpl); // different filenames. ASSERT(tpl2 = cache1.GetTemplate(filename_a, STRIP_BLANK_LINES)); ASSERT(tpl2 != tpl); // different strip. ASSERT(tpl2 = cache1.GetTemplate(filename_b, STRIP_BLANK_LINES)); ASSERT(tpl2 != tpl); // different filenames and strip. ASSERT(tpl2 = cache1.GetTemplate(filename_a, DO_NOT_STRIP)); ASSERT(tpl2 == tpl); // same filename and strip. } static void TestReloadAllIfChangedLazyLoad() { TemplateDictionary dict("empty"); TemplateCache cache1; string filename = StringToTemplateFile("{valid template}"); string nonexistent = StringToTemplateFile("dummy"); unlink(nonexistent.c_str()); const Template* tpl = cache1.GetTemplate(filename, STRIP_WHITESPACE); assert(tpl); const Template* tpl2 = cache1.GetTemplate(nonexistent, STRIP_WHITESPACE); assert(!tpl2); StringToFile("exists now!", nonexistent); tpl2 = cache1.GetTemplate(nonexistent, STRIP_WHITESPACE); ASSERT(!tpl2); cache1.ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); tpl = cache1.GetTemplate(filename, STRIP_WHITESPACE); // force the reload tpl2 = cache1.GetTemplate(nonexistent, STRIP_WHITESPACE); ASSERT(tpl2); // file exists now unlink(nonexistent.c_str()); // here today... cache1.ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); ASSERT(cache1.GetTemplate(filename, STRIP_WHITESPACE)); ASSERT(!cache1.GetTemplate(nonexistent, STRIP_WHITESPACE)); StringToFile("lazarus", nonexistent); StringToFile("{new template}", filename); tpl = cache1.GetTemplate(filename, STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "{valid template}", true); // haven't reloaded // But a different cache (say, the default) should load the new content. const Template* tpl3 = Template::GetTemplate(filename, STRIP_WHITESPACE); AssertExpandIs(tpl3, &dict, "{new template}", true); cache1.ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); tpl = cache1.GetTemplate(filename, STRIP_WHITESPACE); // needed AssertExpandIs(tpl, &dict, "{new template}", true); tpl2 = cache1.GetTemplate(nonexistent, STRIP_WHITESPACE); ASSERT(tpl2); AssertExpandIs(tpl2, &dict, "lazarus", true); // Ensure that string templates don't reload const string cache_key_a = "cache key a"; const string text = "Test template 1"; const Template *str_tpl; ASSERT(cache1.StringToTemplateCache(cache_key_a, text, DO_NOT_STRIP)); str_tpl = cache1.GetTemplate(cache_key_a, DO_NOT_STRIP); AssertExpandIs(str_tpl, &dict, text, true); cache1.ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); ASSERT(cache1.GetTemplate(cache_key_a, DO_NOT_STRIP) == str_tpl); cache1.ClearCache(); } static void TestReloadAllIfChangedImmediateLoad() { TemplateDictionary dict("empty"); TemplateCache cache1; TemplateCachePeer cache_peer(&cache1); // Add templates string filename1 = StringToTemplateFile("{valid template}"); string filename2 = StringToTemplateFile("{another valid template}"); const Template* tpl1 = cache1.GetTemplate(filename1, STRIP_WHITESPACE); assert(tpl1); const Template* tpl2 = cache1.GetTemplate(filename2, STRIP_WHITESPACE); assert(tpl2); StringToFile("{file1 contents changed}", filename1); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); TemplateCachePeer::TemplateCacheKey cache_key1(filename1, STRIP_WHITESPACE); ASSERT(cache_peer.TemplateIsCached(cache_key1)); const Template* tpl1_post_reload = cache_peer.GetTemplate(filename1, STRIP_WHITESPACE); ASSERT(tpl1_post_reload != tpl1); // Check that cache1's tpl1 has the new contents AssertExpandIs(tpl1_post_reload, &dict, "{file1 contents changed}", true); // Ensure tpl2 is unchanged TemplateCachePeer::TemplateCacheKey cache_key2(filename2, STRIP_WHITESPACE); ASSERT(cache_peer.TemplateIsCached(cache_key2)); const Template* tpl2_post_reload = cache_peer.GetTemplate(filename2, STRIP_WHITESPACE); ASSERT(tpl2_post_reload == tpl2); // Test delete & re-add: delete tpl2, and reload. unlink(filename2.c_str()); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); ASSERT(!cache_peer.GetTemplate(filename2, STRIP_WHITESPACE)); // Re-add tpl2 and ensure it reloads. StringToFile("{re-add valid template contents}", filename2); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); ASSERT(cache_peer.GetTemplate(filename2, STRIP_WHITESPACE)); // Ensure that string templates don't reload const string cache_key_a = "cache key a"; const string text = "Test template 1"; const Template *str_tpl; ASSERT(cache1.StringToTemplateCache(cache_key_a, text, DO_NOT_STRIP)); str_tpl = cache1.GetTemplate(cache_key_a, DO_NOT_STRIP); AssertExpandIs(str_tpl, &dict, text, true); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); ASSERT(cache1.GetTemplate(cache_key_a, DO_NOT_STRIP) == str_tpl); cache1.ClearCache(); } static void TestReloadImmediateWithDifferentSearchPaths() { TemplateDictionary dict("empty"); TemplateCache cache1; TemplateCachePeer cache_peer(&cache1); const string pathA = PathJoin(FLAGS_test_tmpdir, "a/"); const string pathB = PathJoin(FLAGS_test_tmpdir, "b/"); CreateOrCleanTestDir(pathA); CreateOrCleanTestDir(pathB); cache1.SetTemplateRootDirectory(pathA); cache1.AddAlternateTemplateRootDirectory(pathB); ASSERT(cache1.template_root_directory() == pathA); // Add b/foo const string path_b_foo = PathJoin(pathB, "template_foo"); StringToFile("b/template_foo", path_b_foo); ASSERT_STREQ(path_b_foo.c_str(), cache1.FindTemplateFilename("template_foo").c_str()); // Add b/foo to the template cache. cache1.GetTemplate("template_foo", DO_NOT_STRIP); // Add a/foo const string path_a_foo = PathJoin(pathA, "template_foo"); StringToFile("a/template_foo", path_a_foo); ASSERT_STREQ(path_a_foo.c_str(), cache1.FindTemplateFilename("template_foo").c_str()); // Now, on reload we pick up foo from the earlier search path: a/foo cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); const Template* foo_post_reload = cache_peer.GetTemplate("template_foo", STRIP_WHITESPACE); AssertExpandIs(foo_post_reload, &dict, "a/template_foo", true); // Delete a/foo and reload. Now we pick up the next available foo: b/foo unlink(path_a_foo.c_str()); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); foo_post_reload = cache_peer.GetTemplate("template_foo", STRIP_WHITESPACE); AssertExpandIs(foo_post_reload, &dict, "b/template_foo", true); } static void TestReloadLazyWithDifferentSearchPaths() { // Identical test as above with but with LAZY_RELOAD TemplateDictionary dict("empty"); TemplateCache cache1; TemplateCachePeer cache_peer(&cache1); const string pathA = PathJoin(FLAGS_test_tmpdir, "a/"); const string pathB = PathJoin(FLAGS_test_tmpdir, "b/"); CreateOrCleanTestDir(pathA); CreateOrCleanTestDir(pathB); cache1.SetTemplateRootDirectory(pathA); cache1.AddAlternateTemplateRootDirectory(pathB); ASSERT(cache1.template_root_directory() == pathA); // Add b/foo const string path_b_foo = PathJoin(pathB, "template_foo"); StringToFile("b/template_foo", path_b_foo); ASSERT_STREQ(path_b_foo.c_str(), cache1.FindTemplateFilename("template_foo").c_str()); // Add b/foo to the template cache. cache1.GetTemplate("template_foo", DO_NOT_STRIP); // Add a/foo const string path_a_foo = PathJoin(pathA, "template_foo"); StringToFile("a/template_foo", path_a_foo); ASSERT_STREQ(path_a_foo.c_str(), cache1.FindTemplateFilename("template_foo").c_str()); // Now, on reload we pick up foo from the earlier search path: a/foo cache1.ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); const Template* foo_post_reload = cache_peer.GetTemplate("template_foo", STRIP_WHITESPACE); AssertExpandIs(foo_post_reload, &dict, "a/template_foo", true); // Delete a/foo and reload. Now we pick up the next available foo: b/foo unlink(path_a_foo.c_str()); cache1.ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); foo_post_reload = cache_peer.GetTemplate("template_foo", STRIP_WHITESPACE); AssertExpandIs(foo_post_reload, &dict, "b/template_foo", true); } static void TestRefcounting() { TemplateCache cache1; TemplateCachePeer cache_peer(&cache1); TemplateDictionary dict("dict"); // Add templates string filename1 = StringToTemplateFile("{valid template}"); string filename2 = StringToTemplateFile("{another valid template}"); const Template* cache1_tpl1 = cache1.GetTemplate(filename1, STRIP_WHITESPACE); assert(cache1_tpl1); const Template* cache1_tpl2 = cache1.GetTemplate(filename2, STRIP_WHITESPACE); assert(cache1_tpl2); // Check refcount. It should be 2 -- one for the originalvalue // when it's constructed, and one for the call to GetTemplate. TemplateCachePeer::TemplateCacheKey cache_key1(filename1, STRIP_WHITESPACE); ASSERT(cache_peer.Refcount(cache_key1) == 2); TemplateCachePeer::TemplateCacheKey cache_key2(filename2, STRIP_WHITESPACE); ASSERT(cache_peer.Refcount(cache_key2) == 2); // Clone cache2 from cache1 TemplateCache* cache2 = cache1.Clone(); TemplateCachePeer cache_peer2(cache2); // Check refcount was incremented. It should be the same for both caches. ASSERT(cache_peer.Refcount(cache_key1) == 3); ASSERT(cache_peer2.Refcount(cache_key1) == 3); ASSERT(cache_peer.Refcount(cache_key2) == 3); ASSERT(cache_peer2.Refcount(cache_key2) == 3); // Check that the template ptrs in both caches are the same. const Template* cache2_tpl1 = cache2->GetTemplate(filename1, STRIP_WHITESPACE); const Template* cache2_tpl2 = cache2->GetTemplate(filename2, STRIP_WHITESPACE); ASSERT(cache2_tpl1 == cache1_tpl1); ASSERT(cache2_tpl2 == cache1_tpl2); // GetTemplate should have augmented the refcount. ASSERT(cache_peer.Refcount(cache_key1) == 4); ASSERT(cache_peer2.Refcount(cache_key1) == 4); ASSERT(cache_peer.Refcount(cache_key2) == 4); ASSERT(cache_peer2.Refcount(cache_key2) == 4); // Change tpl1 file contents and reload. StringToFile("{file1 contents changed}", filename1); cache2->ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); // Since the template will be reloaded into a new instance, // GetTemplate will return new pointers. The older template // pointer was moved to the freelist. const Template* cache2_tpl1_post_reload = cache2->GetTemplate( filename1, STRIP_WHITESPACE); ASSERT(cache2_tpl1_post_reload != cache2_tpl1); // Check that cache1's tpl1 has the new contents AssertExpandIs(cache2_tpl1_post_reload, &dict, "{file1 contents changed}", true); // Ensure tpl2 is unchanged const Template* cache2_tpl2_post_reload = cache2->GetTemplate( filename2, STRIP_WHITESPACE); ASSERT(cache2_tpl2_post_reload == cache2_tpl2); // Now key1 points to different templates in cache1 and cache2. // cache1's version should have a refcount of 3 (was 4, went down // by 1 when cache2 dropped its reference to it). cache2's // version should be 2 (one for the new file, 1 for the call to // GetTemplate() that followed it), while key2 should have a // refcount of 5 in both caches (due to the new call, above, to // GetTemplate()). ASSERT(cache_peer.Refcount(cache_key1) == 3); ASSERT(cache_peer2.Refcount(cache_key1) == 2); ASSERT(cache_peer.Refcount(cache_key2) == 5); ASSERT(cache_peer2.Refcount(cache_key2) == 5); const int old_delete_count = cache_peer.NumTotalTemplateDeletes(); // Clear up the cache2's freelist, this should drop all refcounts, // due to the calls cache_peer2 made to // GetTemplate(the-old-filename1), GetTemplate(the-new-filename1), // and GetTemplate(filename2) (twice!) cache_peer2.DoneWithGetTemplatePtrs(); ASSERT(cache_peer.Refcount(cache_key1) == 2); ASSERT(cache_peer2.Refcount(cache_key1) == 1); ASSERT(cache_peer.Refcount(cache_key2) == 3); ASSERT(cache_peer2.Refcount(cache_key2) == 3); // Make sure that deleting from the cache causes deletion. // ClearCache() on peer1 should finally get rid of the old filename1. cache_peer.ClearCache(); ASSERT(cache_peer.NumTotalTemplateDeletes() == old_delete_count + 1); cache_peer2.ClearCache(); // Delete-count should go up by 2 as both the new tpl1, and tpl2, go away. ASSERT(cache_peer.NumTotalTemplateDeletes() == old_delete_count + 3); delete cache2; } static void TestDoneWithGetTemplatePtrs() { TemplateCache cache1; TemplateCachePeer cache_peer1(&cache1); TemplateDictionary dict("dict"); // Add templates string fname = StringToTemplateFile("{valid template}"); TemplateCachePeer::TemplateCacheKey cache_key(fname, STRIP_WHITESPACE); string out; int old_delete_count = cache_peer1.NumTotalTemplateDeletes(); // OK, let's get the templates in the cache. ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); // This should not have changed the delete-count. ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); // And the refcount should be 1. ASSERT(cache_peer1.Refcount(cache_key) == 1); // Same holds if we expand again. ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); ASSERT(cache_peer1.Refcount(cache_key) == 1); // Now we delete from the cache. Should up the delete_count. ASSERT(cache1.Delete(fname)); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // Calling DoneWithGetTemplatePtrs() should be a noop -- we // haven't called GetTemplate() yet. cache1.DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); // Now do the same thing, but throw in a GetTemplate(). Now // DoneWithGetTemplatePtrs() should still cause a delete, but only // after a call to Delete() deletes the cache's refcount too. ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); cache1.GetTemplate(fname, STRIP_WHITESPACE); cache1.DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); ASSERT(cache1.Delete(fname)); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); cache1.ClearCache(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); // Now load in a replacement. The loading itself should cause a // delete (no GetTemplate calls, so no need to involve the freelist). ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); StringToFile("{file1 contents changed}", fname); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // DoneWithGetTemplatePtrs() should just be a noop. cache1.DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); // Delete the new version of fname too! cache1.Delete(fname); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // Now load in a replacement, but having done a GetTemplate() first. // We need DoneWithGetTemplatePtrs() to delete, in this case. ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); cache1.GetTemplate(fname, STRIP_WHITESPACE); ASSERT(cache_peer1.Refcount(cache_key) == 2); StringToFile("{file1 contents changed}", fname); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); cache1.DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // Delete the new version of fname too! cache1.Delete(fname); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // Add a Clone() into the mix. Now Delete() calls, even from both // caches, won't up the delete-count until we DoneWithGetTemplatePtrs() // -- but only from the cache that called GetTemplate(). ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); cache1.GetTemplate(fname, STRIP_WHITESPACE); ASSERT(cache_peer1.Refcount(cache_key) == 2); { TemplateCache* cache2 = cache1.Clone(); TemplateCachePeer cache_peer2(cache2); ASSERT(cache_peer1.Refcount(cache_key) == 3); ASSERT(cache_peer2.Refcount(cache_key) == 3); // Do all sorts of Delete()s. StringToFile("{file1 contents changed}", fname); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); ASSERT(cache_peer1.Refcount(cache_key) == 1); // the new file ASSERT(cache_peer2.Refcount(cache_key) == 2); // the old file cache2->ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); // Each cache has a different copy of the new file. ASSERT(cache_peer1.Refcount(cache_key) == 1); // the new file ASSERT(cache_peer2.Refcount(cache_key) == 1); // the new file ASSERT(cache1.Delete(fname)); // should delete the new file ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); ASSERT(cache2->Delete(fname)); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); cache2->DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); cache1.DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); cache1.ClearCache(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); delete cache2; } // If we call DoneWithGetTemplatePtrs() while a clone points to the // template, it won't delete the template yet. ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); { TemplateCache* cache2 = cache1.Clone(); TemplateCachePeer cache_peer2(cache2); StringToFile("{file1 contents changed}", fname); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); delete cache2; ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); } cache1.ClearCache(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // If we throw an explicit GetTemplate() in, we still need // DoneWithGetTemplatePtrs(). ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); cache1.GetTemplate(fname, STRIP_WHITESPACE); { TemplateCache* cache2 = cache1.Clone(); TemplateCachePeer cache_peer2(cache2); StringToFile("{file1 contents changed}", fname); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); cache1.DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); delete cache2; ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); } cache1.ClearCache(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // Multiple GetTemplate()s should still all be cleared by // DoneWithGetTemplatePtrs(). ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); cache1.GetTemplate(fname, STRIP_WHITESPACE); cache1.GetTemplate(fname, STRIP_WHITESPACE); ASSERT(cache_peer1.Refcount(cache_key) == 3); StringToFile("{file1 contents changed}", fname); cache1.ReloadAllIfChanged(TemplateCache::IMMEDIATE_RELOAD); cache1.DoneWithGetTemplatePtrs(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); cache1.ClearCache(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // Calling ClearCache() deletes old templates too -- we don't even // need to change the content. ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); cache1.GetTemplate(fname, STRIP_WHITESPACE); cache1.GetTemplate(fname, STRIP_WHITESPACE); cache1.ClearCache(); ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); // So does deleting the cache object. ASSERT(cache1.ExpandWithData(fname, STRIP_WHITESPACE, &dict, NULL, &out)); { TemplateCache* cache2 = cache1.Clone(); TemplateCachePeer cache_peer2(cache2); ASSERT(cache_peer1.Refcount(cache_key) == 2); cache2->GetTemplate(fname, STRIP_WHITESPACE); ASSERT(cache_peer1.Refcount(cache_key) == 3); ASSERT(cache_peer2.Refcount(cache_key) == 3); ASSERT(cache1.Delete(fname)); ASSERT(cache_peer1.NumTotalTemplateDeletes() == old_delete_count); ASSERT(cache_peer2.Refcount(cache_key) == 2); delete cache2; } ASSERT(cache_peer1.NumTotalTemplateDeletes() == ++old_delete_count); } static void TestCloneStringTemplates() { TemplateCache cache1; // Create & insert a string template const string cache_key_a = "cache key a"; const string text = "Test template 1"; TemplateDictionary empty_dict("dict"); ASSERT(cache1.StringToTemplateCache(cache_key_a, text, DO_NOT_STRIP)); // Clone cache2 from cache1 TemplateCache* cache2 = cache1.Clone(); // Check that the string template was copied into cache2 const Template* cache2_tpl = cache2->GetTemplate(cache_key_a, DO_NOT_STRIP); ASSERT(cache2_tpl); AssertExpandIs(cache2_tpl, &empty_dict, text, true); delete cache2; } static void TestInclude() { TemplateCache cache; string incname = StringToTemplateFile("include & print file\n"); string tpl_file = StringToTemplateFile("hi {{>INC:h}} bar\n"); const Template* tpl = cache.GetTemplate(tpl_file, DO_NOT_STRIP); ASSERT(tpl); TemplateDictionary dict("dict"); AssertExpandWithCacheIs(&cache, tpl_file, DO_NOT_STRIP, &dict, NULL, "hi bar\n", true); dict.AddIncludeDictionary("INC")->SetFilename(incname); AssertExpandWithCacheIs(&cache, tpl_file, DO_NOT_STRIP, &dict, NULL, "hi include & print file bar\n", true); } // Make sure we don't deadlock when a template includes itself. // This also tests we handle recursive indentation properly. static void TestRecursiveInclude() { TemplateCache cache; string incname = StringToTemplateFile("hi {{>INC}} bar\n {{>INC}}!"); const Template* tpl = cache.GetTemplate(incname, DO_NOT_STRIP); ASSERT(tpl); TemplateDictionary dict("dict"); dict.AddIncludeDictionary("INC")->SetFilename(incname); // Note the last line is indented 4 spaces instead of 2. This is // because the last sub-include is indented. AssertExpandWithCacheIs(&cache, incname, DO_NOT_STRIP, &dict, NULL, "hi hi bar\n ! bar\n hi bar\n !!", true); } static void TestStringTemplateInclude() { const string cache_key = "TestStringTemplateInclude"; const string cache_key_inc = "TestStringTemplateInclude-inc"; const string text = "{{>INC}}"; const string text_inc = "
\n

\nUser {{USER}}\n

"; TemplateCache cache; ASSERT(cache.StringToTemplateCache(cache_key, text, DO_NOT_STRIP)); ASSERT(cache.StringToTemplateCache(cache_key_inc, text_inc, DO_NOT_STRIP)); const Template *tpl = cache.GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(tpl); TemplateDictionary dict("dict"); TemplateDictionary* sub_dict = dict.AddIncludeDictionary("INC"); sub_dict->SetFilename(cache_key_inc); sub_dict->SetValue("USER", "John<>Doe"); string expected = "
\n

\nUser John<>Doe\n

"; AssertExpandWithCacheIs(&cache, cache_key, DO_NOT_STRIP, &dict, NULL, expected, true); } static void TestTemplateString() { TemplateCache cache; ASSERT(cache.StringToTemplateCache(kKey, kContent, DO_NOT_STRIP)); const Template *tpl = cache.GetTemplate(kKey, DO_NOT_STRIP); ASSERT(tpl); TemplateDictionary dict("dict"); AssertExpandWithCacheIs(&cache, "MY_KEY", DO_NOT_STRIP, &dict, NULL, "content", true); // Try retrieving with a char* rather than a TemplateString*. tpl = cache.GetTemplate("MY_KEY", DO_NOT_STRIP); ASSERT(tpl); AssertExpandWithCacheIs(&cache, "MY_KEY", DO_NOT_STRIP, &dict, NULL, "content", true); // Delete with a char* rather than a TemplateString*. cache.Delete("MY_KEY"); tpl = cache.GetTemplate("MY_KEY", DO_NOT_STRIP); ASSERT(!tpl); ASSERT(cache.StringToTemplateCache("MY_KEY", "content", DO_NOT_STRIP)); tpl = cache.GetTemplate(kKey, DO_NOT_STRIP); ASSERT(tpl); cache.Delete(kKey); tpl = cache.GetTemplate("MY_KEY", DO_NOT_STRIP); ASSERT(!tpl); } static void TestFreeze() { TemplateCache cache; TemplateDictionary dict("dict"); // Load some templates string filename1 = StringToTemplateFile("{valid template}"); string filename2 = StringToTemplateFile("hi {{>INC:h}} bar\n"); const Template* cache_tpl1 = cache.GetTemplate(filename1, STRIP_WHITESPACE); assert(cache_tpl1); AssertExpandIs(cache_tpl1, &dict, "{valid template}", true); const Template* cache_tpl2 = cache.GetTemplate(filename2, DO_NOT_STRIP); assert(cache_tpl2); static_cast(cache_tpl2); // avoid unused var warning in opt mode AssertExpandWithCacheIs(&cache, filename2, DO_NOT_STRIP, &dict, NULL, "hi bar\n", true); // Set the root directory const string pathA = PathJoin(FLAGS_test_tmpdir, "a/"); CreateOrCleanTestDir(pathA); cache.SetTemplateRootDirectory(pathA); ASSERT(cache.template_root_directory() == pathA); // Freeze the cache now, and test its impact. cache.Freeze(); // 1. Loading new templates fails. string filename3 = StringToTemplateFile("{yet another valid template}"); const Template* cache_tpl3 = cache.GetTemplate(filename3, STRIP_WHITESPACE); assert(!cache_tpl3); static_cast(cache_tpl3); // avoid unused var warning in opt mode // 2. Reloading existing templates fails. StringToFile("{file1 contents changed}", filename1); cache.ReloadAllIfChanged(TemplateCache::LAZY_RELOAD); const Template* cache_tpl1_post_reload = cache.GetTemplate( filename1, STRIP_WHITESPACE); ASSERT(cache_tpl1_post_reload == cache_tpl1); // Check that cache's tpl1 has the same old contents AssertExpandIs(cache_tpl1_post_reload, &dict, "{valid template}", true); // 3. Cannot delete from a frozen cache. cache.Delete(filename1); ASSERT(cache.GetTemplate(filename1, STRIP_WHITESPACE)); // 4. Expand won't load an included template on-demand. string incname = StringToTemplateFile("include & print file\n"); dict.AddIncludeDictionary("INC")->SetFilename(incname); AssertExpandWithCacheIs(&cache, filename2, DO_NOT_STRIP, &dict, NULL, "hi bar\n", false); // 5. Cannot change template root directory. const string pathB = PathJoin(FLAGS_test_tmpdir, "b/"); CreateOrCleanTestDir(pathB); cache.SetTemplateRootDirectory(pathB); ASSERT(cache.template_root_directory() == pathA); // Still the old path CreateOrCleanTestDir(pathA); CreateOrCleanTestDir(pathB); } }; int main(int argc, char** argv) { CreateOrCleanTestDirAndSetAsTmpdir(FLAGS_test_tmpdir); TemplateCacheUnittest::TestGetTemplate(); TemplateCacheUnittest::TestLoadTemplate(); TemplateCacheUnittest::TestStringGetTemplate(); TemplateCacheUnittest::TestStringToTemplateCacheWithStrip(); TemplateCacheUnittest::TestExpandNoLoad(); TemplateCacheUnittest::TestTemplateSearchPath(); TemplateCacheUnittest::TestDelete(); TemplateCacheUnittest::TestTemplateCache(); TemplateCacheUnittest::TestReloadAllIfChangedLazyLoad(); TemplateCacheUnittest::TestReloadAllIfChangedImmediateLoad(); TemplateCacheUnittest::TestReloadImmediateWithDifferentSearchPaths(); TemplateCacheUnittest::TestReloadLazyWithDifferentSearchPaths(); TemplateCacheUnittest::TestRefcounting(); TemplateCacheUnittest::TestDoneWithGetTemplatePtrs(); TemplateCacheUnittest::TestCloneStringTemplates(); TemplateCacheUnittest::TestInclude(); TemplateCacheUnittest::TestRecursiveInclude(); TemplateCacheUnittest::TestStringTemplateInclude(); TemplateCacheUnittest::TestTemplateString(); TemplateCacheUnittest::TestFreeze(); printf("DONE\n"); return 0; } ctemplate-ctemplate-2.4/src/tests/template_dictionary_unittest.cc000077500000000000000000001166071363122316600255710ustar00rootroot00000000000000// Copyright (c) 2005, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // This code is written to not use the google testing framework // as much as possible, to make it easier to opensource. #include "config_for_unittests.h" #include #include #include #include #include "base/arena.h" #include #include #include #include "tests/template_test_util.h" #include "base/util.h" TEST_INIT // defines RUN_ALL_TESTS using std::string; using std::vector; using ctemplate::UnsafeArena; using ctemplate::DO_NOT_STRIP; using ctemplate::ExpandEmitter; using ctemplate::PerExpandData; using ctemplate::StaticTemplateString; using ctemplate::StringToTemplateCache; using ctemplate::TemplateDictionary; using ctemplate::TemplateDictionaryInterface; using ctemplate::TemplateDictionaryPeer; using ctemplate::TemplateString; #define ASSERT_STRSTR(text, substr) do { \ if (!strstr((text), (substr))) { \ printf("%s: %d: ASSERT FAILED: '%s' not in '%s'\n", \ __FILE__, __LINE__, (substr), (text)); \ assert(strstr((text), (substr))); \ exit(1); \ } \ } while (0) // test escape-functor that replaces all input with "foo" class FooEscaper : public ctemplate::TemplateModifier { public: void Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* outbuf, const string& arg) const { assert(arg.empty()); // we don't take an argument outbuf->Emit("foo"); } }; // test escape-functor that replaces all input with "" class NullEscaper : public ctemplate::TemplateModifier { public: void Modify(const char* in, size_t inlen, const PerExpandData*, ExpandEmitter* outbuf, const string& arg) const { assert(arg.empty()); // we don't take an argument } }; // first does javascript-escaping, then html-escaping class DoubleEscaper : public ctemplate::TemplateModifier { public: void Modify(const char* in, size_t inlen, const PerExpandData* data, ExpandEmitter* outbuf, const string& arg) const { assert(arg.empty()); // we don't take an argument string tmp = ctemplate::javascript_escape(in, inlen); ctemplate::html_escape.Modify(tmp.data(), tmp.size(), data, outbuf, ""); } }; namespace { static const TemplateDictionary* GetSectionDict( const TemplateDictionary* d, const char* name, int i) { TemplateDictionaryPeer peer(d); vector dicts; EXPECT_GE(peer.GetSectionDictionaries(name, &dicts), i); return dicts[i]; } static const TemplateDictionary* GetIncludeDict( const TemplateDictionary* d, const char* name, int i) { TemplateDictionaryPeer peer(d); vector dicts; EXPECT_GE(peer.GetIncludeDictionaries(name, &dicts), i); return dicts[i]; } static void SetUp() { TemplateDictionary::SetGlobalValue("GLOBAL", "top"); } TEST(TemplateDictionary, SetValueAndTemplateStringAndArena) { // Try both with the arena, and without. UnsafeArena arena(100); // We run the test with arena twice to double-check we don't ever delete it UnsafeArena* arenas[] = {&arena, &arena, NULL}; for (int i = 0; i < sizeof(arenas)/sizeof(*arenas); ++i) { TemplateDictionary dict(string("test_arena") + char('0'+i), arenas[i]); // Test copying char*s, strings, and explicit TemplateStrings dict.SetValue("FOO", "foo"); dict.SetValue(string("FOO2"), TemplateString("foo2andmore", 4)); dict["FOO3"] = "foo3"; dict[string("FOO4")] = TemplateString("foo4andmore", 4); dict["FOO5"] = string("Olaf"); dict["FOO6"] = 6; dict["FOO7"] = long(7); TemplateDictionaryPeer peer(&dict); // verify what happened EXPECT_TRUE(peer.ValueIs("FOO", "foo")); EXPECT_TRUE(peer.ValueIs("FOO2", "foo2")); string dump; dict.DumpToString(&dump); char expected[256]; snprintf(expected, sizeof(expected), ("global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" "};\n" "dictionary 'test_arena%d' {\n" " FOO: >foo<\n" " FOO2: >foo2<\n" " FOO3: >foo3<\n" " FOO4: >foo4<\n" " FOO5: >Olaf<\n" " FOO6: >6<\n" " FOO7: >7<\n" "}\n"), i); EXPECT_STREQ(dump.c_str(), expected); } } TEST(TemplateDictionary, SetValueWithoutCopy) { UnsafeArena arena(100); TemplateDictionary dict("Test arena", &arena); char value[32]; snprintf(value, sizeof(value), "%s", "value"); const void* const ptr = arena.Alloc(0); dict.SetValueWithoutCopy("key", value); // We shouldn't have copied the value string. EXPECT_EQ(ptr, arena.Alloc(0)); TemplateDictionaryPeer peer(&dict); EXPECT_TRUE(peer.ValueIs("key", "value")); // If our content changes, so does what's in the dictionary -- but // only the contents of the buffer, not its length! snprintf(value, sizeof(value), "%s", "not_value"); EXPECT_TRUE(peer.ValueIs("key", "not_v")); // sizeof("not_v") == sizeof("value") } TEST(TemplateDictionary, SetIntValue) { TemplateDictionary dict("test_SetIntValue", NULL); TemplateDictionaryPeer peer(&dict); dict.SetIntValue("INT", 5); // - is an illegal varname in templates, but perfectly fine in dicts dict.SetIntValue("-INT", -5); EXPECT_TRUE(peer.ValueIs("INT", "5")); EXPECT_TRUE(peer.ValueIs("-INT", "-5")); string dump; dict.DumpToString(&dump); ASSERT_STRSTR(dump.c_str(), "\n INT: >5<\n"); ASSERT_STRSTR(dump.c_str(), "\n -INT: >-5<\n"); } TEST(TemplateDictionary, SetFormattedValue) { TemplateDictionary dict("test_SetFormattedValue", NULL); TemplateDictionaryPeer peer(&dict); dict.SetFormattedValue(TemplateString("PRINTF", sizeof("PRINTF")-1), "%s test %04d", "template test", 1); EXPECT_TRUE(peer.ValueIs("PRINTF", "template test test 0001")); string dump; dict.DumpToString(&dump); ASSERT_STRSTR(dump.c_str(), "\n PRINTF: >template test test 0001<\n"); // Now test something of size 4k or so, where we can't use scratchbuf dict.SetFormattedValue(TemplateString("PRINTF", sizeof("PRINTF")-1), "%s test %04444d", "template test", 2); string expected("template test test "); for (int i = 0; i < 4443; ++i) expected.append("0"); expected.append("2"); EXPECT_TRUE(peer.ValueIs("PRINTF", expected)); string dump2; dict.DumpToString(&dump2); expected = string("\n PRINTF: >") + expected + string("<\n"); ASSERT_STRSTR(dump2.c_str(), expected.c_str()); } TEST(TemplateDictionary, SetEscapedValue) { TemplateDictionary dict("test_SetEscapedValue", NULL); TemplateDictionaryPeer peer(&dict); dict.SetEscapedValue("hardest HTML", "", ctemplate::html_escape); dict.SetEscapedValue("hardest JS", ("f = 'foo';\r\n\tprint \"\\&foo = \b\", \"foo\""), ctemplate::javascript_escape); dict.SetEscapedValue("query escape 0", "", ctemplate::url_query_escape); EXPECT_TRUE(peer.ValueIs("hardest HTML", "<A HREF='foo' id="bar && " "baz">")); EXPECT_TRUE(peer.ValueIs("hardest JS", "f \\x3d \\x27foo\\x27;\\r\\n\\tprint \\x22\\\\\\x26" "foo \\x3d \\b\\x22, \\x22foo\\x22")); EXPECT_TRUE(peer.ValueIs("query escape 0", "")); // Test using hand-made modifiers. FooEscaper foo_escaper; dict.SetEscapedValue("easy foo", "hello there!", FooEscaper()); dict.SetEscapedValue("harder foo", "so much to say\nso many foos", foo_escaper); DoubleEscaper double_escaper; dict.SetEscapedValue("easy double", "doo", double_escaper); dict.SetEscapedValue("harder double", "\n", DoubleEscaper()); dict.SetEscapedValue("hardest double", "print \"\";\r\n\\1;", double_escaper); EXPECT_TRUE(peer.ValueIs("easy foo", "foo")); EXPECT_TRUE(peer.ValueIs("harder foo", "foo")); EXPECT_TRUE(peer.ValueIs("easy double", "doo")); EXPECT_TRUE(peer.ValueIs("harder double", "\\x3cA HREF\\x3d\\x27foo\\x27\\x3e\\n")); EXPECT_TRUE(peer.ValueIs("hardest double", "print \\x22\\x3cA HREF\\x3d\\x27foo\\x27\\x3e\\x22;" "\\r\\n\\\\1;")); } TEST(TemplateDictionary, SetEscapedFormattedValue) { TemplateDictionary dict("test_SetEscapedFormattedValue", NULL); TemplateDictionaryPeer peer(&dict); dict.SetEscapedFormattedValue("HTML", ctemplate::html_escape, "This is <%s> #%.4f", "a & b", 1.0/3); dict.SetEscapedFormattedValue("PRE", ctemplate::pre_escape, "if %s x = %.4f;", "(a < 1 && b > 2)\n\t", 1.0/3); dict.SetEscapedFormattedValue("URL", ctemplate::url_query_escape, "pageviews-%s", "r?egex"); dict.SetEscapedFormattedValue("XML", ctemplate::xml_escape, "This&is%s -- ok?", "just&"); EXPECT_TRUE(peer.ValueIs("HTML", "This is <a & b> #0.3333")); EXPECT_TRUE(peer.ValueIs("PRE", "if (a < 1 && b > 2)\n\t x = 0.3333;")); EXPECT_TRUE(peer.ValueIs("URL", "pageviews-r%3Fegex")); EXPECT_TRUE(peer.ValueIs("XML", "This&isjust& -- ok?")); } static const StaticTemplateString kSectName = STS_INIT(kSectName, "test_SetAddSectionDictionary"); TEST(TemplateDictionary, AddSectionDictionary) { // For fun, we'll make this constructor take a static template string. TemplateDictionary dict(kSectName, NULL); TemplateDictionaryPeer peer(&dict); dict.SetValue("TOPLEVEL", "foo"); dict.SetValue("TOPLEVEL2", "foo2"); TemplateDictionary* subdict_1a = dict.AddSectionDictionary("section1"); // This is the same dict, but name is specified a different way. TemplateDictionary* subdict_1b = dict.AddSectionDictionary( TemplateString("section1__ignored__", strlen("section1"))); TemplateDictionaryPeer subdict_1a_peer(subdict_1a); TemplateDictionaryPeer subdict_1b_peer(subdict_1b); subdict_1a->SetValue("SUBLEVEL", "subfoo"); subdict_1b->SetValue("SUBLEVEL", "subbar"); TemplateDictionary* subdict_2 = dict.AddSectionDictionary("section2"); TemplateDictionaryPeer subdict_2_peer(subdict_2); subdict_2->SetValue("TOPLEVEL", "bar"); // overriding top dict TemplateDictionary* subdict_2_1 = subdict_2->AddSectionDictionary("sub"); TemplateDictionaryPeer subdict_2_1_peer(subdict_2_1); subdict_2_1->SetIntValue("GLOBAL", 21); // overrides value in setUp() // Verify that all variables that should be look-up-able are, and that // we have proper precedence. EXPECT_TRUE(peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(peer.ValueIs("TOPLEVEL", "foo")); EXPECT_TRUE(peer.ValueIs("TOPLEVEL2", "foo2")); EXPECT_TRUE(peer.ValueIs("SUBLEVEL", "")); EXPECT_TRUE(subdict_1a_peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(subdict_1a_peer.ValueIs("TOPLEVEL", "foo")); EXPECT_TRUE(subdict_1a_peer.ValueIs("TOPLEVEL2", "foo2")); EXPECT_TRUE(subdict_1a_peer.ValueIs("SUBLEVEL", "subfoo")); EXPECT_TRUE(subdict_1b_peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(subdict_1b_peer.ValueIs("TOPLEVEL", "foo")); EXPECT_TRUE(subdict_1b_peer.ValueIs("TOPLEVEL2", "foo2")); EXPECT_TRUE(subdict_1b_peer.ValueIs("SUBLEVEL", "subbar")); EXPECT_TRUE(subdict_2_peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(subdict_2_peer.ValueIs("TOPLEVEL", "bar")); EXPECT_TRUE(subdict_2_peer.ValueIs("TOPLEVEL2", "foo2")); EXPECT_TRUE(subdict_2_peer.ValueIs("SUBLEVEL", "")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("GLOBAL", "21")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("TOPLEVEL", "bar")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("TOPLEVEL2", "foo2")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("SUBLEVEL", "")); // Verify that everyone knows about its sub-dictionaries, and also // that these go 'up the chain' on lookup failure EXPECT_FALSE(peer.IsHiddenSection("section1")); EXPECT_FALSE(peer.IsHiddenSection("section2")); EXPECT_TRUE(peer.IsHiddenSection("section3")); EXPECT_TRUE(peer.IsHiddenSection("sub")); EXPECT_FALSE(subdict_1a_peer.IsHiddenSection("section1")); EXPECT_TRUE(subdict_1a_peer.IsHiddenSection("sub")); EXPECT_FALSE(subdict_2_peer.IsHiddenSection("sub")); EXPECT_FALSE(subdict_2_1_peer.IsHiddenSection("sub")); // We should get the dictionary-lengths right as well vector dummy; EXPECT_EQ(2, peer.GetSectionDictionaries("section1", &dummy)); EXPECT_EQ(1, peer.GetSectionDictionaries("section2", &dummy)); EXPECT_EQ(1, subdict_2_peer.GetSectionDictionaries("sub", &dummy)); // Test some of the values EXPECT_TRUE(TemplateDictionaryPeer(GetSectionDict(&dict, "section1", 0)) .ValueIs("SUBLEVEL", "subfoo")); EXPECT_TRUE(TemplateDictionaryPeer(GetSectionDict(&dict, "section1", 1)) .ValueIs("SUBLEVEL", "subbar")); EXPECT_TRUE(TemplateDictionaryPeer(GetSectionDict(&dict, "section2", 0)) .ValueIs("TOPLEVEL", "bar")); EXPECT_TRUE(TemplateDictionaryPeer( GetSectionDict(GetSectionDict(&dict, "section2", 0), "sub", 0)) .ValueIs("TOPLEVEL", "bar")); EXPECT_TRUE(TemplateDictionaryPeer( GetSectionDict(GetSectionDict(&dict, "section2", 0), "sub", 0)) .ValueIs("GLOBAL", "21")); // Make sure we're making descriptive names EXPECT_STREQ(dict.name().c_str(), "test_SetAddSectionDictionary"); EXPECT_STREQ(subdict_1a->name().c_str(), "test_SetAddSectionDictionary/section1#1"); EXPECT_STREQ(subdict_1b->name().c_str(), "test_SetAddSectionDictionary/section1#2"); EXPECT_STREQ(subdict_2->name().c_str(), "test_SetAddSectionDictionary/section2#1"); EXPECT_STREQ(subdict_2_1->name().c_str(), "test_SetAddSectionDictionary/section2#1/sub#1"); // Finally, we can test the whole kit and kaboodle string dump; dict.DumpToString(&dump); const char* const expected = ("global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" "};\n" "dictionary 'test_SetAddSectionDictionary' {\n" " TOPLEVEL: >foo<\n" " TOPLEVEL2: >foo2<\n" " section section1 (dict 1 of 2) -->\n" " dictionary 'test_SetAddSectionDictionary/section1#1' {\n" " SUBLEVEL: >subfoo<\n" " }\n" " section section1 (dict 2 of 2) -->\n" " dictionary 'test_SetAddSectionDictionary/section1#2' {\n" " SUBLEVEL: >subbar<\n" " }\n" " section section2 (dict 1 of 1) -->\n" " dictionary 'test_SetAddSectionDictionary/section2#1' {\n" " TOPLEVEL: >bar<\n" " section sub (dict 1 of 1) -->\n" " dictionary 'test_SetAddSectionDictionary/section2#1/sub#1' {\n" " GLOBAL: >21<\n" " }\n" " }\n" "}\n"); EXPECT_STREQ(dump.c_str(), expected); } TEST(TemplateDictionary, ShowSection) { TemplateDictionary dict("test_SetShowSection", NULL); // Let's say what filename dict is associated with dict.SetFilename("bigmamainclude!.tpl"); dict.SetValue("TOPLEVEL", "foo"); dict.SetValue("TOPLEVEL2", "foo2"); dict.ShowSection("section1"); dict.ShowSection("section2"); // Test calling ShowSection twice on the same section dict.ShowSection("section2"); // Test that ShowSection is a no-op if called after AddSectionDictionary() TemplateDictionary* subdict = dict.AddSectionDictionary("section3"); TemplateDictionaryPeer subdict_peer(subdict); subdict->SetValue("TOPLEVEL", "bar"); dict.ShowSection("section3"); EXPECT_TRUE(subdict_peer.ValueIs("TOPLEVEL", "bar")); // Since ShowSection() doesn't return a sub-dict, the only way to // probe what the dicts look like is via Dump() string dump; dict.DumpToString(&dump); const char* const expected = ("global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" "};\n" "dictionary 'test_SetShowSection (intended for bigmamainclude!.tpl)' {\n" " TOPLEVEL: >foo<\n" " TOPLEVEL2: >foo2<\n" " section section1 (dict 1 of 1) -->\n" " dictionary 'empty dictionary' {\n" " }\n" " section section2 (dict 1 of 1) -->\n" " dictionary 'empty dictionary' {\n" " }\n" " section section3 (dict 1 of 1) -->\n" " dictionary 'test_SetShowSection/section3#1' {\n" " TOPLEVEL: >bar<\n" " }\n" "}\n"); EXPECT_STREQ(dump.c_str(), expected); } TEST(TemplateDictionary, SetValueAndShowSection) { TemplateDictionary dict("test_SetValueAndShowSection"); TemplateDictionaryPeer peer(&dict); dict.SetValue("TOPLEVEL", "foo"); dict.SetValueAndShowSection("INSEC", "bar", "SEC1"); dict.SetValueAndShowSection("NOTINSEC", "", "SEC2"); dict.SetValueAndShowSection("NOTINSEC2", NULL, "SEC3"); EXPECT_FALSE(peer.IsHiddenSection("SEC1")); EXPECT_TRUE(peer.IsHiddenSection("SEC2")); EXPECT_TRUE(peer.IsHiddenSection("SEC3")); // Again, we don't get subdicts, so we have to dump to check values string dump; dict.DumpToString(&dump); const char* const expected = ("global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" "};\n" "dictionary 'test_SetValueAndShowSection' {\n" " TOPLEVEL: >foo<\n" " section SEC1 (dict 1 of 1) -->\n" " dictionary 'test_SetValueAndShowSection/SEC1#1' {\n" " INSEC: >bar<\n" " }\n" "}\n"); EXPECT_STREQ(dump.c_str(), expected); } TEST(TemplateDictionary, SetTemplateGlobalValue) { // The functionality involving it passing across the included dictionaries // is also tested in TestAddIncludeDictionary TemplateDictionary dict("test_SetTemplateGlobalValue", NULL); TemplateDictionary* subdict = dict.AddSectionDictionary("section1"); TemplateDictionary* subsubdict = subdict->AddSectionDictionary("section1's child"); TemplateDictionary* includedict = dict.AddIncludeDictionary("include1"); TemplateDictionaryPeer peer(&dict); TemplateDictionaryPeer subdict_peer(subdict); TemplateDictionaryPeer subsubdict_peer(subsubdict); TemplateDictionaryPeer includedict_peer(includedict); // Setting a template value after sub dictionaries are created should // affect the sub dictionaries as well. dict.SetTemplateGlobalValue("TEMPLATEVAL", "templateval"); EXPECT_TRUE(peer.ValueIs("TEMPLATEVAL", "templateval")); EXPECT_TRUE(subdict_peer.ValueIs("TEMPLATEVAL", "templateval")); EXPECT_TRUE(subsubdict_peer.ValueIs("TEMPLATEVAL", "templateval")); EXPECT_TRUE(includedict_peer.ValueIs("TEMPLATEVAL", "templateval")); // sub dictionaries after you set the template value should also // get the template value TemplateDictionary* subdict2 = dict.AddSectionDictionary("section2"); TemplateDictionary* includedict2 = dict.AddIncludeDictionary("include2"); TemplateDictionaryPeer subdict2_peer(subdict2); TemplateDictionaryPeer includedict2_peer(includedict2); EXPECT_TRUE(subdict2_peer.ValueIs("TEMPLATEVAL", "templateval")); EXPECT_TRUE(includedict2_peer.ValueIs("TEMPLATEVAL", "templateval")); // setting a template value on a sub dictionary should affect all the other // sub dictionaries and the parent as well subdict->SetTemplateGlobalValue("TEMPLATEVAL2", "templateval2"); EXPECT_TRUE(peer.ValueIs("TEMPLATEVAL2", "templateval2")); EXPECT_TRUE(subdict_peer.ValueIs("TEMPLATEVAL2", "templateval2")); EXPECT_TRUE(subsubdict_peer.ValueIs("TEMPLATEVAL2", "templateval2")); EXPECT_TRUE(includedict_peer.ValueIs("TEMPLATEVAL2", "templateval2")); EXPECT_TRUE(subdict2_peer.ValueIs("TEMPLATEVAL2", "templateval2")); EXPECT_TRUE(includedict2_peer.ValueIs("TEMPLATEVAL2", "templateval2")); includedict->SetTemplateGlobalValue("TEMPLATEVAL3", "templateval3"); EXPECT_TRUE(peer.ValueIs("TEMPLATEVAL3", "templateval3")); EXPECT_TRUE(subdict_peer.ValueIs("TEMPLATEVAL3", "templateval3")); EXPECT_TRUE(subsubdict_peer.ValueIs("TEMPLATEVAL3", "templateval3")); EXPECT_TRUE(includedict_peer.ValueIs("TEMPLATEVAL3", "templateval3")); EXPECT_TRUE(subdict2_peer.ValueIs("TEMPLATEVAL3", "templateval3")); EXPECT_TRUE(includedict2_peer.ValueIs("TEMPLATEVAL3", "templateval3")); // you should be able to override a template value with a regular value // and the overwritten regular value should pass on to its children subdict->SetValue("TEMPLATEVAL2", "subdictval"); includedict->SetValue("TEMPLATEVAL2", "includedictval"); EXPECT_TRUE(peer.ValueIs("TEMPLATEVAL2", "templateval2")); EXPECT_TRUE(subdict_peer.ValueIs("TEMPLATEVAL2", "subdictval")); EXPECT_TRUE(subsubdict_peer.ValueIs("TEMPLATEVAL2", "subdictval")); EXPECT_TRUE(includedict_peer.ValueIs("TEMPLATEVAL2", "includedictval")); EXPECT_TRUE(subdict2_peer.ValueIs("TEMPLATEVAL2", "templateval2")); EXPECT_TRUE(includedict2_peer.ValueIs("TEMPLATEVAL2", "templateval2")); // A section shown template-globally will be shown in all its children. dict.ShowTemplateGlobalSection("ShownTemplateGlobalSection"); EXPECT_FALSE(peer.IsHiddenSection("ShownTemplateGlobalSection")); EXPECT_FALSE(subdict2_peer.IsHiddenSection("ShownTemplateGlobalSection")); EXPECT_FALSE(subsubdict_peer.IsHiddenSection("ShownTemplateGlobalSection")); // Showing a template-global section in a child will show it in all templates // in the tree subdict->ShowTemplateGlobalSection("ShownFromAChild"); EXPECT_FALSE(peer.IsHiddenSection("ShownFromAChild")); EXPECT_FALSE(subsubdict_peer.IsHiddenSection("ShownFromAChild")); // Asking for a section that doesn't exist shouldn't cause infinite recursion peer.IsHiddenSection("NAVBAR_SECTION"); } TEST(TemplateDictionary, SetTemplateGlobalValueWithoutCopy) { UnsafeArena arena(100); TemplateDictionary dict("Test arena", &arena); TemplateDictionaryPeer peer(&dict); char value[32]; snprintf(value, sizeof(value), "%s", "value"); const void* const ptr = arena.Alloc(0); dict.SetTemplateGlobalValueWithoutCopy("key", value); // We shouldn't have copied the value string. EXPECT_EQ(ptr, arena.Alloc(0)); EXPECT_TRUE(peer.ValueIs("key", "value")); // If our content changes, so does what's in the dictionary -- but // only the contents of the buffer, not its length! snprintf(value, sizeof(value), "%s", "not_value"); EXPECT_TRUE(peer.ValueIs("key", "not_v")); // "not_v" size == value" size } TEST(TemplateDictionary, AddIncludeDictionary) { TemplateDictionary dict("test_SetAddIncludeDictionary", NULL); TemplateDictionaryPeer peer(&dict); dict.SetValue("TOPLEVEL", "foo"); dict.SetValue("TOPLEVEL2", "foo2"); dict.SetTemplateGlobalValue("TEMPLATELEVEL", "foo3"); TemplateDictionary* subdict_1a = dict.AddIncludeDictionary("include1"); TemplateDictionaryPeer subdict_1a_peer(subdict_1a); subdict_1a->SetFilename("incfile1a"); // This is the same dict, but name is specified a different way. TemplateDictionary* subdict_1b = dict.AddIncludeDictionary( TemplateString("include1__ignored__", strlen("include1"))); TemplateDictionaryPeer subdict_1b_peer(subdict_1b); // Let's try not calling SetFilename on this one. subdict_1a->SetValue("SUBLEVEL", "subfoo"); subdict_1b->SetValue("SUBLEVEL", "subbar"); TemplateDictionary* subdict_2 = dict.AddIncludeDictionary("include2"); TemplateDictionaryPeer subdict_2_peer(subdict_2); subdict_2->SetFilename("foo/bar"); subdict_2->SetValue("TOPLEVEL", "bar"); // overriding top dict // overriding template dict subdict_2->SetValue("TEMPLATELEVEL", "subfoo3"); TemplateDictionary* subdict_2_1 = subdict_2->AddIncludeDictionary("sub"); TemplateDictionaryPeer subdict_2_1_peer(subdict_2_1); subdict_2_1->SetFilename("baz"); subdict_2_1->SetIntValue("GLOBAL", 21); // overrides value in setUp() // Verify that all variables that should be look-up-able are, and that // we have proper precedence. Unlike with sections, includes lookups // do not go 'up the chain'. EXPECT_TRUE(peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(peer.ValueIs("TOPLEVEL", "foo")); EXPECT_TRUE(peer.ValueIs("TOPLEVEL2", "foo2")); EXPECT_TRUE(peer.ValueIs("TEMPLATELEVEL", "foo3")); EXPECT_TRUE(peer.ValueIs("SUBLEVEL", "")); EXPECT_TRUE(subdict_1a_peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(subdict_1a_peer.ValueIs("TOPLEVEL", "")); EXPECT_TRUE(subdict_1a_peer.ValueIs("TOPLEVEL2", "")); EXPECT_TRUE(subdict_1a_peer.ValueIs("TEMPLATELEVEL", "foo3")); EXPECT_TRUE(subdict_1a_peer.ValueIs("SUBLEVEL", "subfoo")); EXPECT_TRUE(subdict_1b_peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(subdict_1b_peer.ValueIs("TOPLEVEL", "")); EXPECT_TRUE(subdict_1b_peer.ValueIs("TOPLEVEL2", "")); EXPECT_TRUE(subdict_1b_peer.ValueIs("SUBLEVEL", "subbar")); EXPECT_TRUE(subdict_2_peer.ValueIs("GLOBAL", "top")); EXPECT_TRUE(subdict_2_peer.ValueIs("TOPLEVEL", "bar")); EXPECT_TRUE(subdict_2_peer.ValueIs("TOPLEVEL2", "")); EXPECT_TRUE(subdict_2_peer.ValueIs("TEMPLATELEVEL", "subfoo3")); EXPECT_TRUE(subdict_2_peer.ValueIs("SUBLEVEL", "")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("GLOBAL", "21")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("TOPLEVEL", "")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("TOPLEVEL2", "")); EXPECT_TRUE(subdict_2_1_peer.ValueIs("SUBLEVEL", "")); // Verify that everyone knows about its sub-dictionaries, but that // these do not try to go 'up the chain' on lookup failure EXPECT_FALSE(peer.IsHiddenTemplate("include1")); EXPECT_FALSE(peer.IsHiddenTemplate("include2")); EXPECT_TRUE(peer.IsHiddenTemplate("include3")); EXPECT_TRUE(peer.IsHiddenTemplate("sub")); EXPECT_TRUE(subdict_1a_peer.IsHiddenTemplate("include1")); EXPECT_TRUE(subdict_1a_peer.IsHiddenTemplate("sub")); EXPECT_FALSE(subdict_2_peer.IsHiddenTemplate("sub")); EXPECT_TRUE(subdict_2_1_peer.IsHiddenTemplate("sub")); // We should get the dictionary-lengths right as well vector dummy; EXPECT_EQ(2, peer.GetIncludeDictionaries("include1", &dummy)); EXPECT_EQ(1, peer.GetIncludeDictionaries("include2", &dummy)); EXPECT_EQ(1, subdict_2_peer.GetIncludeDictionaries("sub", &dummy)); // We can also test the include-files are right EXPECT_EQ(2, peer.GetIncludeDictionaries("include1", &dummy)); EXPECT_EQ(1, peer.GetIncludeDictionaries("include2", &dummy)); EXPECT_EQ(1, subdict_2_peer.GetIncludeDictionaries("sub", &dummy)); // Test some of the values EXPECT_TRUE(TemplateDictionaryPeer(GetIncludeDict(&dict, "include1", 0)) .ValueIs("SUBLEVEL", "subfoo")); EXPECT_TRUE(TemplateDictionaryPeer(GetIncludeDict(&dict, "include1", 1)) .ValueIs("SUBLEVEL", "subbar")); EXPECT_TRUE(TemplateDictionaryPeer(GetIncludeDict(&dict, "include2", 0)) .ValueIs("TOPLEVEL", "bar")); EXPECT_TRUE(TemplateDictionaryPeer( GetIncludeDict(GetIncludeDict(&dict, "include2", 0), "sub", 0)) .ValueIs("TOPLEVEL", "")); EXPECT_TRUE(TemplateDictionaryPeer( GetIncludeDict(GetIncludeDict(&dict, "include2", 0), "sub", 0)) .ValueIs("GLOBAL", "21")); // We can test the include-names as well EXPECT_STREQ(peer.GetIncludeTemplateName("include1", 0), "incfile1a"); EXPECT_STREQ(peer.GetIncludeTemplateName("include1", 1), ""); EXPECT_STREQ(peer.GetIncludeTemplateName("include2", 0), "foo/bar"); EXPECT_STREQ(TemplateDictionaryPeer(GetIncludeDict(&dict, "include2", 0)) .GetIncludeTemplateName("sub", 0), "baz"); // Make sure we're making descriptive names EXPECT_STREQ(dict.name().c_str(), "test_SetAddIncludeDictionary"); EXPECT_STREQ(subdict_1a->name().c_str(), "test_SetAddIncludeDictionary/include1#1"); EXPECT_STREQ(subdict_1b->name().c_str(), "test_SetAddIncludeDictionary/include1#2"); EXPECT_STREQ(subdict_2->name().c_str(), "test_SetAddIncludeDictionary/include2#1"); EXPECT_STREQ(subdict_2_1->name().c_str(), "test_SetAddIncludeDictionary/include2#1/sub#1"); // Finally, we can test the whole kit and kaboodle string dump; dict.DumpToString(&dump); const char* const expected = ("global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" "};\n" "template dictionary {\n" " TEMPLATELEVEL: >foo3<\n" "};\n" "dictionary 'test_SetAddIncludeDictionary' {\n" " TOPLEVEL: >foo<\n" " TOPLEVEL2: >foo2<\n" " include-template include1 (dict 1 of 2, from incfile1a) -->\n" " global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" " };\n" " dictionary 'test_SetAddIncludeDictionary/include1#1 (intended for incfile1a)' {\n" " SUBLEVEL: >subfoo<\n" " }\n" " include-template include1 (dict 2 of 2, **NO FILENAME SET; THIS DICT WILL BE IGNORED**) -->\n" " global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" " };\n" " dictionary 'test_SetAddIncludeDictionary/include1#2' {\n" " SUBLEVEL: >subbar<\n" " }\n" " include-template include2 (dict 1 of 1, from foo/bar) -->\n" " global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" " };\n" " dictionary 'test_SetAddIncludeDictionary/include2#1 (intended for foo/bar)' {\n" " TEMPLATELEVEL: >subfoo3<\n" " TOPLEVEL: >bar<\n" " include-template sub (dict 1 of 1, from baz) -->\n" " global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" " };\n" " dictionary 'test_SetAddIncludeDictionary/include2#1/sub#1 (intended for baz)' {\n" " GLOBAL: >21<\n" " }\n" " }\n" "}\n"); EXPECT_STREQ(dump.c_str(), expected); } static void TestMakeCopy(bool use_local_arena) { UnsafeArena local_arena(1024); UnsafeArena* arena = NULL; if (use_local_arena) arena = &local_arena; // First, let's make a non-trivial template dictionary (We use // 'new' because later we'll test deleting this dict but keeping // around the copy.) TemplateDictionary* dict = new TemplateDictionary("testdict", arena); dict->SetValue("TOPLEVEL", "foo"); dict->SetTemplateGlobalValue("TEMPLATELEVEL", "foo3"); TemplateDictionary* subdict_1a = dict->AddIncludeDictionary("include1"); subdict_1a->SetFilename("incfile1a"); subdict_1a->SetValue("SUBLEVEL", "subfoo"); TemplateDictionary* subdict_1b = dict->AddIncludeDictionary("include1"); // Let's try not calling SetFilename on this one. subdict_1b->SetValue("SUBLEVEL", "subbar"); TemplateDictionary* subdict_2a = dict->AddSectionDictionary("section1"); TemplateDictionary* subdict_2b = dict->AddSectionDictionary("section1"); subdict_2a->SetValue("SUBLEVEL", "subfoo"); subdict_2b->SetValue("SUBLEVEL", "subbar"); TemplateDictionary* subdict_3 = dict->AddSectionDictionary("section2"); subdict_3->SetValue("TOPLEVEL", "bar"); // overriding top dict TemplateDictionary* subdict_3_1 = subdict_3->AddSectionDictionary("sub"); subdict_3_1->SetIntValue("GLOBAL", 21); // overrides value in setUp() string orig; dict->DumpToString(&orig); // Make a copy TemplateDictionary* dict_copy = dict->MakeCopy("testdict", NULL); // Make sure it doesn't work to copy a sub-dictionary EXPECT_TRUE(subdict_1a->MakeCopy("copy of subdict") == NULL); EXPECT_TRUE(subdict_2a->MakeCopy("copy of subdict") == NULL); // Delete the original dict, to make sure the copy really is independent delete dict; dict = NULL; string copy; dict_copy->DumpToString(©); delete dict_copy; EXPECT_STREQ(orig.c_str(), copy.c_str()); } TEST(MakeCopy, UseLocalArena) { TestMakeCopy(true); } TEST(MakeCopy, DoNotUseLocalArena) { TestMakeCopy(false); } TEST(TemplateDictionary, SetModifierData) { PerExpandData per_expand_data; const void* data = "test"; per_expand_data.InsertForModifiers("a", data); EXPECT_EQ(data, per_expand_data.LookupForModifiers("a")); } TEST(TemplateDictionary, Iterator) { // Build up a nice community of TemplateDictionaries. TemplateDictionary farm("Farm"); TemplateDictionaryPeer farm_peer(&farm); TemplateDictionaryInterface* grey_barn = farm.AddIncludeDictionary("BARN"); TemplateDictionaryInterface* duck_pond = farm.AddIncludeDictionary("POND"); TemplateDictionaryInterface* cattle_pond = farm.AddIncludeDictionary("POND"); TemplateDictionaryInterface* irrigation_pond = farm.AddIncludeDictionary("POND"); // A section name with repeated sections TemplateDictionaryInterface* lillies = farm.AddSectionDictionary("FLOWERS"); TemplateDictionaryInterface* lilacs = farm.AddSectionDictionary("FLOWERS"); TemplateDictionaryInterface* daisies = farm.AddSectionDictionary("FLOWERS"); // A section name with one repeat TemplateDictionaryInterface* wheat = farm.AddSectionDictionary("WHEAT"); // A section name, just shown farm.ShowSection("CORN"); // Check that the iterators expose all of the dictionaries. TemplateDictionaryPeer::Iterator* barns = farm_peer.CreateTemplateIterator("BARN"); EXPECT_TRUE(barns->HasNext()); EXPECT_EQ(&barns->Next(), grey_barn); EXPECT_FALSE(barns->HasNext()); delete barns; TemplateDictionaryPeer::Iterator* ponds = farm_peer.CreateTemplateIterator("POND"); EXPECT_TRUE(ponds->HasNext()); EXPECT_EQ(&ponds->Next(), duck_pond); EXPECT_TRUE(ponds->HasNext()); EXPECT_EQ(&ponds->Next(), cattle_pond); EXPECT_TRUE(ponds->HasNext()); EXPECT_EQ(&ponds->Next(), irrigation_pond); EXPECT_FALSE(ponds->HasNext()); delete ponds; TemplateDictionaryPeer::Iterator* flowers = farm_peer.CreateSectionIterator("FLOWERS"); EXPECT_TRUE(flowers->HasNext()); EXPECT_EQ(&flowers->Next(), lillies); EXPECT_TRUE(flowers->HasNext()); EXPECT_EQ(&flowers->Next(), lilacs); EXPECT_TRUE(flowers->HasNext()); EXPECT_EQ(&flowers->Next(), daisies); EXPECT_FALSE(flowers->HasNext()); delete flowers; TemplateDictionaryPeer::Iterator* crop = farm_peer.CreateSectionIterator("WHEAT"); EXPECT_TRUE(crop->HasNext()); EXPECT_EQ(&crop->Next(), wheat); EXPECT_FALSE(crop->HasNext()); delete crop; TemplateDictionaryPeer::Iterator* corn_crop = farm_peer.CreateSectionIterator("CORN"); EXPECT_TRUE(corn_crop->HasNext()); EXPECT_TRUE(&corn_crop->Next()); // ShowSection doesn't give us the dict back EXPECT_FALSE(corn_crop->HasNext()); delete corn_crop; } TEST(TemplateDictionary, IsHiddenSectionDefault) { TemplateDictionary dict("dict"); TemplateDictionaryPeer peer(&dict); EXPECT_TRUE(peer.IsHiddenSection("UNDEFINED")); EXPECT_FALSE(peer.IsUnhiddenSection("UNDEFINED")); dict.ShowSection("VISIBLE"); EXPECT_FALSE(peer.IsHiddenSection("VISIBLE")); EXPECT_TRUE(peer.IsUnhiddenSection("VISIBLE")); } // This has to run last, since its SetGlobalValue modifies the global // state, which can affect other tests (especially given the embedded // NUL!) So we don't use the normal TEST() here, and call it manually // in main(). void TestSetValueWithNUL() { TemplateDictionary dict("test_SetValueWithNUL", NULL); TemplateDictionaryPeer peer(&dict); // Test copying char*s, strings, and explicit TemplateStrings dict.SetValue(string("FOO\0BAR", 7), string("QUX\0QUUX", 8)); dict.SetGlobalValue(string("GOO\0GAR", 7), string("GUX\0GUUX", 8)); // FOO should not match FOO\0BAR EXPECT_TRUE(peer.ValueIs("FOO", "")); EXPECT_TRUE(peer.ValueIs("GOO", "")); EXPECT_TRUE(peer.ValueIs(string("FOO\0BAR", 7), string("QUX\0QUUX", 8))); EXPECT_TRUE(peer.ValueIs(string("GOO\0GAR", 7), string("GUX\0GUUX", 8))); string dump; dict.DumpToString(&dump); // We can't use EXPECT_STREQ here because of the embedded NULs. // They also require I count the length of the string by hand. :-( string expected(("global dictionary {\n" " BI_NEWLINE: >\n" "<\n" " BI_SPACE: > <\n" " GLOBAL: >top<\n" " GOO\0GAR: >GUX\0GUUX<\n" "};\n" "dictionary 'test_SetValueWithNUL' {\n" " FOO\0BAR: >QUX\0QUUX<\n" "}\n"), 160); EXPECT_EQ(dump, expected); } TEST(TemplateDictionary, TestShowTemplateGlobalSection) { StringToTemplateCache("test.tpl", "{{#sect}}OK{{/sect}}", DO_NOT_STRIP); TemplateDictionary dict("mydict"); dict.ShowTemplateGlobalSection("sect"); string out; ExpandTemplate("test.tpl", DO_NOT_STRIP, &dict, &out); } TEST(TemplateDictionary, TestShowTemplateGlobalSection_Child) { // The TemplateDictionary::template_global_dict_ behaves differently for child // dictionaries than for the root parent dictionary. StringToTemplateCache("test2.tpl", "{{#foo}}{{#sect}}OK{{/sect}}{{/foo}}", DO_NOT_STRIP); TemplateDictionary dict("mydict"); dict.ShowTemplateGlobalSection("sect"); dict.AddSectionDictionary("foo"); string out; ExpandTemplate("test2.tpl", DO_NOT_STRIP, &dict, &out); } TEST(TemplateDictionary, TestShowTemplateGlobalSection_SectionDoesntExist) { StringToTemplateCache("test3.tpl", "{{#bad}}bad{{/bad}}", DO_NOT_STRIP); TemplateDictionary dict("mydict"); string out; ExpandTemplate("test3.tpl", DO_NOT_STRIP, &dict, &out); } } // unnamed namespace int main(int argc, char** argv) { SetUp(); const int retval = RUN_ALL_TESTS(); // This has to run last, so we run it manually TestSetValueWithNUL(); return retval; } ctemplate-ctemplate-2.4/src/tests/template_modifiers_unittest.cc000077500000000000000000001512701363122316600254000ustar00rootroot00000000000000// Copyright (c) 2007, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // This code is written to not use the google testing framework // as much as possible, to make it easier to opensource. #include "config_for_unittests.h" #include #include #include #include #include #include #include #include #include #include "template_modifiers_internal.h" #include "tests/template_test_util.h" #include "base/util.h" TEST_INIT // defines RUN_ALL_TESTS using std::string; using std::vector; // Rather than put all these tests in the ctemplate namespace, or use // using-declarations, for this test I've decided to manually prepend // ctemplate:: everywhere it's needed. This test can serve as an // example of how that approach looks. TEST(TemplateModifiers, HtmlEscape) { ctemplate::TemplateDictionary dict("TestHtmlEscape", NULL); dict.SetEscapedValue("easy HTML", "foo", ctemplate::html_escape); dict.SetEscapedValue("harder HTML", "foo & bar", ctemplate::html_escape); dict.SetEscapedValue("hardest HTML", "", ctemplate::html_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts // TODO(csilvers): change this (and all other expect_*'s in all files // in this directory) to take the expected value first, not second. EXPECT_STREQ(peer.GetSectionValue("easy HTML"), "foo"); EXPECT_STREQ(peer.GetSectionValue("harder HTML"), "foo & bar"); EXPECT_STREQ(peer.GetSectionValue("hardest HTML"), "<A HREF='foo' id="bar && " "baz">"); } TEST(TemplateModifiers, SnippetEscape) { ctemplate::TemplateDictionary dict("TestSnippetEscape", NULL); dict.SetEscapedValue("easy snippet", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("valid snippet", "foo
& b­ar", ctemplate::snippet_escape); dict.SetEscapedValue("invalid snippet", "", ctemplate::snippet_escape); dict.SetEscapedValue("snippet with italics", "foo
& b­ar", ctemplate::snippet_escape); dict.SetEscapedValue("unclosed snippet", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("snippet with interleaving", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("unclosed interleaving", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("unclosed", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("unterminated 1", "foo<", ctemplate::snippet_escape); dict.SetEscapedValue("unterminated 2", "foofoofoo", ctemplate::snippet_escape); dict.SetEscapedValue("close i b", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("em", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("nested em", "This is foo...", ctemplate::snippet_escape); dict.SetEscapedValue("unclosed em", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("wrongly closed em", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("misnested em", "foo", ctemplate::snippet_escape); dict.SetEscapedValue("span ltr", "bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("span rtl", "bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("span garbage dir attr", "bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("span no dir", "bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("span bad attribute", "bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("span quotes", "bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("nested span", "This is bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("doubly-nested span", "This is " "bidi text", ctemplate::snippet_escape); dict.SetEscapedValue("two spans", "This is text that is " "bidi.", ctemplate::snippet_escape); dict.SetEscapedValue("unclosed span", "This is bidi text", ctemplate::snippet_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy snippet"), "foo"); EXPECT_STREQ(peer.GetSectionValue("valid snippet"), "foo
& b­ar"); EXPECT_STREQ(peer.GetSectionValue("invalid snippet"), "<A HREF='foo' id="bar &&{ " "baz">"); EXPECT_STREQ(peer.GetSectionValue("snippet with italics"), "foo
& b­ar"); EXPECT_STREQ(peer.GetSectionValue("unclosed snippet"), "foo"); EXPECT_STREQ(peer.GetSectionValue("snippet with interleaving"), "foo"); EXPECT_STREQ(peer.GetSectionValue("unclosed interleaving"), "<b>foo"); EXPECT_STREQ(peer.GetSectionValue("unclosed"), "foo"); EXPECT_STREQ(peer.GetSectionValue("unterminated 1"), "foo<"); EXPECT_STREQ(peer.GetSectionValue("unterminated 2"), "foo<b"); EXPECT_STREQ(peer.GetSectionValue("unterminated 3"), "foo</"); EXPECT_STREQ(peer.GetSectionValue("unterminated 4"), "foo</b"); EXPECT_STREQ(peer.GetSectionValue("unterminated 5"), "foo</b"); EXPECT_STREQ(peer.GetSectionValue("close b i"), "foo"); EXPECT_STREQ(peer.GetSectionValue("close i b"), "foo"); EXPECT_STREQ(peer.GetSectionValue("em"), "foo"); EXPECT_STREQ(peer.GetSectionValue("nested em"), "This is foo..."); EXPECT_STREQ(peer.GetSectionValue("unclosed em"), "foo"); EXPECT_STREQ(peer.GetSectionValue("wrongly closed em"), "foo</em>"); EXPECT_STREQ(peer.GetSectionValue("misnested em"), "foo"); EXPECT_STREQ(peer.GetSectionValue("span ltr"), "bidi text"); EXPECT_STREQ(peer.GetSectionValue("span rtl"), "bidi text"); EXPECT_STREQ(peer.GetSectionValue("span garbage dir attr"), "<span dir=foo>bidi text</span>"); EXPECT_STREQ(peer.GetSectionValue("span no dir"), "<span>bidi text</span>"); EXPECT_STREQ(peer.GetSectionValue("span bad attribute"), "<span onclick=alert('foo')>bidi text</span>"); EXPECT_STREQ(peer.GetSectionValue("span quotes"), "<span dir="rtl">bidi text</span>"); EXPECT_STREQ(peer.GetSectionValue("nested span"), "This is bidi text"); EXPECT_STREQ(peer.GetSectionValue("doubly-nested span"), "This is <span dir=rtl>bidi text" "</span>"); EXPECT_STREQ(peer.GetSectionValue("two spans"), "This is text that is " "bidi."); EXPECT_STREQ(peer.GetSectionValue("unclosed span"), "This is bidi text"); } TEST(TemplateModifiers, PreEscape) { ctemplate::TemplateDictionary dict("TestPreEscape", NULL); dict.SetEscapedValue("easy PRE", "foo", ctemplate::pre_escape); dict.SetEscapedValue("harder PRE", "foo & bar", ctemplate::pre_escape); dict.SetEscapedValue("hardest PRE", " \"--\v--\f--\n--\t--&--<-->--'--\"", ctemplate::pre_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy PRE"), "foo"); EXPECT_STREQ(peer.GetSectionValue("harder PRE"), "foo & bar"); EXPECT_STREQ(peer.GetSectionValue("hardest PRE"), " "--\v--\f--\n--\t--&--<-->--'--""); } TEST(TemplateModifiers, XmlEscape) { ctemplate::TemplateDictionary dict("TestXmlEscape", NULL); dict.SetEscapedValue("no XML", "", ctemplate::xml_escape); dict.SetEscapedValue("easy XML", "xoo", ctemplate::xml_escape); dict.SetEscapedValue("harder XML-1", "<>&'\"", ctemplate::xml_escape); dict.SetEscapedValue("harder XML-2", "Hello", ctemplate::xml_escape); dict.SetEscapedValue("hardest XML", "<>&!''\"\"foo", ctemplate::xml_escape); // Characters 0x00-0x1F (except \t, \r and \n) are not valid for XML and // compliant parsers are allowed to die when they encounter them. They // should be replaced with spaces. dict.SetEscapedValue("Spacey XML", " \r\n\f", ctemplate::xml_escape); dict.SetEscapedValue("XML with control chars", "\x01\x02\x03\x09\x0A\x0B\x0D\x15\x16\x1F", ctemplate::xml_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("no XML"), ""); EXPECT_STREQ(peer.GetSectionValue("easy XML"), "xoo"); EXPECT_STREQ(peer.GetSectionValue("harder XML-1"), "<>&'""); EXPECT_STREQ(peer.GetSectionValue("harder XML-2"), "Hello<script>alert('&')</script>"); EXPECT_STREQ(peer.GetSectionValue("hardest XML"), "<<b>>&!''""foo"); EXPECT_STREQ(peer.GetSectionValue("Spacey XML"), " \r\n "); EXPECT_STREQ(peer.GetSectionValue("XML with control chars"), " \t\n \r "); } TEST(TemplateModifiers, ValidateUrlHtmlEscape) { ctemplate::TemplateDictionary dict("TestValidateUrlHtmlEscape", NULL); dict.SetEscapedValue("easy http URL", "http://www.google.com", ctemplate::validate_url_and_html_escape); dict.SetEscapedValue("harder https URL", "https://www.google.com/search?q=f&hl=en", ctemplate::validate_url_and_html_escape); dict.SetEscapedValue("easy javascript URL", "javascript:alert(document.cookie)", ctemplate::validate_url_and_html_escape); dict.SetEscapedValue("harder javascript URL", "javascript:alert(10/5)", ctemplate::validate_url_and_html_escape); dict.SetEscapedValue("easy relative URL", "foobar.html", ctemplate::validate_url_and_html_escape); dict.SetEscapedValue("harder relative URL", "/search?q=green flowers&hl=en", ctemplate::validate_url_and_html_escape); dict.SetEscapedValue("ftp URL", "ftp://ftp.example.org/pub/file.txt", ctemplate::validate_url_and_html_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy http URL"), "http://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("harder https URL"), "https://www.google.com/search?q=f&hl=en"); EXPECT_STREQ(peer.GetSectionValue("easy javascript URL"), ctemplate::ValidateUrl::kUnsafeUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("harder javascript URL"), ctemplate::ValidateUrl::kUnsafeUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("easy relative URL"), "foobar.html"); EXPECT_STREQ(peer.GetSectionValue("harder relative URL"), "/search?q=green flowers&hl=en"); EXPECT_STREQ(peer.GetSectionValue("ftp URL"), "ftp://ftp.example.org/pub/file.txt"); } TEST(TemplateModifiers, ValidateImgSrcUrlHtmlEscape) { ctemplate::TemplateDictionary dict("TestValidateImgSrcUrlHtmlEscape", NULL); dict.SetEscapedValue("easy http URL", "http://www.google.com", ctemplate::validate_img_src_url_and_html_escape); dict.SetEscapedValue("harder https URL", "https://www.google.com/search?q=f&hl=en", ctemplate::validate_img_src_url_and_html_escape); dict.SetEscapedValue("easy javascript URL", "javascript:alert(document.cookie)", ctemplate::validate_img_src_url_and_html_escape); dict.SetEscapedValue("harder javascript URL", "javascript:alert(10/5)", ctemplate::validate_img_src_url_and_html_escape); dict.SetEscapedValue("easy relative URL", "foobar.html", ctemplate::validate_img_src_url_and_html_escape); dict.SetEscapedValue("harder relative URL", "/search?q=green flowers&hl=en", ctemplate::validate_img_src_url_and_html_escape); dict.SetEscapedValue("ftp URL", "ftp://ftp.example.org/pub/file.txt", ctemplate::validate_img_src_url_and_html_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy http URL"), "http://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("harder https URL"), "https://www.google.com/search?q=f&hl=en"); EXPECT_STREQ(peer.GetSectionValue("easy javascript URL"), ctemplate::ValidateUrl::kUnsafeImgSrcUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("harder javascript URL"), ctemplate::ValidateUrl::kUnsafeImgSrcUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("easy relative URL"), "foobar.html"); EXPECT_STREQ(peer.GetSectionValue("harder relative URL"), "/search?q=green flowers&hl=en"); EXPECT_STREQ(peer.GetSectionValue("ftp URL"), "ftp://ftp.example.org/pub/file.txt"); } TEST(TemplateModifiers, ValidateUrlJavascriptEscape) { ctemplate::TemplateDictionary dict("TestValidateUrlJavascriptEscape", NULL); dict.SetEscapedValue( "easy http URL", "http://www.google.com", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "harder https URL", "https://www.google.com/search?q=f&hl=en", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "mangled http URL", "HTTP://www.google.com", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "easy javascript URL", "javascript:alert(document.cookie)", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "harder javascript URL", "javascript:alert(10/5)", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "easy relative URL", "foobar.html", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "harder relative URL", "/search?q=green flowers&hl=en", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "data URL", "data: text/html", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "mangled javascript URL", "javaSCRIPT:alert(5)", ctemplate::validate_url_and_javascript_escape); dict.SetEscapedValue( "harder mangled javascript URL", "java\nSCRIPT:alert(5)", ctemplate::validate_url_and_javascript_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy http URL"), "http://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("harder https URL"), "https://www.google.com/search?q\\x3df\\x26hl\\x3den"); EXPECT_STREQ(peer.GetSectionValue("mangled http URL"), "HTTP://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("easy javascript URL"), "#"); EXPECT_STREQ(peer.GetSectionValue("harder javascript URL"), ctemplate::ValidateUrl::kUnsafeUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("easy relative URL"), "foobar.html"); EXPECT_STREQ(peer.GetSectionValue("harder relative URL"), "/search?q\\x3dgreen flowers\\x26hl\\x3den"); EXPECT_STREQ(peer.GetSectionValue("data URL"), ctemplate::ValidateUrl::kUnsafeUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("mangled javascript URL"), ctemplate::ValidateUrl::kUnsafeUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("harder mangled javascript URL"), ctemplate::ValidateUrl::kUnsafeUrlReplacement); } TEST(TemplateModifiers, ValidateImgSrcUrlJavascriptEscape) { ctemplate::TemplateDictionary dict("TestValidateImgSrcUrlJavascriptEscape", NULL); dict.SetEscapedValue( "easy http URL", "http://www.google.com", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "harder https URL", "https://www.google.com/search?q=f&hl=en", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "mangled http URL", "HTTP://www.google.com", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "easy javascript URL", "javascript:alert(document.cookie)", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "harder javascript URL", "javascript:alert(10/5)", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "easy relative URL", "foobar.html", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "harder relative URL", "/search?q=green flowers&hl=en", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "data URL", "data: text/html", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "mangled javascript URL", "javaSCRIPT:alert(5)", ctemplate::validate_img_src_url_and_javascript_escape); dict.SetEscapedValue( "harder mangled javascript URL", "java\nSCRIPT:alert(5)", ctemplate::validate_img_src_url_and_javascript_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy http URL"), "http://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("harder https URL"), "https://www.google.com/search?q\\x3df\\x26hl\\x3den"); EXPECT_STREQ(peer.GetSectionValue("mangled http URL"), "HTTP://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("easy javascript URL"), ctemplate::ValidateUrl::kUnsafeImgSrcUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("harder javascript URL"), ctemplate::ValidateUrl::kUnsafeImgSrcUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("easy relative URL"), "foobar.html"); EXPECT_STREQ(peer.GetSectionValue("harder relative URL"), "/search?q\\x3dgreen flowers\\x26hl\\x3den"); EXPECT_STREQ(peer.GetSectionValue("data URL"), "/images/cleardot.gif"); EXPECT_STREQ(peer.GetSectionValue("mangled javascript URL"), ctemplate::ValidateUrl::kUnsafeImgSrcUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("harder mangled javascript URL"), ctemplate::ValidateUrl::kUnsafeImgSrcUrlReplacement); } TEST(TemplateModifiers, ValidateUrlCssEscape) { ctemplate::TemplateDictionary dict("TestValidateUrlCssEscape", NULL); dict.SetEscapedValue("easy http URL", "http://www.google.com", ctemplate::validate_url_and_css_escape); dict.SetEscapedValue("harder https URL", "https://www.google.com/search?q=f&hl=en", ctemplate::validate_url_and_css_escape); dict.SetEscapedValue("javascript URL", "javascript:alert(document.cookie)", ctemplate::validate_url_and_css_escape); dict.SetEscapedValue("relative URL", "/search?q=green flowers&hl=en", ctemplate::validate_url_and_css_escape); dict.SetEscapedValue("hardest URL", "http://www.google.com/s?q='bla'" "&a=\"\"&b=()&c=*\r\n\\\\bla", ctemplate::validate_url_and_css_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy http URL"), "http://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("harder https URL"), "https://www.google.com/search?q=f&hl=en"); EXPECT_STREQ(peer.GetSectionValue("javascript URL"), ctemplate::ValidateUrl::kUnsafeUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("relative URL"), "/search?q=green flowers&hl=en"); EXPECT_STREQ(peer.GetSectionValue("hardest URL"), "http://www.google.com/s?q=%27bla%27" "&a=%22%22&b=%28%3Ctag%3E%29&c=%2A%0D%0A%5C%5Cbla"); } TEST(TemplateModifiers, ValidateImgSrcUrlCssEscape) { ctemplate::TemplateDictionary dict("TestValidateImgSrcUrlCssEscape", NULL); dict.SetEscapedValue("easy http URL", "http://www.google.com", ctemplate::validate_img_src_url_and_css_escape); dict.SetEscapedValue("harder https URL", "https://www.google.com/search?q=f&hl=en", ctemplate::validate_img_src_url_and_css_escape); dict.SetEscapedValue("javascript URL", "javascript:alert(document.cookie)", ctemplate::validate_img_src_url_and_css_escape); dict.SetEscapedValue("relative URL", "/search?q=green flowers&hl=en", ctemplate::validate_img_src_url_and_css_escape); dict.SetEscapedValue("hardest URL", "http://www.google.com/s?q='bla'" "&a=\"\"&b=()&c=*\r\n\\\\bla", ctemplate::validate_img_src_url_and_css_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy http URL"), "http://www.google.com"); EXPECT_STREQ(peer.GetSectionValue("harder https URL"), "https://www.google.com/search?q=f&hl=en"); EXPECT_STREQ(peer.GetSectionValue("javascript URL"), ctemplate::ValidateUrl::kUnsafeImgSrcUrlReplacement); EXPECT_STREQ(peer.GetSectionValue("relative URL"), "/search?q=green flowers&hl=en"); EXPECT_STREQ(peer.GetSectionValue("hardest URL"), "http://www.google.com/s?q=%27bla%27" "&a=%22%22&b=%28%3Ctag%3E%29&c=%2A%0D%0A%5C%5Cbla"); } TEST(TemplateModifiers, CleanseAttribute) { ctemplate::TemplateDictionary dict("TestCleanseAttribute", NULL); dict.SetEscapedValue("easy attribute", "top", ctemplate::cleanse_attribute); dict.SetEscapedValue("harder attribute", "foo & bar", ctemplate::cleanse_attribute); dict.SetEscapedValue("hardest attribute", "top onclick='alert(document.cookie)'", ctemplate::cleanse_attribute); dict.SetEscapedValue("equal in middle", "foo = bar", ctemplate::cleanse_attribute); dict.SetEscapedValue("leading equal", "=foo", ctemplate::cleanse_attribute); dict.SetEscapedValue("trailing equal", "foo=", ctemplate::cleanse_attribute); dict.SetEscapedValue("all equals", "===foo===bar===", ctemplate::cleanse_attribute); dict.SetEscapedValue("just equals", "===", ctemplate::cleanse_attribute); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy attribute"), "top"); EXPECT_STREQ(peer.GetSectionValue("harder attribute"), "foo___bar"); EXPECT_STREQ(peer.GetSectionValue("hardest attribute"), "top_onclick=_alert_document.cookie__"); EXPECT_STREQ(peer.GetSectionValue("equal in middle"), "foo_=_bar"); EXPECT_STREQ(peer.GetSectionValue("leading equal"), "_foo"); EXPECT_STREQ(peer.GetSectionValue("trailing equal"), "foo_"); EXPECT_STREQ(peer.GetSectionValue("just equals"), "_=_"); EXPECT_STREQ(peer.GetSectionValue("all equals"), "_==foo===bar==_"); } TEST(TemplateModifiers, CleanseCss) { ctemplate::TemplateDictionary dict("TestCleanseCss", NULL); dict.SetEscapedValue("easy css", "top", ctemplate::cleanse_css); dict.SetEscapedValue("harder css", "foo & bar", ctemplate::cleanse_css); dict.SetEscapedValue("hardest css", ";width:expression(document.cookie)", ctemplate::cleanse_css); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy css"), "top"); EXPECT_STREQ(peer.GetSectionValue("harder css"), "foo bar"); EXPECT_STREQ(peer.GetSectionValue("hardest css"), "widthexpressiondocument.cookie"); } TEST(TemplateModifiers, JavascriptEscape) { ctemplate::TemplateDictionary dict("TestJavascriptEscape", NULL); dict.SetEscapedValue("easy JS", "joo", ctemplate::javascript_escape); dict.SetEscapedValue("harder JS", "f = 'joo';", ctemplate::javascript_escape); dict.SetEscapedValue("hardest JS", ("f = 'foo\f';\r\n\tprint \"\\&foo = \b\", \"foo\""), ctemplate::javascript_escape); dict.SetEscapedValue("close script JS", "//-->", ctemplate::javascript_escape); dict.SetEscapedValue("unicode codepoints", ("line1" "\xe2\x80\xa8" "line2" "\xe2\x80\xa9" "line3" /* \u2027 */ "\xe2\x80\xa7" /* \u202A */ "\xe2\x80\xaa" /* malformed */ "\xe2" "\xe2\x80\xa8" /* truncated */ "\xe2\x80"), ctemplate::javascript_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy JS"), "joo"); EXPECT_STREQ(peer.GetSectionValue("harder JS"), "f \\x3d \\x27joo\\x27;"); EXPECT_STREQ(peer.GetSectionValue("hardest JS"), "f \\x3d \\x27foo\\f\\x27;\\r\\n\\tprint \\x22\\\\\\x26foo " "\\x3d \\b\\x22, \\x22foo\\x22"); EXPECT_STREQ(peer.GetSectionValue("close script JS"), "//--\\x3e\\x3c/script\\x3e\\x3cscript\\x3e" "alert(123);\\x3c/script\\x3e"); EXPECT_STREQ(peer.GetSectionValue("unicode codepoints"), "line1" "\\u2028" "line2" "\\u2029" "line3" "\xe2\x80\xa7" "\xe2\x80\xaa" "\xe2" "\\u2028" "\xe2\x80"); } TEST(TemplateModifiers, JavascriptNumber) { ctemplate::TemplateDictionary dict("TestJavascriptNumber", NULL); dict.SetEscapedValue("empty string", "", ctemplate::javascript_number); dict.SetEscapedValue("boolean true", "true", ctemplate::javascript_number); dict.SetEscapedValue("boolean false", "false", ctemplate::javascript_number); dict.SetEscapedValue("bad boolean 1", "tfalse", ctemplate::javascript_number); dict.SetEscapedValue("bad boolean 2", "tru", ctemplate::javascript_number); dict.SetEscapedValue("bad boolean 3", "truee", ctemplate::javascript_number); dict.SetEscapedValue("bad boolean 4", "invalid", ctemplate::javascript_number); // Check that our string comparisons for booleans do not // assume input is null terminated. dict.SetEscapedValue("good boolean 5", ctemplate::TemplateString("truee", 4), ctemplate::javascript_number); dict.SetEscapedValue("bad boolean 6", ctemplate::TemplateString("true", 3), ctemplate::javascript_number); dict.SetEscapedValue("hex number 1", "0x123456789ABCDEF", ctemplate::javascript_number); dict.SetEscapedValue("hex number 2", "0X123456789ABCDEF", ctemplate::javascript_number); dict.SetEscapedValue("bad hex number 1", "0x123GAC", ctemplate::javascript_number); dict.SetEscapedValue("bad hex number 2", "0x", ctemplate::javascript_number); dict.SetEscapedValue("number zero", "0", ctemplate::javascript_number); dict.SetEscapedValue("invalid number", "A9", ctemplate::javascript_number); dict.SetEscapedValue("decimal zero", "0.0", ctemplate::javascript_number); dict.SetEscapedValue("octal number", "01234567", ctemplate::javascript_number); dict.SetEscapedValue("decimal number", "799.123", ctemplate::javascript_number); dict.SetEscapedValue("negative number", "-244", ctemplate::javascript_number); dict.SetEscapedValue("positive number", "+244", ctemplate::javascript_number); dict.SetEscapedValue("valid float 1", ".55", ctemplate::javascript_number); dict.SetEscapedValue("valid float 2", "8.55e-12", ctemplate::javascript_number); dict.SetEscapedValue("invalid float", "8.55ABC", ctemplate::javascript_number); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("empty string"), ""); EXPECT_STREQ(peer.GetSectionValue("boolean true"), "true"); EXPECT_STREQ(peer.GetSectionValue("boolean false"), "false"); EXPECT_STREQ(peer.GetSectionValue("bad boolean 1"), "null"); EXPECT_STREQ(peer.GetSectionValue("bad boolean 2"), "null"); EXPECT_STREQ(peer.GetSectionValue("bad boolean 3"), "null"); EXPECT_STREQ(peer.GetSectionValue("bad boolean 4"), "null"); EXPECT_STREQ(peer.GetSectionValue("good boolean 5"), "true"); EXPECT_STREQ(peer.GetSectionValue("bad boolean 6"), "null"); EXPECT_STREQ(peer.GetSectionValue("hex number 1"), "0x123456789ABCDEF"); EXPECT_STREQ(peer.GetSectionValue("hex number 2"), "0X123456789ABCDEF"); EXPECT_STREQ(peer.GetSectionValue("bad hex number 1"), "null"); EXPECT_STREQ(peer.GetSectionValue("bad hex number 2"), "null"); EXPECT_STREQ(peer.GetSectionValue("number zero"), "0"); EXPECT_STREQ(peer.GetSectionValue("invalid number"), "null"); EXPECT_STREQ(peer.GetSectionValue("decimal zero"), "0.0"); EXPECT_STREQ(peer.GetSectionValue("octal number"), "01234567"); EXPECT_STREQ(peer.GetSectionValue("decimal number"), "799.123"); EXPECT_STREQ(peer.GetSectionValue("negative number"), "-244"); EXPECT_STREQ(peer.GetSectionValue("positive number"), "+244"); EXPECT_STREQ(peer.GetSectionValue("valid float 1"), ".55"); EXPECT_STREQ(peer.GetSectionValue("valid float 2"), "8.55e-12"); EXPECT_STREQ(peer.GetSectionValue("invalid float"), "null"); } TEST(TemplateModifiers, JsonEscape) { ctemplate::TemplateDictionary dict("TestJsonEscape", NULL); dict.SetEscapedValue("easy JSON", "joo", ctemplate::json_escape); dict.SetEscapedValue("harder JSON", "f = \"joo\"; e = 'joo';", ctemplate::json_escape); dict.SetEscapedValue("hardest JSON", "f = 'foo<>';\r\n\t\fprint \"\\&foo = /\b\", \"foo\"", ctemplate::json_escape); dict.SetEscapedValue("html in JSON", " ", ctemplate::json_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("easy JSON"), "joo"); EXPECT_STREQ(peer.GetSectionValue("harder JSON"), "f = \\\"joo\\\"; " "e = 'joo';"); EXPECT_STREQ(peer.GetSectionValue("html in JSON"), "\\u003Chtml\\u003E\\u0026nbsp;\\u003C\\/html\\u003E"); // There's a bug in MSVC 7.1 where you can't pass a literal string // with more than one \" in it to a macro (!) -- see // http://marc.info/?t=110853662500001&r=1&w=2 // We work around this by assigning the string to a variable first. const char* expected = ("f = 'foo\\u003C\\u003E';\\r\\n\\t\\fprint \\\"" "\\\\\\u0026foo = \\/\\b\\\", \\\"foo\\\""); EXPECT_STREQ(peer.GetSectionValue("hardest JSON"), expected); } TEST(TemplateModifiers, UrlQueryEscape) { ctemplate::TemplateDictionary dict("TestUrlQueryEscape", NULL); // The first three tests do not need escaping. dict.SetEscapedValue("query escape 0", "", ctemplate::url_query_escape); dict.SetEscapedValue("query escape 1", "noop", ctemplate::url_query_escape); dict.SetEscapedValue("query escape 2", "0123456789abcdefghjijklmnopqrstuvwxyz" "ABCDEFGHIJKLMNOPQRSTUVWXYZ.-_*/~!(),", ctemplate::url_query_escape); dict.SetEscapedValue("query escape 3", " ?a=b;c#d ", ctemplate::url_query_escape); dict.SetEscapedValue("query escape 4", "#$%&+<=>?@[\\]^`{|}", ctemplate::url_query_escape); dict.SetEscapedValue("query escape 5", "\xDE\xAD\xCA\xFE", ctemplate::url_query_escape); dict.SetEscapedValue("query escape 6", "\"':", ctemplate::url_query_escape); ctemplate::TemplateDictionaryPeer peer(&dict); // peer can look inside dicts EXPECT_STREQ(peer.GetSectionValue("query escape 0"), ""); EXPECT_STREQ(peer.GetSectionValue("query escape 1"), "noop"); EXPECT_STREQ(peer.GetSectionValue("query escape 2"), "0123456789abcdefghjijklmnopqrstuvwxyz" "ABCDEFGHIJKLMNOPQRSTUVWXYZ.-_*/~!(),"); EXPECT_STREQ(peer.GetSectionValue("query escape 3"), "+%3Fa%3Db%3Bc%23d+"); EXPECT_STREQ(peer.GetSectionValue("query escape 4"), "%23%24%25%26%2B%3C%3D%3E%3F%40%5B%5C%5D%5E%60%7B%7C%7D"); EXPECT_STREQ(peer.GetSectionValue("query escape 5"), "%DE%AD%CA%FE"); EXPECT_STREQ(peer.GetSectionValue("query escape 6"), "%22%27%3A"); } TEST(TemplateModifiers, PrefixLine) { ctemplate::TemplateDictionary dict("TestPrefixLine", NULL); // These don't escape: we don't put the prefix before the first line EXPECT_STREQ(ctemplate::prefix_line("pt 1", " ").c_str(), "pt 1"); EXPECT_STREQ(ctemplate::prefix_line("pt 1", "::").c_str(), "pt 1"); EXPECT_STREQ(ctemplate::prefix_line("pt 1\npt 2", ":").c_str(), "pt 1\n:pt 2"); EXPECT_STREQ(ctemplate::prefix_line("pt 1\npt 2", " ").c_str(), "pt 1\n pt 2"); EXPECT_STREQ(ctemplate::prefix_line("pt 1\npt 2", "\n").c_str(), "pt 1\n\npt 2"); EXPECT_STREQ(ctemplate::prefix_line("pt 1\npt 2\n", " ").c_str(), "pt 1\n pt 2\n "); EXPECT_STREQ(ctemplate::prefix_line("pt 1\rpt 2\n", ":").c_str(), "pt 1\r:pt 2\n:"); EXPECT_STREQ(ctemplate::prefix_line("pt 1\npt 2\r", ":").c_str(), "pt 1\n:pt 2\r:"); EXPECT_STREQ(ctemplate::prefix_line("pt 1\r\npt 2\r", ":").c_str(), "pt 1\r\n:pt 2\r:"); } TEST(TemplateModifiers, FindModifier) { const ctemplate::ModifierInfo* info; EXPECT_TRUE((info = ctemplate::FindModifier("html_escape", 11, "", 0))); EXPECT_EQ(info->modifier, &ctemplate::html_escape); EXPECT_TRUE((info = ctemplate::FindModifier("h", 1, "", 0))); EXPECT_EQ(info->modifier, &ctemplate::html_escape); EXPECT_TRUE((info = ctemplate::FindModifier("html_escape_with_arg", 20, "=pre", 4))); EXPECT_EQ(info->modifier, &ctemplate::pre_escape); EXPECT_TRUE((info = ctemplate::FindModifier("H", 1, "=pre", 4))); EXPECT_EQ(info->modifier, &ctemplate::pre_escape); EXPECT_TRUE((info = ctemplate::FindModifier("javascript_escape_with_arg", 26, "=number", 7))); EXPECT_TRUE((info = ctemplate::FindModifier("J", 1, "=number", 7))); EXPECT_EQ(info->modifier, &ctemplate::javascript_number); // html_escape_with_arg doesn't have a default value, so these should fail. EXPECT_FALSE(ctemplate::FindModifier("H", 1, "=pre", 2)); // "=p" EXPECT_FALSE(ctemplate::FindModifier("H", 1, "=pree", 5)); EXPECT_FALSE(ctemplate::FindModifier("H", 1, "=notpresent", 11)); // If we don't have a modifier-value when we ought, we should fail. EXPECT_FALSE(ctemplate::FindModifier("html_escape", 11, "=p", 2)); EXPECT_FALSE(ctemplate::FindModifier("h", 1, "=p", 2)); EXPECT_FALSE(ctemplate::FindModifier("html_escape_with_arg", 20, "", 0)); EXPECT_FALSE(ctemplate::FindModifier("H", 1, "", 0)); // Test with added modifiers as well. ctemplate::NullModifier foo_modifier1; ctemplate::NullModifier foo_modifier2; ctemplate::NullModifier foo_modifier3; ctemplate::NullModifier foo_modifier4; EXPECT_TRUE(ctemplate::AddModifier("x-test", &foo_modifier1)); EXPECT_TRUE(ctemplate::AddModifier("x-test-arg=", &foo_modifier2)); EXPECT_TRUE(ctemplate::AddModifier("x-test-arg=h", &foo_modifier3)); EXPECT_TRUE(ctemplate::AddModifier("x-test-arg=json", &foo_modifier4)); EXPECT_TRUE((info = ctemplate::FindModifier("x-test", 6, "", 0))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier1); EXPECT_EQ(info->xss_class, ctemplate::XSS_UNIQUE); EXPECT_TRUE((info = ctemplate::FindModifier("x-test", 6, "=h", 2))); EXPECT_FALSE(info->is_registered); // This tests default values EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "=p", 2))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier2); EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "=h", 2))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier3); EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "=html", 5))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier2); EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "=json", 5))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier4); // The value is required to start with an '=' to match the // specialization. If it doesn't, it will match the default. EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "json", 4))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier2); EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "=jsonnabbe", 5))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier4); EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "=jsonnabbe", 6))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier2); EXPECT_TRUE((info = ctemplate::FindModifier("x-test-arg", 10, "=jsonnabbe", 4))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->modifier, &foo_modifier2); // If we try to find an x- modifier that wasn't added, we should get // a legit but "unknown" modifier back. EXPECT_TRUE((info = ctemplate::FindModifier("x-foo", 5, "", 0))); EXPECT_FALSE(info->is_registered); EXPECT_TRUE((info = ctemplate::FindModifier("x-bar", 5, "=p", 2))); EXPECT_FALSE(info->is_registered); // Basic test with added XssSafe modifier. ctemplate::NullModifier foo_modifier5; EXPECT_TRUE(ctemplate::AddXssSafeModifier("x-safetest", &foo_modifier5)); EXPECT_TRUE((info = ctemplate::FindModifier("x-safetest", 10, "", 0))); EXPECT_TRUE(info->is_registered); EXPECT_EQ(info->xss_class, ctemplate::XSS_SAFE); EXPECT_EQ(info->modifier, &foo_modifier5); } TEST(TemplateModifiers, AddModifier) { EXPECT_TRUE(ctemplate::AddModifier("x-atest", &ctemplate::html_escape)); EXPECT_TRUE(ctemplate::AddModifier("x-atest-arg=", &ctemplate::html_escape)); EXPECT_TRUE(ctemplate::AddModifier("x-atest-arg=h", &ctemplate::html_escape)); EXPECT_TRUE(ctemplate::AddModifier("x-atest-arg=html", &ctemplate::html_escape)); EXPECT_TRUE(ctemplate::AddModifier("x-atest-arg=json", &ctemplate::json_escape)); EXPECT_TRUE(ctemplate::AddModifier("x-atest-arg=j", &ctemplate::json_escape)); EXPECT_TRUE(ctemplate::AddModifier("x-atest-arg=J", &ctemplate::json_escape)); // Make sure AddModifier fails with an invalid name. EXPECT_FALSE(ctemplate::AddModifier("test", &ctemplate::html_escape)); // Make sure AddModifier fails with a duplicate name. EXPECT_FALSE(ctemplate::AddModifier("x-atest", &ctemplate::html_escape)); EXPECT_FALSE(ctemplate::AddModifier("x-atest-arg=", &ctemplate::html_escape)); EXPECT_FALSE(ctemplate::AddModifier("x-atest-arg=h", &ctemplate::html_escape)); EXPECT_FALSE(ctemplate::AddModifier("x-atest-arg=html", &ctemplate::html_escape)); const ctemplate::ModifierInfo* info; EXPECT_TRUE((info = ctemplate::FindModifier("x-atest", 7, "", 0))); EXPECT_FALSE(info->modval_required); // Make sure we can still add a modifier after having already // searched for it. EXPECT_TRUE((info = ctemplate::FindModifier("x-foo", 5, "", 0))); EXPECT_FALSE(info->is_registered); ctemplate::NullModifier foo_modifier; EXPECT_TRUE(ctemplate::AddModifier("x-foo", &foo_modifier)); EXPECT_TRUE((info = ctemplate::FindModifier("x-foo", 5, "", 0))); EXPECT_EQ(info->modifier, &foo_modifier); } TEST(TemplateModifiers, AddXssSafeModifier) { // For shorter lines. const ctemplate::TemplateModifier* esc_fn = &ctemplate::html_escape; EXPECT_TRUE(ctemplate::AddXssSafeModifier("x-asafetest", esc_fn)); EXPECT_TRUE(ctemplate::AddXssSafeModifier("x-asafetest-arg=", esc_fn)); EXPECT_TRUE(ctemplate::AddXssSafeModifier("x-asafetest-arg=h", esc_fn)); // Make sure AddXssSafeModifier fails with an invalid name. EXPECT_FALSE(ctemplate::AddXssSafeModifier("test", esc_fn)); // Make sure AddXssSafeModifier fails with a duplicate name. EXPECT_FALSE(ctemplate::AddXssSafeModifier("x-asafetest", esc_fn)); EXPECT_FALSE(ctemplate::AddXssSafeModifier("x-asafetest-arg=", esc_fn)); EXPECT_FALSE(ctemplate::AddXssSafeModifier("x-asafetest-arg=h", esc_fn)); // Make sure AddXssSafeModifier fails if the same modifier was // previously added via AddModifier. EXPECT_TRUE(ctemplate::AddModifier("x-safetest2", esc_fn)); EXPECT_TRUE(ctemplate::AddModifier("x-safetest2-arg=", esc_fn)); EXPECT_TRUE(ctemplate::AddModifier("x-safetest2-arg=h", esc_fn)); EXPECT_FALSE(ctemplate::AddXssSafeModifier("x-safetest2", esc_fn)); EXPECT_FALSE(ctemplate::AddXssSafeModifier("x-safetest2-arg=", esc_fn)); EXPECT_FALSE(ctemplate::AddXssSafeModifier("x-safetest2-arg=h", esc_fn)); // and vice versa. EXPECT_FALSE(ctemplate::AddModifier("x-asafetest", esc_fn)); EXPECT_FALSE(ctemplate::AddModifier("x-asafetest-arg=", esc_fn)); EXPECT_FALSE(ctemplate::AddModifier("x-asafetest-arg=h", esc_fn)); } // Helper function. Determines whether the Modifier specified by // alt_modname/alt_modval is a safe XSS alternative to // the Modifier specified by modname/modval. static bool CheckXSSAlternative(const string& modname, const string& modval, const string& alt_modname, const string& alt_modval) { const ctemplate::ModifierInfo *mod, *alt_mod; mod = ctemplate::FindModifier(modname.c_str(), modname.length(), modval.c_str(), modval.length()); alt_mod = ctemplate::FindModifier(alt_modname.c_str(), alt_modname.length(), alt_modval.c_str(), alt_modval.length()); EXPECT_TRUE(mod != NULL && alt_mod != NULL); return IsSafeXSSAlternative(*mod, *alt_mod); } TEST(TemplateModifiers, XSSAlternatives) { // A modifier is always a safe replacement to itself, even non built-in. EXPECT_TRUE(CheckXSSAlternative("h", "", "h", "")); EXPECT_TRUE(CheckXSSAlternative("url_escape_with_arg", "=javascript", "url_escape_with_arg", "=javascript")); EXPECT_TRUE(CheckXSSAlternative("x-bla", "", "x-bla", "")); // A built-in modifier is always a safe replacement to // another with the same function. EXPECT_TRUE(CheckXSSAlternative("H", "=pre", "p", "")); EXPECT_TRUE(CheckXSSAlternative("url_query_escape", "", "url_escape_with_arg", "=query")); // H=(pre|snippet|attribute), p, u, U=query, U=html (a.k.a H=url) // and I=html are all alternatives to h. EXPECT_TRUE(CheckXSSAlternative("h", "", "H", "=pre")); EXPECT_TRUE(CheckXSSAlternative("h", "", "H", "=snippet")); EXPECT_TRUE(CheckXSSAlternative("h", "", "H", "=attribute")); EXPECT_TRUE(CheckXSSAlternative("h", "", "H", "=url")); EXPECT_TRUE(CheckXSSAlternative("h", "", "p", "")); EXPECT_TRUE(CheckXSSAlternative("h", "", "u", "")); EXPECT_TRUE(CheckXSSAlternative("h", "", "U", "=query")); EXPECT_TRUE(CheckXSSAlternative("h", "", "U", "=html")); EXPECT_TRUE(CheckXSSAlternative("h", "", "I", "=html")); // But h is not an alternative to H=attribute and I=html, // nor is json_escape an alternative to h. EXPECT_FALSE(CheckXSSAlternative("H", "=attribute", "h", "")); EXPECT_FALSE(CheckXSSAlternative("I", "=html", "h", "")); EXPECT_FALSE(CheckXSSAlternative("h", "", "json_escape", "")); // H=snippet and H=attribute are alternatives to H=pre // But H=pre is not an alternative to H=attribute. EXPECT_TRUE(CheckXSSAlternative("H", "=pre", "H", "=snippet")); EXPECT_TRUE(CheckXSSAlternative("H", "=pre", "H", "=attribute")); EXPECT_FALSE(CheckXSSAlternative("H", "=attribute", "H", "=pre")); // javascript_escape is an alternative to json_escape and vice versa EXPECT_TRUE(CheckXSSAlternative("json_escape", "", "javascript_escape", "")); EXPECT_TRUE(CheckXSSAlternative("javascript_escape", "", "json_escape", "")); // I=javascript is an alternative to :j and :U=javascript but not // vice versa EXPECT_TRUE(CheckXSSAlternative("javascript_escape", "", "I", "=javascript")); EXPECT_TRUE(CheckXSSAlternative("U", "=javascript", "I", "=javascript")); EXPECT_FALSE(CheckXSSAlternative("I", "=javascript", "javascript_escape", "")); EXPECT_FALSE(CheckXSSAlternative("I", "=javascript", "U", "=javascript")); // U=css and I=css are alternatives to :c but not vice versa EXPECT_TRUE(CheckXSSAlternative("c", "", "U", "=css")); EXPECT_TRUE(CheckXSSAlternative("c", "", "I", "=css")); EXPECT_FALSE(CheckXSSAlternative("U", "=css", "c", "")); EXPECT_FALSE(CheckXSSAlternative("I", "=css", "c", "")); // Extended modifier should not match any other except itself. EXPECT_FALSE(CheckXSSAlternative("x-bla", "", "x-foo", "")); } // This is a basic sanity check for the GetDefaultModifierForXXX() functions. // More testing happens in AutoEscaper code which uses them. TEST(TemplateModifiers, DefaultModifiersForContext) { const ctemplate::ModifierAndValue* modval; string print_mods; const vector modvals_html = ctemplate::GetDefaultModifierForHtml(); EXPECT_EQ(1, modvals_html.size()); print_mods = ctemplate::PrettyPrintModifiers(modvals_html, ";"); EXPECT_STREQ(":h", print_mods.c_str()); modval = modvals_html.front(); EXPECT_EQ(modval->modifier_info->modifier, &ctemplate::html_escape); const vector modvals_js = ctemplate::GetDefaultModifierForJs(); EXPECT_EQ(1, modvals_js.size()); print_mods = ctemplate::PrettyPrintModifiers(modvals_js, ";"); EXPECT_STREQ(":j", print_mods.c_str()); modval = modvals_js.front(); EXPECT_EQ(modval->modifier_info->modifier, &ctemplate::javascript_escape); const vector modvals_xml = ctemplate::GetDefaultModifierForXml(); EXPECT_EQ(1, modvals_xml.size()); print_mods = ctemplate::PrettyPrintModifiers(modvals_xml, ";"); EXPECT_STREQ(":xml_escape", print_mods.c_str()); modval = modvals_xml.front(); EXPECT_EQ(modval->modifier_info->modifier, &ctemplate::xml_escape); const vector modvals_json = ctemplate::GetDefaultModifierForJson(); EXPECT_EQ(1, modvals_json.size()); print_mods = ctemplate::PrettyPrintModifiers(modvals_json, ";"); EXPECT_STREQ(":j", print_mods.c_str()); modval = modvals_json.front(); EXPECT_EQ(modval->modifier_info->modifier, &ctemplate::javascript_escape); } // This tests for a bug we had where we were returning a pointer into // a vector that became invalid after the vector was resized. TEST(TemplateModifiers, ManyUnknownModifiers) { string tpl_str1 = "{{from_name:x-test=4}} sent you a message"; const ctemplate::Template* tpl1 = ctemplate::Template::StringToTemplate( tpl_str1, ctemplate::DO_NOT_STRIP); string tpl_str2 = "{{from_name:x-test=4}} sent you a message:"; string expected_out = "me sent you a message:"; // All those new unknown varnames should cause g_unknown_modifiers // to resize. 1111 is an arbitrary large number. for (int i = 0; i < 1111; i++) { tpl_str2.append("{{from_name:x-" + string(i, 't') + "=4}}"); expected_out.append("me"); } const ctemplate::Template* tpl2 = ctemplate::Template::StringToTemplate( tpl_str2, ctemplate::DO_NOT_STRIP); // Even after the resizing, the references to the unknown // modifiers in tpl1 and tpl2 should still be valid. ctemplate::TemplateDictionary dict("test"); dict.SetValue("from_name", "me"); string out; out.clear(); tpl1->Expand(&out, &dict); EXPECT_STREQ("me sent you a message", out.c_str()); delete tpl1; out.clear(); tpl2->Expand(&out, &dict); EXPECT_STREQ(expected_out.c_str(), out.c_str()); delete tpl2; } int main(int argc, char** argv) { return RUN_ALL_TESTS(); } ctemplate-ctemplate-2.4/src/tests/template_regtest.cc000077500000000000000000000505061363122316600231350ustar00rootroot00000000000000// Copyright (c) 2005, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // This test consists of creating a pretty complicated // dictionary, and then applying it to a bunch of templates // (specified in the testdata dir) and making sure the output // is as expected. We actually support testing with multiple // dictionaries. We glob the testdat dir, so it's possible to // add a new test just by creating a template and expected-output // file in the testdata directory. Files are named // template_unittest_testXX.in // template_unittest_testXX_dictYY.out // YY should start with 01 (not 00). XX can be an arbitrary string. #include "config_for_unittests.h" #include #include #include #include #include #ifdef HAVE_DIRENT_H # include // for opendir() etc #else # define dirent direct # ifdef HAVE_SYS_NDIR_H # include # endif # ifdef HAVE_SYS_DIR_H # include # endif # ifdef HAVE_NDIR_H # include # endif #endif // for opendir() etc #include // for sort() and stable_partition #include #include #include #include #include #include #include "base/util.h" using std::vector; using std::string; using std::sort; using ctemplate::DO_NOT_STRIP; using ctemplate::PerExpandData; using ctemplate::STRIP_BLANK_LINES; using ctemplate::STRIP_WHITESPACE; using ctemplate::TC_HTML; using ctemplate::TC_MANUAL; using ctemplate::Template; using ctemplate::TemplateDictionary; #define ASSERT(cond) do { \ if (!(cond)) { \ printf("%s: %d: ASSERT FAILED: %s\n", __FILE__, __LINE__, \ #cond); \ assert(cond); \ exit(1); \ } \ } while (0) #define ASSERT_STRING_EQ(a, b) do { \ assert(StringEq(a, b, __FILE__, __LINE__, #a, #b)); \ } while (0) bool StringEq(const string& a, const string& b, const char* filename, int lineno, const char* namea, const char* nameb) { if (a != b) { printf("%s: %d: ASSERT FAILED: %s == %s:\n", filename, lineno, namea, nameb); printf("EXPECTED:\n%s\n", a.c_str()); printf("ACTUAL:\n%s\n", b.c_str()); return false; } return true; } #define ASSERT_STREQ_EXCEPT(a, b, except) ASSERT(StreqExcept(a, b, except)) #define ASSERT_STREQ(a, b) ASSERT(strcmp(a, b) == 0) #define ASSERT_NOT_STREQ(a, b) ASSERT(strcmp(a, b) != 0) // First, (conceptually) remove all chars in "except" from both a and b. // Then return true iff munged_a == munged_b. bool StreqExcept(const char* a, const char* b, const char* except) { const char* pa = a, *pb = b; const size_t exceptlen = strlen(except); while (1) { // Use memchr instead of strchr because strchr(foo, '\0') always fails while (memchr(except, *pa, exceptlen)) pa++; // ignore "except" chars in a while (memchr(except, *pb, exceptlen)) pb++; // ignore "except" chars in b if ((*pa == '\0') && (*pb == '\0')) return true; if (*pa++ != *pb++) // includes case where one is at \0 return false; } } RegisterTemplateFilename(VALID1_FN, "template_unittest_test_valid1.in"); RegisterTemplateFilename(INVALID1_FN, "template_unittest_test_invalid1.in"); RegisterTemplateFilename(INVALID2_FN, "template_unittest_test_invalid2.in"); RegisterTemplateFilename(NONEXISTENT_FN, "nonexistent__file.tpl"); struct Testdata { string input_template_name; // the filename of the input template string input_template; // the contents of the input template vector output; // entry i is the output of using dict i. vector annotated_output; // used to test annotations }; static void ReadToString(const string& filename, string* s) { const int bufsize = 8092; char buffer[bufsize]; size_t n; FILE* fp = fopen(filename.c_str(), "rb"); if (!fp) PFATAL(filename.c_str()); while ((n=fread(buffer, 1, bufsize, fp)) > 0) { if (ferror(fp)) PFATAL(filename.c_str()); s->append(string(buffer, n)); } fclose(fp); } static bool EndsWith(const string& s, const string& suffix) { return (s.length() >= suffix.length() && s.substr(s.length() - suffix.length()) == suffix); } #ifndef USING_PORT_CC /* windows defines its own version in windows/port.cc */ static void GetNamelist(const char* testdata_dir, vector* namelist) { DIR* dir = opendir(testdata_dir); struct dirent* dir_entry; if (dir == NULL) PFATAL("opendir"); while ( (dir_entry=readdir(dir)) != NULL ) { if (!strncmp(dir_entry->d_name, "template_unittest_test", sizeof("template_unittest_test")-1)) { namelist->push_back(dir_entry->d_name); // collect test files } } if (closedir(dir) != 0) PFATAL("closedir"); } #endif // expensive to resize this vector and copy it and all, but that's ok static vector ReadDataFiles(const char* testdata_dir) { vector retval; vector namelist; GetNamelist(testdata_dir, &namelist); sort(namelist.begin(), namelist.end()); for (vector::const_iterator it = namelist.begin(); it != namelist.end(); ++it) { vector* new_output = NULL; const string fname = string(testdata_dir) + "/" + it->c_str(); if (EndsWith(fname, ".in")) { retval.push_back(Testdata()); retval.back().input_template_name = *it; ReadToString(fname, &retval.back().input_template); } else if (EndsWith(fname, ".out")) { new_output = &retval.back().output; } else if (EndsWith(fname, ".anno_out")) { new_output = &retval.back().annotated_output; } else { ASSERT(false); // Filename must end in either .in, .out, or .anno_out. } if (new_output) { // the .out and .anno_out cases ASSERT(!retval.empty()); // an .out without any corresponding .in? ASSERT(it->length() > retval.back().input_template_name.length() + 4); // input file is foo.in, and output is foo_dictYY.out. This gets to YY. const char* dictnum_pos = (it->c_str() + retval.back().input_template_name.length() + 2); int dictnum = atoi32(dictnum_pos); // just ignore chars after the YY ASSERT(dictnum); // dictnums should start with 01 while (new_output->size() < static_cast::size_type>(dictnum)) new_output->push_back(string()); ReadToString(fname, &((*new_output)[dictnum-1])); } } return retval; } // Creates a complicated dictionary, using every TemplateDictionary // command under the sun. Returns a pointer to the new dictionary-root. // Should be freed by the caller. static TemplateDictionary* MakeDict1() { TemplateDictionary* dict = new TemplateDictionary("dict1", NULL); dict->SetFilename("just used for debugging, so doesn't matter.txt"); // --- These are used by template_unittest_test_simple.in dict->SetValue("HEAD", " This is the head "); // We leave BODY undefined, to make sure that expansion works properly. // --- These are used by template_unittest_test_footer.in TemplateDictionary* fbt = dict->AddSectionDictionary("FOOTER_BAR_TEXT"); fbt->SetValue("BODY", "Should never be shown"); // this is part of simple fbt->SetEscapedValue("HOME_LINK", "Time to go home!", ctemplate::html_escape); // Note: you should never have code like this in real life! The // and should be part of the template proper. fbt->SetFormattedValue("ADVERTISE_LINK", "Be advertiser #%d", 2); fbt->SetValue("ABOUT_GOOGLE_LINK", "About Google!"); // We show PROMO_LICENSING_SECTION in the main dict, even though // it's defined in the fbt subsection. This will still work: section // showing goes to the parent dict if not found in the current dict. dict->ShowSection("PROMO_LICENSING_SECTION"); dict->SetValue("PROMO_LICENSING_LINK", ""); // We don't show the TRIM_LINE section, so these vars shouldn't be seen dict->SetValue("TRIM_LINE_COLOR", "Who cares?"); dict->SetIntValue("TRIM_LINE_HEIGHT", 10); dict->SetIntValue("MODIFIED_BY_GOOGLE", 2005); dict->SetValue("MSG_copyright", "© Google Inc. (all rights reserved)"); // We don't set ODP_ATTRIBUTION, so this include is ignored. dict->ShowSection("CLOSING_DIV_SECTION"); // We won't set any of the includes that follow, just to keep things simple // First, call SetValueAndShowSection on a non-existence section, should noop dict->SetValueAndShowSection("LATENCY_PREFETCH_URL", "/huh?", "UNUSED_SECTION_NAME"); // Now try the real URL dict->SetValueAndShowSection("LATENCY_PREFETCH_URL", string("/latency"), "LATENCY_PREFETCH"); // JAVASCRIPT_FOOTER_SECTION was meant to be either shown or hidden, but // hey, let's try showing it several times, each with a different include. // And let's include each one several times. TemplateDictionary* jfs1 = dict->AddSectionDictionary( "JAVASCRIPT_FOOTER_SECTION"); // This first dictionary should have an empty HEAD and BODY TemplateDictionary* inc1a = jfs1->AddIncludeDictionary("FAST_NEXT_JAVASCRIPT"); inc1a->SetFilename("template_unittest_test_simple.in"); // For the second dict, let's set an illegal filename: should be ignored TemplateDictionary* inc1b = jfs1->AddIncludeDictionary("FAST_NEXT_JAVASCRIPT"); inc1b->SetFilename(INVALID1_FN); // For the third dict, let's do the same as the first, but with a HEAD TemplateDictionary* inc1c = jfs1->AddIncludeDictionary("FAST_NEXT_JAVASCRIPT"); inc1c->SetFilename("template_unittest_test_simple.in"); inc1c->SetValue("HEAD", "head"); // Let's expand the section again with two different includes, and again a // third template not meant to be expanded (in this case, don't set filename) TemplateDictionary* jfs2 = dict->AddSectionDictionary( "JAVASCRIPT_FOOTER_SECTION"); TemplateDictionary* inc2a = jfs2->AddIncludeDictionary("FAST_NEXT_JAVASCRIPT"); inc2a->SetFilename("template_unittest_test_simple.in"); inc2a->SetValue("HEAD", "include-head"); inc2a->SetEscapedFormattedValue("BODY", ctemplate::html_escape, "%s: %.4f", "", 1.0/3); inc2a->SetValue("BI_NEWLINE", ""); // override the global default TemplateDictionary* inc2b = jfs2->AddIncludeDictionary("FAST_NEXT_JAVASCRIPT"); inc2b->SetFilename("template_unittest_test_html.in"); inc2b->SetValue("HEAD", "should be ignored"); jfs2->AddIncludeDictionary("FAST_NEXT_JAVASCRIPT"); // ignored: no filename // --- These are used by template_unittest_test_html.in // This should returns in NO_MOUSEOVER_FUNCTIONS remaining hidden dict->SetValueAndShowSection("DUMMY", "", "NO_MOUSEOVER_FUNCTIONS"); dict->ShowSection("MOUSEOVER_FUNCTIONS"); TemplateDictionary* foo = dict->AddIncludeDictionary("MOUSEOVER_JAVASCRIPT"); foo->SetFilename(string("not_a_template")); foo->SetValue("BI_NEWLINE", "not gonna matter"); dict->SetEscapedValue("GOTO_MESSAGE", "print \"Go home\"", ctemplate::javascript_escape); dict->SetEscapedValue("UPDATE", "monday & tuesday", ctemplate::html_escape); dict->ShowSection("UPDATE_SECTION"); dict->SetValue("ALIGNMENT", "\"right\""); // all results sections see this for (int i = 0; i < 3; ++i) { // we'll do three results TemplateDictionary* result = dict->AddSectionDictionary("RESULTS"); if (i % 2 == 0) result->ShowSection("WHITE_BG"); // gives us striped results! const char* res = "<&>\"result\" #%d'&'"; result->SetFormattedValue("RESULT", res, i); result->SetEscapedFormattedValue("XML_RESULT", ctemplate::xml_escape, res, i); result->SetIntValue("GOODNESS", i + 5); } // For testing auto-escape. dict->SetValue("AE_TITLE_GOOD", "Hello World!"); dict->SetValue("AE_TITLE_BAD", "Hello World!"); dict->SetValue("AE_URL_GOOD", "http://www.google.com/"); dict->SetValue("AE_URL_BAD", "javascript:alert(1)"); dict->SetValue("AE_BG_COLOR_GOOD", "red"); dict->SetValue("AE_BG_COLOR_BAD", "evil! &"); dict->SetValue("AE_JS_GOOD", "your text here"); dict->SetValue("AE_JS_BAD", "your text'is clever'thanks"); dict->SetValue("AE_USERNAME_GOOD", "Mr. Nice"); dict->SetValue("AE_USERNAME_BAD", "DoctorEvil"); dict->SetValue("AE_START_EDGE", "left"); dict->SetValue("AE_END_EDGE", ";:center()$$"); // Some invalid chars. dict->SetValue("AE_FONT_SIZE_PC", "120%"); dict->SetValue("AE_FONT_SIZE_PT", "12pt"); dict->SetValue("AE_MAUVE_RGB", "#FF7BD5"); dict->SetValue("AE_ITALIC", "italic"); // This won't see any of the vars *we* set TemplateDictionary* footer_dict = dict->AddIncludeDictionary("FOOTER"); footer_dict->SetFilename("template_unittest_test_footer.in"); // --- These are used by template_unittest_test_modifiers.in // UPDATE and UPDATE_SECTION we inherit from test_html.in TemplateDictionary* inc_simple = dict->AddIncludeDictionary("SIMPLE"); inc_simple->SetFilename("template_unittest_test_simple.in"); return dict; } // Quite opposite of dict1, dict2 is a dictionary that has nothing in it static TemplateDictionary* MakeDict2() { return new TemplateDictionary("dict2"); } // dict3 tests just the handling of whitespace static TemplateDictionary* MakeDict3() { TemplateDictionary* dict = new TemplateDictionary("dict3"); dict->SetValue("HEAD", " "); dict->SetValue("BODY", "\r\n"); return dict; } static TemplateDictionary* MakeDictionary(int i) { switch (i) { case 1: return MakeDict1(); case 2: return MakeDict2(); case 3: return MakeDict3(); default: ASSERT(false); // No dictionary with this number yet. } return NULL; } static void TestExpand(const vector::const_iterator& begin, const vector::const_iterator& end) { for (vector::const_iterator one_test = begin; one_test != end; ++one_test) { Template* tpl_none = Template::GetTemplate(one_test->input_template_name, DO_NOT_STRIP); Template* tpl_lines = Template::GetTemplate(one_test->input_template_name, STRIP_BLANK_LINES); Template* tpl_ws = Template::GetTemplate(one_test->input_template_name, STRIP_WHITESPACE); // Test TemplateToString while we're at it. Template* tplstr_none = Template::StringToTemplate( one_test->input_template, DO_NOT_STRIP); Template* tplstr_lines = Template::StringToTemplate( one_test->input_template, STRIP_BLANK_LINES); Template* tplstr_ws = Template::StringToTemplate( one_test->input_template, STRIP_WHITESPACE); for (vector::const_iterator out = one_test->output.begin(); out != one_test->output.end(); ++out) { int dictnum = out - one_test->output.begin() + 1; // first dict is 01 // If output is the empty string, we assume the file does not exist if (out->empty()) continue; printf("Testing template %s on dict #%d\n", one_test->input_template_name.c_str(), dictnum); // If we're expecting output, the template better not have had an error ASSERT(tpl_none && tpl_lines && tpl_ws); ASSERT(tplstr_none && tplstr_lines && tplstr_ws); TemplateDictionary* dict = MakeDictionary(dictnum); string stroutput_none, stroutput_lines, stroutput_ws; string stroutput_strnone, stroutput_strlines, stroutput_strws; tpl_none->Expand(&stroutput_none, dict); tpl_lines->Expand(&stroutput_lines, dict); tpl_ws->Expand(&stroutput_ws, dict); tplstr_none->Expand(&stroutput_strnone, dict); tplstr_lines->Expand(&stroutput_strlines, dict); tplstr_ws->Expand(&stroutput_strws, dict); // "out" is the output for STRIP_WHITESPACE mode. ASSERT_STRING_EQ(*out, stroutput_ws); // Now compare the variants against each other. // NONE and STRIP_LINES may actually be the same on simple inputs //ASSERT(output_none != output_lines); ASSERT(stroutput_none != stroutput_ws); ASSERT(stroutput_lines != stroutput_ws); ASSERT_STREQ_EXCEPT(stroutput_none.c_str(), stroutput_lines.c_str(), " \t\v\f\r\n"); ASSERT_STREQ_EXCEPT(stroutput_none.c_str(), stroutput_ws.c_str(), " \t\v\f\r\n"); // It shouldn't matter if we read stuff from a file or a string. ASSERT(stroutput_none == stroutput_strnone); ASSERT(stroutput_lines == stroutput_strlines); ASSERT(stroutput_ws == stroutput_strws); delete dict; // it's our responsibility } // The annotation test is a bit simpler; we only strip one way for (vector::const_iterator out = one_test->annotated_output.begin(); out != one_test->annotated_output.end(); ++out) { int dictnum = out - one_test->annotated_output.begin() + 1; // If output is the empty string, we assume the file does not exist if (out->empty()) continue; printf("Testing template %s on dict #%d (annotated)\n", one_test->input_template_name.c_str(), dictnum); TemplateDictionary* dict = MakeDictionary(dictnum); PerExpandData per_expand_data; per_expand_data.SetAnnotateOutput("template_unittest_test"); string output; tpl_lines->ExpandWithData(&output, dict, &per_expand_data); ASSERT_STREQ_EXCEPT(out->c_str(), output.c_str(), "\r\n"); delete dict; // it's our responsibility } delete tplstr_none; // these are our responsibility too delete tplstr_lines; delete tplstr_ws; } } int main(int argc, char** argv) { // If TEMPLATE_ROOTDIR is set in the environment, it overrides the // default of ".". We use an env-var rather than argv because // that's what automake supports most easily. const char* template_rootdir = getenv("TEMPLATE_ROOTDIR"); if (template_rootdir == NULL) template_rootdir = DEFAULT_TEMPLATE_ROOTDIR; // probably "." string rootdir = ctemplate::PathJoin(template_rootdir, "src"); rootdir = ctemplate::PathJoin(rootdir, "tests"); Template::SetTemplateRootDirectory(rootdir); vector testdata = ReadDataFiles( Template::template_root_directory().c_str()); if (testdata.empty()) { printf("FATAL ERROR: No test files found for template_regtest\n"); return 1; } TestExpand(testdata.begin(), testdata.end()); printf("DONE\n"); return 0; } ctemplate-ctemplate-2.4/src/tests/template_setglobals_unittest.cc000077500000000000000000000056631363122316600255620ustar00rootroot00000000000000// Copyright (c) 2002, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- #include "config_for_unittests.h" #include #include #include #include #include "tests/template_test_util.h" #include "base/util.h" TEST_INIT // defines RUN_ALL_TESTS() using ctemplate::Template; using ctemplate::TemplateDictionary; TEST(SetGlobalValue, TemplateDictionary) { // Test to see that the global dictionary object gets created when you // first call the static function TemplateDictionary::SetGlobalValue(). TemplateDictionary::SetGlobalValue("TEST_GLOBAL_VAR", "test_value"); TemplateDictionary tpl("empty"); ctemplate::TemplateDictionaryPeer peer(&tpl); EXPECT_STREQ(peer.GetSectionValue("TEST_GLOBAL_VAR"), "test_value"); } TEST(SetGlobalValue, SetRootDirectory) { // Test to see that the Template static variables get created when you // first call the static function Template::SetRootDirectory(). Template::SetTemplateRootDirectory("/some/directory/path"); // We don't know if we appended a / or a \, so we test indirectly EXPECT_EQ(strlen("/some/directory/path")+1, // assert they added a char Template::template_root_directory().size()); EXPECT_EQ(0, memcmp(Template::template_root_directory().c_str(), "/some/directory/path", strlen("/some/directory/path"))); } int main(int argc, char **argv) { return RUN_ALL_TESTS(); } ctemplate-ctemplate-2.4/src/tests/template_test_util.cc000066400000000000000000000244661363122316600234770ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- #include "config_for_unittests.h" #include "base/mutex.h" // must come first, for _XOPEN_SOURCE #include "tests/template_test_util.h" #include // for assert() #ifdef HAVE_DIRENT_H # include // for opendir() etc #else # define dirent direct # ifdef HAVE_SYS_NDIR_H # include # endif # ifdef HAVE_SYS_DIR_H # include # endif # ifdef HAVE_NDIR_H # include # endif #endif // for DIR, dirent, closedir(), opendir(), etc #include // for printf(), FILE, fclose(), fopen(), etc #include // for exit() #include // for strcmp(), strcpy(), strstr() #include // for mkdir() #include // for mode_t #include // for time_t #ifdef HAVE_UTIME_H # include #endif // for utime() #ifdef HAVE_UNISTD_H # include #endif // for unlink() #include // for vector<>, vector<>::size_type #include // for Template #include // for TemplateDictionary #include #include // for Strip #include // for TemplateNamelist, etc #include // for PathJoin() #include "base/util.h" // for down_cast() using std::string; using std::vector; #ifdef ASSERT # undef ASSERT #endif #define ASSERT(cond) do { \ if (!(cond)) { \ printf("ASSERT FAILED, line %d: %s\n", __LINE__, #cond); \ assert(cond); \ exit(1); \ } \ } while (0) namespace ctemplate { // Deletes all files named *template* in dir, and sets up dir as the // place where StringToTemplate writes. static char* g_tmpdir = NULL; #ifndef USING_PORT_CC /* windows defines its own version in windows/port.cc */ void CreateOrCleanTestDir(const string& dirname) { DIR* dir = opendir(dirname.c_str()); if (!dir) { // directory doesn't exist or something like that mkdir(dirname.c_str(), 0755); // make the dir if we can return; } while (struct dirent* d = readdir(dir)) { if (strstr(d->d_name, "template")) unlink(PathJoin(dirname, d->d_name).c_str()); } closedir(dir); } static string TmpFile(const char* basename) { return string("/tmp/") + basename; } #endif // #ifndef USING_PORT_CC void CreateOrCleanTestDirAndSetAsTmpdir(const string& dirname) { CreateOrCleanTestDir(dirname); delete[] g_tmpdir; g_tmpdir = new char[dirname.length() + 1]; strcpy(g_tmpdir, dirname.c_str()); } const string FLAGS_test_tmpdir(TmpFile("template_unittest_dir")); // This writes s to the given file. We want to make sure that every // time we create a file, it has a different mtime (just like would // be the case in real life), so we use a mock clock. static Mutex g_time_mutex(base::LINKER_INITIALIZED); static time_t mock_time = 946713600; // jan 1, 2000, in california void StringToFile(const string& s, const string& filename) { FILE* fp = fopen(filename.c_str(), "wb"); ASSERT(fp); size_t r = fwrite(s.data(), 1, s.length(), fp); ASSERT(r == s.length()); fclose(fp); g_time_mutex.Lock(); const time_t file_time = mock_time++; g_time_mutex.Unlock(); struct utimbuf timbuf = { file_time, file_time }; utime(filename.c_str(), &timbuf); } time_t Now() { g_time_mutex.Lock(); const time_t now = mock_time; g_time_mutex.Unlock(); return now; } // This writes s to a file and returns the filename. string StringToTemplateFile(const string& s) { static int filenum = 0; char buf[16]; snprintf(buf, sizeof(buf), "%03d", ++filenum); string filename = PathJoin(g_tmpdir ? g_tmpdir : "", string("template.") + buf); StringToFile(s, filename); return filename; } // This writes s to a file and then loads it into a template object. Template* StringToTemplate(const string& s, Strip strip) { return Template::GetTemplate(StringToTemplateFile(s), strip); } // This is esp. useful for calling from within gdb. // The gdb nice-ness is balanced by the need for the caller to delete the buf. const char* ExpandIs(const Template* tpl, const TemplateDictionary *dict, PerExpandData* per_expand_data, bool expected) { string outstring; if (per_expand_data) ASSERT(expected == tpl->ExpandWithData(&outstring, dict, per_expand_data)); else ASSERT(expected == tpl->Expand(&outstring, dict)); char* buf = new char[outstring.size()+1]; strcpy(buf, outstring.c_str()); return buf; } const char* ExpandWithCacheIs(TemplateCache* cache, const string& filename, Strip strip, const TemplateDictionary *dict, PerExpandData* per_expand_data, bool expected) { string outstring; ASSERT(expected == cache->ExpandWithData(filename, strip, dict, per_expand_data, &outstring)); char* buf = new char[outstring.size()+1]; strcpy(buf, outstring.c_str()); return buf; } void AssertExpandWithDataIs(const Template* tpl, const TemplateDictionary *dict, PerExpandData* per_expand_data, const string& is, bool expected) { const char* buf = ExpandIs(tpl, dict, per_expand_data, expected); if (strcmp(buf, is.c_str())) { printf("expected = '%s'\n", is.c_str()); printf("actual = '%s'\n", buf); } ASSERT(string(buf) == is); delete [] buf; } void AssertExpandIs(const Template* tpl, const TemplateDictionary *dict, const string& is, bool expected) { AssertExpandWithDataIs(tpl, dict, NULL, is, expected); } void AssertExpandWithCacheIs(TemplateCache* cache, const string& filename, Strip strip, const TemplateDictionary *dict, PerExpandData* per_expand_data, const string& is, bool expected) { const char* buf = ExpandWithCacheIs(cache, filename, strip, dict, per_expand_data, expected); if (strcmp(buf, is.c_str())) { printf("expected = '%s'\n", is.c_str()); printf("actual = '%s'\n", buf); } ASSERT(string(buf) == is); delete [] buf; } TemporaryRegisterTemplate::TemporaryRegisterTemplate(const char* name) { old_namelist_ = TemplateNamelist::namelist_; if (old_namelist_) { namelist_ = *old_namelist_; } namelist_.insert(name); TemplateNamelist::namelist_ = &namelist_; } TemporaryRegisterTemplate::~TemporaryRegisterTemplate() { TemplateNamelist::namelist_ = old_namelist_; } const char* TemplateDictionaryPeer::GetSectionValue( const TemplateString& variable) const { // Luckily, TemplateDictionary stores all values with a trailing NUL. return dict_->GetValue(variable).data(); } bool TemplateDictionaryPeer::ValueIs(const TemplateString& variable, const TemplateString& expected) const { return dict_->GetValue(variable) == expected; } bool TemplateDictionaryPeer::IsHiddenSection( const TemplateString& name) const { return dict_->IsHiddenSection(name); } bool TemplateDictionaryPeer::IsUnhiddenSection( const TemplateString& name) const { return dict_->IsUnhiddenSection(name); } bool TemplateDictionaryPeer::IsHiddenTemplate( const TemplateString& name) const { return dict_->IsHiddenTemplate(name); } int TemplateDictionaryPeer::GetSectionDictionaries( const TemplateString& section_name, vector* dicts) const { dicts->clear(); if (dict_->IsHiddenSection(section_name)) return 0; TemplateDictionaryInterface::Iterator* di = dict_->CreateSectionIterator(section_name); while (di->HasNext()) dicts->push_back(down_cast(&di->Next())); delete di; return static_cast(dicts->size()); } int TemplateDictionaryPeer::GetIncludeDictionaries( const TemplateString& section_name, vector* dicts) const { dicts->clear(); if (dict_->IsHiddenTemplate(section_name)) return 0; TemplateDictionaryInterface::Iterator* di = dict_->CreateTemplateIterator(section_name); while (di->HasNext()) dicts->push_back(down_cast(&di->Next())); delete di; return static_cast(dicts->size()); } const char* TemplateDictionaryPeer::GetIncludeTemplateName( const TemplateString& variable, int dictnum) const { return dict_->GetIncludeTemplateName(variable, dictnum); } const char* TemplateDictionaryPeer::GetFilename() const { return dict_->filename_; } } ctemplate-ctemplate-2.4/src/tests/template_test_util.h000077500000000000000000000253041363122316600233340ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // Intended usage of TemplateDictionaryPeer: // Use this class if you need to TEST that a dictionary has certain // expected contents. This should be fairly uncommon outside the // template directory. // #ifndef TEMPLATE_TEMPLATE_TEST_UTIL_H_ #define TEMPLATE_TEMPLATE_TEST_UTIL_H_ #include "config_for_unittests.h" #include // for time_t #include // for string #include // for vector<> #include HASH_MAP_H // UNUSED #include // for Template::num_deletes_ #include // for TemplateCache #include // for TemplateDictionary #include #include // for Strip #include #include // for TemplateString, TemplateId namespace ctemplate { using std::string; class PerExpandData; class TemplateCache; class TemplateDictionary; inline TemplateId GlobalIdForTest(const char* ptr, int len) { return TemplateString(ptr, len).GetGlobalId(); } // Call this to create a StaticTemplateString for testing when the ptr is // not guaranteed to be allocated for the entire length of the test. #define STS_INIT_FOR_TEST(ptr, len, arena) \ { { arena->Memdup(ptr, len), len, ctemplate::GlobalIdForTest(ptr, len) } }; extern const std::string FLAGS_test_tmpdir; // These are routines that are useful for creating template files for testing. // Deletes all files named *template* in dir. void CreateOrCleanTestDir(const string& dirname); // This delets all files named *template*, and also sets dirname to be // the directory that all future StringToFile calls will place their // templates. void CreateOrCleanTestDirAndSetAsTmpdir(const string& dirname); // This writes s to the given file. We want to make sure that every // time we create a file, it has a different mtime (just like would // be the case in real life), so we use a mock clock. Filenames created // by this routine will all have an mtime of around Jan 1, 2000. void StringToFile(const string& s, const string& filename); // This is the (mock) time used when creating the last file in StringToFile. time_t Now(); // This writes s to a file and returns the filename. string StringToTemplateFile(const string& s); // This writes s to a file and then loads it into a template object. Template* StringToTemplate(const string& s, Strip strip); // This is esp. useful for calling from within gdb. // The gdb nice-ness is balanced by the need for the caller to delete the buf. const char* ExpandIs(const Template* tpl, const TemplateDictionary *dict, PerExpandData* per_expand_data, bool expected); void AssertExpandWithDataIs(const Template* tpl, const TemplateDictionary *dict, PerExpandData* per_expand_data, const string& is, bool expected); void AssertExpandIs(const Template* tpl, const TemplateDictionary *dict, const string& is, bool expected); void AssertExpandWithCacheIs(TemplateCache* cache, const string& filename, Strip strip, const TemplateDictionary *dict, PerExpandData* per_expand_data, const string& is, bool expected); class TemporaryRegisterTemplate { public: explicit TemporaryRegisterTemplate(const char* name); ~TemporaryRegisterTemplate(); private: ctemplate::TemplateNamelist::NameListType* old_namelist_; ctemplate::TemplateNamelist::NameListType namelist_; // disallow copy constructor and assignment TemporaryRegisterTemplate(const TemporaryRegisterTemplate&); void operator=(const TemporaryRegisterTemplate&); }; // For friendship reasons, we make this a top-level class rather // than a nested class. It's used only in TemplateDictionaryPeer. // We take ownership of the iterator passed to us. To make sure that // isn't a problem, we make this class not-copyable. class TemplateDictionaryPeerIterator { public: explicit TemplateDictionaryPeerIterator( TemplateDictionaryInterface::Iterator* it) : it_(it) { } ~TemplateDictionaryPeerIterator() { delete it_; } bool HasNext() const { return it_->HasNext(); } const TemplateDictionaryInterface& Next() { return it_->Next(); } private: TemplateDictionaryInterface::Iterator* it_; TemplateDictionaryPeerIterator(const TemplateDictionaryPeerIterator&); TemplateDictionaryPeerIterator& operator=( const TemplateDictionaryPeerIterator&); }; // This class is meant for use in unittests. This class wraps the // TemplateDictionary and provides access to internal data that should // not be used in production code. If you need this kind of // functionality in production, use TemplateDictionaryWrapper or // TemplateDictionaryInterface; see top of file for details. // // Example Usage: // TemplateDictionary dict("test dictionary"); // FillDictionaryValues(&dict); // // TemplateDictionaryPeer peer(&dict); // EXPECT_EQ("5", peer.GetSectionValue("width")); class TemplateDictionaryPeer { public: explicit TemplateDictionaryPeer(const TemplateDictionary* dict) : dict_(dict) {} // Returns whether the named variable has value equal to "expected". bool ValueIs(const TemplateString& variable, const TemplateString& expected) const; // DEPRECATED: Returns the value of the named variable. Does not // deal properly with values that have an internal NUL. Use ValueIs // for new code. const char* GetSectionValue(const TemplateString& variable) const; // Returns true if the named section is hidden. bool IsHiddenSection(const TemplateString& name) const; // IsUnhiddenSection // Returns true if the section has been marked visible and false otherwise. bool IsUnhiddenSection(const TemplateString& name) const; // Returns true if the named sub-template is hidden. bool IsHiddenTemplate(const TemplateString& name) const; // Retrieves TemplateDictionary instances for the given section name. The // caller does not assume ownership of the returned TemplateDictionary // instances. The number of instances is returned. All prior entries in // the dicts vector are cleared. // // NOTE: This method assumes that old-style template dictionaries are not in // use. That is, it assumes that all section dictionaries have been added // with AddSectionDictionary rather than AddOldstyleSectionDictionary. int GetSectionDictionaries(const TemplateString& section_name, std::vector* dicts) const; // Retrieves included TemplateDictionary instances for the given name. The // caller does not assume ownership of the returned TemplateDictionary // instances. The number of instances is returned. All prior entries in // the dicts vector are cleared. // // NOTE: This method assumes that old-style template dictionaries are not in // use. That is, it assumes that all section dictionaries have been added // with AddIncludeDictionary rather than AddOldstyleIncludeDictionary. int GetIncludeDictionaries(const TemplateString& section_name, std::vector* dicts) const; const char* GetIncludeTemplateName(const TemplateString& variable, int dictnum) const; typedef TemplateDictionaryPeerIterator Iterator; Iterator* CreateTemplateIterator(const TemplateString& section) const { return new Iterator(dict_->CreateTemplateIterator(section)); } Iterator* CreateSectionIterator(const TemplateString& section) const { return new Iterator(dict_->CreateSectionIterator(section)); } // Returns the filename associated with the TemplateDictionary. const char* GetFilename() const; private: const TemplateDictionary* dict_; // Not owned. // disallow copy constructor and assignment TemplateDictionaryPeer(const TemplateDictionaryPeer&); void operator=(const TemplateDictionaryPeer&); }; class TemplateCachePeer { public: TemplateCachePeer(TemplateCache* cache) : cache_(cache) {} struct TemplateCacheKey : public TemplateCache::TemplateCacheKey { TemplateCacheKey(const string& key, int strip) { this->first = GlobalIdForTest(key.data(), key.length()); this->second = strip; } }; TemplateCache::TemplateMap* parsed_template_cache() { return cache_->parsed_template_cache_; } bool TemplateIsCached(const TemplateCacheKey key) const { return cache_->TemplateIsCached(key); } const Template* GetTemplate(const TemplateString& key, Strip strip) const { return cache_->GetTemplate(key, strip); } int Refcount(const TemplateCacheKey key) const { return cache_->Refcount(key); } void DoneWithGetTemplatePtrs() { cache_->DoneWithGetTemplatePtrs(); } void ClearCache() { cache_->ClearCache(); } static int NumTotalTemplateDeletes() { return Template::num_deletes(); } private: TemplateCache* cache_; // Not owned. // Don't allow copying TemplateCachePeer(const TemplateCachePeer&); void operator=(const TemplateCachePeer&); }; } #endif // TEMPLATE_TEMPLATE_TEST_UTIL_H_ ctemplate-ctemplate-2.4/src/tests/template_test_util_test.cc000077500000000000000000000222371363122316600245330ustar00rootroot00000000000000// Copyright (c) 2008, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. #include "config_for_unittests.h" #include "tests/template_test_util.h" #include #include #include #include "base/arena.h" #include #include #include "base/util.h" TEST_INIT // defines RUN_ALL_TESTS() #define ASSERT_EQ(a, b) EXPECT_EQ(a, b) using std::vector; using std::string; using ctemplate::UnsafeArena; using ctemplate::TemplateDictionary; using ctemplate::TemplateDictionaryPeer; using ctemplate::TemplateString; using ctemplate::StaticTemplateString; namespace { TEST(TemplateTestUtilTest, GetSectionValue) { TemplateDictionary dict("test_GetSectionValue"); dict.SetValue("VALUE", "value"); TemplateDictionaryPeer peer(&dict); EXPECT_STREQ("value", peer.GetSectionValue("VALUE")); } TEST(TemplateTestUtilTest, IsHiddenSection) { TemplateDictionary dict("test_IsHiddenSection"); { TemplateDictionaryPeer peer(&dict); EXPECT_TRUE(peer.IsHiddenSection("SECTION")); } dict.AddSectionDictionary("SECTION"); { TemplateDictionaryPeer peer(&dict); EXPECT_FALSE(peer.IsHiddenSection("SECTION")); } } TEST(TemplateTestUtilTest, GetSectionDictionaries) { TemplateDictionary dict("test_GetSectionDictionaries"); { TemplateDictionaryPeer peer(&dict); vector dicts; // Add some dummy value into the vector to confirm that the call to // GetSectionDictionaries will correctly clear the vector. dicts.push_back(NULL); EXPECT_EQ(0, peer.GetSectionDictionaries("SECTION", &dicts)); EXPECT_TRUE(dicts.empty()); } dict.AddSectionDictionary("SECTION")->SetValue("SECTION_VALUE", "0"); { TemplateDictionaryPeer peer(&dict); vector dicts; ASSERT_EQ(1, peer.GetSectionDictionaries("SECTION", &dicts)); TemplateDictionaryPeer peer_section(dicts[0]); EXPECT_STREQ("0", peer_section.GetSectionValue("SECTION_VALUE")); } dict.AddSectionDictionary("SECTION")->SetValue("SECTION_VALUE", "1"); dict.AddSectionDictionary("ANOTHER_SECTION")->SetValue("ANOTHER_VALUE", "2"); { TemplateDictionaryPeer peer(&dict); vector dicts; ASSERT_EQ(2, peer.GetSectionDictionaries("SECTION", &dicts)); TemplateDictionaryPeer peer_section0(dicts[0]); EXPECT_STREQ("0", peer_section0.GetSectionValue("SECTION_VALUE")); TemplateDictionaryPeer peer_section1(dicts[1]); EXPECT_STREQ("1", peer_section1.GetSectionValue("SECTION_VALUE")); } } TEST(TemplateTestUtilTest, GetIncludeDictionaries) { TemplateDictionary dict("test_GetIncludeDictionaries"); { TemplateDictionaryPeer peer(&dict); vector dicts; // Add some dummy value into the vector to confirm that the call to // GetSectionDictionaries will correctly clear the vector. dicts.push_back(NULL); EXPECT_EQ(0, peer.GetIncludeDictionaries("SECTION", &dicts)); EXPECT_TRUE(dicts.empty()); } dict.AddIncludeDictionary("SECTION")->SetValue("SECTION_VALUE", "0"); { TemplateDictionaryPeer peer(&dict); vector dicts; ASSERT_EQ(1, peer.GetIncludeDictionaries("SECTION", &dicts)); TemplateDictionaryPeer peer_section(dicts[0]); EXPECT_STREQ("0", peer_section.GetSectionValue("SECTION_VALUE")); } dict.AddIncludeDictionary("SECTION")->SetValue("SECTION_VALUE", "1"); dict.AddIncludeDictionary("ANOTHER_SECTION")->SetValue("ANOTHER_VALUE", "2"); { TemplateDictionaryPeer peer(&dict); vector dicts; ASSERT_EQ(2, peer.GetIncludeDictionaries("SECTION", &dicts)); TemplateDictionaryPeer peer_section0(dicts[0]); EXPECT_STREQ("0", peer_section0.GetSectionValue("SECTION_VALUE")); TemplateDictionaryPeer peer_section1(dicts[1]); EXPECT_STREQ("1", peer_section1.GetSectionValue("SECTION_VALUE")); } } TEST(TemplateTestUtilTest, GetIncludeAndSectionDictionaries) { TemplateDictionary dict("test_GetIncludeAndSectionDictionaries"); { TemplateDictionaryPeer peer(&dict); vector dicts; EXPECT_EQ(0, peer.GetIncludeDictionaries("SECTION", &dicts)); EXPECT_EQ(0, peer.GetSectionDictionaries("SECTION", &dicts)); } dict.AddIncludeDictionary("SECTION")->SetValue("SECTION_VALUE", "0"); dict.AddSectionDictionary("SECTION")->SetValue("SECTION_VALUE", "1"); { TemplateDictionaryPeer peer(&dict); vector include_dicts; ASSERT_EQ(1, peer.GetIncludeDictionaries("SECTION", &include_dicts)); TemplateDictionaryPeer include_peer(include_dicts[0]); EXPECT_STREQ("0", include_peer.GetSectionValue("SECTION_VALUE")); vector section_dicts; ASSERT_EQ(1, peer.GetSectionDictionaries("SECTION", §ion_dicts)); TemplateDictionaryPeer section_peer(section_dicts[0]); EXPECT_STREQ("1", section_peer.GetSectionValue("SECTION_VALUE")); } dict.AddIncludeDictionary("SECTION")->SetValue("SECTION_VALUE", "2"); dict.AddIncludeDictionary("ANOTHER_SECTION")->SetValue("ANOTHER_VALUE", "3"); dict.AddSectionDictionary("SECTION")->SetValue("SECTION_VALUE", "4"); dict.AddSectionDictionary("ONE_MORE_SECTION")->SetValue("ANOTHER_VALUE", "5"); { TemplateDictionaryPeer peer(&dict); vector dicts; ASSERT_EQ(2, peer.GetIncludeDictionaries("SECTION", &dicts)); TemplateDictionaryPeer include_peer0(dicts[0]); EXPECT_STREQ("0", include_peer0.GetSectionValue("SECTION_VALUE")); TemplateDictionaryPeer include_peer1(dicts[1]); EXPECT_STREQ("2", include_peer1.GetSectionValue("SECTION_VALUE")); EXPECT_EQ(1, peer.GetIncludeDictionaries("ANOTHER_SECTION", &dicts)); EXPECT_EQ(0, peer.GetIncludeDictionaries("ONE_MORE_SECTION", &dicts)); vector section_dicts; ASSERT_EQ(2, peer.GetSectionDictionaries("SECTION", §ion_dicts)); TemplateDictionaryPeer section_peer0(section_dicts[0]); EXPECT_STREQ("1", section_peer0.GetSectionValue("SECTION_VALUE")); TemplateDictionaryPeer section_peer1(section_dicts[1]); EXPECT_STREQ("4", section_peer1.GetSectionValue("SECTION_VALUE")); EXPECT_EQ(0, peer.GetSectionDictionaries("ANOTHER_SECTION", &dicts)); EXPECT_EQ(1, peer.GetSectionDictionaries("ONE_MORE_SECTION", &dicts)); } } TEST(TemplateTestUtilTest, GetFilename) { TemplateDictionary parent("test_GetFilename"); TemplateDictionary* child = parent.AddIncludeDictionary("INCLUDE_marker"); child->SetFilename("included_filename"); TemplateDictionaryPeer parent_peer(&parent); EXPECT_EQ(NULL, parent_peer.GetFilename()); TemplateDictionaryPeer child_peer(child); EXPECT_STREQ("included_filename", child_peer.GetFilename()); } StaticTemplateString GetTestTemplateString(UnsafeArena* arena) { string will_go_out_of_scope("VALUE"); // We want to ensure that the STS_INIT_FOR_TEST macro: // - Can produce a StaticTemplateString (guard again its format changing). // - Produces a StaticTemplateString that is still valid after the string // used to initialize it goes out-of-scope. StaticTemplateString sts = STS_INIT_FOR_TEST(will_go_out_of_scope.c_str(), will_go_out_of_scope.length(), arena); return sts; } TEST(TemplateUtilTest, InitStaticTemplateStringForTest) { UnsafeArena arena(1024); StaticTemplateString kValue = GetTestTemplateString(&arena); TemplateDictionary dict("test_GetSectionValue"); dict.SetValue(kValue, "value"); TemplateDictionaryPeer peer(&dict); EXPECT_STREQ("value", peer.GetSectionValue(kValue)); } } // namespace anonymous int main(int argc, char **argv) { return RUN_ALL_TESTS(); } ctemplate-ctemplate-2.4/src/tests/template_unittest.cc000077500000000000000000002670221363122316600233420ustar00rootroot00000000000000// Copyright (c) 2005, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) #include "config_for_unittests.h" #include #include // for assert() #if defined(HAVE_PTHREAD) && !defined(NO_THREADS) # include #endif // for pthread_t, pthread_create(), etc #include // for size_t #include // for printf(), FILE, snprintf(), fclose(), etc #include // for exit() #include // for strcmp(), memchr(), strlen(), strstr() #include // for mode_t #include // for time_t, time() #ifdef HAVE_UNISTD_H # include #endif // for link(), unlink() #include // for list<>::size_type #include // for vector<> #include // for PerExpandData #include // for TextTemplateAnnotator #include // for TemplateDictionary #include // for ExpandEmitter #include // for STRIP_WHITESPACE, Strip, etc #include // for AddModifier(), HtmlEscape, etc #include // for TemplateNamelist, etc #include // for PathJoin(), IsAbspath(), etc #include // for TemplateString, StringHash, etc #include "tests/template_test_util.h" // for StringToTemplate(), etc #include "base/util.h" TEST_INIT // defines RUN_ALL_TESTS() using std::vector; using std::string; using ctemplate::FLAGS_test_tmpdir; using ctemplate::AssertExpandIs; using ctemplate::AssertExpandWithDataIs; using ctemplate::CreateOrCleanTestDir; using ctemplate::CreateOrCleanTestDirAndSetAsTmpdir; using ctemplate::DO_NOT_STRIP; using ctemplate::ExpandEmitter; using ctemplate::IsAbspath; using ctemplate::Now; using ctemplate::PathJoin; using ctemplate::PerExpandData; using ctemplate::STRIP_BLANK_LINES; using ctemplate::STRIP_WHITESPACE; using ctemplate::StaticTemplateString; using ctemplate::StringToFile; using ctemplate::StringToTemplate; using ctemplate::StringToTemplateFile; using ctemplate::Strip; using ctemplate::TC_CSS; using ctemplate::TC_HTML; using ctemplate::TC_JS; using ctemplate::TC_JSON; using ctemplate::TC_MANUAL; using ctemplate::TC_UNUSED; using ctemplate::TC_XML; using ctemplate::Template; using ctemplate::TemplateContext; using ctemplate::TemplateDictionary; using ctemplate::TemplateNamelist; using ctemplate::TemplateString; using ctemplate::kRootdir; using ctemplate::ExpandTemplate; using ctemplate::ExpandWithData; using ctemplate::StringToTemplateCache; static const StaticTemplateString kHello = STS_INIT(kHello, "Hello"); static const StaticTemplateString kWorld = STS_INIT(kWorld, "World"); static const char* kPragmaHtml = "{{%AUTOESCAPE context=\"HTML\"}}\n"; static const char* kPragmaJs = "{{%AUTOESCAPE context=\"JAVASCRIPT\"}}\n"; static const char* kPragmaCss = "{{%AUTOESCAPE context=\"CSS\"}}\n"; static const char* kPragmaXml = "{{%AUTOESCAPE context=\"XML\"}}\n"; static const char* kPragmaJson = "{{%AUTOESCAPE context=\"JSON\"}}\n"; // How many threads to use for our threading test. // This is a #define instead of a const int so we can use it in array-sizes // even on c++ compilers that don't support var-length arrays. #define kNumThreads 10 #define PFATAL(s) do { perror(s); exit(1); } while (0) // TODO(csilvers): rewrite to be more gunit-like: use expectations // instead of asserts, and move assert-checking out of helper routines // and into tests proper. Ideally, replace AssertExpandIs() with // VerifyExpandIs(). #define ASSERT(cond) do { \ if (!(cond)) { \ printf("ASSERT FAILED, line %d: %s\n", __LINE__, #cond); \ assert(cond); \ exit(1); \ } \ } while (0) #define ASSERT_STREQ_EXCEPT(a, b, except) ASSERT(StreqExcept(a, b, except)) #define ASSERT_STREQ(a, b) ASSERT(strcmp(a, b) == 0) #define ASSERT_NOT_STREQ(a, b) ASSERT(strcmp(a, b) != 0) #define ASSERT_STREQ_VERBOSE(a, b, c) ASSERT(StrEqVerbose(a, b, c)) #define ASSERT_INTEQ(a, b) ASSERT(IntEqVerbose(a, b)) namespace { // First, (conceptually) remove all chars in "except" from both a and b. // Then return true iff munged_a == munged_b. bool StreqExcept(const char* a, const char* b, const char* except) { const char* pa = a, *pb = b; const size_t exceptlen = strlen(except); while (1) { // Use memchr isntead of strchr because memchr(foo, '\0') always fails while (memchr(except, *pa, exceptlen)) pa++; // ignore "except" chars in a while (memchr(except, *pb, exceptlen)) pb++; // ignore "except" chars in b if ((*pa == '\0') && (*pb == '\0')) return true; if (*pa++ != *pb++) // includes case where one is at \0 return false; } } // If a and b do not match, print their values and that of text // and return false. bool StrEqVerbose(const string& a, const string& b, const string& text) { if (a != b) { printf("EXPECTED: %s\n", a.c_str()); printf("ACTUAL: %s\n", b.c_str()); printf("TEXT: %s\n", text.c_str()); return false; } return true; } bool IntEqVerbose(int a, int b) { if (a != b) { printf("EXPECTED: %d\n", a); printf("ACTUAL: %d\n", b); return false; } return true; } // This test emitter writes to a string, but writes X's of the right // length, rather than the actual content passed in. class SizeofEmitter : public ExpandEmitter { string* const outbuf_; public: SizeofEmitter(string* outbuf) : outbuf_(outbuf) {} virtual void Emit(char c) { Emit(&c, 1); } virtual void Emit(const string& s) { Emit(s.data(), s.length()); } virtual void Emit(const char* s) { Emit(s, strlen(s)); } virtual void Emit(const char*, size_t slen) { outbuf_->append(slen, 'X'); } }; } // unnamed namespace RegisterTemplateFilename(VALID1_FN, "template_unittest_test_valid1.in"); RegisterTemplateFilename(INVALID1_FN, "template_unittest_test_invalid1.in"); RegisterTemplateFilename(INVALID2_FN, "template_unittest_test_invalid2.in"); RegisterTemplateFilename(NONEXISTENT_FN, "nonexistent__file.tpl"); // Returns the proper AUTOESCAPE pragma that corresponds to the // given TemplateContext. static string GetPragmaForContext(TemplateContext context) { switch(context) { case TC_HTML: return kPragmaHtml; case TC_JS: return kPragmaJs; case TC_CSS: return kPragmaCss; case TC_JSON: return kPragmaJson; case TC_XML: return kPragmaXml; case TC_MANUAL: return ""; // No AUTOESCAPE pragma. case TC_UNUSED: ASSERT(false); // Developer error, this TC is not to be used. } ASSERT(false); // Developer error - invalid TemplateContext. return ""; } // This writes s to a file with the AUTOESCAPE pragma corresponding // to the given TemplateContext and then loads it into a template object. static Template* StringToTemplateWithAutoEscaping(const string& s, Strip strip, TemplateContext context) { string text = GetPragmaForContext(context) + s; return Template::GetTemplate(StringToTemplateFile(text), strip); } // A helper method used by TestCorrectModifiersForAutoEscape. // Populates out with lines of the form: // VARNAME:mod1[=val1][:mod2[=val2]]...\n from the dump of the template // and compares against the expected string. static void AssertCorrectModifiersInTemplate(Template* tpl, const string& text, const string& expected_out) { ASSERT(tpl); string dump_out, out; tpl->DumpToString("bogus_filename", &dump_out); string::size_type i, j; i = 0; while ((i = dump_out.find("Variable Node: ", i)) != string::npos) { i += strlen("Variable Node: "); j = dump_out.find("\n", i); out.append(dump_out.substr(i, j - i)); // should be safe. out.append("\n"); } ASSERT_STREQ_VERBOSE(expected_out, out, text); } // Wrapper on top of AssertCorrectModifiersInTemplate which first // obtains a template from the given contents and template context. static void AssertCorrectModifiers(TemplateContext template_type, const string& text, const string& expected_out) { Strip strip = STRIP_WHITESPACE; Template *tpl = StringToTemplateWithAutoEscaping(text, strip, template_type); AssertCorrectModifiersInTemplate(tpl, text, expected_out); } // A helper method used by TestCorrectModifiersForAutoEscape. // Initializes the template in the Auto Escape mode with the // given TemplateContext, expands it with the given dictionary // and checks that the output matches the expected value. static void AssertCorrectEscaping(TemplateContext template_type, const TemplateDictionary& dict, const string& text, const string& expected_out) { Strip strip = STRIP_WHITESPACE; Template *tpl = StringToTemplateWithAutoEscaping(text, strip, template_type); string outstring; tpl->Expand(&outstring, &dict); ASSERT_STREQ_VERBOSE(expected_out, outstring, text); } class DynamicModifier : public ctemplate::TemplateModifier { public: void Modify(const char* in, size_t inlen, const PerExpandData* per_expand_data, ExpandEmitter* outbuf, const string& arg) const { assert(arg.empty()); // we don't take an argument assert(per_expand_data); const char* value = per_expand_data->LookupForModifiersAsString("value"); if (value) outbuf->Emit(value); } }; class EmphasizeTemplateModifier : public ctemplate::TemplateModifier { public: EmphasizeTemplateModifier(const string& match) : match_(match) { } bool MightModify(const PerExpandData* per_expand_data, const string& arg) const { return strstr(arg.c_str(), match_.c_str()); } void Modify(const char* in, size_t inlen, const PerExpandData* per_expand_data, ExpandEmitter* outbuf, const string& arg) const { outbuf->Emit(">>"); outbuf->Emit(in, inlen); outbuf->Emit("<<"); } private: string match_; }; // This is used by TestAnnotation(). It behaves like // TextTemplateAnnotator but just to test our ability to customize // annotation, and with stateful one, it prefixes each text annotation // with an event (call) count. class CustomTestAnnotator : public ctemplate::TextTemplateAnnotator { public: CustomTestAnnotator() : event_count_(0) { } void Reset() { event_count_ = 0; } virtual void EmitOpenInclude(ExpandEmitter* emitter, const string& value) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitOpenInclude(emitter, value); } virtual void EmitCloseInclude(ExpandEmitter* emitter) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitCloseInclude(emitter); } virtual void EmitOpenFile(ExpandEmitter* emitter, const string& value) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitOpenFile(emitter, value); } virtual void EmitCloseFile(ExpandEmitter* emitter) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitCloseFile(emitter); } virtual void EmitOpenSection(ExpandEmitter* emitter, const string& value) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitOpenSection(emitter, value); } virtual void EmitCloseSection(ExpandEmitter* emitter) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitCloseSection(emitter); } virtual void EmitOpenVariable(ExpandEmitter* emitter, const string& value) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitOpenVariable(emitter, value); } virtual void EmitCloseVariable(ExpandEmitter* emitter) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitCloseVariable(emitter); } virtual void EmitFileIsMissing(ExpandEmitter* emitter, const string& value) { EmitTestPrefix(emitter); ctemplate::TextTemplateAnnotator::EmitFileIsMissing(emitter, value); } private: void EmitTestPrefix(ExpandEmitter* emitter) { char buf[128]; snprintf(buf, sizeof(buf), "{{EVENT=%d}}", ++event_count_); emitter->Emit(buf); } int event_count_; DISALLOW_COPY_AND_ASSIGN(CustomTestAnnotator); }; class TemplateForTest : public Template { public: using Template::kSafeWhitelistedVariables; using Template::kNumSafeWhitelistedVariables; private: // This quiets gcc3, which otherwise complains: "base `Template' // with only non-default constructor in class without a constructor". TemplateForTest(); }; // Tests annotation, in particular inheriting annotation among children // This should be called first, so the filenames don't change as we add // more tests. static void TestAnnotation() { string incname = StringToTemplateFile("include {{#ISEC}}file{{/ISEC}}\n"); string incname2 = StringToTemplateFile("include #2\n"); Template* tpl = StringToTemplate( "boo!\n{{>INC}}\nhi {{#SEC}}lo{{#SUBSEC}}jo{{/SUBSEC}}{{/SEC}} bar " "{{VAR:x-foo}}", DO_NOT_STRIP); TemplateDictionary dict("dict"); PerExpandData per_expand_data; dict.ShowSection("SEC"); TemplateDictionary* incdict = dict.AddIncludeDictionary("INC"); incdict->SetFilename(incname); incdict->ShowSection("ISEC"); dict.AddIncludeDictionary("INC")->SetFilename(incname2); dict.SetValue("VAR", "var"); // This string is equivalent to "/template." (at least on unix) string slash_tpl(PathJoin(kRootdir, "template.")); per_expand_data.SetAnnotateOutput(""); char expected[10240]; // 10k should be big enough! snprintf(expected, sizeof(expected), "{{#FILE=%s003}}{{#SEC=__{{MAIN}}__}}boo!\n" "{{#INC=INC}}{{#FILE=%s001}}" "{{#SEC=__{{MAIN}}__}}include {{#SEC=ISEC}}file{{/SEC}}\n" "{{/SEC}}{{/FILE}}{{/INC}}" "{{#INC=INC}}{{#FILE=%s002}}" "{{#SEC=__{{MAIN}}__}}include #2\n{{/SEC}}{{/FILE}}{{/INC}}" "\nhi {{#SEC=SEC}}lo{{/SEC}} bar " "{{#VAR=VAR:x-foo}}var{{/VAR}}{{/SEC}}{{/FILE}}", (FLAGS_test_tmpdir + slash_tpl).c_str(), (FLAGS_test_tmpdir + slash_tpl).c_str(), (FLAGS_test_tmpdir + slash_tpl).c_str()); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, expected, true); // Test ability to set custom annotator. CustomTestAnnotator custom_annotator; per_expand_data.SetAnnotator(&custom_annotator); snprintf(expected, sizeof(expected), "{{EVENT=1}}{{#FILE=%s003}}" "{{EVENT=2}}{{#SEC=__{{MAIN}}__}}boo!\n" "{{EVENT=3}}{{#INC=INC}}" "{{EVENT=4}}{{#FILE=%s001}}" "{{EVENT=5}}{{#SEC=__{{MAIN}}__}}include " "{{EVENT=6}}{{#SEC=ISEC}}file" "{{EVENT=7}}{{/SEC}}\n" "{{EVENT=8}}{{/SEC}}" "{{EVENT=9}}{{/FILE}}" "{{EVENT=10}}{{/INC}}" "{{EVENT=11}}{{#INC=INC}}" "{{EVENT=12}}{{#FILE=%s002}}" "{{EVENT=13}}{{#SEC=__{{MAIN}}__}}include #2\n" "{{EVENT=14}}{{/SEC}}" "{{EVENT=15}}{{/FILE}}" "{{EVENT=16}}{{/INC}}\nhi " "{{EVENT=17}}{{#SEC=SEC}}lo" "{{EVENT=18}}{{/SEC}} bar " "{{EVENT=19}}{{#VAR=VAR:x-foo}}var" "{{EVENT=20}}{{/VAR}}" "{{EVENT=21}}{{/SEC}}" "{{EVENT=22}}{{/FILE}}", (FLAGS_test_tmpdir + slash_tpl).c_str(), (FLAGS_test_tmpdir + slash_tpl).c_str(), (FLAGS_test_tmpdir + slash_tpl).c_str()); // We can't use AssertExpandWithDataIs() on our deliberately stateful // test annotator because it internally does a second expansion // assuming no state change between calls. string custom_outstring; ASSERT(tpl->ExpandWithData(&custom_outstring, &dict, &per_expand_data)); ASSERT_STREQ(custom_outstring.c_str(), expected); // Unset annotator and continue with next test as test of ability // to revert to built-in annotator. per_expand_data.SetAnnotator(NULL); per_expand_data.SetAnnotateOutput(slash_tpl.c_str()); snprintf(expected, sizeof(expected), "{{#FILE=%s003}}{{#SEC=__{{MAIN}}__}}boo!\n" "{{#INC=INC}}{{#FILE=%s001}}" "{{#SEC=__{{MAIN}}__}}include {{#SEC=ISEC}}file{{/SEC}}\n" "{{/SEC}}{{/FILE}}{{/INC}}" "{{#INC=INC}}{{#FILE=%s002}}" "{{#SEC=__{{MAIN}}__}}include #2\n{{/SEC}}{{/FILE}}{{/INC}}" "\nhi {{#SEC=SEC}}lo{{/SEC}} bar " "{{#VAR=VAR:x-foo}}var{{/VAR}}{{/SEC}}{{/FILE}}", (slash_tpl).c_str(), (slash_tpl).c_str(), (slash_tpl).c_str()); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, expected, true); per_expand_data.SetAnnotateOutput(NULL); // should turn off annotations AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "boo!\ninclude file\ninclude #2\n\nhi lo bar var", true); // Test that even if we set an annotator we shouldn't get annotation // if it is not turned on with SetAnnotateOutput(). per_expand_data.SetAnnotator(&custom_annotator); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "boo!\ninclude file\ninclude #2\n\nhi lo bar var", true); // Test annotation of "missing include" condition. Template* one_inc_tpl = StringToTemplate("File contents: {{>INC}}\n", DO_NOT_STRIP); TemplateDictionary dict_missing_file("dict_with_missing_file"); dict_missing_file.AddIncludeDictionary("INC")->SetFilename("missing.tpl"); per_expand_data.SetAnnotateOutput(""); per_expand_data.SetAnnotator(NULL); snprintf(expected, sizeof(expected), "{{#FILE=%s004}}{{#SEC=__{{MAIN}}__}}File contents: " "{{#INC=INC}}{{MISSING_FILE=missing.tpl}}{{/INC}}\n" "{{/SEC}}{{/FILE}}", (FLAGS_test_tmpdir + slash_tpl).c_str()); // We expect a false return value because of the missing file. AssertExpandWithDataIs(one_inc_tpl, &dict_missing_file, &per_expand_data, expected, false); // Same missing include test with custom annotator custom_annotator.Reset(); per_expand_data.SetAnnotator(&custom_annotator); snprintf(expected, sizeof(expected), "{{EVENT=1}}{{#FILE=%s004}}" "{{EVENT=2}}{{#SEC=__{{MAIN}}__}}File contents: " "{{EVENT=3}}{{#INC=INC}}" "{{EVENT=4}}{{MISSING_FILE=missing.tpl}}" "{{EVENT=5}}{{/INC}}\n" "{{EVENT=6}}{{/SEC}}" "{{EVENT=7}}{{/FILE}}", (FLAGS_test_tmpdir + slash_tpl).c_str()); // See comment above on why we can't use AssertExpandWithDataIs() for // our stateful test annotator. custom_outstring.clear(); ASSERT(!one_inc_tpl->ExpandWithData(&custom_outstring, &dict_missing_file, &per_expand_data)); ASSERT_STREQ(custom_outstring.c_str(), expected); } TEST(Template, CheckWhitelistedVariablesSorted) { // NOTE(williasr): kSafeWhitelistedVariables must be sorted, it's accessed // using binary search. for (size_t i = 1; i < TemplateForTest::kNumSafeWhitelistedVariables; i++) { assert(strcmp(TemplateForTest::kSafeWhitelistedVariables[i-1], TemplateForTest::kSafeWhitelistedVariables[i]) < 0); } } // The following tests test various aspects of how Expand() should behave. TEST(Template, WeirdSyntax) { TemplateDictionary dict("dict"); // When we see {{{, we should match the second {{, not the first. Template* tpl1 = StringToTemplate("hi {{{! VAR {{!VAR} }} lo", STRIP_WHITESPACE); AssertExpandIs(tpl1, &dict, "hi { lo", true); // Likewise for }}} Template* tpl2 = StringToTemplate("fn(){{{BI_NEWLINE}} x=4;{{BI_NEWLINE}}}", DO_NOT_STRIP); AssertExpandIs(tpl2, &dict, "fn(){\n x=4;\n}", true); // Try lots of {'s! Template* tpl3 = StringToTemplate("{{{{{{VAR}}}}}}}}", DO_NOT_STRIP); AssertExpandIs(tpl3, &dict, "{{{{}}}}}}", true); } TEST(Template, Comment) { TemplateDictionary dict("dict"); Template* tpl1 = StringToTemplate("hi {{!VAR}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl1, &dict, "hi lo", true); Template* tpl2 = StringToTemplate("hi {{!VAR {VAR} }} lo", STRIP_WHITESPACE); AssertExpandIs(tpl2, &dict, "hi lo", true); Template* tpl3 = StringToTemplate("hi {{! VAR {{!VAR} }} lo", STRIP_WHITESPACE); AssertExpandIs(tpl3, &dict, "hi lo", true); } TEST(Template, SetMarkerDelimiters) { TemplateDictionary dict("dict"); dict.SetValue("VAR", "yo"); Template* tpl1 = StringToTemplate("{{=| |=}}\nhi |VAR| {{lo}}", STRIP_WHITESPACE); AssertExpandIs(tpl1, &dict, "hi yo {{lo}}", true); Template* tpl2 = StringToTemplate("{{=| |=}}hi |VAR| {{lo}}", STRIP_WHITESPACE); AssertExpandIs(tpl2, &dict, "hi yo {{lo}}", true); Template* tpl3 = StringToTemplate("{{=| ||=}}hi ||VAR|||VAR|| {{lo}}", STRIP_WHITESPACE); AssertExpandIs(tpl3, &dict, "hi |yoyo {{lo}}", true); Template* tpl4 = StringToTemplate("{{=< >=}}hi <> {{lo}}", STRIP_WHITESPACE); AssertExpandIs(tpl4, &dict, "hi {{lo}}", true); Template* tpl4b = StringToTemplate("{{=<< >>=}}hi <> {{lo}}", STRIP_WHITESPACE); AssertExpandIs(tpl4b, &dict, "hi yo {{lo}}", true); Template* tpl4c = StringToTemplate("{{=<< <<=}}hi <=}}\n" "hi {{VAR}} lo\n" "hi lo\n<={ }=>\n" "hi {{VAR}} lo\n{={{ }}=}\n" "hi {{VAR}} lo\n", STRIP_WHITESPACE); AssertExpandIs(tpl5, &dict, "hi yo lohi {{VAR}} lohi yo lohi {yo} lohi yo lo", true); Template* tpl6 = StringToTemplate("hi {{VAR}} lo\n{{=< >}}\n", STRIP_WHITESPACE); ASSERT(tpl6 == NULL); Template* tpl7 = StringToTemplate("hi {{VAR}} lo\n{{=<>}}\n", STRIP_WHITESPACE); ASSERT(tpl7 == NULL); Template* tpl8 = StringToTemplate("hi {{VAR}} lo\n{{=< >=}}\n", STRIP_WHITESPACE); ASSERT(tpl8 == NULL); Template* tpl9 = StringToTemplate("hi {{VAR}} lo\n{{==}}\n", STRIP_WHITESPACE); ASSERT(tpl9 == NULL); Template* tpl10 = StringToTemplate("hi {{VAR}} lo\n{{=}}\n", STRIP_WHITESPACE); ASSERT(tpl10 == NULL); // Test that {{= =}} is a "removable" marker. Template* tpl11 = StringToTemplate("line\n {{=| |=}} \nhi |VAR| {{lo}}\n", STRIP_BLANK_LINES); AssertExpandIs(tpl11, &dict, "line\nhi yo {{lo}}\n", true); // Test that "removable" markers survive marker-modification. Template* tpl12 = StringToTemplate(" {{#SEC1}} \n" "{{=| |=}} |VAR|\n" " |/SEC1|\ntada! |VAR|\n" "hello|=<< >>=|\n" " <> \n" "done", STRIP_BLANK_LINES); AssertExpandIs(tpl12, &dict, "tada! yo\nhello\ndone", true); } TEST(Template, Variable) { Template* tpl = StringToTemplate("hi {{VAR}} lo", STRIP_WHITESPACE); TemplateDictionary dict("dict"); AssertExpandIs(tpl, &dict, "hi lo", true); dict.SetValue("VAR", "yo"); AssertExpandIs(tpl, &dict, "hi yo lo", true); dict.SetValue("VAR", "yoyo"); AssertExpandIs(tpl, &dict, "hi yoyo lo", true); dict.SetValue("VA", "noyo"); dict.SetValue("VAR ", "noyo2"); dict.SetValue("var", "noyo3"); AssertExpandIs(tpl, &dict, "hi yoyo lo", true); // Sanity check string template behaviour while we're at it. Template* tpl2 = Template::StringToTemplate("hi {{VAR}} lo", STRIP_WHITESPACE); TemplateDictionary dict2("dict"); AssertExpandIs(tpl2, &dict2, "hi lo", true); dict2.SetValue("VAR", "yo"); AssertExpandIs(tpl2, &dict2, "hi yo lo", true); dict2.SetValue("VAR", "yoyo"); AssertExpandIs(tpl2, &dict2, "hi yoyo lo", true); dict2.SetValue("VA", "noyo"); dict2.SetValue("VAR ", "noyo2"); dict2.SetValue("var", "noyo3"); AssertExpandIs(tpl2, &dict2, "hi yoyo lo", true); delete tpl2; // You have to delete StringToTemplate strings } TEST(Template, VariableWithModifiers) { Template* tpl = StringToTemplate("hi {{VAR:html_escape}} lo", STRIP_WHITESPACE); TemplateDictionary dict("dict"); // Test with no modifiers. dict.SetValue("VAR", "yo"); AssertExpandIs(tpl, &dict, "hi yo lo", true); dict.SetValue("VAR", "yo&yo"); AssertExpandIs(tpl, &dict, "hi yo&yo lo", true); // Test with URL escaping. tpl = StringToTemplate("", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "", true); tpl = StringToTemplate("", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "", true); // Test with multiple URL escaping. tpl = StringToTemplate("", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "", true); // Test HTML escaping. tpl = StringToTemplate("hi {{VAR:h}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo&yo lo", true); tpl = StringToTemplate("hi {{VAR:h:h}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo&amp;yo lo", true); // Test special HTML escaping dict.SetValue("URL_VAR", "javascript:void"); dict.SetValue("SNIPPET_VAR", "foo & bar"); tpl = StringToTemplate("hi {{VAR:H=attribute}} {{URL_VAR:H=url}} " "{{SNIPPET_VAR:H=snippet}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo_yo # foo & bar lo", true); // Test with custom modifiers [regular or XssSafe should not matter]. ASSERT(ctemplate::AddModifier("x-test", &ctemplate::html_escape)); ASSERT(ctemplate::AddModifier("x-test-arg=", &ctemplate::html_escape)); ASSERT(ctemplate::AddXssSafeModifier("x-test-arg=snippet", &ctemplate::snippet_escape)); tpl = StringToTemplate("hi {{VAR:x-test}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo&yo lo", true); tpl = StringToTemplate("hi {{SNIPPET_VAR:x-test-arg=snippet}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi foo & bar lo", true); tpl = StringToTemplate("hi {{VAR:x-unknown}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo&yo lo", true); // Test with a modifier taking per-expand data DynamicModifier dynamic_modifier; ASSERT(ctemplate::AddModifier("x-dynamic", &dynamic_modifier)); PerExpandData per_expand_data; tpl = StringToTemplate("hi {{VAR:x-dynamic}} lo", STRIP_WHITESPACE); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "hi lo", true); per_expand_data.InsertForModifiers("value", "foo"); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "hi foo lo", true); per_expand_data.InsertForModifiers("value", "bar"); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "hi bar lo", true); per_expand_data.InsertForModifiers("value", NULL); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "hi lo", true); // Test with no modifiers. tpl = StringToTemplate("hi {{VAR}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo&yo lo", true); // Check that ordering is right dict.SetValue("VAR", "yo\nyo"); tpl = StringToTemplate("hi {{VAR:h}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo yo lo", true); tpl = StringToTemplate("hi {{VAR:p}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo\nyo lo", true); tpl = StringToTemplate("hi {{VAR:j}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo\\nyo lo", true); tpl = StringToTemplate("hi {{VAR:h:j}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo yo lo", true); tpl = StringToTemplate("hi {{VAR:j:h}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo\\nyo lo", true); // Check more complicated modifiers using fullname tpl = StringToTemplate("hi {{VAR:javascript_escape:h}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo\\nyo lo", true); tpl = StringToTemplate("hi {{VAR:j:html_escape}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo\\nyo lo", true); tpl = StringToTemplate("hi {{VAR:pre_escape:j}} lo", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "hi yo\\nyo lo", true); // Check that illegal modifiers are rejected tpl = StringToTemplate("hi {{VAR:j:h2}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); tpl = StringToTemplate("hi {{VAR:html_ecap}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); tpl = StringToTemplate("hi {{VAR:javascript_escaper}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); tpl = StringToTemplate("hi {{VAR:js:j}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); tpl = StringToTemplate("hi {{VAR:}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); // Check we reject modifier-values when we ought to tpl = StringToTemplate("hi {{VAR:j=4}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); tpl = StringToTemplate("hi {{VAR:html_escape=yes}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); tpl = StringToTemplate("hi {{VAR:url_query_escape=wombats}} lo", STRIP_WHITESPACE); ASSERT(tpl == NULL); // Check we don't allow modifiers on sections tpl = StringToTemplate("hi {{#VAR:h}} lo {{/VAR}}", STRIP_WHITESPACE); ASSERT(tpl == NULL); // Test when expanded grows by more than 12% per modifier. dict.SetValue("VAR", "http://a.com?b=c&d=e&f=g&q=a>b"); tpl = StringToTemplate("{{VAR:u:j:h}}", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "http%3A//a.com%3Fb%3Dc%26d%3De%26f%3Dg%26q%3Da%3Eb", true); // As above with 4 modifiers. dict.SetValue("VAR", "http://a.com?b=c&d=e&f=g&q=a>b"); tpl = StringToTemplate("{{VAR:u:j:h:h}}", STRIP_WHITESPACE); AssertExpandIs(tpl, &dict, "http%3A//a.com%3Fb%3Dc%26d%3De%26f%3Dg%26q%3Da%3Eb", true); } TEST(Template, Section) { Template* tpl = StringToTemplate( "boo!\nhi {{#SEC}}lo{{#SUBSEC}}jo{{/SUBSEC}}{{/SEC}} bar", STRIP_WHITESPACE); TemplateDictionary dict("dict"); AssertExpandIs(tpl, &dict, "boo!hi bar", true); dict.ShowSection("SEC"); AssertExpandIs(tpl, &dict, "boo!hi lo bar", true); dict.ShowSection("SEC"); AssertExpandIs(tpl, &dict, "boo!hi lo bar", true); // This should work even though subsec isn't a child of the main dict dict.ShowSection("SUBSEC"); AssertExpandIs(tpl, &dict, "boo!hi lojo bar", true); TemplateDictionary dict2("dict2"); dict2.AddSectionDictionary("SEC"); AssertExpandIs(tpl, &dict2, "boo!hi lo bar", true); dict2.AddSectionDictionary("SEC"); AssertExpandIs(tpl, &dict2, "boo!hi lolo bar", true); dict2.AddSectionDictionary("sec"); AssertExpandIs(tpl, &dict2, "boo!hi lolo bar", true); dict2.ShowSection("SUBSEC"); AssertExpandIs(tpl, &dict2, "boo!hi lojolojo bar", true); } TEST(Template, SectionSeparator) { Template* tpl = StringToTemplate( "hi {{#SEC}}lo{{#SEC_separator}}jo{{JO}}{{/SEC_separator}}{{/SEC}} bar", STRIP_WHITESPACE); TemplateDictionary dict("dict"); AssertExpandIs(tpl, &dict, "hi bar", true); // Since SEC is only expanded once, the separator section shouldn't show. dict.ShowSection("SEC"); AssertExpandIs(tpl, &dict, "hi lo bar", true); dict.ShowSection("SEC"); AssertExpandIs(tpl, &dict, "hi lo bar", true); // This should work even though SEC_separator isn't a child of the // main dict. It verifies SEC_separator is just a normal section, too. dict.ShowSection("SEC_separator"); AssertExpandIs(tpl, &dict, "hi lojo bar", true); TemplateDictionary dict2("dict2"); dict2.AddSectionDictionary("SEC"); AssertExpandIs(tpl, &dict2, "hi lo bar", true); dict2.AddSectionDictionary("SEC"); AssertExpandIs(tpl, &dict2, "hi lojolo bar", true); // This is a weird case: using separator and specifying manually. dict2.ShowSection("SEC_separator"); AssertExpandIs(tpl, &dict2, "hi lojojolojo bar", true); TemplateDictionary dict3("dict3"); TemplateDictionary* sec1 = dict3.AddSectionDictionary("SEC"); TemplateDictionary* sec2 = dict3.AddSectionDictionary("SEC"); TemplateDictionary* sec3 = dict3.AddSectionDictionary("SEC"); dict3.SetValue("JO", "J"); AssertExpandIs(tpl, &dict3, "hi lojoJlojoJlo bar", true); sec1->SetValue("JO", "JO"); AssertExpandIs(tpl, &dict3, "hi lojoJOlojoJlo bar", true); sec2->SetValue("JO", "JOO"); AssertExpandIs(tpl, &dict3, "hi lojoJOlojoJOOlo bar", true); dict3.AddSectionDictionary("SEC"); AssertExpandIs(tpl, &dict3, "hi lojoJOlojoJOOlojoJlo bar", true); sec3->AddSectionDictionary("SEC_separator"); AssertExpandIs(tpl, &dict3, "hi lojoJOlojoJOOlojoJjoJlo bar", true); // Make sure we don't do anything special with var or include names Template* tpl2 = StringToTemplate( "hi {{#SEC}}lo{{>SEC_separator}}{{/SEC}} bar", STRIP_WHITESPACE); AssertExpandIs(tpl2, &dict2, "hi lolo bar", true); Template* tpl3 = StringToTemplate( "hi {{#SEC}}lo{{SEC_separator}}{{/SEC}} bar", STRIP_WHITESPACE); dict2.SetValue("SEC_separator", "-"); AssertExpandIs(tpl3, &dict2, "hi lo-lo- bar", true); } TEST(Template, Include) { string incname = StringToTemplateFile("include file\n"); string incname2 = StringToTemplateFile("inc2a\ninc2b\n"); string incname_bad = StringToTemplateFile("{{syntax_error"); Template* tpl = StringToTemplate("hi {{>INC}} bar\n", STRIP_WHITESPACE); TemplateDictionary dict("dict"); AssertExpandIs(tpl, &dict, "hi bar", true); dict.AddIncludeDictionary("INC"); AssertExpandIs(tpl, &dict, "hi bar", true); // noop: no filename was set dict.AddIncludeDictionary("INC")->SetFilename("/notarealfile "); AssertExpandIs(tpl, &dict, "hi bar", false); // noop: illegal filename dict.AddIncludeDictionary("INC")->SetFilename(incname); AssertExpandIs(tpl, &dict, "hi include file bar", false); dict.AddIncludeDictionary("INC")->SetFilename(incname_bad); AssertExpandIs(tpl, &dict, "hi include file bar", false); // noop: syntax error dict.AddIncludeDictionary("INC")->SetFilename(incname); AssertExpandIs(tpl, &dict, "hi include fileinclude file bar", false); dict.AddIncludeDictionary("inc")->SetFilename(incname); AssertExpandIs(tpl, &dict, "hi include fileinclude file bar", false); dict.AddIncludeDictionary("INC")->SetFilename(incname2); AssertExpandIs(tpl, &dict, "hi include fileinclude fileinc2ainc2b bar", false); // Now test that includes preserve Strip Template* tpl2 = StringToTemplate("hi {{>INC}} bar", DO_NOT_STRIP); AssertExpandIs(tpl2, &dict, "hi include file\ninclude file\ninc2a\ninc2b\n bar", false); // Test that if we indent the include, every line on the include // is indented. Template* tpl3 = StringToTemplate("hi\n {{>INC}} bar", DO_NOT_STRIP); AssertExpandIs(tpl3, &dict, "hi\n include file\n include file\n" " inc2a\n inc2b\n bar", false); // But obviously, if we strip leading whitespace, no indentation. Template* tpl4 = StringToTemplate("hi\n {{>INC}} bar", STRIP_WHITESPACE); AssertExpandIs(tpl4, &dict, "hiinclude fileinclude fileinc2ainc2b bar", false); // And if it's not a whitespace indent, we don't indent either. Template* tpl5 = StringToTemplate("hi\n - {{>INC}} bar", DO_NOT_STRIP); AssertExpandIs(tpl5, &dict, "hi\n - include file\ninclude file\n" "inc2a\ninc2b\n bar", false); // Make sure we indent properly at the beginning. Template* tpl6 = StringToTemplate(" {{>INC}}\nbar", DO_NOT_STRIP); AssertExpandIs(tpl6, &dict, " include file\n include file\n" " inc2a\n inc2b\n \nbar", false); // And deal correctly when we include twice in a row. Template* tpl7 = StringToTemplate(" {{>INC}}-{{>INC}}", DO_NOT_STRIP); AssertExpandIs(tpl7, &dict, " include file\n include file\n inc2a\n inc2b\n " "-include file\ninclude file\ninc2a\ninc2b\n", false); } TEST(Template, IncludeWithModifiers) { string incname = StringToTemplateFile("include & print file\n"); string incname2 = StringToTemplateFile("inc2\n"); string incname3 = StringToTemplateFile("yo&yo"); // Note this also tests that html-escape, but not javascript-escape or // pre-escape, escapes \n to Template* tpl1 = StringToTemplate("hi {{>INC:h}} bar\n", DO_NOT_STRIP); Template* tpl2 = StringToTemplate("hi {{>INC:javascript_escape}} bar\n", DO_NOT_STRIP); Template* tpl3 = StringToTemplate("hi {{>INC:pre_escape}} bar\n", DO_NOT_STRIP); Template* tpl4 = StringToTemplate("hi {{>INC:u}} bar\n", DO_NOT_STRIP); // Test that if we include the same template twice, once with a modifer // and once without, they each get applied properly. Template* tpl5 = StringToTemplate("hi {{>INC:h}} bar {{>INC}} baz\n", DO_NOT_STRIP); TemplateDictionary dict("dict"); AssertExpandIs(tpl1, &dict, "hi bar\n", true); dict.AddIncludeDictionary("INC")->SetFilename(incname); AssertExpandIs(tpl1, &dict, "hi include & print file bar\n", true); dict.AddIncludeDictionary("INC")->SetFilename(incname2); AssertExpandIs(tpl1, &dict, "hi include & print file inc2 bar\n", true); AssertExpandIs(tpl2, &dict, "hi include \\x26 print file\\ninc2\\n bar\n", true); AssertExpandIs(tpl3, &dict, "hi include & print file\ninc2\n bar\n", true); dict.AddIncludeDictionary("INC")->SetFilename(incname3); AssertExpandIs(tpl4, &dict, "hi include+%26+print+file%0Ainc2%0Ayo%26yo bar\n", true); AssertExpandIs(tpl5, &dict, "hi include & print file inc2 yo&yo bar " "include & print file\ninc2\nyo&yo baz\n", true); // Don't test modifier syntax here; that's in TestVariableWithModifiers() } // Make sure we don't deadlock when a template includes itself. // This also tests we handle recursive indentation properly. TEST(Template, RecursiveInclude) { string incname = StringToTemplateFile("hi {{>INC}} bar\n {{>INC}}!"); Template* tpl = Template::GetTemplate(incname, DO_NOT_STRIP); TemplateDictionary dict("dict"); dict.AddIncludeDictionary("INC")->SetFilename(incname); // Note the last line is indented 4 spaces instead of 2. This is // because the last sub-include is indented. AssertExpandIs(tpl, &dict, "hi hi bar\n ! bar\n hi bar\n !!", true); } // Tests that vars inherit/override their parents properly TEST(Template, Inheritence) { Template* tpl = StringToTemplate("{{FOO}}{{#SEC}}{{FOO}}{{#SEC}}{{FOO}}{{/SEC}}{{/SEC}}", STRIP_WHITESPACE); TemplateDictionary dict("dict"); dict.SetValue("FOO", "foo"); dict.ShowSection("SEC"); AssertExpandIs(tpl, &dict, "foofoofoo", true); TemplateDictionary dict2("dict2"); dict2.SetValue("FOO", "foo"); TemplateDictionary* sec = dict2.AddSectionDictionary("SEC"); AssertExpandIs(tpl, &dict2, "foofoofoo", true); sec->SetValue("FOO", "bar"); AssertExpandIs(tpl, &dict2, "foobarbar", true); TemplateDictionary* sec2 = sec->AddSectionDictionary("SEC"); AssertExpandIs(tpl, &dict2, "foobarbar", true); sec2->SetValue("FOO", "baz"); AssertExpandIs(tpl, &dict2, "foobarbaz", true); // Now test an include template, which shouldn't inherit from its parents tpl = StringToTemplate("{{FOO}}{{#SEC}}hi{{/SEC}}\n{{>INC}}", STRIP_WHITESPACE); string incname = StringToTemplateFile( "include {{FOO}}{{#SEC}}invisible{{/SEC}}file\n"); TemplateDictionary incdict("dict"); incdict.ShowSection("SEC"); incdict.SetValue("FOO", "foo"); incdict.AddIncludeDictionary("INC")->SetFilename(incname); AssertExpandIs(tpl, &incdict, "foohiinclude file", true); } TEST(Template, TemplateString) { // Make sure using TemplateString and StaticTemplateString for the // dictionary expands the same as using char*'s. Template* tpl = StringToTemplate("hi {{VAR}} lo", STRIP_WHITESPACE); TemplateDictionary dict("dict"); dict.SetValue("VAR", TemplateString("short-lived", strlen("short"))); AssertExpandIs(tpl, &dict, "hi short lo", true); dict.SetValue("VAR", kHello); AssertExpandIs(tpl, &dict, "hi Hello lo", true); } // Tests that we append to the output string, rather than overwrite TEST(Template, Expand) { Template* tpl = StringToTemplate("hi", STRIP_WHITESPACE); TemplateDictionary dict("test_expand"); string output("premade"); ASSERT(tpl->Expand(&output, &dict)); ASSERT_STREQ(output.c_str(), "premadehi"); tpl = StringToTemplate(" lo ", STRIP_WHITESPACE); ASSERT(tpl->Expand(&output, &dict)); ASSERT_STREQ(output.c_str(), "premadehilo"); } TEST(Template, ExpandTemplate) { string filename = StringToTemplateFile(" hi {{THERE}}"); TemplateDictionary dict("test_expand"); dict.SetValue("THERE", "test"); string output; ASSERT(ExpandTemplate(filename, STRIP_WHITESPACE, &dict, &output)); ASSERT_STREQ(output.c_str(), "hi test"); // This will append to output, so we see both together. ASSERT(ExpandWithData(filename, DO_NOT_STRIP, &dict, NULL, &output)); ASSERT_STREQ(output.c_str(), "hi test hi test"); ASSERT(!ExpandTemplate(filename + " not found", DO_NOT_STRIP, &dict, &output)); } TEST(Template, ExpandWithCustomEmitter) { Template* tpl = StringToTemplate("{{VAR}} {{VAR}}", STRIP_WHITESPACE); TemplateDictionary dict("test_expand"); dict.SetValue("VAR", "this song is just six words long"); string output; SizeofEmitter e(&output); ASSERT(tpl->Expand(&e, &dict)); ASSERT_STREQ("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", output.c_str()); } TEST(Template, TemplateExpansionModifier) { string parent_tpl_name = StringToTemplateFile("before {{>INC}} after"); string child_tpl_name1 = StringToTemplateFile("child1"); string child_tpl_name2 = StringToTemplateFile("child2"); Template* tpl = Template::GetTemplate(parent_tpl_name, DO_NOT_STRIP); TemplateDictionary dict("parent dict"); dict.AddIncludeDictionary("INC")->SetFilename(child_tpl_name1); dict.AddIncludeDictionary("INC")->SetFilename(child_tpl_name2); PerExpandData per_expand_data; EmphasizeTemplateModifier modifier1(child_tpl_name1); per_expand_data.SetTemplateExpansionModifier(&modifier1); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "before >>child1<>child2<< after", true); EmphasizeTemplateModifier modifier3(parent_tpl_name); per_expand_data.SetTemplateExpansionModifier(&modifier3); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, ">>before child1child2 after<<", true); per_expand_data.SetTemplateExpansionModifier(NULL); AssertExpandWithDataIs(tpl, &dict, &per_expand_data, "before child1child2 after", true); } TEST(Template, GetTemplate) { // Tests the cache string filename = StringToTemplateFile("{This is perfectly valid} yay!"); Template* tpl1 = Template::GetTemplate(filename, DO_NOT_STRIP); Template* tpl2 = Template::GetTemplate(filename.c_str(), DO_NOT_STRIP); Template* tpl3 = Template::GetTemplate(filename, STRIP_WHITESPACE); ASSERT(tpl1 && tpl2 && tpl3); ASSERT(tpl1 == tpl2); ASSERT(tpl1 != tpl3); // Tests that a nonexistent template returns NULL Template* tpl4 = Template::GetTemplate("/yakakak", STRIP_WHITESPACE); ASSERT(!tpl4); // Tests that syntax errors cause us to return NULL Template* tpl5 = StringToTemplate("{{This has spaces in it}}", DO_NOT_STRIP); ASSERT(!tpl5); Template* tpl6 = StringToTemplate("{{#SEC}}foo", DO_NOT_STRIP); ASSERT(!tpl6); Template* tpl7 = StringToTemplate("{{#S1}}foo{{/S2}}", DO_NOT_STRIP); ASSERT(!tpl7); Template* tpl8 = StringToTemplate("{{#S1}}foo{{#S2}}bar{{/S1}{{/S2}", DO_NOT_STRIP); ASSERT(!tpl8); Template* tpl9 = StringToTemplate("{{noend", DO_NOT_STRIP); ASSERT(!tpl9); } TEST(Template, StringCacheKey) { // If you use these same cache keys somewhere else, // call Template::ClearCache first. const string cache_key_a = "cache key a"; const string text = "Test template 1"; TemplateDictionary empty_dict("dict"); // When a string template is registered via StringToTemplateCache, // we can use GetTemplate for that same cache-key under any other // Strip because we cache the contents. Template *tpl1, *tpl2; ASSERT(Template::StringToTemplateCache(cache_key_a, text)); tpl1 = Template::GetTemplate(cache_key_a, DO_NOT_STRIP); AssertExpandIs(tpl1, &empty_dict, text, true); // Different strip. ASSERT(tpl2 = Template::GetTemplate(cache_key_a, STRIP_BLANK_LINES)); ASSERT(tpl2 != tpl1); AssertExpandIs(tpl2, &empty_dict, text, true); Template::ClearCache(); } TEST(Template, StringGetTemplate) { TemplateDictionary dict("dict"); // Test cache lookups const char* const tpltext = "{This is perfectly valid} yay!"; ASSERT(Template::StringToTemplateCache("tgt", tpltext)); Template* tpl1 = Template::GetTemplate("tgt", DO_NOT_STRIP); Template* tpl2 = Template::GetTemplate("tgt", STRIP_WHITESPACE); ASSERT(tpl1 && tpl2); ASSERT(tpl1 != tpl2); AssertExpandIs(tpl1, &dict, tpltext, true); AssertExpandIs(tpl2, &dict, tpltext, true); // If we register a new string under the same text, it should be // ignored. ASSERT(!Template::StringToTemplateCache("tgt", tpltext)); ASSERT(!Template::StringToTemplateCache("tgt", "new text")); Template* tpl3 = Template::GetTemplate("tgt", DO_NOT_STRIP); ASSERT(tpl3 == tpl1); AssertExpandIs(tpl3, &dict, tpltext, true); // Tests that syntax errors cause us to return NULL ASSERT(!Template::StringToTemplateCache("tgt2", "{{This has spaces}}")); ASSERT(!Template::StringToTemplateCache("tgt3", "{{#SEC}}foo")); ASSERT(!Template::StringToTemplateCache("tgt4", "{{#S1}}foo{{/S2}}")); ASSERT(!Template::StringToTemplateCache("tgt5", "{{#S1}}foo{{#S2}}bar{{/S1}{{/S2}")); ASSERT(!Template::StringToTemplateCache("tgt6", "{{noend")); // And that we didn't cache them by mistake ASSERT(!Template::GetTemplate("tgt2", STRIP_WHITESPACE)); Template::ClearCache(); } TEST(Template, StringTemplateInclude) { Template::ClearCache(); // just for exercise. const string cache_key = "TestStringTemplateInclude"; const string cache_key_inc = "TestStringTemplateInclude-inc"; const string cache_key_indent = "TestStringTemplateInclude-indent"; const string text = "{{>INC}}"; const string text_inc = "
\n

\nUser {{USER}}\n

"; const string text_indent = "\n {{>INC}}"; ASSERT(Template::StringToTemplateCache(cache_key, text)); ASSERT(Template::StringToTemplateCache(cache_key_inc, text_inc)); ASSERT(Template::StringToTemplateCache(cache_key_indent, text_indent)); Template *tpl = Template::GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(tpl); TemplateDictionary dict("dict"); TemplateDictionary* sub_dict = dict.AddIncludeDictionary("INC"); sub_dict->SetFilename(cache_key_inc); sub_dict->SetValue("USER", "John<>Doe"); string expected = "
\n

\nUser John<>Doe\n

"; AssertExpandIs(tpl, &dict, expected, true); // Repeat the same except that now the parent has a template-level // directive (by way of the automatic-line-indenter). tpl = Template::GetTemplate(cache_key_indent, DO_NOT_STRIP); ASSERT(tpl); expected = "\n" "
\n" "

\n" " User John<>Doe\n" "

" ""; AssertExpandIs(tpl, &dict, expected, true); Template::ClearCache(); } TEST(Template, TemplateSearchPath) { const string pathA = PathJoin(FLAGS_test_tmpdir, "a/"); const string pathB = PathJoin(FLAGS_test_tmpdir, "b/"); CreateOrCleanTestDir(pathA); CreateOrCleanTestDir(pathB); TemplateDictionary dict(""); Template::SetTemplateRootDirectory(pathA); Template::AddAlternateTemplateRootDirectory(pathB); // 1. Show that a template in the secondary path can be found. const string path_b_bar = PathJoin(pathB, "template_bar"); StringToFile("b/template_bar", path_b_bar); ASSERT_STREQ(path_b_bar.c_str(), Template::FindTemplateFilename("template_bar").c_str()); Template* b_bar = Template::GetTemplate("template_bar", DO_NOT_STRIP); ASSERT(b_bar); AssertExpandIs(b_bar, &dict, "b/template_bar", true); // 2. Show that the search stops once the first match is found. // Create two templates in separate directories with the same name. const string path_a_foo = PathJoin(pathA, "template_foo"); StringToFile("a/template_foo", path_a_foo); StringToFile("b/template_foo", PathJoin(pathB, "template_foo")); ASSERT_STREQ(path_a_foo.c_str(), Template::FindTemplateFilename("template_foo").c_str()); Template* a_foo = Template::GetTemplate("template_foo", DO_NOT_STRIP); ASSERT(a_foo); AssertExpandIs(a_foo, &dict, "a/template_foo", true); // 3. Show that attempting to find a non-existent template gives an // empty path. ASSERT(Template::FindTemplateFilename("baz").empty()); CreateOrCleanTestDir(pathA); CreateOrCleanTestDir(pathB); } TEST(Template, RemoveStringFromTemplateCache) { Template::ClearCache(); // just for exercise. const string cache_key = "TestRemoveStringFromTemplateCache"; const string text = "here today..."; TemplateDictionary dict("test"); ASSERT(Template::StringToTemplateCache(cache_key, text)); Template* tpl = Template::GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(tpl); AssertExpandIs(tpl, &dict, text, true); tpl = Template::GetTemplate(cache_key, STRIP_WHITESPACE); ASSERT(tpl); AssertExpandIs(tpl, &dict, text, true); Template::RemoveStringFromTemplateCache(cache_key); tpl = Template::GetTemplate(cache_key, DO_NOT_STRIP); ASSERT(!tpl); tpl = Template::GetTemplate(cache_key, STRIP_WHITESPACE); ASSERT(!tpl); tpl = Template::GetTemplate(cache_key, STRIP_BLANK_LINES); ASSERT(!tpl); } TEST(Template, TemplateCache) { const string filename_a = StringToTemplateFile("Test template 1"); const string filename_b = StringToTemplateFile("Test template 2."); Template *tpl, *tpl2; ASSERT(tpl = Template::GetTemplate(filename_a, DO_NOT_STRIP)); ASSERT(tpl2 = Template::GetTemplate(filename_b, DO_NOT_STRIP)); ASSERT(tpl2 != tpl); // different filenames. ASSERT(tpl2 = Template::GetTemplate(filename_a, STRIP_BLANK_LINES)); ASSERT(tpl2 != tpl); // different strip. ASSERT(tpl2 = Template::GetTemplate(filename_b, STRIP_BLANK_LINES)); ASSERT(tpl2 != tpl); // different filenames and strip. ASSERT(tpl2 = Template::GetTemplate(filename_a, DO_NOT_STRIP)); ASSERT(tpl2 == tpl); // same filename and strip. } // Tests that the various strip values all do the expected thing. TEST(Template, Strip) { TemplateDictionary dict("dict"); dict.SetValue("FOO", "foo"); const char* tests[][4] = { // 0: in, 1: do-not-strip, 2: blanklines, 3: ws {"hi!\n", "hi!\n", "hi!\n", "hi!"}, {"hi!", "hi!", "hi!", "hi!"}, // These test strip-blank-lines, primarily {"{{FOO}}\n\n{{FOO}}", "foo\n\nfoo", "foo\nfoo", "foofoo"}, {"{{FOO}}\r\n\r\n{{FOO}}", "foo\r\n\r\nfoo", "foo\r\nfoo", "foofoo"}, {"{{FOO}}\n \n{{FOO}}\n", "foo\n \nfoo\n", "foo\nfoo\n", "foofoo"}, {"{{FOO}}\n{{BI_NEWLINE}}\nb", "foo\n\n\nb", "foo\n\n\nb", "foo\nb"}, {"{{FOO}}\n{{!comment}}\nb", "foo\n\nb", "foo\nb", "foob"}, {"{{FOO}}\n{{!comment}}{{!comment2}}\nb", "foo\n\nb", "foo\n\nb", "foob"}, {"{{FOO}}\n{{>ONE_INC}}\nb", "foo\n\nb", "foo\nb", "foob"}, {"{{FOO}}\n\t{{>ONE_INC}} \nb", "foo\n\t \nb", "foo\nb", "foob"}, {"{{FOO}}\n{{>ONE_INC}}{{>TWO_INC}}\nb", "foo\n\nb", "foo\n\nb", "foob"}, {"{{FOO}}\n {{#SEC}}\ntext \n {{/SEC}}\n", "foo\n \n", "foo\n", "foo"}, {"{{%AUTOESCAPE context=\"HTML\"}}\nBLA", "\nBLA", "BLA", "BLA"}, // These test strip-whitespace {"foo\nbar\n", "foo\nbar\n", "foo\nbar\n", "foobar"}, {"{{FOO}}\nbar\n", "foo\nbar\n", "foo\nbar\n", "foobar"}, {" {{FOO}} {{!comment}}\nb", " foo \nb", " foo \nb", "foo b"}, {" {{FOO}} {{BI_SPACE}}\n", " foo \n", " foo \n", "foo "}, {" \t \f\v \n\r\n ", " \t \f\v \n\r\n ", "", ""}, }; for (int i = 0; i < sizeof(tests)/sizeof(*tests); ++i) { Template* tpl1 = StringToTemplate(tests[i][0], DO_NOT_STRIP); Template* tpl2 = StringToTemplate(tests[i][0], STRIP_BLANK_LINES); Template* tpl3 = StringToTemplate(tests[i][0], STRIP_WHITESPACE); AssertExpandIs(tpl1, &dict, tests[i][1], true); AssertExpandIs(tpl2, &dict, tests[i][2], true); AssertExpandIs(tpl3, &dict, tests[i][3], true); } } TEST(Template, TemplateRootDirectory) { string filename = StringToTemplateFile("Test template"); ASSERT(IsAbspath(filename)); Template* tpl1 = Template::GetTemplate(filename, DO_NOT_STRIP); Template::SetTemplateRootDirectory(kRootdir); // "/" // template-root shouldn't matter for absolute directories Template* tpl2 = Template::GetTemplate(filename, DO_NOT_STRIP); Template::SetTemplateRootDirectory("/sadfadsf/waerfsa/safdg"); Template* tpl3 = Template::GetTemplate(filename, DO_NOT_STRIP); ASSERT(tpl1 != NULL); ASSERT(tpl1 == tpl2); ASSERT(tpl1 == tpl3); // Now test it actually works by breaking the abspath in various places. // We do it twice, since we don't know if the path-sep is "/" or "\". // NOTE: this depends on filename not using "/" or "\" except as a // directory separator (so nothing like "/var/tmp/foo\a/weirdfile"). const char* const kPathSeps = "/\\"; for (const char* path_sep = kPathSeps; *path_sep; path_sep++) { for (string::size_type sep_pos = filename.find(*path_sep, 0); sep_pos != string::npos; sep_pos = filename.find(*path_sep, sep_pos + 1)) { Template::SetTemplateRootDirectory(filename.substr(0, sep_pos + 1)); Template* tpl = Template::GetTemplate(filename.substr(sep_pos + 1), DO_NOT_STRIP); ASSERT(string(tpl->template_file()) == tpl1->template_file()); } } } #if defined(HAVE_PTHREAD) && !defined(NO_THREADS) struct ThreadReturn { Template* file_template; bool string_to_template_cache_return; Template* string_template; }; // RunThread returns a ThreadReturn* that should be deleted. static void* RunThread(void* vfilename) { const char* filename = reinterpret_cast(vfilename); ThreadReturn* ret = new ThreadReturn; ret->file_template = Template::GetTemplate(filename, DO_NOT_STRIP); ASSERT(ret->file_template != NULL); const char* const key = "RunThread key"; ret->string_to_template_cache_return = StringToTemplateCache(key, " RunThread text ", STRIP_WHITESPACE); ret->string_template = Template::GetTemplate(key, STRIP_WHITESPACE); ASSERT(ret->string_template != NULL); return ret; } TEST(Template, ThreadSafety) { string filename = StringToTemplateFile("(testing thread-safety)"); // GetTemplate() is the most thread-contended routine. We get a // template in many threads, and assert we get the same template // from each. pthread_t thread_ids[kNumThreads]; for (int i = 0; i < kNumThreads; ++i) { ASSERT(pthread_create(thread_ids+i, NULL, RunThread, (void*)filename.c_str()) == 0); } // Wait for all the threads to terminate (should be very quick!) ThreadReturn* first_thread_return = NULL; int num_times_string_to_template_cache_returned_true = 0; for (int i = 0; i < kNumThreads; ++i) { void* vthread_return; ASSERT(pthread_join(thread_ids[i], &vthread_return) == 0); ThreadReturn* thread_return = reinterpret_cast(vthread_return); if (thread_return->string_to_template_cache_return) { ++num_times_string_to_template_cache_returned_true; } if (first_thread_return == NULL) { // we're the first thread first_thread_return = thread_return; } else { ASSERT(thread_return->file_template == first_thread_return->file_template); ASSERT(thread_return->string_template == first_thread_return->string_template); delete thread_return; } } delete first_thread_return; ASSERT_INTEQ(1, num_times_string_to_template_cache_returned_true); Template::ClearCache(); } #endif // #if defined(HAVE_PTHREAD) && !defined(NO_THREADS) // Tests all the static methods in TemplateNamelist TEST(Template, TemplateNamelist) { time_t before_time = Now(); // in template_test_util.cc string f1 = StringToTemplateFile("{{This has spaces in it}}"); string f2 = StringToTemplateFile("{{#SEC}}foo"); string f3 = StringToTemplateFile("{This is ok"); // Where we'll copy f1 - f3 to: these are names known at compile-time string f1_copy = PathJoin(FLAGS_test_tmpdir, INVALID1_FN); string f2_copy = PathJoin(FLAGS_test_tmpdir, INVALID2_FN); string f3_copy = PathJoin(FLAGS_test_tmpdir, VALID1_FN); Template::SetTemplateRootDirectory(FLAGS_test_tmpdir); time_t after_time = Now(); // f1, f2, f3 all written by now TemplateNamelist::NameListType names = TemplateNamelist::GetList(); ASSERT(names.size() == 4); ASSERT(names.count(NONEXISTENT_FN)); ASSERT(names.count(INVALID1_FN)); ASSERT(names.count(INVALID2_FN)); ASSERT(names.count(VALID1_FN)); // Before creating the files INVALID1_FN, etc., all should be missing. for (int i = 0; i < 3; ++i) { // should be consistent all 3 times const TemplateNamelist::MissingListType& missing = TemplateNamelist::GetMissingList(false); ASSERT(missing.size() == 4); } // Everyone is missing, but nobody should have bad syntax ASSERT(!TemplateNamelist::AllDoExist()); ASSERT(TemplateNamelist::IsAllSyntaxOkay(DO_NOT_STRIP)); // Now create those files ASSERT(link(f1.c_str(), f1_copy.c_str()) == 0); ASSERT(link(f2.c_str(), f2_copy.c_str()) == 0); ASSERT(link(f3.c_str(), f3_copy.c_str()) == 0); // We also have to clear the template cache, since we created a new file. // ReloadAllIfChanged() would probably work, too. Template::ClearCache(); // When GetMissingList is false, we don't reload, so you still get all-gone TemplateNamelist::MissingListType missing = TemplateNamelist::GetMissingList(false); ASSERT(missing.size() == 4); // But with true, we should have a different story missing = TemplateNamelist::GetMissingList(true); ASSERT(missing.size() == 1); missing = TemplateNamelist::GetMissingList(false); ASSERT(missing.size() == 1); ASSERT(missing[0] == NONEXISTENT_FN); ASSERT(!TemplateNamelist::AllDoExist()); // IsAllSyntaxOK did a badsyntax check, before the files were created. // So with a false arg, should still say everything is ok TemplateNamelist::SyntaxListType badsyntax = TemplateNamelist::GetBadSyntaxList(false, DO_NOT_STRIP); ASSERT(badsyntax.size() == 0); // But IsAllSyntaxOK forces a refresh ASSERT(!TemplateNamelist::IsAllSyntaxOkay(DO_NOT_STRIP)); badsyntax = TemplateNamelist::GetBadSyntaxList(false, DO_NOT_STRIP); ASSERT(badsyntax.size() == 2); ASSERT(badsyntax[0] == INVALID1_FN || badsyntax[1] == INVALID1_FN); ASSERT(badsyntax[0] == INVALID2_FN || badsyntax[1] == INVALID2_FN); ASSERT(!TemplateNamelist::IsAllSyntaxOkay(DO_NOT_STRIP)); badsyntax = TemplateNamelist::GetBadSyntaxList(true, DO_NOT_STRIP); ASSERT(badsyntax.size() == 2); time_t modtime = TemplateNamelist::GetLastmodTime(); ASSERT(modtime >= before_time && modtime <= after_time); // Now update a file and make sure lastmod time is updated. // Note that since TemplateToFile uses "fake" timestamps way // in the past, this append should definitely give a time // that's after after_time. FILE* fp = fopen(f1_copy.c_str(), "ab"); ASSERT(fp); fwrite("\n", 1, 1, fp); fclose(fp); modtime = TemplateNamelist::GetLastmodTime(); ASSERT(modtime > after_time); // Checking if we can register templates at run time. string f4 = StringToTemplateFile("{{ONE_GOOD_TEMPLATE}}"); TemplateNamelist::RegisterTemplate(f4.c_str()); names = TemplateNamelist::GetList(); ASSERT(names.size() == 5); string f5 = StringToTemplateFile("{{ONE BAD TEMPLATE}}"); TemplateNamelist::RegisterTemplate(f5.c_str()); names = TemplateNamelist::GetList(); ASSERT(names.size() == 6); badsyntax = TemplateNamelist::GetBadSyntaxList(false, DO_NOT_STRIP); ASSERT(badsyntax.size() == 2); // we did not refresh the bad syntax list badsyntax = TemplateNamelist::GetBadSyntaxList(true, DO_NOT_STRIP); // After refresh, the file we just registerd also added in bad syntax list ASSERT(badsyntax.size() == 3); TemplateNamelist::RegisterTemplate("A_non_existant_file.tpl"); names = TemplateNamelist::GetList(); ASSERT(names.size() == 7); missing = TemplateNamelist::GetMissingList(false); ASSERT(missing.size() == 1); // we did not refresh the missing list missing = TemplateNamelist::GetMissingList(true); // After refresh, the file we just registerd also added in missing list ASSERT(missing.size() == 2); } // This test is not "end-to-end", it doesn't use a dictionary // and only outputs what the template system thinks is the // correct modifier for variables. TEST(Template, CorrectModifiersForAutoEscape) { string text, expected_out; // template with no variable, nothing to emit. text = "Static template."; AssertCorrectModifiers(TC_HTML, text, ""); // Simple templates with one variable substitution. // 1. No in-template modifiers. Auto Escaper sets correct ones. text = "Hello {{USER}}"; AssertCorrectModifiers(TC_HTML, text, "USER:h\n"); // Complete URLs in different attributes that take URLs. text = "bla"; AssertCorrectModifiers(TC_HTML, text, "URL:U=html\n"); text = ""; AssertCorrectModifiers(TC_HTML, text, "URL:U=html\n"); text = ""; AssertCorrectModifiers(TC_HTML, text, "URL:U=html\n"); // URL fragment only so just html_escape. text = ""; AssertCorrectModifiers(TC_HTML, text, "QUERY:h\n"); // URL fragment not quoted, so url_escape. text = ""; AssertCorrectModifiers(TC_HTML, text, "QUERY:u\n"); text = "
"; AssertCorrectModifiers(TC_HTML, text, "CLASS:h\n"); text = "
"; AssertCorrectModifiers(TC_HTML, text, "CLASS:H=attribute\n"); text = "
"; // CLASS here is name/value pair. AssertCorrectModifiers(TC_HTML, text, "CLASS:H=attribute\n"); text = "
"; // Style attribute. AssertCorrectModifiers(TC_HTML, text, "DISPLAY:c\n"); // Content inside a style tag should have :c regardless of quoting. text = ""; AssertCorrectModifiers(TC_HTML, text, "COLOR:c\nFONT:c\n"); // onMouseEvent and onKeyUp accept javascript. text = ""; // ID quoted AssertCorrectModifiers(TC_HTML, text, "ID:j\n"); text = ""; // ID not quoted AssertCorrectModifiers(TC_HTML, text, "ID:J=number\n"); text = ""; // not common AssertCorrectModifiers(TC_HTML, text, "ID:j\n"); // If ID is javascript code, J=number will break it, for good and bad. text = ""; AssertCorrectModifiers(TC_HTML, text, "ID:J=number\n"); // Target just needs html escaping. text = ""; AssertCorrectModifiers(TC_HTML, text, "TARGET:h\n"); // Test a parsing corner case which uses TemplateDirective // call in the parser to change state properly. To reproduce // both variables should be unquoted and the first should // have no value except the variable substitution. text = ""; AssertCorrectModifiers(TC_HTML, text, "CLASS:H=attribute\nQUERY:u\n"); // TODO(jad): Once we have a fix for it in code, fix me. // Javascript URL is not properly supported, we currently // apply :h which is not sufficient. text = "bla"; AssertCorrectModifiers(TC_HTML, text, "VAR:h\n"); // Special handling for BI_SPACE and BI_NEWLINE. text = "{{BI_SPACE}}"; AssertCorrectModifiers(TC_HTML, text, "BI_SPACE\n"); // Untouched. text = "{{BI_NEWLINE}}"; AssertCorrectModifiers(TC_HTML, text, "BI_NEWLINE\n"); // Untouched. // Check that the parser is parsing BI_SPACE, if not, it would have failed. text = "text"; AssertCorrectModifiers(TC_HTML, text, "BI_SPACE\nVAR:c\n"); // XML and JSON modes. text = "{{DATA}}"; AssertCorrectModifiers(TC_XML, text, "VAL:xml_escape\nDATA:xml_escape\n"); text = "{ x = \"{{VAL}}\"}"; AssertCorrectModifiers(TC_JSON, text, "VAL:j\n"); // 2. Escaping modifiers were set, handle them. // 2a: Modifier :none is honored whether the escaping is correct or not. text = "Hello {{USER:none}}"; // :none on its own. AssertCorrectModifiers(TC_HTML, text, "USER:none\n"); text = "Hello {{USER:h:none}}"; // correct escaping. AssertCorrectModifiers(TC_HTML, text, "USER:h:none\n"); text = "Hello {{USER:j:none}}"; // incorrect escaping. AssertCorrectModifiers(TC_HTML, text, "USER:j:none\n"); text = ""; AssertCorrectModifiers(TC_HTML, text, "ID:none\n"); // 2b: Correct modifiers, nothing to change. text = "Hello {{USER:h}}"; AssertCorrectModifiers(TC_HTML, text, "USER:h\n"); text = "Hello {{USER:U=html}}"; // :U=html is a valid replacement for .h AssertCorrectModifiers(TC_HTML, text, "USER:U=html\n"); text = "Hello {{USER:H=url}}"; // :H=url (a.k.a. U=html) is valid too AssertCorrectModifiers(TC_HTML, text, "USER:H=url\n"); text = "Hello {{USER:h:j}}"; // Extra :j, honor it. AssertCorrectModifiers(TC_HTML, text, "USER:h:j\n"); text = "bla"; AssertCorrectModifiers(TC_HTML, text, "URL:U=html\n"); text = "bla"; // :h is valid. AssertCorrectModifiers(TC_HTML, text, "QUERY:h\n"); text = "bla"; // so is :u. AssertCorrectModifiers(TC_HTML, text, "QUERY:u\n"); text = ""; AssertCorrectModifiers(TC_HTML, text, "ID:j\n"); text = ""; AssertCorrectModifiers(TC_HTML, text, "ID:J=number\n"); text = ""; // correct :U=css AssertCorrectModifiers(TC_HTML, text, "URL:U=css\n"); // 2c: Incorrect modifiers, add our own. text = "Hello {{USER:j}}"; // Missing :h AssertCorrectModifiers(TC_HTML, text, "USER:j:h\n"); text = "Hello {{USER:c:c:c:c:c:j}}"; // Still missing :h AssertCorrectModifiers(TC_HTML, text, "USER:c:c:c:c:c:j:h\n"); text = ""; // Missing :j AssertCorrectModifiers(TC_HTML, text, "VAR:h:j\n"); text = ""; // Extra :h:j AssertCorrectModifiers(TC_HTML, text, "VAR:j:h:j\n"); text = ""; // Unquoted AssertCorrectModifiers(TC_HTML, text, "ID:j:J=number\n"); // 2d: Custom modifiers are maintained. text = "Hello {{USER:x-bla}}"; // Missing :h AssertCorrectModifiers(TC_HTML, text, "USER:x-bla:h\n"); text = "Hello {{USER:x-bla:h}}"; // Correct, accept it. AssertCorrectModifiers(TC_HTML, text, "USER:x-bla:h\n"); text = "Hello {{USER:x-bla:x-foo}}"; // Missing :h AssertCorrectModifiers(TC_HTML, text, "USER:x-bla:x-foo:h\n"); text = "Hello {{USER:x-bla:none}}"; // Complete due to :none AssertCorrectModifiers(TC_HTML, text, "USER:x-bla:none\n"); text = "Hello {{USER:h:x-bla}}"; // Still missing :h. AssertCorrectModifiers(TC_HTML, text, "USER:h:x-bla:h\n"); text = "Hello {{USER:x-bla:h:x-foo}}"; // Still missing :h AssertCorrectModifiers(TC_HTML, text, "USER:x-bla:h:x-foo:h\n"); text = "Hello {{USER:x-bla:h:x-foo:h}}"; // Valid, accept it. AssertCorrectModifiers(TC_HTML, text, "USER:x-bla:h:x-foo:h\n"); // 2e: Equivalent modifiers are honored. All HTML Escapes. text = "Hello {{USER:p}}"; AssertCorrectModifiers(TC_HTML, text, "USER:p\n"); text = "Hello {{USER:H=attribute}}"; AssertCorrectModifiers(TC_HTML, text, "USER:H=attribute\n"); text = "Hello {{USER:H=snippet}}"; AssertCorrectModifiers(TC_HTML, text, "USER:H=snippet\n"); text = "Hello {{USER:H=pre}}"; AssertCorrectModifiers(TC_HTML, text, "USER:H=pre\n"); // All URL + HTML Escapes. text = "bla"; AssertCorrectModifiers(TC_HTML, text, "URL:H=url\n"); text = "bla"; AssertCorrectModifiers(TC_HTML, text, "URL:U=html\n"); // 2f: Initialize template in Javascript Context. text = "var a = '{{VAR}}'"; // Escaping not given. AssertCorrectModifiers(TC_JS, text, "VAR:j\n"); text = "var a = '{{VAR:none}}'"; // Variable safe. AssertCorrectModifiers(TC_JS, text, "VAR:none\n"); text = "var a = '{{VAR:j}}'"; // Escaping correct. AssertCorrectModifiers(TC_JS, text, "VAR:j\n"); text = "var a = '{{VAR:h}}'"; // Escaping incorrect. AssertCorrectModifiers(TC_JS, text, "VAR:h:j\n"); text = "var a = '{{VAR:J=number}}'"; // Not considered equiv. AssertCorrectModifiers(TC_JS, text, "VAR:J=number:j\n"); // 2g: Honor any modifiers for BI_SPACE and BI_NEWLINE. text = "{{BI_NEWLINE:j}}"; // An invalid modifier for the context. AssertCorrectModifiers(TC_HTML, text, "BI_NEWLINE:j\n"); text = "{{BI_SPACE:h}}"; // An otherwise valid modifier. AssertCorrectModifiers(TC_HTML, text, "BI_SPACE:h\n"); text = "{{BI_SPACE:x-bla}}"; // Also support custom modifiers. AssertCorrectModifiers(TC_HTML, text, "BI_SPACE:x-bla\n"); // 2h: TC_CSS, TC_XML and TC_JSON text = "H1{margin-{{START_EDGE}}:0;\n text-align:{{END_EDGE}}\n}"; AssertCorrectModifiers(TC_CSS, text, "START_EDGE:c\nEND_EDGE:c\n"); text = "body{background:url('{{URL:U=css}}')}"; // :U=css valid substitute AssertCorrectModifiers(TC_CSS, text, "URL:U=css\n"); text = "body{background:url('{{URL:U=html}}')}"; // Not valid, will add :c. AssertCorrectModifiers(TC_CSS, text, "URL:U=html:c\n"); text = ""; // Correct escaping AssertCorrectModifiers(TC_XML, text, "VAL:xml_escape\n"); text = ""; // XSS equivalent AssertCorrectModifiers(TC_XML, text, "VAL:H=attribute\n"); text = ""; // XSS equivalent AssertCorrectModifiers(TC_XML, text, "VAL:h\n"); text = ""; // Not XSS equivalent AssertCorrectModifiers(TC_XML, text, "VAL:H=pre:xml_escape\n"); text = ""; // Not XSS equivalent AssertCorrectModifiers(TC_XML, text, "VAL:c:xml_escape\n"); text = "{user={{USER:j}}"; // Correct escaping AssertCorrectModifiers(TC_JSON, text, "USER:j\n"); text = "{user={{USER:o}}"; // json_escape is XSS equivalent AssertCorrectModifiers(TC_JSON, text, "USER:o\n"); text = "{user={{USER:h}}"; // but html_escape is not AssertCorrectModifiers(TC_JSON, text, "USER:h:j\n"); // 2i: Variables with XssSafe Custom modifiers are untouched. ASSERT(ctemplate::AddXssSafeModifier("x-test-cm", &ctemplate::html_escape)); text = "Hello {{USER:x-test-cm}}"; // Missing :h AssertCorrectModifiers(TC_HTML, text, "USER:x-test-cm\n"); text = "Hello {{USER:x-test-cm:j}}"; // Extra :j AssertCorrectModifiers(TC_HTML, text, "USER:x-test-cm:j\n"); text = "Hello {{USER:x-test-cm:x-foo}}"; // Non-safe modifier AssertCorrectModifiers(TC_HTML, text, "USER:x-test-cm:x-foo\n"); text = "Hello {{USER:x-foo:x-test-cm}}"; // Non-safe modifier AssertCorrectModifiers(TC_HTML, text, "USER:x-foo:x-test-cm\n"); text = "Hello {{USER:x-test-cm:none}}"; // Complete due to :none AssertCorrectModifiers(TC_HTML, text, "USER:x-test-cm:none\n"); text = "Hello {{USER:h:x-test-cm}}"; // Prior escaping AssertCorrectModifiers(TC_HTML, text, "USER:h:x-test-cm\n"); // 3. Larger test with close to every escaping case. text = "\n" "\n" "

{{TITLE}}

\n" "\n" "
\n" " \n" " \n" "
\n" "
\n" "
\n" "\n" "bla\n" "Goodbye friend {{USER}}!\n\n"; expected_out = "CSS_URL:U=css\n" "COLOR:c\n" "TITLE:h\n" "IMG_URL:U=html\n" "HL:H=attribute\n" "FORM_MSG:h\n" "BG_COLOR:c\n" "MSG_TEXT:j\n" "MOUSE:j\n" // :j also escapes html entities "USER:h\n"; AssertCorrectModifiers(TC_HTML, text, expected_out); } // More "end-to-end" test to ensure that variables are // escaped as expected with auto-escape mode enabled. // Obviously there is a lot more we can test. TEST(Template, VariableWithAutoEscape) { string text, expected_out; TemplateDictionary dict("dict"); string good_url("http://www.google.com/"); string bad_url("javascript:alert();"); text = "hi {{VAR}} lo"; dict.SetValue("VAR", "yo"); AssertCorrectEscaping(TC_HTML, dict, text, "hi <bad>yo lo"); text = "bla"; dict.SetValue("URL", good_url); expected_out = "bla"; AssertCorrectEscaping(TC_HTML, dict, text, expected_out); dict.SetValue("URL", bad_url); expected_out = "bla"; AssertCorrectEscaping(TC_HTML, dict, text, expected_out); text = "
"; dict.SetValue("DISPLAY", "none"); expected_out = "
"; AssertCorrectEscaping(TC_HTML, dict, text, expected_out); // Bad characters are simply removed in CleanseCss. dict.SetValue("URL", "!#none_ "); expected_out = "
"; AssertCorrectEscaping(TC_HTML, dict, text, expected_out); text = ""; dict.SetValue("EVENT", "safe"); expected_out = ""; AssertCorrectEscaping(TC_HTML, dict, text, expected_out); dict.SetValue("EVENT", "f = 'y';"); expected_out = ""; // Check special handling of BI_SPACE and BI_NEWLINE. text = "Hello\n{{BI_SPACE}}bla{{BI_NEWLINE}}foo."; expected_out = "Hello bla\nfoo."; AssertCorrectEscaping(TC_HTML, dict, text, expected_out); // TC_CSS text = "H1{margin-{{EDGE}}:0; text-align:{{BAD_EDGE}}}"; dict.SetValue("EDGE", "left"); dict.SetValue("BAD_EDGE", "$$center()!!"); // Bad chars are removed. AssertCorrectEscaping(TC_CSS, dict, text, "H1{margin-left:0; text-align:center!!}"); // TC_XML and TC_JSON text = "{{DATA}}"; dict.SetValue("DATA", "good-data"); AssertCorrectEscaping(TC_XML, dict, text, "good-data"); dict.SetValue("DATA", "FOO"); AssertCorrectEscaping(TC_XML, dict, text, "<BAD>FOO</BAD>"); text = "{user = \"{{USER}}\"}"; dict.SetValue("USER", "good-user"); AssertCorrectEscaping(TC_JSON, dict, text, "{user = \"good-user\"}"); dict.SetValue("USER", "evil'<>\""); AssertCorrectEscaping(TC_JSON, dict, text, "{user = \"evil\\x27\\x3c\\x3e\\x22\"}"); } // Test that the template initialization fails in auto-escape // mode if the parser failed to parse. TEST(Template, FailedInitWithAutoEscape) { Strip strip = STRIP_WHITESPACE; // Taken from HTML Parser test suite. string bad_html = "\n"; ASSERT(NULL == StringToTemplateWithAutoEscaping(bad_html, strip, TC_HTML)); // Missing quotes around URL, not accepted in URL-taking attributes. bad_html = "bla"; ASSERT(NULL == StringToTemplateWithAutoEscaping(bad_html, strip, TC_HTML)); // Missing quotes around STYLE, not accepted in style-taking attributes. bad_html = "
"; ASSERT(NULL == StringToTemplateWithAutoEscaping(bad_html, strip, TC_HTML)); } TEST(Template, AutoEscaping) { Strip strip = STRIP_WHITESPACE; Template *tpl; string filename; string text; string user = "John<>Doe"; string user_esc = "John<>Doe"; // Positive test cases -- template initialization succeeds. // We also check that modifiers that were missing or given incorrect // have been updated as expected. // TODO(jad): Cut-down redundancy by merging with // TestCorrectModifiersForAutoEscape. text = "{{%AUTOESCAPE context=\"HTML\"}}" // HTML "{{USER:o}}"; ASSERT(tpl = StringToTemplate(text, strip)); string expected_mods = "USER:o:h\nURL:U=html\nCLASS:h:H=attribute\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{%AUTOESCAPE context=\"HTML\" state=\"IN_TAG\"}}" // HTML in tag "href=\"{{URL}}\" class={{CLASS:h}} style=\"font:{{COLOR}}\""; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "URL:U=html\nCLASS:h:H=attribute\nCOLOR:c\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{%AUTOESCAPE context=\"HTML\" state=\"in_tag\"}}" // lowercase ok "href=\"{{URL}}\" class={{CLASS:h}} style=\"font:{{COLOR}}\""; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "URL:U=html\nCLASS:h:H=attribute\nCOLOR:c\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); // Repeat the test with trailing HTML that closes the tag. This is // undefined behavior. We test it to ensure the parser does not choke. text += ">HelloSome text"; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "URL:U=html\nCLASS:h:H=attribute\nCOLOR:c\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{%AUTOESCAPE context=\"JAVASCRIPT\"}}" // JAVASCRIPT "var a = {{A}}; var b = '{{B:h}}';"; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "A:J=number\nB:h:j\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{%AUTOESCAPE context=\"CSS\"}}" // CSS "body {color:\"{{COLOR}}\"; font-size:{{SIZE:j}}"; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "COLOR:c\nSIZE:j:c\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{%AUTOESCAPE context=\"JSON\"}}" // JSON "{ 'id': {{ID:j}}, 'value': {{VALUE:h}} }"; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "ID:j\nVALUE:h:j\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{%AUTOESCAPE context=\"XML\"}}" // XML "{{DATA:h}}"; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "VAL:xml_escape\nDATA:h\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{%AUTOESCAPE context=\"xml\"}}" // lower-case XML "{{DATA:h}}"; ASSERT(tpl = StringToTemplate(text, strip)); expected_mods = "VAL:xml_escape\nDATA:h\n"; AssertCorrectModifiersInTemplate(tpl, text, expected_mods); text = "{{!bla}}{{%AUTOESCAPE context=\"HTML\"}}"; // after comment ASSERT(tpl = StringToTemplate(text, strip)); text = "{{%AUTOESCAPE context=\"HTML\" state=\"default\"}}"; ASSERT(tpl = StringToTemplate(text, strip)); // adding state // Negative test cases - template initialization fails due to errors // in the marker. Also checks that our parsing is defensive. text = "{{%AUTOESCAPE}}"; // missing context ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPER context=\"HTML\"}}"; // invalid id ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%}}"; // missing id ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{% }}"; // missing id ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{% =}}"; // missing id ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE =\"HTML\"}}"; // missing name ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE foo=\"HTML\"}}"; // bogus name ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE =}}"; // lone '=' ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=HTML}}"; // val not quoted ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"HTML}}"; // no end quotes ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"\\\"HTML\"}}"; // Unescape val ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"\\\"HT\\\"\\\"ML\\\"\"}}"; // more complex ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"\"HTML\"}}"; // Unescape val ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"JAVASCRIPT\" bla}}"; // extra attr ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"JAVASCRIPT\"bla}}"; // invalid value ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"JAVASCRIPT\" foo=bla}}"; // extra attr/val ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"HTML\"}}"; // extra whitesp ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context =\"HTML\"}}"; // extra whitesp ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context= \"HTML\"}}"; // extra whitesp ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"HTML\" }}"; // extra whitesp ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"Xml\"}}"; // mixed-case xml ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"HTML\" state=\"tag\"}}"; // bad state ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{%AUTOESCAPE context=\"CSS\" state=\"IN_TAG\"}}"; // invalid state ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "Hello{{%AUTOESCAPE context=\"HTML\"}}"; // after text ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{USER}}{{%AUTOESCAPE context=\"HTML\"}}"; // after variable ASSERT((tpl = StringToTemplate(text, strip)) == NULL); text = "{{#SEC}}{{%AUTOESCAPE context=\"HTML\"}}{{/SEC}}"; // not in MAIN ASSERT((tpl = StringToTemplate(text, strip)) == NULL); string kAutoescapeHtmlPragma = "{{%AUTOESCAPE context=\"HTML\"}}"; // Check that Selective Auto-Escape does not auto-escape included templates // unless these are also marked for auto-escape. To attest that, // we check that when no escaping was given in the included template, none // will be applied to it. USER will not get html-escaped. text = kAutoescapeHtmlPragma + "{{>INC}}"; tpl = StringToTemplate(text, strip); ASSERT(tpl); string inc_text = "{{USER}}"; // missing :h escaping. TemplateDictionary dict("dict"); TemplateDictionary *inc_dict = dict.AddIncludeDictionary("INC"); inc_dict->SetFilename(StringToTemplateFile(inc_text)); inc_dict->SetValue("USER", user); AssertExpandIs(tpl, &dict, user, true); // Add AUTOESCAPE pragma to included template and check that it works. inc_text = kAutoescapeHtmlPragma + inc_text; filename = StringToTemplateFile(inc_text); inc_dict->SetFilename(filename); AssertExpandIs(tpl, &dict, user_esc, true); // Check that Selective Auto-Escape works with Template::StringToTemplate. tpl = Template::StringToTemplate(inc_text, strip); ASSERT(tpl); TemplateDictionary dict2("dict2"); dict2.SetValue("USER", user); AssertExpandIs(tpl, &dict2, user_esc, true); delete tpl; // Test that Selective AutoEscape follows included templates: Included // templates 2 and 4 are registered for auto-escaping but not included // templates 1 and 3. Check that only templates 2 and 4 get escaped. text = "Parent: {{USER}}; {{>INCONE}}"; string text_inc1 = "INC1: {{USER1}}; {{>INCTWO}}"; string text_inc2 = kAutoescapeHtmlPragma + "INC2: {{USER2}}; {{>INCTHREE}}"; string text_inc3 = "INC3: {{USER3}}; {{>INCFOUR}}"; string text_inc4 = kAutoescapeHtmlPragma + "INC4: {{USER4}}"; dict.SetValue("USER", user); TemplateDictionary *dict_inc1 = dict.AddIncludeDictionary("INCONE"); dict_inc1->SetFilename(StringToTemplateFile(text_inc1)); dict_inc1->SetValue("USER1", user); TemplateDictionary *dict_inc2 = dict_inc1->AddIncludeDictionary("INCTWO"); filename = StringToTemplateFile(text_inc2); dict_inc2->SetFilename(filename); dict_inc2->SetValue("USER2", user); TemplateDictionary *dict_inc3 = dict_inc2->AddIncludeDictionary("INCTHREE"); dict_inc3->SetFilename(StringToTemplateFile(text_inc3)); dict_inc3->SetValue("USER3", user); TemplateDictionary *dict_inc4 = dict_inc3->AddIncludeDictionary("INCFOUR"); filename = StringToTemplateFile(text_inc4); dict_inc4->SetFilename(filename); dict_inc4->SetValue("USER4", user); tpl = StringToTemplate(text, strip); string expected_out = "Parent: " + user + "; INC1: " + user + "; INC2: " + user_esc + "; INC3: " + user + "; INC4: " + user_esc; AssertExpandIs(tpl, &dict, expected_out, true); // Check that we do not modify template-includes. // Here, xml_escape would have been changed to :h:xml_escape // causing a double-escaping of the USER. text = kAutoescapeHtmlPragma + "{{>INC:xml_escape}}"; inc_text = "{{USER}}"; tpl = StringToTemplate(text, strip); ASSERT(tpl); TemplateDictionary dict3("dict"); inc_dict = dict3.AddIncludeDictionary("INC"); inc_dict->SetFilename(StringToTemplateFile(inc_text)); inc_dict->SetValue("USER", user); AssertExpandIs(tpl, &dict3, user_esc, true); // Test that {{%...}} is a "removable" marker. A related test is // also added to TestStrip(). tpl = StringToTemplate("{{%AUTOESCAPE context=\"HTML\"}}\nText\n Text", STRIP_BLANK_LINES); AssertExpandIs(tpl, &dict, "Text\n Text", true); } TEST(Template, RegisterString) { ASSERT(Template::StringToTemplateCache("file1", "Some text")); Template* tpl = Template::GetTemplate("file1", STRIP_WHITESPACE); ASSERT(tpl); ASSERT(Template::GetTemplate("file1", STRIP_WHITESPACE) == tpl); ASSERT(Template::StringToTemplateCache("file2", "Top {{>INC}}")); TemplateDictionary dict("dict"); string expected = "Some text"; AssertExpandIs(tpl, &dict, expected, true); TemplateDictionary* sub_dict = dict.AddIncludeDictionary("INC"); sub_dict->SetFilename("file1"); tpl = Template::GetTemplate("file2", STRIP_WHITESPACE); expected = "Top Some text"; AssertExpandIs(tpl, &dict, expected, true); } // This tests that StaticTemplateString is sufficiently initialized at // static-initialization time (as opposed to dynamic-initialization // time, which comes later), that we can safely expand templates // during dynamic initialization. This is worth testing, because some // parts of a StaticTemplateString -- especially the hash value, *do* // get computed later at dynamic-initialization time, and we want to // make sure that things still work properly even if we access the // StaticTemplateString before that happens. extern const StaticTemplateString kLateDefine; class DynamicInitializationTemplateExpander { public: DynamicInitializationTemplateExpander() { Template* tpl = Template::StringToTemplate("hi {{VAR}} lo", STRIP_WHITESPACE); TemplateDictionary dict("dict"); dict.SetValue("VAR", TemplateString("short-lived", strlen("short"))); AssertExpandIs(tpl, &dict, "hi short lo", true); dict.SetValue("VAR", kHello); AssertExpandIs(tpl, &dict, "hi Hello lo", true); dict.SetValue("VAR", kLateDefine); AssertExpandIs(tpl, &dict, "hi laterz lo", true); delete tpl; } }; DynamicInitializationTemplateExpander sts_tester; // this runs before main() const StaticTemplateString kLateDefine = STS_INIT(kLateDefine, "laterz"); int main(int argc, char** argv) { CreateOrCleanTestDirAndSetAsTmpdir(FLAGS_test_tmpdir); // This goes first so that future tests don't mess up the filenames. // So we make it a normal function rather than using TEST() on it. TestAnnotation(); return RUN_ALL_TESTS(); } ctemplate-ctemplate-2.4/src/tests/template_unittest_test_footer.in000066400000000000000000000024201363122316600257620ustar00rootroot00000000000000

{{#TRIM_LINE}}
{{CLEARDOT}}
{{/TRIM_LINE}}
{{#FOOTER_BAR_TEXT}} {{HOME_LINK}} -{{BI_SPACE}} {{ADVERTISE_LINK}} -{{BI_SPACE}} {{#PROMO_LICENSING_SECTION}} {{PROMO_LICENSING_LINK}} -{{BI_SPACE}} {{/PROMO_LICENSING_SECTION}} {{ABOUT_GOOGLE_LINK}} {{/FOOTER_BAR_TEXT}} {{#EMPTY_FOOTER_BAR_TEXT}}  {{/EMPTY_FOOTER_BAR_TEXT}}

{{MODIFIED_BY_GOOGLE}}{{MSG_copyright}} {{>ODP_ATTRIBUTION}}
{{#CLOSING_DIV_SECTION}}
{{/CLOSING_DIV_SECTION}} {{>GOOGLE_COMPLETE_JS}} {{>SITE_SPEED_SCRIPT_FOOTER}} {{>AD_WIDE_WRAP_JAVASCRIPT}} {{>BROWSER_STATS_INCLUDE}} {{#LATENCY_PREFETCH}} {{/LATENCY_PREFETCH}} {{#JAVASCRIPT_FOOTER_SECTION}} {{/JAVASCRIPT_FOOTER_SECTION}} ctemplate-ctemplate-2.4/src/tests/template_unittest_test_footer_dict01.out000066400000000000000000000013061363122316600273310ustar00rootroot00000000000000

<b>Time to go home!</b> - Be advertiser #2 - - About Google!

2005© Google Inc. (all rights reserved)
ctemplate-ctemplate-2.4/src/tests/template_unittest_test_footer_dict02.out000066400000000000000000000002521363122316600273310ustar00rootroot00000000000000


ctemplate-ctemplate-2.4/src/tests/template_unittest_test_html.in000066400000000000000000000034741363122316600254420ustar00rootroot00000000000000 {{BI_NEWLINE}} {{TAG_LINE}} {{#UPDATE_SECTION}}Last updated: {{UPDATE}}
{{/UPDATE_SECTION}} {{#RESULTS}}
  1. Result: {{RESULT}}
  2. Goodness of result: {{GOODNESS}}
  3. xml-safe result: {{XML_RESULT}}
{{/RESULTS}} {{>FOOTER}} ctemplate-ctemplate-2.4/src/tests/template_unittest_test_html_dict01.anno_out000066400000000000000000000047271363122316600300240ustar00rootroot00000000000000{{#FILE=template_unittest_test_html.in}}{{#SEC=__{{MAIN}}__}} {{#VAR=BI_NEWLINE}} {{/VAR}} {{#VAR=TAG_LINE}}{{/VAR}} {{#SEC=UPDATE_SECTION}}Last updated: {{#VAR=UPDATE}}monday & tuesday{{/VAR}}
{{/SEC}} {{#SEC=RESULTS}}
  1. Result: {{#VAR=RESULT}}<&>"result" #0'&'{{/VAR}}
  2. Goodness of result: {{#VAR=GOODNESS}}5{{/VAR}}
  3. xml-safe result: {{#VAR=XML_RESULT}}<&>"result" #0'&'{{/VAR}}
{{/SEC}}{{#SEC=RESULTS}}
  1. Result: {{#VAR=RESULT}}<&>"result" #1'&'{{/VAR}}
  2. Goodness of result: {{#VAR=GOODNESS}}6{{/VAR}}
  3. xml-safe result: {{#VAR=XML_RESULT}}<&>"result" #1'&'{{/VAR}}
{{/SEC}}{{#SEC=RESULTS}}
  1. Result: {{#VAR=RESULT}}<&>"result" #2'&'{{/VAR}}
  2. Goodness of result: {{#VAR=GOODNESS}}7{{/VAR}}
  3. xml-safe result: {{#VAR=XML_RESULT}}<&>"result" #2'&'{{/VAR}}
{{/SEC}}{{#INC=FOOTER}}{{#FILE=template_unittest_test_footer.in}}{{#SEC=__{{MAIN}}__}}


{{#VAR=MODIFIED_BY_GOOGLE}}{{/VAR}}{{#VAR=MSG_copyright}}{{/VAR}}
{{/SEC}}{{/FILE}}{{/INC}} {{/SEC}}{{/FILE}} ctemplate-ctemplate-2.4/src/tests/template_unittest_test_html_dict01.out000066400000000000000000000020231363122316600267740ustar00rootroot00000000000000 Last updated: monday & tuesday
  1. Result: <&>"result" #0'&'
  2. Goodness of result: 5
  3. xml-safe result: <&>"result" #0'&'
  1. Result: <&>"result" #1'&'
  2. Goodness of result: 6
  3. xml-safe result: <&>"result" #1'&'
  1. Result: <&>"result" #2'&'
  2. Goodness of result: 7
  3. xml-safe result: <&>"result" #2'&'


ctemplate-ctemplate-2.4/src/tests/template_unittest_test_html_dict02.out000066400000000000000000000001611363122316600267760ustar00rootroot00000000000000 ctemplate-ctemplate-2.4/src/tests/template_unittest_test_invalid1.in000066400000000000000000000001421363122316600261720ustar00rootroot00000000000000This is html {This is fine} {{This is not so fine}} {{! Can't have spaces in a variable name!}} ctemplate-ctemplate-2.4/src/tests/template_unittest_test_invalid2.in000066400000000000000000000001201363122316600261670ustar00rootroot00000000000000This is html. {{#SECTION}} Happy section {{/SEC}} But the section never ends! ctemplate-ctemplate-2.4/src/tests/template_unittest_test_markerdelim.in000066400000000000000000000037331363122316600267700ustar00rootroot00000000000000{{! Use <<< and >>> to delimit template stuff, rather than {{ and }} {{=<<< >>>=}} <<>> >>> <<>> <<<#UPDATE_SECTION>>>Last updated: <<>>
<<>> <<<#RESULTS>>> >> align=<<>><<>> <<<#WHITE_BG>>>bgColor=#ffffff <<>>border=0>
  1. Result: <<>>
  2. Goodness of result: <<>>
  3. xml-safe result: <<>>
<<>> <<<>FOOTER>>> ctemplate-ctemplate-2.4/src/tests/template_unittest_test_markerdelim_dict01.out000066400000000000000000000020231363122316600303240ustar00rootroot00000000000000 Last updated: monday & tuesday
  1. Result: <&>"result" #0'&'
  2. Goodness of result: 5
  3. xml-safe result: <&>"result" #0'&'
  1. Result: <&>"result" #1'&'
  2. Goodness of result: 6
  3. xml-safe result: <&>"result" #1'&'
  1. Result: <&>"result" #2'&'
  2. Goodness of result: 7
  3. xml-safe result: <&>"result" #2'&'


ctemplate-ctemplate-2.4/src/tests/template_unittest_test_markerdelim_dict02.out000066400000000000000000000001611363122316600303260ustar00rootroot00000000000000 ctemplate-ctemplate-2.4/src/tests/template_unittest_test_modifiers.in000066400000000000000000000005501363122316600264470ustar00rootroot00000000000000 {{#UPDATE_SECTION}} {{UPDATE}} {{UPDATE:h}} {{UPDATE:javascript_escape}} {{UPDATE:h:u}} {{/UPDATE_SECTION}} {{! There should be no problem with this comment having a : in it. }} {{>SIMPLE:html_escape}} {{BI_NEWLINE}}ctemplate-ctemplate-2.4/src/tests/template_unittest_test_modifiers_dict01.anno_out000066400000000000000000000015461363122316600310350ustar00rootroot00000000000000{{#FILE=template_unittest_test_modifiers.in}}{{#SEC=__{{MAIN}}__}} {{#SEC=UPDATE_SECTION}} {{#VAR=UPDATE}}monday & tuesday{{/VAR}} {{#VAR=UPDATE:html_escape}}monday &amp; tuesday{{/VAR}} {{#VAR=UPDATE:javascript_escape}}monday \x26amp; tuesday{{/VAR}} {{#VAR=UPDATE:html_escape:url_query_escape}}monday+%26amp%3Bamp%3B+tuesday{{/VAR}} {{/SEC}} {{#INC=SIMPLE:html_escape}}{{#FILE=template_unittest_test_simple.in}}{{#SEC=__{{MAIN}}__}}<html> <head> {{#VAR=HEAD}}{{/VAR}} </head> <body> {{#VAR=BODY}}{{/VAR}} </body> </html>{{#VAR=BI_NEWLINE}} {{/VAR}}{{/SEC}}{{/FILE}}{{/INC}} {{#VAR=BI_NEWLINE}} {{/VAR}}{{/SEC}}{{/FILE}}ctemplate-ctemplate-2.4/src/tests/template_unittest_test_modifiers_dict01.out000066400000000000000000000004441363122316600300160ustar00rootroot00000000000000monday & tuesdaymonday &amp; tuesdaymonday \x26amp; tuesdaymonday+%26amp%3Bamp%3B+tuesday<html><head></head><body></body></html> ctemplate-ctemplate-2.4/src/tests/template_unittest_test_nul.in000066400000000000000000000001611363122316600252620ustar00rootroot00000000000000Thisdocument{{HAS}}{{#LOTS}}of{{NULLS}}{{/LOTS}}. But none is in a {{VARIABLE}}{{NAME}}{{>EH}}! ctemplate-ctemplate-2.4/src/tests/template_unittest_test_nul_dict01.out000066400000000000000000000000551363122316600266310ustar00rootroot00000000000000Thisdocument.But none is in a !ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_css.in000066400000000000000000000004471363122316600273260ustar00rootroot00000000000000{{%AUTOESCAPE context="CSS"}} P.abstract { margin-{{AE_START_EDGE}}:0; text-align:{{AE_END_EDGE}}; font-size:{{AE_FONT_SIZE_PC}}; } .italic {font-style:{{AE_ITALIC}}} H1 { font-size:{{AE_FONT_SIZE_PT}}; color:{{AE_MAUVE_RGB}}; } BODY {background:transparent url('{{AE_URL_GOOD}}');}ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_css_dict01.out000066400000000000000000000002611363122316600306650ustar00rootroot00000000000000P.abstract {margin-left:0;text-align:center;font-size:120%;}.italic {font-style:italic}H1 {font-size:12pt;color:#FF7BD5;}BODY {background:transparent url('httpwww.google.com');}ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_css_dict02.out000066400000000000000000000002001363122316600306570ustar00rootroot00000000000000P.abstract {margin-:0;text-align:;font-size:;}.italic {font-style:}H1 {font-size:;color:;}BODY {background:transparent url('');}ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_html.in000066400000000000000000000006751363122316600275050ustar00rootroot00000000000000{{%AUTOESCAPE context="HTML"}} {{!Is a copy of template_unittest_test_autoescape_simple.in}}

{{AE_TITLE_GOOD}}

{{AE_TITLE_BAD}}

Goodbye {{AE_USERNAME_GOOD}}! Goodbye {{AE_USERNAME_BAD}}!ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_html_dict01.out000066400000000000000000000005541363122316600310460ustar00rootroot00000000000000

Hello World!

Hello <script>alert(1)</script> World!

Goodbye Mr. Nice!Goodbye Doctor<script>alert(2)</script>Evil!ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_html_dict02.out000066400000000000000000000002511363122316600310410ustar00rootroot00000000000000

Goodbye !Goodbye !ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_js.in000066400000000000000000000003711363122316600271460ustar00rootroot00000000000000{{%AUTOESCAPE context="JAVASCRIPT"}} var msg_text1 = '{{AE_JS_GOOD}}'; var msg_text2 = '{{AE_JS_BAD}}'; {{!Below variable is not quoted}} var msg_text3 = {{AE_JS_BAD}}; {{!Below variable ends up with :h:j}} var msg_text4 = '{{AE_JS_BAD:h}}';ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_js_dict01.out000066400000000000000000000002441363122316600305120ustar00rootroot00000000000000var msg_text1 = 'your text here';var msg_text2 = 'your text\x27is clever\x27thanks';var msg_text3 = null;var msg_text4 = 'your text\x26#39;is clever\x26#39;thanks';ctemplate-ctemplate-2.4/src/tests/template_unittest_test_selective_js_dict02.out000066400000000000000000000001121363122316600305050ustar00rootroot00000000000000var msg_text1 = '';var msg_text2 = '';var msg_text3 = ;var msg_text4 = '';ctemplate-ctemplate-2.4/src/tests/template_unittest_test_simple.in000066400000000000000000000001201363122316600257500ustar00rootroot00000000000000 {{HEAD}} {{BODY}} {{BI_NEWLINE}}ctemplate-ctemplate-2.4/src/tests/template_unittest_test_simple_dict01.out000066400000000000000000000000761363122316600273270ustar00rootroot00000000000000 This is the head ctemplate-ctemplate-2.4/src/tests/template_unittest_test_simple_dict02.out000066400000000000000000000000501363122316600273200ustar00rootroot00000000000000 ctemplate-ctemplate-2.4/src/tests/template_unittest_test_simple_dict03.out000066400000000000000000000000551363122316600273260ustar00rootroot00000000000000 ctemplate-ctemplate-2.4/src/tests/template_unittest_test_valid1.in000066400000000000000000000001131363122316600256410ustar00rootroot00000000000000This is ok. {This is also ok.} Look ma!, no template substitutions at all! ctemplate-ctemplate-2.4/src/tests/template_unittest_test_valid1_dict01.out000066400000000000000000000001101363122316600272030ustar00rootroot00000000000000This is ok.{This is also ok.}Look ma!, no template substitutions at all!ctemplate-ctemplate-2.4/src/windows/000077500000000000000000000000001363122316600175755ustar00rootroot00000000000000ctemplate-ctemplate-2.4/src/windows/config.h000077500000000000000000000146441363122316600212270ustar00rootroot00000000000000/* A manual version of config.h fit for windows machines. */ /* Sometimes we accidentally #include this config.h instead of the one in .. -- this is particularly true for msys/mingw, which uses the unix config.h but also runs code in the windows directory. */ #if defined(__MINGW32__) || defined(__MING64__) #define CTEMPLATE_DLL_DECL // These two lines make sure we read the unix-style config.h, and not the // windows-style config.h -- it would be bad if we tried to read both! #include "../config.h" #define GOOGLE_CTEMPLATE_WINDOWS_CONFIG_H_ #endif #ifndef GOOGLE_CTEMPLATE_WINDOWS_CONFIG_H_ #define GOOGLE_CTEMPLATE_WINDOWS_CONFIG_H_ /* Namespace for Google classes */ #define GOOGLE_NAMESPACE ctemplate /* the location of or */ #define HASH_MAP_H /* the namespace of hash_map/hash_set */ #define HASH_NAMESPACE std /* the location of or */ #define HASH_SET_H /* Define to 1 if you have the header file. */ #undef HAVE_BYTESWAP_H /* Define to 1 if you have the header file, and it defines `DIR'. */ #undef HAVE_DIRENT_H /* Define to 1 if you have the header file. */ #undef HAVE_DLFCN_H /* Define to 1 if you have the header file. */ #undef HAVE_ENDIAN_H /* Define to 1 if you have the `getopt' function. */ #undef HAVE_GETOPT /* Define to 1 if you have the header file. */ #undef HAVE_GETOPT_H /* Define to 1 if you have the `getopt_long' function. */ #undef HAVE_GETOPT_LONG /* define if the compiler has hash_map */ #define HAVE_HASH_MAP 1 /* define if the compiler has hash_set */ #define HAVE_HASH_SET 1 /* Define to 1 if you have the header file. */ #undef HAVE_INTTYPES_H /* Define to 1 if you have the header file. */ #undef HAVE_LIBKERN_OSBYTEORDER_H /* Define to 1 if you have the header file. */ #undef HAVE_MACHINE_ENDIAN_H /* Define to 1 if you have the header file. */ #define HAVE_MEMORY_H 1 /* define if the compiler implements namespaces */ #define HAVE_NAMESPACES 1 /* Define to 1 if you have the header file, and it defines `DIR'. */ #undef HAVE_NDIR_H /* Define if you have POSIX threads libraries and header files. */ #undef HAVE_PTHREAD /* define if the compiler implements pthread_rwlock_* */ #undef HAVE_RWLOCK /* Define to 1 if you have the header file. */ #undef HAVE_STDINT_H /* Define to 1 if you have the header file. */ #define HAVE_STDLIB_H 1 /* Define to 1 if you have the header file. */ #undef HAVE_STRINGS_H /* Define to 1 if you have the header file. */ #define HAVE_STRING_H 1 /* Define to 1 if you have the header file. */ #undef HAVE_SYS_BYTEORDER_H /* Define to 1 if you have the header file, and it defines `DIR'. */ #undef HAVE_SYS_DIR_H /* Define to 1 if you have the header file. */ #undef HAVE_SYS_ENDIAN_H /* Define to 1 if you have the header file. */ #undef HAVE_SYS_ISA_DEFS_H /* Define to 1 if you have the header file, and it defines `DIR'. */ #undef HAVE_SYS_NDIR_H /* Define to 1 if you have the header file. */ #define HAVE_SYS_STAT_H 1 /* Define to 1 if you have the header file. */ #define HAVE_SYS_TYPES_H 1 /* Define to 1 if the system has the type `uint32_t'. */ #undef HAVE_UINT32_T /* Define to 1 if the system has the type `uint64_t'. */ #undef HAVE_UINT64_T /* Define to 1 if you have the header file. */ #undef HAVE_UNISTD_H /* define if the compiler supports unordered_{map,set} */ #define HAVE_UNORDERED_MAP 1 /* Define to 1 if you have the header file. */ #undef HAVE_UTIME_H /* Define to 1 if the system has the type `u_int32_t'. */ #undef HAVE_U_INT32_T /* Define to 1 if the system has the type `u_int64_t'. */ #undef HAVE_U_INT64_T /* define if your compiler has __attribute__ */ #undef HAVE___ATTRIBUTE__ /* Define to 1 if the system has the type `__uint32. */ #define HAVE___INT32 1 /* Define to 1 if the system has the type `__uint64. */ #define HAVE___INT64 1 /* The namespace to put the htmlparser code. */ #define HTMLPARSER_NAMESPACE google_ctemplate_streamhtmlparser /* define if first argument to InterlockedExchange is just LONG */ #undef INTERLOCKED_EXCHANGE_NONVOLATILE /* Define to the sub-directory in which libtool stores uninstalled libraries. */ #undef LT_OBJDIR /* Name of package */ #undef PACKAGE /* Define to the address where bug reports for this package should be sent. */ #undef PACKAGE_BUGREPORT /* Define to the full name of this package. */ #undef PACKAGE_NAME /* Define to the full name and version of this package. */ #define PACKAGE_STRING "ctemplate 1.1" /* Define to the one symbol short name of this package. */ #undef PACKAGE_TARNAME /* Define to the home page for this package. */ #undef PACKAGE_URL /* Define to the version of this package. */ #undef PACKAGE_VERSION /* printf format code for printing a size_t and ssize_t */ #define PRIdS "Id" /* printf format code for printing a size_t and ssize_t */ #define PRIuS "Iu" /* printf format code for printing a size_t and ssize_t */ #define PRIxS "Ix" /* Define to necessary symbol if this constant uses a non-standard name on your system. */ #undef PTHREAD_CREATE_JOINABLE /* Define to 1 if you have the ANSI C header files. */ #define STDC_HEADERS 1 /* the namespace where STL code like vector<> is defined */ #define STL_NAMESPACE std /* Version number of package */ #undef VERSION /* Stops putting the code inside the Google namespace */ #define _END_GOOGLE_NAMESPACE_ } /* Puts following code inside the Google namespace */ #define _START_GOOGLE_NAMESPACE_ namespace ctemplate { // --------------------------------------------------------------------- // Extra stuff not found in config.h.in // This must be defined before anything else in our project: make sure // that when compiling the dll, we export our functions/classes. Safe // to define this here because this file is only used internally, to // compile the DLL, and every dll source file #includes "config.h" // before anything else. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllexport) # define CTEMPLATE_DLL_DECL_FOR_UNITTESTS __declspec(dllimport) #endif // TODO(csilvers): include windows/port.h in every relevant source file instead? #include "windows/port.h" #endif /* GOOGLE_CTEMPLATE_WINDOWS_CONFIG_H_ */ ctemplate-ctemplate-2.4/src/windows/ctemplate/000077500000000000000000000000001363122316600215535ustar00rootroot00000000000000ctemplate-ctemplate-2.4/src/windows/ctemplate/find_ptr.h000066400000000000000000000056431363122316600235410ustar00rootroot00000000000000// Copyright (c) 2012, Olaf van der Spek // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: Olaf van der Spek #ifndef TEMPLATE_FIND_PTR_H_ #define TEMPLATE_FIND_PTR_H_ #include // NOTE: if you are statically linking the template library into your binary // (rather than using the template .dll), set '/D CTEMPLATE_DLL_DECL=' // as a compiler flag in your project file to turn off the dllimports. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllimport) #endif namespace ctemplate { template const typename T::value_type* find_ptr0(const T& c, U v) { typename T::const_iterator i = c.find(v); return i == c.end() ? NULL : &*i; } template typename T::value_type::second_type* find_ptr(T& c, U v) { typename T::iterator i = c.find(v); return i == c.end() ? NULL : &i->second; } template const typename T::value_type::second_type* find_ptr(const T& c, U v) { typename T::const_iterator i = c.find(v); return i == c.end() ? NULL : &i->second; } template typename T::value_type::second_type find_ptr2(T& c, U v) { typename T::iterator i = c.find(v); return i == c.end() ? NULL : i->second; } template const typename T::value_type::second_type find_ptr2(const T& c, U v) { typename T::const_iterator i = c.find(v); return i == c.end() ? NULL : i->second; } } #endif // TEMPLATE_FIND_PTR_H_ ctemplate-ctemplate-2.4/src/windows/ctemplate/per_expand_data.h000066400000000000000000000137051363122316600250500ustar00rootroot00000000000000// Copyright (c) 2008, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) // // In addition to a TemplateDictionary, there is also a PerExpandData // dictionary. This dictionary holds information that applies to one // call to Expand, such as whether to annotate the template expansion // output. A template dictionary is associated with a template (.tpl) // file; a per-expand dictionary is associated to a particular call to // Expand() in a .cc file. // // For (many) more details, see the doc/ directory. #ifndef TEMPLATE_PER_EXPAND_DATA_H_ #define TEMPLATE_PER_EXPAND_DATA_H_ #include // for NULL #include // for strcmp #include #include #include // for StringHash // NOTE: if you are statically linking the template library into your binary // (rather than using the template .dll), set '/D CTEMPLATE_DLL_DECL=' // as a compiler flag in your project file to turn off the dllimports. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllimport) #endif namespace ctemplate { class TemplateModifier; class TemplateAnnotator; class CTEMPLATE_DLL_DECL PerExpandData { public: PerExpandData() : annotate_path_(NULL), annotator_(NULL), expand_modifier_(NULL), map_(NULL) { } ~PerExpandData(); // Indicate that annotations should be inserted during template expansion. // template_path_start - the start of a template path. When // printing the filename for template-includes, anything before and // including template_path_start is elided. This can make the // output less dependent on filesystem location for template files. void SetAnnotateOutput(const char* template_path_start) { annotate_path_ = template_path_start; } // Whether to annotate the expanded output. bool annotate() const { return annotate_path_ != NULL; } // The annotate-path; undefined if annotate() != true const char* annotate_path() const { return annotate_path_; } // This sets the TemplateAnnotator to be used when annotating is on. // This allows you to override the default text-based annotator // that will be used if you do not call this. The passed annotator // will be aliased by this object and returned by annotator(). // Passing NULL has the special behavior of causing annotator() to // revert to returning its built-in instance. void SetAnnotator(TemplateAnnotator* annotator) { annotator_ = annotator; } // This returns the TemplateAnnotator to be used when annotating is on. // The value returned will be either an instance previously provided // to SetAnnotator() or the callable built-in text-based annotator. TemplateAnnotator* annotator() const; // This is a TemplateModifier to be applied to all templates // expanded via this call to Expand(). That is, this modifier is // applies to the template (.tpl) file we expand, as well as // sub-templates that are expanded due to {{>INCLUDE}} directives. // Caller is responsible for ensuring that modifier exists for the // lifetime of this object. void SetTemplateExpansionModifier(const TemplateModifier* modifier) { expand_modifier_ = modifier; } const TemplateModifier* template_expansion_modifier() const { return expand_modifier_; } // Store data in this structure, to be used by template modifiers // (see template_modifiers.h). Call with value set to NULL to clear // any value previously set. Caller is responsible for ensuring key // and value point to valid data for the lifetime of this object. void InsertForModifiers(const char* key, const void* value); // Retrieve data specific to this Expand call. Returns NULL if key // is not found. This should only be used by template modifiers. const void* LookupForModifiers(const char* key) const; // Same as Lookup, but casts the result to a c string. const char* LookupForModifiersAsString(const char* key) const { return static_cast(LookupForModifiers(key)); } private: struct DataEq { bool operator()(const char* s1, const char* s2) const; }; typedef std::unordered_map DataMap; const char* annotate_path_; TemplateAnnotator* annotator_; const TemplateModifier* expand_modifier_; DataMap* map_; PerExpandData(const PerExpandData&); // disallow evil copy constructor void operator=(const PerExpandData&); // disallow evil operator= }; } #endif // TEMPLATE_PER_EXPAND_DATA_H_ ctemplate-ctemplate-2.4/src/windows/ctemplate/str_ref.h000066400000000000000000000063031363122316600233720ustar00rootroot00000000000000// Copyright (c) 2012, Olaf van der Spek // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: Olaf van der Spek #ifndef TEMPLATE_STR_REF_H_ #define TEMPLATE_STR_REF_H_ #include // NOTE: if you are statically linking the template library into your binary // (rather than using the template .dll), set '/D CTEMPLATE_DLL_DECL=' // as a compiler flag in your project file to turn off the dllimports. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllimport) #endif namespace ctemplate { template class str_ref_basic { public: str_ref_basic() { clear(); } template str_ref_basic(const U& c) { if (c.end() != c.begin()) assign(&*c.begin(), c.end() - c.begin() + &*c.begin()); else clear(); } str_ref_basic(const void* b, const void* e) { assign(b, e); } str_ref_basic(const void* b, size_t sz) { assign(b, sz); } str_ref_basic(const char* b) { if (b) assign(b, strlen(b)); else clear(); } void clear() { begin_ = end_ = NULL; } void assign(const void* b, const void* e) { begin_ = reinterpret_cast(b); end_ = reinterpret_cast(e); } void assign(const void* b, size_t sz) { begin_ = reinterpret_cast(b); end_ = begin_ + sz; } T begin() const { return begin_; } T end() const { return end_; } T data() const { return begin(); } size_t size() const { return end() - begin(); } bool empty() const { return begin() == end(); } private: T begin_; T end_; }; typedef str_ref_basic data_ref; typedef str_ref_basic str_ref; } #endif // TEMPLATE_STR_REF_H_ ctemplate-ctemplate-2.4/src/windows/ctemplate/template.h000066400000000000000000000534031363122316600235440ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // This file implements the Template class. For information about // how to use this class, and to write the templates it takes as input, // see the doc/ directory. #ifndef CTEMPLATE_TEMPLATE_H_ #define CTEMPLATE_TEMPLATE_H_ #include // for time_t #include #include #include #include // We include this just so folks don't have to include both template.h // and template_dictionary.h, or template_namelist.h etc, to use the // template system; we don't actually use anything in these files // ourselves. #if 1 #include #include #include #else namespace ctemplate { class TemplateDictionaryInterface; class PerExpandData; } #endif namespace ctemplate_htmlparser { class HtmlParser; } // NOTE: if you are statically linking the template library into your binary // (rather than using the template .dll), set '/D CTEMPLATE_DLL_DECL=' // as a compiler flag in your project file to turn off the dllimports. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllimport) #endif namespace ctemplate { // These free functions form the "simple" template API, and support // the most common operations (expanding a template from a file, and // from a string). They all just delegate to a default instance of // the TemplateCache object. // // For more sophisticated use of the template system, you may need // to create your own TemplateCache object, and work directly with // it. See template_cache.h for details. extern CTEMPLATE_DLL_DECL const TemplateCache* default_template_cache(); extern CTEMPLATE_DLL_DECL TemplateCache* mutable_default_template_cache(); // ---- EXPANDING A TEMPLATE ------- // ExpandTemplate // ExpandWithData // Loads the template named filename from disk if necessary -- it // gets it from the cache instead, if the template had been loaded // before or if it had been put explicitly in the cache via a call // to StringToTemplateCache() -- and expands it using the given // dictionary. // The first version is the most general, followed by common-case code. inline bool ExpandTemplate(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface *dictionary, ExpandEmitter* output) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, NULL, output); } inline bool ExpandTemplate(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface* dictionary, std::string* output_buffer) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, NULL, output_buffer); } // If you want any per-expand data to be used at expand time, call // this routine instead of Expand. You pass in an extra // PerExpandData structure (see per_expand_data.h) which sets this // data: whether or not you want the template to be annotated, and // any data you want to pass in to template modifers. If // per_expand_data is NULL, this is exactly the same as Expand(). // The first version is the most general, followed by common-case code. inline bool ExpandWithData(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface *dictionary, PerExpandData* per_expand_data, ExpandEmitter* output) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, per_expand_data, output); } inline bool ExpandWithData(const TemplateString& filename, Strip strip, const TemplateDictionaryInterface* dictionary, PerExpandData* per_expand_data, std::string* output_buffer) { return mutable_default_template_cache()->ExpandWithData( filename, strip, dictionary, per_expand_data, output_buffer); } // ---- INSERTING INTO THE CACHE ------- // LoadTemplate // StringToTemplateCache // Reads a file from disk and inserts it into the template, if it's // not already there. Returns true on success or false if the // template could not be found, or could not be parsed. It's never // necessary to call this -- Expand() will load templates lazily if // needed -- but you may want to if you want to make sure templates // exist before trying to expand them, or because you want to // control disk access patterns, or for some other reason. inline bool LoadTemplate(const TemplateString& filename, Strip strip) { return mutable_default_template_cache()->LoadTemplate(filename, strip); } // Inserts the given string into the default template cache, as if // it were a file read from disk. You can call Expand() with its // first arg (filename) the same as the key you use here. You can // also use this key as the 'filename' for sub-included templates, // in TemplateDictionary::SetFilename(). inline bool StringToTemplateCache(const TemplateString& key, const TemplateString& content, Strip strip) { return mutable_default_template_cache()->StringToTemplateCache( key, content, strip); } inline bool StringToTemplateCache(const TemplateString& key, const char* content, size_t content_len, Strip strip) { return mutable_default_template_cache()->StringToTemplateCache( key, content, content_len, strip); } // --------------------------------------------------------------------- // The follow are deprecated. // TODO(csilvers): move to parsed_template.h // TemplateState of a template is: // - TS_EMPTY before parsing is complete, // - TS_ERROR if a syntax error was found during parsing, and // - TS_READY if parsing has completed successfully // (TS_UNUSED is not used) enum TemplateState { TS_UNUSED, TS_EMPTY, TS_ERROR, TS_READY }; // Used for Auto-Escape. It represents the different contexts a template may // be initialized in via the AUTOESCAPE pragma in the template file // (or string). It is only public for testing. The contexts are: // - TC_HTML: The template contains HTML code. Need not be a complete HTML // page just content the browser interprets in the context of // HTML parsing. This should be the most common context to use. // This mode activates our HTML parser. // - TC_JS: The template contains raw javascript. If your template // starts with OR // // For variables that are quoted (i.e. string literals) use javascript_escape. // // Limitations: // . NaN, +/-Infinity and null are not recognized. // . Output is not guaranteed to be a valid literal, // e.g: +55+-e34 will output as is. // e.g: trueeee will output nothing as it is not a valid boolean. // // Details: // . For Hex numbers, it checks for case-insensitive 0x[0-9A-F]+ // that should be a proper check. // . For other numbers, it checks for case-insensitive [0-9eE+-.]* // so can also accept invalid numbers such as the number 5..45--10. // . "true" and "false" (without quotes) are also accepted and that's it. // class CTEMPLATE_DLL_DECL JavascriptNumber : public TemplateModifier { MODIFY_SIGNATURE_; }; extern CTEMPLATE_DLL_DECL JavascriptNumber javascript_number; // Escapes characters not in [0-9a-zA-Z.,_:*/~!()-] as %-prefixed hex. // Space is encoded as a +. class CTEMPLATE_DLL_DECL UrlQueryEscape : public TemplateModifier { MODIFY_SIGNATURE_; }; extern CTEMPLATE_DLL_DECL UrlQueryEscape url_query_escape; // Escapes " \ / to \" \\ \/ \f \r \n \b \t // Also escapes < > & to their corresponding \uXXXX representation // (\u003C, \u003E, \u0026 respectively). class CTEMPLATE_DLL_DECL JsonEscape : public TemplateModifier { MODIFY_SIGNATURE_; }; extern CTEMPLATE_DLL_DECL JsonEscape json_escape; // Inserts the given prefix (given as the argument to this modifier) // after every newline in the text. Note that it does *not* insert // prefix at the very beginning of the text -- in its expected use, // that prefix will already be present before this text, in the // template. This is meant to be used internally, and is not exported // via the g_modifiers list. class CTEMPLATE_DLL_DECL PrefixLine : public TemplateModifier { MODIFY_SIGNATURE_; }; extern CTEMPLATE_DLL_DECL PrefixLine prefix_line; #undef MODIFY_SIGNATURE_ // Registers a new template modifier. // long_name must start with "x-". // If the modifier takes a value (eg "{{VAR:x-name=value}}"), then // long_name should end with "=". This is similar to getopt(3) syntax. // We also allow value-specializations, with specific values specified // as part of long-name. For instance: // AddModifier("x-mod=", &my_modifierA); // AddModifier("x-mod=bar", &my_modifierB); // AddModifier("x-mod2", &my_modifierC); // For the template // {{VAR1:x-mod=foo}} {{VAR2:x-mod=bar}} {{VAR3:x-mod=baz}} {{VAR4:x-mod2}} // VAR1 and VAR3 would get modified by my_modifierA, VAR2 by my_modifierB, // and VAR4 by my_modifierC. The order of the AddModifier calls is not // significant. extern CTEMPLATE_DLL_DECL bool AddModifier(const char* long_name, const TemplateModifier* modifier); // Same as AddModifier() above except that the modifier is considered // to produce safe output that can be inserted in any context without // the need for additional escaping. This difference only impacts // the Auto-Escape mode: In that mode, when a variable (or template-include) // has a modifier added via AddXssSafeModifier(), it is excluded from // further escaping, effectively treated as though it had the :none modifier. // Because Auto-Escape is disabled for any variable and template-include // that includes such a modifier, use this function with care and ensure // that it may not emit harmful output that could lead to XSS. // // Some valid uses of AddXssSafeModifier: // . A modifier that converts a string to an integer since // an integer is generally safe in any context. // . A modifier that returns one of a fixed number of safe values // depending on properties of the input. // // Some not recommended uses of AddXssSafeModifier: // . A modifier that applies some extra formatting to the input // before returning it since the output will still contain // harmful content if the input does. // . A modifier that applies one type of escaping to the input // (say HTML-escape). This may be dangerous when the modifier // is used in a different context (say Javascript) where this // escaping may be inadequate. extern CTEMPLATE_DLL_DECL bool AddXssSafeModifier(const char* long_name, const TemplateModifier* modifier); } #endif // TEMPLATE_TEMPLATE_MODIFIERS_H_ ctemplate-ctemplate-2.4/src/windows/ctemplate/template_namelist.h000066400000000000000000000165551363122316600254470ustar00rootroot00000000000000// Copyright (c) 2006, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // // This class implements some template helper classes, that manage // template files and make it easier to monitor them. // // For information about how to use these classes and macros, and to // write the templates it takes as input, // see the doc/ directory. #ifndef TEMPLATE_TEMPLATE_NAMELIST_H_ #define TEMPLATE_TEMPLATE_NAMELIST_H_ #include // for time_t #include #include #include #include // for Strip #include // for StringHash // NOTE: if you are statically linking the template library into your binary // (rather than using the template .dll), set '/D CTEMPLATE_DLL_DECL=' // as a compiler flag in your project file to turn off the dllimports. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllimport) #endif namespace ctemplate { // RegisterTemplateFilename // Used to define a reference variable for the name of a template file. E.g: // RegisterTemplateFilename(EXAMPLE_FN, "example.tpl"); // This should be used rather than the seemingly equivalent: // #define EXAMPLE_FN "example.tpl" // As in the latter, any call to GetTemplate may then reference the name // via the first argument. In the example, that would be: // Template::GetTemplate(EXAMPLE_FN, DO_NOT_STRIP); // By using this macro, rather than the #define, all templates can // later be tested for either existence or correct syntax after program // start-up. // TODO (we wish): Make this macro produce the #include for the auto-generated // header files, when and if the macro pre-processor supports that #define RegisterTemplateFilename(var, name) \ const char* const var = ctemplate::TemplateNamelist::RegisterTemplate(name); // Class: TemplateNamelist // Each time this class is instantiated, the name passed to // the constructor is added to the class's static list of names. The // entire list may be retrieved later by calling the GetList accessor // method. Or they may all be checked for existence or for correct // syntax via the other methods. We use this in our // sanity-checking code to make sure all the templates used by a program // exist and are syntactically correct. class CTEMPLATE_DLL_DECL TemplateNamelist { friend class TemporaryRegisterTemplate; public: // These types should be taken as 'generic' containers. The only // thing you should do with them is call size() and/or iterate // between begin() and end(), and the only operations we promise // the iterators will support are operator* and operator++. typedef std::unordered_set NameListType; typedef std::vector MissingListType; typedef std::vector SyntaxListType; public: // Takes a name and pushes it onto the static namelist // Returns: a pointer to the entry in namelist which holds the name static const char* RegisterTemplate(const char* name); // GetList // Description: Returns the collected list of names. static const NameListType& GetList(); // GetMissingList // If refresh is true or if it is the first time the function is called // in the execution of the program, it creates (or clears) the missing // list and then fills it with the list of // templates that the program knows about but are missing from // the template directory. // If refresh is false and it is not the first time the function is // called, it merely returns the list created in the // call when the last refresh was done. // NOTE: The templates are NOT read, parsed, or cached // by this function. static const MissingListType& GetMissingList(bool refresh); // GetBadSyntaxList // If refresh is true or if it is the first time the function is called // in the execution of the program, it creates (or clears) the "bad // syntax" list and then fills it with the list of // templates that the program knows about but contain syntax errors. // A missing file is not considered a syntax error, and thus is // not included in this list. // If refresh is false and it is not the first time the function is // called, it merely returns the list created in the // call when the last refresh was done. // NOTE: The side effect of calling this the first time or // with refresh equal true is that all templates are parsed and cached. // Hence they need to be retrieved with the flags that // the program needs them loaded with (i.e, the strip parameter // passed to Template::GetTemplate.) static const SyntaxListType& GetBadSyntaxList(bool refresh, Strip strip); // GetLastmodTime // Iterates through all non-missing templates, and returns the latest // last-modification time for the template files, as returned by stat(). // This can be used to make sure template files are getting refreshed. static time_t GetLastmodTime(); // AllDoExist // Retrieves the missing list (always refreshing the list) // and returns true if it contains any names. // Else, returns false. static bool AllDoExist(); // IsAllSyntaxOkay // Retrieves the "bad syntax" list (always refreshing the list) // and returns true if it contains any names. // Else, returns false. // NOTE: The side effect of calling this is that all templates are parsed // and cached, hence they need to be retrieved with the flags that // the program needs them loaded with. (I.e, the strip parameter // ultimately passed to Template::GetTemplate.) static bool IsAllSyntaxOkay(Strip strip); protected: // The static list of names static NameListType *namelist_; static MissingListType *missing_list_; static SyntaxListType *bad_syntax_list_; private: TemplateNamelist(const TemplateNamelist&); // disallow copying void operator=(const TemplateNamelist&); }; } #endif // TEMPLATE_TEMPLATE_NAMELIST_H_ ctemplate-ctemplate-2.4/src/windows/ctemplate/template_pathops.h000066400000000000000000000061471363122316600253050ustar00rootroot00000000000000// Copyright (c) 2007, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // --- // Author: csilvers@google.com (Craig Silverstein) #ifndef TEMPLATE_TEMPLATE_PATHOPS_H_ #define TEMPLATE_TEMPLATE_PATHOPS_H_ #include // NOTE: if you are statically linking the template library into your binary // (rather than using the template .dll), set '/D CTEMPLATE_DLL_DECL=' // as a compiler flag in your project file to turn off the dllimports. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllimport) #endif namespace ctemplate { extern CTEMPLATE_DLL_DECL const char kCWD[]; // a string that's equivalent to "./" extern CTEMPLATE_DLL_DECL const char kRootdir[]; // a string that's equivalent to "/" extern CTEMPLATE_DLL_DECL std::string PathJoin(const std::string& a, const std::string& b); extern CTEMPLATE_DLL_DECL bool IsAbspath(const std::string& path); extern CTEMPLATE_DLL_DECL bool IsDirectory(const std::string& path); // checks if path ends with / extern CTEMPLATE_DLL_DECL void NormalizeDirectory(std::string* dir); // adds trailing / if needed extern CTEMPLATE_DLL_DECL std::string Basename(const std::string& path); // everything after last / // Returns true iff text contains the word as a full word, i.e. delimited by one // of [.,_-#*?:] on both the sides. // This is used while loading a template, to check that the file's name matches // the auto-escape mode specified by it. // NOTE: This assumes that the word doesn't contain any of the delimiter // characters. extern CTEMPLATE_DLL_DECL bool ContainsFullWord(const std::string& text, const std::string& word); } #endif // TEMPLATE_TEMPLATE_PATHOPS_H_ ctemplate-ctemplate-2.4/src/windows/ctemplate/template_string.h000066400000000000000000000325051363122316600251320ustar00rootroot00000000000000// Copyright (c) 2008, Google Inc. // All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions are // met: // // * Redistributions of source code must retain the above copyright // notice, this list of conditions and the following disclaimer. // * Redistributions in binary form must reproduce the above // copyright notice, this list of conditions and the following disclaimer // in the documentation and/or other materials provided with the // distribution. // * Neither the name of Google Inc. nor the names of its // contributors may be used to endorse or promote products derived from // this software without specific prior written permission. // // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. // // --- // Author: csilvers@google.com (Craig Silerstein) #ifndef TEMPLATE_TEMPLATE_STRING_H_ #define TEMPLATE_TEMPLATE_STRING_H_ #include // for memcmp() and size_t #include #include #include #include #include class TemplateStringTest; // needed for friendship declaration class StaticTemplateStringTest; #if 0 extern char _start[] __attribute__((weak)); // linker emits: start of .text extern char data_start[] __attribute__((weak)); // start of .data #endif // NOTE: if you are statically linking the template library into your binary // (rather than using the template .dll), set '/D CTEMPLATE_DLL_DECL=' // as a compiler flag in your project file to turn off the dllimports. #ifndef CTEMPLATE_DLL_DECL # define CTEMPLATE_DLL_DECL __declspec(dllimport) #endif namespace ctemplate { // Most methods of TemplateDictionary take a TemplateString rather than a // C++ string. This is for efficiency: it can avoid extra string copies. // For any argument that takes a TemplateString, you can pass in any of: // * A C++ string // * A char* // * A StringPiece // * TemplateString(char*, length) // The last of these is the most efficient, though it requires more work // on the call site (you have to create the TemplateString explicitly). class TemplateString; // If you have a string constant (e.g. the string literal "foo") that // you need to pass into template routines repeatedly, it is more // efficient if you convert it into a TemplateString only once. The // way to do this is to use a global StaticTemplateString via STS_INIT // (note: do this at global scope *only*!): // static const StaticTemplateString kMyVar = STS_INIT(kMyVar, "MY_VALUE"); struct StaticTemplateString; #define STS_INIT(name, str) STS_INIT_WITH_HASH(name, str, 0) // Let's define a convenient hash_compare function for hashing 'normal' // strings: char* and string. We'll use MurmurHash, which is probably // better than the STL default. We don't include TemplateString or // StaticTemplateString here, since they are hashed more efficiently // based on their id. struct CTEMPLATE_DLL_DECL StringHash { inline size_t operator()(const char* s) const { return Hash(s, strlen(s)); }; inline size_t operator()(const std::string& s) const { return Hash(s.data(), s.size()); } inline bool operator()(const char* a, const char* b) const { return (a != b) && (strcmp(a, b) < 0); // <, for MSVC } inline bool operator()(const std::string& a, const std::string& b) const { return a < b; } static const size_t bucket_size = 4; // These are required by MSVC static const size_t min_buckets = 8; // 4 and 8 are the defaults private: size_t Hash(const char* s, size_t slen) const; }; // ----------------------- THE CLASSES ------------------------------- typedef uint64_t TemplateId; const TemplateId kIllegalTemplateId = 0; struct CTEMPLATE_DLL_DECL StaticTemplateString { // Do not define a constructor! We use only brace-initialization, // so the data is constructed at static-initialization time. // Anything you want to put in a constructor, put in // StaticTemplateStringInitializer instead. // These members shouldn't be accessed directly, except in the // internals of the template code. They are public because that is // the only way we can brace-initialize them. struct { const char* ptr_; size_t length_; mutable TemplateId id_; // sometimes lazily-initialized. } do_not_use_directly_; // This class is a good hash_compare functor to pass in as the third // argument to unordered_map<>, when creating a map whose keys are // StaticTemplateString. NOTE: This class isn't that safe to use, // because it requires that StaticTemplateStringInitializer has done // its job. Unfortunately, even when you use the STS_INIT macro // (which is always, right??), dynamic initialiation does not happen // in a particular order, and objects in different .cc files may // reference a StaticTemplateString before the corresponding // StaticTemplateStringInitializer sets the id. struct Hasher { inline size_t operator()(const StaticTemplateString& sts) const; inline bool operator()(const StaticTemplateString& a, // <, for MSVC const StaticTemplateString& b) const; static const size_t bucket_size = 4; // These are required by MSVC static const size_t min_buckets = 8; // 4 and 8 are the defaults }; inline bool empty() const { return do_not_use_directly_.length_ == 0; } // Allows comparisons of StaticTemplateString objects as if they were // strings. This is useful for STL. inline bool operator==(const StaticTemplateString& x) const; }; class CTEMPLATE_DLL_DECL TemplateString { public: TemplateString(const char* s) : ptr_(s ? s : ""), length_(strlen(ptr_)), is_immutable_(InTextSegment(ptr_)), id_(kIllegalTemplateId) { } TemplateString(const std::string& s) : ptr_(s.data()), length_(s.size()), is_immutable_(false), id_(kIllegalTemplateId) { } TemplateString(const char* s, size_t slen) : ptr_(s), length_(slen), is_immutable_(InTextSegment(s)), id_(kIllegalTemplateId) { } TemplateString(const StaticTemplateString& s) : ptr_(s.do_not_use_directly_.ptr_), length_(s.do_not_use_directly_.length_), is_immutable_(true), id_(s.do_not_use_directly_.id_) { } const char* begin() const { return ptr_; } const char* end() const { return ptr_ + length_; } const char* data() const { return ptr_; } size_t size() const { return length_; } inline bool empty() const { return length_ == 0; }; inline bool is_immutable() const { return is_immutable_; } // STL requires this to be public for hash_map, though I'd rather not. inline bool operator==(const TemplateString& x) const { return GetGlobalId() == x.GetGlobalId(); } private: // Only TemplateDictionaries and template expansion code can read these. friend class TemplateDictionary; friend class TemplateCache; // for GetGlobalId friend class StaticTemplateStringInitializer; // for AddToGlo... friend struct TemplateStringHasher; // for GetGlobalId friend TemplateId GlobalIdForTest(const char* ptr, int len); friend TemplateId GlobalIdForSTS_INIT(const TemplateString& s); TemplateString(const char* s, size_t slen, bool is_immutable, TemplateId id) : ptr_(s), length_(slen), is_immutable_(is_immutable), id_(id) { } // This returns true if s is in the .text segment of the binary. // (Note this only checks .text of the main executable, not of // shared libraries. So it may not be all that useful.) // This requires the gnu linker (and probably elf), to define // _start and data_start. static bool InTextSegment(const char* s) { #if 0 return (s >= _start && s < data_start); // in .text #else return false; // the conservative choice: assume it's not static memory #endif } protected: inline void CacheGlobalId() { // used by HashedTemplateString id_ = GetGlobalId(); }; private: // Returns the global id, computing it for the first time if // necessary. Note that since this is a const method, we don't // store the computed value in id_, even if id_ is 0. TemplateId GetGlobalId() const; // Adds this TemplateString to the map from global-id to name. void AddToGlobalIdToNameMap(); // Use sparingly. Converting to a string loses information about the // id of the template string, making operations require extra hash_compare // computations. std::string ToString() const { return std::string(ptr_, length_); } // Does the reverse map from TemplateId to TemplateString contents. // Returns a TemplateString(kStsEmpty) if id isn't found. Note that // the TemplateString returned is not necessarily NUL terminated. static TemplateString IdToString(TemplateId id); const char* ptr_; size_t length_; // Do we need to manage memory for this string? bool is_immutable_; // Id for hash_compare lookups. If 0, we don't have one and it should be // computed as-needed. TemplateId id_; }; // ----------------------- THE CODE ------------------------------- // Use the low-bit from TemplateId as the "initialized" flag. Note // that since all initialized TemplateId have the lower bit set, it's // safe to have used 0 for kIllegalTemplateId, as we did above. const TemplateId kTemplateStringInitializedFlag = 1; inline bool IsTemplateIdInitialized(TemplateId id) { return id & kTemplateStringInitializedFlag; } // This is a helper struct used in TemplateString::Hasher/TemplateStringHasher struct TemplateIdHasher { size_t operator()(TemplateId id) const { // The shift has two effects: it randomizes the "initialized" flag, // and slightly improves the randomness of the low bits. This is // slightly useful when size_t is 32 bits, or when using a small // hash_compare tables with power-of-2 sizes. return static_cast(id ^ (id >> 33)); } bool operator()(TemplateId a, TemplateId b) const { // <, for MSVC return a < b; } static const size_t bucket_size = 4; // These are required by MSVC static const size_t min_buckets = 8; // 4 and 8 are the defaults }; inline size_t StaticTemplateString::Hasher::operator()( const StaticTemplateString& sts) const { TemplateId id = sts.do_not_use_directly_.id_; assert(IsTemplateIdInitialized(id)); return TemplateIdHasher()(id); } inline bool StaticTemplateString::Hasher::operator()( const StaticTemplateString& a, const StaticTemplateString& b) const { TemplateId id_a = a.do_not_use_directly_.id_; TemplateId id_b = b.do_not_use_directly_.id_; assert(IsTemplateIdInitialized(id_a)); assert(IsTemplateIdInitialized(id_b)); return TemplateIdHasher()(id_a, id_b); } inline bool StaticTemplateString::operator==( const StaticTemplateString& x) const { return (do_not_use_directly_.length_ == x.do_not_use_directly_.length_ && (do_not_use_directly_.ptr_ == x.do_not_use_directly_.ptr_ || memcmp(do_not_use_directly_.ptr_, x.do_not_use_directly_.ptr_, do_not_use_directly_.length_) == 0)); } // We set up as much of StaticTemplateString as we can at // static-initialization time (using brace-initialization), but some // things can't be set up then. This class is for those things; it // runs at dynamic-initialization time. If you add logic here, only // do so as an optimization: this may be called rather late (though // before main), so other code should not depend on this being called // before them. class CTEMPLATE_DLL_DECL StaticTemplateStringInitializer { public: // This constructor operates on a const StaticTemplateString - we should // only change those things that are mutable. explicit StaticTemplateStringInitializer(const StaticTemplateString* sts); }; // Don't use this. This is used only in auto-generated .varnames.h files. #define STS_INIT_WITH_HASH(name, str, hash_compare) \ { { str, sizeof("" str "")-1, hash_compare } }; \ namespace ctemplate_sts_init { \ static const ctemplate::StaticTemplateStringInitializer name##_init(&name); \ } // We computed this hash_compare value for the empty string online. In debug // mode, we verify it's correct during runtime (that is, that we // verify the hash_compare function used by make_tpl_varnames_h hasn't changed // since we computed this number). Note this struct is logically // static, but since it's in a .h file, we don't say 'static' but // instead rely on the linker to provide the POD-with-internal-linkage // magic. const StaticTemplateString kStsEmpty = STS_INIT_WITH_HASH(kStsEmpty, "", 1457976849674613049ULL); } #endif // TEMPLATE_TEMPLATE_STRING_H_ ctemplate-ctemplate-2.4/src/windows/port.cc000077500000000000000000000107231363122316600210760ustar00rootroot00000000000000/* Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * --- * Author: Craig Silverstein */ #include "windows/config.h" #ifndef _WIN32 # error You should only be including windows/port.cc in a windows environment! #endif #include // for va_list, va_start, va_end #include // for strstr() #include #include "port.h" // These call the windows _vsnprintf, but always NUL-terminate. #if !defined(__MINGW32__) && !defined(__MINGW64__) && (!defined(_MSC_VER) || _MSC_VER < 1900) /* mingw already defines */ int safe_vsnprintf(char *str, size_t size, const char *format, va_list ap) { if (size == 0) // not even room for a \0? return -1; // not what C99 says to do, but what windows does str[size-1] = '\0'; return _vsnprintf(str, size-1, format, ap); } int snprintf(char *str, size_t size, const char *format, ...) { int r; va_list ap; va_start(ap, format); r = vsnprintf(str, size, format, ap); va_end(ap); return r; } #endif /* #if !defined(__MINGW32__) && !defined(__MINGW64__) */ #ifdef __cplusplus #include #include #include using std::string; using std::vector; namespace ctemplate { // defined (for unix) in template_test_utils.cc string TmpFile(const char* basename) { char tmppath_buffer[1024]; int tmppath_len = GetTempPathA(sizeof(tmppath_buffer), tmppath_buffer); if (tmppath_len <= 0 || tmppath_len >= sizeof(tmppath_buffer)) { return basename; // an error, so just bail on tmppath } assert(tmppath_buffer[tmppath_len - 1] == '\\'); // API guarantees it return string(tmppath_buffer) + basename; } // A replacement for template_unittest.cc:CleanTestDir() void CreateOrCleanTestDir(const string& dirname) { string glob(PathJoin(dirname, "*")); WIN32_FIND_DATAA found; // that final A is for Ansi (as opposed to Unicode) HANDLE hFind = FindFirstFileA(glob.c_str(), &found); // A is for Ansi if (hFind == INVALID_HANDLE_VALUE) { // directory doesn't exist or some such _mkdir(dirname.c_str()); hFind = FindFirstFileA(glob.c_str(), &found); // Try again... } if (hFind != INVALID_HANDLE_VALUE) { do { if (strstr(found.cFileName, "template")) _unlink(PathJoin(dirname, found.cFileName).c_str()); } while (FindNextFileA(hFind, &found) != FALSE); // A is for Ansi FindClose(hFind); } } } void GetNamelist(const char* testdata_dir, vector* namelist) { string glob(ctemplate::PathJoin(testdata_dir, "template_unittest_test*")); WIN32_FIND_DATAA found; // that final A is for Ansi (as opposed to Unicode) HANDLE hFind = FindFirstFileA(glob.c_str(), &found); if (hFind == INVALID_HANDLE_VALUE) // no files matching the glob, probably return; // if we don't find any files, nothing to add to namelist do { namelist->push_back(found.cFileName); } while (FindNextFileA(hFind, &found) != FALSE); // A is for Ansi FindClose(hFind); } #endif /* __cplusplus */ ctemplate-ctemplate-2.4/src/windows/port.h000066400000000000000000000126021363122316600207330ustar00rootroot00000000000000/* Copyright (c) 2007, Google Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are * met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above * copyright notice, this list of conditions and the following disclaimer * in the documentation and/or other materials provided with the * distribution. * * Neither the name of Google Inc. nor the names of its * contributors may be used to endorse or promote products derived from * this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. * * --- * Author: Craig Silverstein * * These are some portability typedefs and defines to make it a bit * easier to compile this code under VC++. * * Several of these are taken from glib: * http://developer.gnome.org/doc/API/glib/glib-windows-compatability-functions.html */ #ifndef CTEMPLATE_WINDOWS_PORT_H_ #define CTEMPLATE_WINDOWS_PORT_H_ #include "windows/config.h" #ifdef _WIN32 #define USING_PORT_CC #define WIN32_LEAN_AND_MEAN /* We always want minimal includes */ #include #include /* because we so often use open/close/etc */ #include /* for _getcwd() */ #include /* for _utime() */ #include /* read in vsnprintf decl. before redifining it */ #include /* template_dictionary.cc uses va_copy */ #include /* for _strnicmp */ /* Note: the C++ #includes are all together at the bottom. This file is * used by both C and C++ code, so we put all the C++ together. */ /* 4244: otherwise we get problems when substracting two size_t's to an int * 4251: it's complaining about a private struct I've chosen not to dllexport * 4355: we use this in a constructor, but we do it safely * 4715: for some reason VC++ stopped realizing you can't return after abort() * 4800: we know we're casting ints/char*'s to bools, and we're ok with that * 4996: Yes, we're ok using "unsafe" functions like fopen() and strerror() */ #ifdef _MSC_VER #pragma warning(disable:4244 4251 4355 4715 4800 4996) #endif /* file I/O */ #define PATH_MAX 1024 #define access _access #define getcwd _getcwd #define open _open #define read _read #define write _write #define lseek _lseek #define close _close #define popen _popen #define pclose _pclose #ifndef R_OK /* mingw defines this, for instance */ #define R_OK 04 /* read-only (for access()) */ #endif #define S_ISDIR(m) (((m) & _S_IFMT) == _S_IFDIR) #define utime _utime #define utimbuf _utimbuf /* Not quite as lightweight as a hard-link, but more than good enough for us. */ #define link(oldpath, newpath) (!CopyFileA(oldpath, newpath, false)) #define strcasecmp _stricmp #define strncasecmp _strnicmp /* Sleep is in ms, on windows */ #define sleep(secs) Sleep((secs) * 1000) /* We can't just use _vsnprintf and _snprintf as drop-in-replacements, * because they don't always NUL-terminate. :-( We also can't use the * name vsnprintf, since windows defines that (but not snprintf (!)). */ #if !defined(__MINGW32__) && !defined(__MINGW64__) && (!defined(_MSC_VER) || _MSC_VER < 1900) /* mingw already defines */ extern CTEMPLATE_DLL_DECL int snprintf(char *str, size_t size, const char *format, ...); extern int CTEMPLATE_DLL_DECL safe_vsnprintf(char *str, size_t size, const char *format, va_list ap); #define vsnprintf(str, size, format, ap) safe_vsnprintf(str, size, format, ap) #define va_copy(dst, src) (dst) = (src) #endif /* #if !defined(__MINGW32__) && !defined(__MINGW64__) */ /* Windows doesn't support specifying the number of buckets as a * hash_map constructor arg, so we leave this blank. */ #define CTEMPLATE_SMALL_HASHTABLE #define DEFAULT_TEMPLATE_ROOTDIR ".." /* These are functions we have to override because they're O/S-specific */ #ifdef __cplusplus #include #include namespace ctemplate { extern CTEMPLATE_DLL_DECL std::string TmpFile(const char* basename); void CTEMPLATE_DLL_DECL CreateOrCleanTestDir(const std::string& dirname); } void CTEMPLATE_DLL_DECL GetNamelist(const char* testdata_dir, std::vector* namelist); #endif /* __cplusplus */ #ifndef __cplusplus /* I don't see how to get inlining for C code in MSVC. Ah well. */ #define inline #endif #endif /* _WIN32 */ #endif /* CTEMPLATE_WINDOWS_PORT_H_ */ ctemplate-ctemplate-2.4/src/windows/preprocess.sh000077500000000000000000000117341363122316600223270ustar00rootroot00000000000000#!/bin/sh # Copyright (c) 2007, Google Inc. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are # met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above # copyright notice, this list of conditions and the following disclaimer # in the documentation and/or other materials provided with the # distribution. # * Neither the name of Google Inc. nor the names of its # contributors may be used to endorse or promote products derived from # this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # --- # Author: Craig Silverstein # # This script is meant to be run at distribution-generation time, for # instance by autogen.sh. It does some of the work configure would # normally do, for windows systems. In particular, it expands all the # @...@ variables found in .in files, and puts them here, in the windows # directory. # # This script should be run before any new release. if [ -z "$1" ]; then echo "USAGE: $0 " exit 1 fi DLLDEF_MACRO_NAME="CTEMPLATE_DLL_DECL" # The text we put in every .h files we create. As a courtesy, we'll # include a helpful comment for windows users as to how to use # CTEMPLATE_DLL_DECL. Apparently sed expands \n into a newline. Good! DLLDEF_DEFINES="\ // NOTE: if you are statically linking the template library into your binary\n\ // (rather than using the template .dll), set '/D $DLLDEF_MACRO_NAME='\n\ // as a compiler flag in your project file to turn off the dllimports.\n\ #ifndef $DLLDEF_MACRO_NAME\n\ # define $DLLDEF_MACRO_NAME __declspec(dllimport)\n\ #endif" # template_cache.h gets a special DEFINE to work around the # difficulties in dll-exporting stl containers. Ugh. TEMPLATE_CACHE_DLLDEF_DEFINES="\ // NOTE: if you are statically linking the template library into your binary\n\ // (rather than using the template .dll), set '/D $DLLDEF_MACRO_NAME='\n\ // as a compiler flag in your project file to turn off the dllimports.\n\ #ifndef $DLLDEF_MACRO_NAME\n\ # define $DLLDEF_MACRO_NAME __declspec(dllimport)\n\ extern template class __declspec(dllimport) std::allocator;\n\ extern template class __declspec(dllimport) std::vector;\n\ #else\n\ template class __declspec(dllexport) std::allocator;\n\ template class __declspec(dllexport) std::vector;\n\ #endif" # Read all the windows config info into variables # In order for the 'set' to take, this requires putting all in a subshell. ( while read define varname value; do [ "$define" != "#define" ] && continue eval "$varname='$value'" done # Process all the .in files in the "ctemplate" subdirectory mkdir -p "$1/windows/ctemplate" for file in "$1"/ctemplate/*.in; do echo "Processing $file" outfile="$1/windows/ctemplate/`basename $file .in`" if test "`basename $file`" = template_cache.h.in; then MY_DLLDEF_DEFINES=$TEMPLATE_CACHE_DLLDEF_DEFINES else MY_DLLDEF_DEFINES=$DLLDEF_DEFINES fi # Besides replacing @...@, we also need to turn on dllimport # We also need to replace hash by hash_compare (annoying we hard-code :-( ) sed -e "s!@ac_windows_dllexport@!$DLLDEF_MACRO_NAME!g" \ -e "s!@ac_windows_dllexport_defines@!$MY_DLLDEF_DEFINES!g" \ -e "s!@ac_cv_cxx_hash_map@!$HASH_MAP_H!g" \ -e "s!@ac_cv_cxx_hash_set@!$HASH_SET_H!g" \ -e "s!@ac_cv_cxx_hash_namespace@!$HASH_NAMESPACE!g" \ -e "s!@ac_google_attribute@!${HAVE___ATTRIBUTE__:-0}!g" \ -e "s!@ac_google_end_namespace@!$_END_GOOGLE_NAMESPACE_!g" \ -e "s!@ac_google_namespace@!$GOOGLE_NAMESPACE!g" \ -e "s!@ac_google_start_namespace@!$_START_GOOGLE_NAMESPACE_!g" \ -e "s!@ac_htmlparser_namespace@!$HTMLPARSER_NAMESPACE!g" \ -e "s!@ac_cv_uint64@!unsigned __int64!g" \ -e "s!@ac_cv_have_stdint_h@!0!g" \ -e "s!@ac_cv_have_inttypes_h@!0!g" \ -e "s!@ac_have_attribute_weak@!0!g" \ -e "s!\\bhash\\b!hash_compare!g" \ "$file" > "$outfile" done ) < "$1/windows/config.h" echo "DONE" ctemplate-ctemplate-2.4/vsprojects/000077500000000000000000000000001363122316600175165ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/compile_test/000077500000000000000000000000001363122316600222055ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/compile_test/compile_test.vcxproj000077500000000000000000000141111363122316600263120ustar00rootroot00000000000000 Debug Win32 Release Win32 {4B263748-5F0F-468C-8C5C-ED2682BB6BE3} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)compile_test.exe true $(OutDir)compile_test.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)compile_test.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/diff_tpl_auto_escape/000077500000000000000000000000001363122316600236555ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/diff_tpl_auto_escape/diff_tpl_auto_escape.vcxproj000077500000000000000000000143521363122316600314410ustar00rootroot00000000000000 Debug Win32 Release Win32 {72CD1C2A-56F6-4DDA-957B-BCF181BB558E} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)diff_tpl_auto_escape.exe true $(OutDir)diff_tpl_auto_escape.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)diff_tpl_auto_escape.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/generate_fsm_c_test/000077500000000000000000000000001363122316600235165ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/generate_fsm_c_test/generate_fsm_c_test.vcxproj000077500000000000000000000123051363122316600311370ustar00rootroot00000000000000 Debug Win32 Release Win32 {8FB1935F-FC23-465D-8C18-C1466A99D08D} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)generate_fsm_c_test.exe true $(OutDir)generate_fsm_c_test.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)generate_fsm_c_test.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ctemplate-ctemplate-2.4/vsprojects/htmlparser_test/000077500000000000000000000000001363122316600227365ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/htmlparser_test/htmlparser_test.vcxproj000077500000000000000000000130111363122316600275720ustar00rootroot00000000000000 Debug Win32 Release Win32 {B105C0FF-C28A-480B-8298-A66AB4F1F0CD} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)htmlparser_test.exe true $(OutDir)htmlparser_test.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)htmlparser_test.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/libctemplate/000077500000000000000000000000001363122316600221635ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/libctemplate/libctemplate.vcxproj000077500000000000000000000366011363122316600262560ustar00rootroot00000000000000 Debug Win32 Release Win32 {FB27FBDB-E6C0-4D00-A7F8-1EEEF1B48ABC} libctemplate Win32Proj DynamicLibrary v140 MultiByte DynamicLibrary v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_WINDOWS;_USRDLL;LIBCTEMPLATE_EXPORTS;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)libctemplate-debug.dll true $(OutDir)libctemplate-debug.pdb Windows $(OutDir)libctemplate-debug.lib MachineX86 WIN32;NDEBUG;_WINDOWS;_USRDLL;LIBCTEMPLATE_EXPORTS;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)libctemplate.dll true Windows true true $(OutDir)libctemplate.lib MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ctemplate-ctemplate-2.4/vsprojects/make_tpl_varname_h/000077500000000000000000000000001363122316600233325ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/make_tpl_varname_h/make_tpl_varname_h.vcxproj000077500000000000000000000143431363122316600305730ustar00rootroot00000000000000 Debug Win32 Release Win32 {CFD560F2-1B16-4CEE-985D-B19FDE74513F} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)make_tpl_varname_h.exe true $(OutDir)make_tpl_varname_h.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)make_tpl_varname_h.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/statemachine_test/000077500000000000000000000000001363122316600232225ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/statemachine_test/statemachine_test.vcxproj000077500000000000000000000160511363122316600303510ustar00rootroot00000000000000 Debug Win32 Release Win32 {A105C0FF-C28A-480B-8298-A66AB4F1F0CD} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)statemachine_test.exe true $(OutDir)statemachine_test.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)statemachine_test.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL CompileAsC ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL CompileAsC ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL CompileAsC ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL CompileAsC ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL CompileAsC ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL CompileAsC ctemplate-ctemplate-2.4/vsprojects/template_cache_test/000077500000000000000000000000001363122316600235135ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_cache_test/template_cache_test.vcxproj000077500000000000000000000160341363122316600311340ustar00rootroot00000000000000 Debug Win32 Release Win32 {5105C0FF-C28A-480B-8298-A66AB4F1F0C0} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_cache_test.exe true $(OutDir)template_cache_test.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_cache_test.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/template_dictionary_unittest/000077500000000000000000000000001363122316600255155ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_dictionary_unittest/template_dictionary_unittest.vcxproj000077500000000000000000000161001363122316600351320ustar00rootroot00000000000000 Debug Win32 Release Win32 {9160CC7F-3BC6-49F1-A158-70DF579376CA} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_dictionary_unittest.exe true $(OutDir)template_dictionary_unittest.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_dictionary_unittest.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/template_modifiers_unittest/000077500000000000000000000000001363122316600253315ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_modifiers_unittest/template_modifiers_unittest.vcxproj000077500000000000000000000161671363122316600345770ustar00rootroot00000000000000 Debug Win32 Release Win32 {D9E45FD2-07AC-4EE8-9DA4-26E0427D619E} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_modifiers_unittest.exe true $(OutDir)template_modifiers_unittest.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_modifiers_unittest.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/template_regtest/000077500000000000000000000000001363122316600230665ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_regtest/template_regtest.vcxproj000077500000000000000000000160201363122316600300550ustar00rootroot00000000000000 Debug Win32 Release Win32 {0072B37E-DCDA-4128-BC12-7C0A7EF59016} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_regtest.exe true $(OutDir)template_regtest.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_regtest.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/template_setglobals_unittest/000077500000000000000000000000001363122316600255075ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_setglobals_unittest/template_setglobals_unittest.vcxproj000077500000000000000000000160671363122316600351320ustar00rootroot00000000000000 Debug Win32 Release Win32 {686099C0-A778-4D27-80B5-A7E051658D2F} template_setglobals_unittest Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_setglobals_unittest.exe true $(OutDir)template_setglobals_unittest.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_setglobals_unittest.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/template_test_util_test/000077500000000000000000000000001363122316600244645ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_test_util_test/template_test_util_test.vcxproj000077500000000000000000000160541363122316600330600ustar00rootroot00000000000000 Debug Win32 Release Win32 {87132061-D584-4388-A80B-D8560F8D0895} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_test_util_test.exe true $(OutDir)template_test_util_test.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_test_util_test.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/template_unittest/000077500000000000000000000000001363122316600232705ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_unittest/template_unittest.vcxproj000077500000000000000000000160241363122316600304650ustar00rootroot00000000000000 Debug Win32 Release Win32 {5105C0FF-C28A-480B-8298-A66AB4F1F0CD} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_unittest.exe true $(OutDir)template_unittest.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_unittest.exe true Console true true MachineX86 ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL {fb27fbdb-e6c0-4d00-a7f8-1eeef1b48abc} false ctemplate-ctemplate-2.4/vsprojects/template_unittest_static/000077500000000000000000000000001363122316600246375ustar00rootroot00000000000000ctemplate-ctemplate-2.4/vsprojects/template_unittest_static/template_unittest_static.vcxproj000077500000000000000000000474761363122316600334220ustar00rootroot00000000000000 Debug Win32 Release Win32 {ED69C82B-F593-418A-8D17-C64CBB11E603} Win32Proj Application v140 MultiByte Application v140 MultiByte <_ProjectFileVersion>14.0.25123.0 $(SolutionDir)$(Configuration)\ $(Configuration)\ true $(SolutionDir)$(Configuration)\ $(Configuration)\ false Disabled WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) true EnableFastChecks MultiThreadedDebug Level3 $(OutDir)template_unittest_static.exe true $(OutDir)template_unittest_static.pdb Console MachineX86 WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) MultiThreaded Level3 ProgramDatabase $(OutDir)template_unittest_static.exe true Console true true MachineX86 /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDebugDLL /D CTEMPLATE_DLL_DECL= %(AdditionalOptions) ..\..\src\windows; ..\..\src;%(AdditionalIncludeDirectories) MultiThreadedDLL